2024-09-13T16:11:03.511 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-13T16:11:03.535 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T16:11:03.639 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903803 branch: wip-rishabh-testing-20240910.095512-reef description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/iozone}} email: null first_in_suite: false job_id: '7903803' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240910.095512-reef ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 50bbe7f395997dcd871813949c49addbbf630361 subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported dry_run: true install: ceph: flavor: default sha1: 50bbe7f395997dcd871813949c49addbbf630361 kclient: client.0: mount_subvol_num: 1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240910.095512-reef sha1: 50bbe7f395997dcd871813949c49addbbf630361 owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1152 sha1: 50bbe7f395997dcd871813949c49addbbf630361 sleep_before_teardown: 0 subset: 984/1024 suite: fs suite_branch: wip-rishabh-testing-20240910.095512-reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_50bbe7f395997dcd871813949c49addbbf630361/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 50bbe7f395997dcd871813949c49addbbf630361 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi016.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI44qTDClOa0s2uxnYxPGBwUSEz1piedTBcZp+7+W4uj/SQmM5L8HH8b6/7jRLQq6bZIiNMlDXUN+Yiwtq0qYq8= smithi107.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE9bZ2/tBWHFS5fHcW9PB7rqJe0et0m3SC6TBicRVYUDx3CwAGr1nY4cAQfrJ5ewbPbR50RZ7z5cDUsTZrrvaRE= smithi111.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA+wVRUhDg1DLFziJsy8AVCPAMlKVRg0GRPb4koSGdJ6Y47Qy4/oARXOD1oJa6Xfa82i08ymlv2BxsZ2Z6szszY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/iozone.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-13_14:36:41 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-13T16:11:03.639 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_50bbe7f395997dcd871813949c49addbbf630361/qa; will attempt to use it 2024-09-13T16:11:03.639 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_50bbe7f395997dcd871813949c49addbbf630361/qa/tasks 2024-09-13T16:11:03.640 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-13T16:11:03.641 INFO:teuthology.task.internal:Checking packages... 2024-09-13T16:11:03.663 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '50bbe7f395997dcd871813949c49addbbf630361' 2024-09-13T16:11:03.663 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T16:11:03.663 INFO:teuthology.packaging:ref: None 2024-09-13T16:11:03.663 INFO:teuthology.packaging:tag: None 2024-09-13T16:11:03.663 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240910.095512-reef 2024-09-13T16:11:03.663 INFO:teuthology.packaging:sha1: 50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T16:11:03.664 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240910.095512-reef 2024-09-13T16:11:03.817 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-850.g50bbe7f3 2024-09-13T16:11:03.819 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-13T16:11:03.826 INFO:teuthology.task.internal:no buildpackages task found 2024-09-13T16:11:03.826 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-13T16:11:03.832 INFO:teuthology.task.internal:Saving configuration 2024-09-13T16:11:03.851 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-13T16:11:03.857 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-13T16:11:03.888 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903803', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 16:05:38.663361', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI44qTDClOa0s2uxnYxPGBwUSEz1piedTBcZp+7+W4uj/SQmM5L8HH8b6/7jRLQq6bZIiNMlDXUN+Yiwtq0qYq8='} 2024-09-13T16:11:03.936 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903803', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 16:05:38.665875', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE9bZ2/tBWHFS5fHcW9PB7rqJe0et0m3SC6TBicRVYUDx3CwAGr1nY4cAQfrJ5ewbPbR50RZ7z5cDUsTZrrvaRE='} 2024-09-13T16:11:03.967 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi111.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903803', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 16:05:38.664734', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA+wVRUhDg1DLFziJsy8AVCPAMlKVRg0GRPb4koSGdJ6Y47Qy4/oARXOD1oJa6Xfa82i08ymlv2BxsZ2Z6szszY='} 2024-09-13T16:11:03.968 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-13T16:11:03.973 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-13T16:11:03.973 INFO:teuthology.task.internal:roles: ubuntu@smithi107.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-13T16:11:03.973 INFO:teuthology.task.internal:roles: ubuntu@smithi111.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-13T16:11:03.973 INFO:teuthology.run_tasks:Running task console_log... 2024-09-13T16:11:04.082 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fbdedc3bbe0>, signals=[15]) 2024-09-13T16:11:04.082 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-13T16:11:04.088 INFO:teuthology.task.internal:Opening connections... 2024-09-13T16:11:04.088 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2024-09-13T16:11:04.089 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:11:04.175 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi107.front.sepia.ceph.com 2024-09-13T16:11:04.175 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:11:04.258 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi111.front.sepia.ceph.com 2024-09-13T16:11:04.259 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:11:04.343 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-13T16:11:04.349 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2024-09-13T16:11:04.368 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2024-09-13T16:11:04.368 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2024-09-13T16:11:04.426 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="CentOS Stream" 2024-09-13T16:11:04.426 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="9" 2024-09-13T16:11:04.426 INFO:teuthology.orchestra.run.smithi016.stdout:ID="centos" 2024-09-13T16:11:04.426 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="rhel fedora" 2024-09-13T16:11:04.426 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="9" 2024-09-13T16:11:04.426 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el9" 2024-09-13T16:11:04.426 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-13T16:11:04.427 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2024-09-13T16:11:04.427 INFO:teuthology.orchestra.run.smithi016.stdout:LOGO="fedora-logo-icon" 2024-09-13T16:11:04.427 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-13T16:11:04.427 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://centos.org/" 2024-09-13T16:11:04.427 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-13T16:11:04.427 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-13T16:11:04.427 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-13T16:11:04.427 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2024-09-13T16:11:04.459 DEBUG:teuthology.orchestra.run.smithi107:> uname -m 2024-09-13T16:11:04.477 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2024-09-13T16:11:04.477 DEBUG:teuthology.orchestra.run.smithi107:> cat /etc/os-release 2024-09-13T16:11:04.535 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="CentOS Stream" 2024-09-13T16:11:04.535 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="9" 2024-09-13T16:11:04.536 INFO:teuthology.orchestra.run.smithi107.stdout:ID="centos" 2024-09-13T16:11:04.536 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE="rhel fedora" 2024-09-13T16:11:04.536 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="9" 2024-09-13T16:11:04.536 INFO:teuthology.orchestra.run.smithi107.stdout:PLATFORM_ID="platform:el9" 2024-09-13T16:11:04.536 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-13T16:11:04.536 INFO:teuthology.orchestra.run.smithi107.stdout:ANSI_COLOR="0;31" 2024-09-13T16:11:04.536 INFO:teuthology.orchestra.run.smithi107.stdout:LOGO="fedora-logo-icon" 2024-09-13T16:11:04.536 INFO:teuthology.orchestra.run.smithi107.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-13T16:11:04.536 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://centos.org/" 2024-09-13T16:11:04.536 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-13T16:11:04.536 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-13T16:11:04.536 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-13T16:11:04.537 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2024-09-13T16:11:04.559 DEBUG:teuthology.orchestra.run.smithi111:> uname -m 2024-09-13T16:11:04.577 INFO:teuthology.orchestra.run.smithi111.stdout:x86_64 2024-09-13T16:11:04.578 DEBUG:teuthology.orchestra.run.smithi111:> cat /etc/os-release 2024-09-13T16:11:04.636 INFO:teuthology.orchestra.run.smithi111.stdout:NAME="CentOS Stream" 2024-09-13T16:11:04.636 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION="9" 2024-09-13T16:11:04.637 INFO:teuthology.orchestra.run.smithi111.stdout:ID="centos" 2024-09-13T16:11:04.637 INFO:teuthology.orchestra.run.smithi111.stdout:ID_LIKE="rhel fedora" 2024-09-13T16:11:04.637 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION_ID="9" 2024-09-13T16:11:04.637 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM_ID="platform:el9" 2024-09-13T16:11:04.637 INFO:teuthology.orchestra.run.smithi111.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-13T16:11:04.637 INFO:teuthology.orchestra.run.smithi111.stdout:ANSI_COLOR="0;31" 2024-09-13T16:11:04.637 INFO:teuthology.orchestra.run.smithi111.stdout:LOGO="fedora-logo-icon" 2024-09-13T16:11:04.637 INFO:teuthology.orchestra.run.smithi111.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-13T16:11:04.637 INFO:teuthology.orchestra.run.smithi111.stdout:HOME_URL="https://centos.org/" 2024-09-13T16:11:04.637 INFO:teuthology.orchestra.run.smithi111.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-13T16:11:04.637 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-13T16:11:04.637 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-13T16:11:04.638 INFO:teuthology.lock.ops:Updating smithi111.front.sepia.ceph.com on lock server 2024-09-13T16:11:04.667 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-13T16:11:04.684 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-13T16:11:04.764 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-13T16:11:04.764 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2024-09-13T16:11:04.767 DEBUG:teuthology.orchestra.run.smithi107:> test '!' -e /home/ubuntu/cephtest 2024-09-13T16:11:04.769 DEBUG:teuthology.orchestra.run.smithi111:> test '!' -e /home/ubuntu/cephtest 2024-09-13T16:11:04.787 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-13T16:11:04.793 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-13T16:11:04.793 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2024-09-13T16:11:04.825 DEBUG:teuthology.orchestra.run.smithi107:> test -z $(ls -A /var/lib/ceph) 2024-09-13T16:11:04.827 DEBUG:teuthology.orchestra.run.smithi111:> test -z $(ls -A /var/lib/ceph) 2024-09-13T16:11:04.921 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-13T16:11:05.020 INFO:teuthology.run_tasks:Running task kernel... 2024-09-13T16:11:05.036 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-13T16:11:05.036 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-13T16:11:05.036 DEBUG:teuthology.orchestra.run.smithi016:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T16:11:05.037 DEBUG:teuthology.orchestra.run.smithi107:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T16:11:05.038 DEBUG:teuthology.orchestra.run.smithi111:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T16:11:05.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:11:05.056 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-09-13T16:11:05.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:11:05.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:11:05.059 DEBUG:teuthology.orchestra.run.smithi111:> uname -r 2024-09-13T16:11:05.060 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2024-09-13T16:11:05.114 INFO:teuthology.orchestra.run.smithi016.stdout:5.14.0-503.el9.x86_64 2024-09-13T16:11:05.114 INFO:teuthology.task.kernel:Running kernel on smithi016: 5.14.0-503.el9.x86_64 2024-09-13T16:11:05.114 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum install -y kernel 2024-09-13T16:11:05.117 INFO:teuthology.orchestra.run.smithi111.stdout:5.14.0-503.el9.x86_64 2024-09-13T16:11:05.117 INFO:teuthology.task.kernel:Running kernel on smithi111: 5.14.0-503.el9.x86_64 2024-09-13T16:11:05.117 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum install -y kernel 2024-09-13T16:11:05.118 INFO:teuthology.orchestra.run.smithi107.stdout:5.14.0-503.el9.x86_64 2024-09-13T16:11:05.118 INFO:teuthology.task.kernel:Running kernel on smithi107: 5.14.0-503.el9.x86_64 2024-09-13T16:11:05.118 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum install -y kernel 2024-09-13T16:11:07.750 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - BaseOS 8.8 MB/s | 8.2 MB 00:00 2024-09-13T16:11:07.761 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 9.4 MB/s | 8.2 MB 00:00 2024-09-13T16:11:08.844 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.2 MB 00:01 2024-09-13T16:11:09.652 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-13T16:11:11.061 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 20 MB/s | 20 MB 00:01 2024-09-13T16:11:11.320 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 8.4 MB/s | 20 MB 00:02 2024-09-13T16:11:15.285 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.5 MB 00:01 2024-09-13T16:11:15.545 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-13T16:11:16.695 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.5 MB 00:01 2024-09-13T16:11:16.869 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 19 kB 00:00 2024-09-13T16:11:17.179 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 19 kB 00:00 2024-09-13T16:11:18.373 INFO:teuthology.orchestra.run.smithi111.stdout:Extra Packages for Enterprise Linux 16 MB/s | 23 MB 00:01 2024-09-13T16:11:18.471 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 19 kB 00:00 2024-09-13T16:11:18.549 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 18 MB/s | 23 MB 00:01 2024-09-13T16:11:19.541 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-13T16:11:23.754 INFO:teuthology.orchestra.run.smithi111.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-09-13T16:11:23.955 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-13T16:11:25.139 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-09-13T16:11:25.718 INFO:teuthology.orchestra.run.smithi111.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-13T16:11:25.718 INFO:teuthology.orchestra.run.smithi111.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-13T16:11:25.820 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-13T16:11:25.831 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-13T16:11:25.831 INFO:teuthology.orchestra.run.smithi111.stdout: Package Architecture Version Repository Size 2024-09-13T16:11:25.831 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-13T16:11:25.831 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2024-09-13T16:11:25.832 INFO:teuthology.orchestra.run.smithi111.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-13T16:11:25.832 INFO:teuthology.orchestra.run.smithi111.stdout:Installing dependencies: 2024-09-13T16:11:25.832 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-13T16:11:25.832 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-13T16:11:25.832 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-13T16:11:25.832 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T16:11:25.832 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2024-09-13T16:11:25.832 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-13T16:11:25.832 INFO:teuthology.orchestra.run.smithi111.stdout:Install 4 Packages 2024-09-13T16:11:25.832 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T16:11:25.833 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 79 M 2024-09-13T16:11:25.833 INFO:teuthology.orchestra.run.smithi111.stdout:Installed size: 126 M 2024-09-13T16:11:25.833 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2024-09-13T16:11:25.961 INFO:teuthology.orchestra.run.smithi107.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-13T16:11:25.961 INFO:teuthology.orchestra.run.smithi107.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-13T16:11:26.052 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-13T16:11:26.059 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-13T16:11:26.059 INFO:teuthology.orchestra.run.smithi107.stdout: Package Architecture Version Repository Size 2024-09-13T16:11:26.060 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-13T16:11:26.060 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-13T16:11:26.060 INFO:teuthology.orchestra.run.smithi107.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-13T16:11:26.060 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-13T16:11:26.060 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-13T16:11:26.060 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-13T16:11:26.061 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-13T16:11:26.061 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-13T16:11:26.061 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-13T16:11:26.061 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-13T16:11:26.061 INFO:teuthology.orchestra.run.smithi107.stdout:Install 4 Packages 2024-09-13T16:11:26.061 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-13T16:11:26.061 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 79 M 2024-09-13T16:11:26.062 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 126 M 2024-09-13T16:11:26.062 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-13T16:11:26.230 INFO:teuthology.orchestra.run.smithi111.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 77 kB/s | 23 kB 00:00 2024-09-13T16:11:26.373 INFO:teuthology.orchestra.run.smithi107.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 78 kB/s | 23 kB 00:00 2024-09-13T16:11:27.238 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-13T16:11:27.238 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-13T16:11:27.335 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-09-13T16:11:27.345 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-09-13T16:11:27.345 INFO:teuthology.orchestra.run.smithi016.stdout: Package Architecture Version Repository Size 2024-09-13T16:11:27.346 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-09-13T16:11:27.346 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-09-13T16:11:27.346 INFO:teuthology.orchestra.run.smithi016.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-13T16:11:27.346 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-09-13T16:11:27.346 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-13T16:11:27.346 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-13T16:11:27.346 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-13T16:11:27.346 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:11:27.346 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-09-13T16:11:27.346 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-09-13T16:11:27.346 INFO:teuthology.orchestra.run.smithi016.stdout:Install 4 Packages 2024-09-13T16:11:27.346 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:11:27.347 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 79 M 2024-09-13T16:11:27.347 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 126 M 2024-09-13T16:11:27.347 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-09-13T16:11:27.549 INFO:teuthology.orchestra.run.smithi016.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 219 kB/s | 23 kB 00:00 2024-09-13T16:11:27.840 INFO:teuthology.orchestra.run.smithi107.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 8.9 MB/s | 16 MB 00:01 2024-09-13T16:11:30.058 INFO:teuthology.orchestra.run.smithi016.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 6.0 MB/s | 16 MB 00:02 2024-09-13T16:11:30.924 INFO:teuthology.orchestra.run.smithi016.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 8.5 MB/s | 29 MB 00:03 2024-09-13T16:11:32.363 INFO:teuthology.orchestra.run.smithi111.stdout:(2/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 5.4 MB/s | 35 MB 00:06 2024-09-13T16:11:33.116 INFO:teuthology.orchestra.run.smithi016.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 6.1 MB/s | 35 MB 00:05 2024-09-13T16:11:33.116 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-09-13T16:11:33.116 INFO:teuthology.orchestra.run.smithi016.stdout:Total 14 MB/s | 79 MB 00:05 2024-09-13T16:11:33.588 INFO:teuthology.orchestra.run.smithi111.stdout:(3/4): kernel-core-5.14.0-505.el9.x86_64.rpm 2.0 MB/s | 16 MB 00:07 2024-09-13T16:11:33.832 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-09-13T16:11:34.235 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-09-13T16:11:34.235 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-09-13T16:11:35.192 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-09-13T16:11:35.193 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-09-13T16:11:36.388 INFO:teuthology.orchestra.run.smithi111.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 2.8 MB/s | 29 MB 00:10 2024-09-13T16:11:36.388 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-09-13T16:11:36.388 INFO:teuthology.orchestra.run.smithi111.stdout:Total 7.5 MB/s | 79 MB 00:10 2024-09-13T16:11:37.065 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2024-09-13T16:11:37.449 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2024-09-13T16:11:37.449 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2024-09-13T16:11:37.784 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-09-13T16:11:38.387 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2024-09-13T16:11:38.387 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2024-09-13T16:11:38.435 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-13T16:11:38.525 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T16:11:40.841 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2024-09-13T16:11:41.321 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T16:11:41.454 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T16:11:41.576 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-13T16:11:41.680 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T16:11:42.537 INFO:teuthology.orchestra.run.smithi107.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 1.8 MB/s | 29 MB 00:16 2024-09-13T16:11:43.397 INFO:teuthology.orchestra.run.smithi107.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 2.0 MB/s | 35 MB 00:17 2024-09-13T16:11:43.397 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-13T16:11:43.397 INFO:teuthology.orchestra.run.smithi107.stdout:Total 4.5 MB/s | 79 MB 00:17 2024-09-13T16:11:44.081 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-13T16:11:44.453 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-13T16:11:44.453 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-13T16:11:44.477 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T16:11:44.599 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T16:11:45.405 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-13T16:11:45.406 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-13T16:11:46.577 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T16:11:46.622 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T16:11:47.842 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-13T16:11:48.623 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-13T16:11:48.712 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T16:11:49.449 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T16:11:49.625 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T16:11:51.530 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T16:11:51.672 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T16:11:51.762 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T16:11:54.472 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T16:11:56.510 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T16:11:56.641 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T16:12:01.470 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T16:12:30.980 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T16:12:31.020 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-13T16:12:31.376 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T16:12:31.376 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-13T16:12:31.376 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T16:12:31.376 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T16:12:32.205 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T16:12:32.205 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:12:32.206 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-09-13T16:12:32.206 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-13T16:12:32.206 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-13T16:12:32.206 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-13T16:12:32.206 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-13T16:12:32.206 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:12:32.206 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-09-13T16:12:32.557 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T16:12:32.656 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T16:12:32.656 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T16:12:32.656 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-13T16:12:32.656 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T16:12:32.656 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T16:12:32.657 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-13T16:12:32.657 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi016.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T16:12:32.657 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T16:12:32.679 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T16:12:32.715 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T16:12:32.715 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T16:12:32.715 DEBUG:teuthology.orchestra.run.smithi016:> sudo rpm -qi grub2-tools 2024-09-13T16:12:32.720 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-13T16:12:32.749 INFO:teuthology.orchestra.run.smithi016.stdout:Name : grub2-tools 2024-09-13T16:12:32.749 INFO:teuthology.orchestra.run.smithi016.stdout:Epoch : 1 2024-09-13T16:12:32.749 INFO:teuthology.orchestra.run.smithi016.stdout:Version : 2.06 2024-09-13T16:12:32.749 INFO:teuthology.orchestra.run.smithi016.stdout:Release : 82.el9 2024-09-13T16:12:32.749 INFO:teuthology.orchestra.run.smithi016.stdout:Architecture: x86_64 2024-09-13T16:12:32.749 INFO:teuthology.orchestra.run.smithi016.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-13T16:12:32.749 INFO:teuthology.orchestra.run.smithi016.stdout:Group : Unspecified 2024-09-13T16:12:32.749 INFO:teuthology.orchestra.run.smithi016.stdout:Size : 8274670 2024-09-13T16:12:32.750 INFO:teuthology.orchestra.run.smithi016.stdout:License : GPLv3+ 2024-09-13T16:12:32.750 INFO:teuthology.orchestra.run.smithi016.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-13T16:12:32.750 INFO:teuthology.orchestra.run.smithi016.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-13T16:12:32.750 INFO:teuthology.orchestra.run.smithi016.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-13T16:12:32.750 INFO:teuthology.orchestra.run.smithi016.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-13T16:12:32.750 INFO:teuthology.orchestra.run.smithi016.stdout:Packager : builder@centos.org 2024-09-13T16:12:32.750 INFO:teuthology.orchestra.run.smithi016.stdout:Vendor : CentOS 2024-09-13T16:12:32.750 INFO:teuthology.orchestra.run.smithi016.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-13T16:12:32.750 INFO:teuthology.orchestra.run.smithi016.stdout:Summary : Support tools for GRUB. 2024-09-13T16:12:32.750 INFO:teuthology.orchestra.run.smithi016.stdout:Description : 2024-09-13T16:12:32.750 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:12:32.750 INFO:teuthology.orchestra.run.smithi016.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-13T16:12:32.750 INFO:teuthology.orchestra.run.smithi016.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-13T16:12:32.750 INFO:teuthology.orchestra.run.smithi016.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-13T16:12:32.751 INFO:teuthology.orchestra.run.smithi016.stdout:hardware devices. 2024-09-13T16:12:32.751 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:12:32.751 INFO:teuthology.orchestra.run.smithi016.stdout:This subpackage provides tools for support of all platforms. 2024-09-13T16:12:32.751 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-13T16:12:32.752 INFO:teuthology.task.kernel:Updating grub on smithi016 to boot 5.14.0-505.el9.x86_64 2024-09-13T16:12:32.752 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-13T16:12:33.126 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T16:12:33.126 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-13T16:12:33.127 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T16:12:33.127 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T16:12:33.449 INFO:teuthology.orchestra.run.smithi016.stderr:Generating grub configuration file ... 2024-09-13T16:12:34.009 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T16:12:34.009 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T16:12:34.009 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2024-09-13T16:12:34.009 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-13T16:12:34.009 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-13T16:12:34.009 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-13T16:12:34.010 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-13T16:12:34.010 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T16:12:34.010 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-13T16:12:34.369 DEBUG:teuthology.orchestra.run.smithi111:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T16:12:34.459 INFO:teuthology.orchestra.run.smithi111.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T16:12:34.459 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T16:12:34.460 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-13T16:12:34.460 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T16:12:34.460 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T16:12:34.460 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-09-13T16:12:34.460 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi111.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T16:12:34.460 DEBUG:teuthology.orchestra.run.smithi111:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T16:12:34.518 INFO:teuthology.orchestra.run.smithi111.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T16:12:34.519 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T16:12:34.519 DEBUG:teuthology.orchestra.run.smithi111:> sudo rpm -qi grub2-tools 2024-09-13T16:12:34.557 INFO:teuthology.orchestra.run.smithi111.stdout:Name : grub2-tools 2024-09-13T16:12:34.557 INFO:teuthology.orchestra.run.smithi111.stdout:Epoch : 1 2024-09-13T16:12:34.557 INFO:teuthology.orchestra.run.smithi111.stdout:Version : 2.06 2024-09-13T16:12:34.557 INFO:teuthology.orchestra.run.smithi111.stdout:Release : 82.el9 2024-09-13T16:12:34.558 INFO:teuthology.orchestra.run.smithi111.stdout:Architecture: x86_64 2024-09-13T16:12:34.558 INFO:teuthology.orchestra.run.smithi111.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-13T16:12:34.558 INFO:teuthology.orchestra.run.smithi111.stdout:Group : Unspecified 2024-09-13T16:12:34.558 INFO:teuthology.orchestra.run.smithi111.stdout:Size : 8274670 2024-09-13T16:12:34.558 INFO:teuthology.orchestra.run.smithi111.stdout:License : GPLv3+ 2024-09-13T16:12:34.558 INFO:teuthology.orchestra.run.smithi111.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-13T16:12:34.558 INFO:teuthology.orchestra.run.smithi111.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-13T16:12:34.558 INFO:teuthology.orchestra.run.smithi111.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-13T16:12:34.558 INFO:teuthology.orchestra.run.smithi111.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-13T16:12:34.558 INFO:teuthology.orchestra.run.smithi111.stdout:Packager : builder@centos.org 2024-09-13T16:12:34.558 INFO:teuthology.orchestra.run.smithi111.stdout:Vendor : CentOS 2024-09-13T16:12:34.558 INFO:teuthology.orchestra.run.smithi111.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-13T16:12:34.558 INFO:teuthology.orchestra.run.smithi111.stdout:Summary : Support tools for GRUB. 2024-09-13T16:12:34.558 INFO:teuthology.orchestra.run.smithi111.stdout:Description : 2024-09-13T16:12:34.559 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T16:12:34.559 INFO:teuthology.orchestra.run.smithi111.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-13T16:12:34.559 INFO:teuthology.orchestra.run.smithi111.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-13T16:12:34.559 INFO:teuthology.orchestra.run.smithi111.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-13T16:12:34.559 INFO:teuthology.orchestra.run.smithi111.stdout:hardware devices. 2024-09-13T16:12:34.559 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T16:12:34.559 INFO:teuthology.orchestra.run.smithi111.stdout:This subpackage provides tools for support of all platforms. 2024-09-13T16:12:34.560 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-13T16:12:34.560 INFO:teuthology.task.kernel:Updating grub on smithi111 to boot 5.14.0-505.el9.x86_64 2024-09-13T16:12:34.560 DEBUG:teuthology.orchestra.run.smithi111:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-13T16:12:34.786 INFO:teuthology.orchestra.run.smithi016.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-13T16:12:34.827 INFO:teuthology.orchestra.run.smithi016.stderr:done 2024-09-13T16:12:34.829 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2024-09-13T16:12:34.844 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.T2mdwKBOBo 2024-09-13T16:12:34.844 DEBUG:teuthology.orchestra.run.smithi016:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.T2mdwKBOBo 2024-09-13T16:12:34.910 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.T2mdwKBOBo 2024-09-13T16:12:35.050 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.T2mdwKBOBo is 6KB 2024-09-13T16:12:35.099 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.T2mdwKBOBo 2024-09-13T16:12:35.114 DEBUG:teuthology.orchestra.run.smithi016:> sudo /bin/ls /boot/loader/entries || true 2024-09-13T16:12:35.176 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-13T16:12:35.176 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-13T16:12:35.176 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-13T16:12:35.176 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-13T16:12:35.177 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-13T16:12:35.201 INFO:teuthology.orchestra.run.smithi111.stderr:Generating grub configuration file ... 2024-09-13T16:12:35.314 DEBUG:teuthology.orchestra.run.smithi016:> sudo shutdown -r now 2024-09-13T16:12:36.566 INFO:teuthology.orchestra.run.smithi111.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-13T16:12:36.603 INFO:teuthology.orchestra.run.smithi111.stderr:done 2024-09-13T16:12:36.605 DEBUG:teuthology.orchestra.run.smithi111:> mktemp 2024-09-13T16:12:36.621 INFO:teuthology.orchestra.run.smithi111.stdout:/tmp/tmp.Vzg0XG7714 2024-09-13T16:12:36.621 DEBUG:teuthology.orchestra.run.smithi111:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Vzg0XG7714 2024-09-13T16:12:36.687 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0666 /tmp/tmp.Vzg0XG7714 2024-09-13T16:12:36.827 DEBUG:teuthology.orchestra.remote:smithi111:/tmp/tmp.Vzg0XG7714 is 6KB 2024-09-13T16:12:36.875 DEBUG:teuthology.orchestra.run.smithi111:> rm -fr /tmp/tmp.Vzg0XG7714 2024-09-13T16:12:36.889 DEBUG:teuthology.orchestra.run.smithi111:> sudo /bin/ls /boot/loader/entries || true 2024-09-13T16:12:36.953 INFO:teuthology.orchestra.run.smithi111.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-13T16:12:36.953 INFO:teuthology.orchestra.run.smithi111.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-13T16:12:36.953 INFO:teuthology.orchestra.run.smithi111.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-13T16:12:36.953 INFO:teuthology.orchestra.run.smithi111.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-13T16:12:36.954 DEBUG:teuthology.orchestra.run.smithi111:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-13T16:12:37.128 DEBUG:teuthology.orchestra.run.smithi111:> sudo shutdown -r now 2024-09-13T16:12:39.718 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T16:12:39.757 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-13T16:12:40.109 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T16:12:40.109 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-13T16:12:40.109 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T16:12:40.110 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T16:12:40.938 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T16:12:40.938 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-13T16:12:40.938 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-13T16:12:40.938 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-13T16:12:40.938 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-13T16:12:40.938 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-13T16:12:40.939 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-13T16:12:40.939 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-13T16:12:40.939 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-13T16:12:41.295 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T16:12:41.420 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T16:12:41.421 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T16:12:41.421 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-13T16:12:41.421 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T16:12:41.421 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T16:12:41.421 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-13T16:12:41.421 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi107.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T16:12:41.421 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T16:12:41.479 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T16:12:41.480 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T16:12:41.480 DEBUG:teuthology.orchestra.run.smithi107:> sudo rpm -qi grub2-tools 2024-09-13T16:12:41.522 INFO:teuthology.orchestra.run.smithi107.stdout:Name : grub2-tools 2024-09-13T16:12:41.522 INFO:teuthology.orchestra.run.smithi107.stdout:Epoch : 1 2024-09-13T16:12:41.522 INFO:teuthology.orchestra.run.smithi107.stdout:Version : 2.06 2024-09-13T16:12:41.522 INFO:teuthology.orchestra.run.smithi107.stdout:Release : 82.el9 2024-09-13T16:12:41.522 INFO:teuthology.orchestra.run.smithi107.stdout:Architecture: x86_64 2024-09-13T16:12:41.522 INFO:teuthology.orchestra.run.smithi107.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-13T16:12:41.522 INFO:teuthology.orchestra.run.smithi107.stdout:Group : Unspecified 2024-09-13T16:12:41.522 INFO:teuthology.orchestra.run.smithi107.stdout:Size : 8274670 2024-09-13T16:12:41.523 INFO:teuthology.orchestra.run.smithi107.stdout:License : GPLv3+ 2024-09-13T16:12:41.523 INFO:teuthology.orchestra.run.smithi107.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-13T16:12:41.523 INFO:teuthology.orchestra.run.smithi107.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-13T16:12:41.523 INFO:teuthology.orchestra.run.smithi107.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-13T16:12:41.523 INFO:teuthology.orchestra.run.smithi107.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-13T16:12:41.523 INFO:teuthology.orchestra.run.smithi107.stdout:Packager : builder@centos.org 2024-09-13T16:12:41.523 INFO:teuthology.orchestra.run.smithi107.stdout:Vendor : CentOS 2024-09-13T16:12:41.523 INFO:teuthology.orchestra.run.smithi107.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-13T16:12:41.523 INFO:teuthology.orchestra.run.smithi107.stdout:Summary : Support tools for GRUB. 2024-09-13T16:12:41.523 INFO:teuthology.orchestra.run.smithi107.stdout:Description : 2024-09-13T16:12:41.523 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-13T16:12:41.523 INFO:teuthology.orchestra.run.smithi107.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-13T16:12:41.523 INFO:teuthology.orchestra.run.smithi107.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-13T16:12:41.523 INFO:teuthology.orchestra.run.smithi107.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-13T16:12:41.523 INFO:teuthology.orchestra.run.smithi107.stdout:hardware devices. 2024-09-13T16:12:41.524 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-13T16:12:41.524 INFO:teuthology.orchestra.run.smithi107.stdout:This subpackage provides tools for support of all platforms. 2024-09-13T16:12:41.525 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-13T16:12:41.525 INFO:teuthology.task.kernel:Updating grub on smithi107 to boot 5.14.0-505.el9.x86_64 2024-09-13T16:12:41.525 DEBUG:teuthology.orchestra.run.smithi107:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-13T16:12:42.171 INFO:teuthology.orchestra.run.smithi107.stderr:Generating grub configuration file ... 2024-09-13T16:12:43.513 INFO:teuthology.orchestra.run.smithi107.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-13T16:12:43.542 INFO:teuthology.orchestra.run.smithi107.stderr:done 2024-09-13T16:12:43.544 DEBUG:teuthology.orchestra.run.smithi107:> mktemp 2024-09-13T16:12:43.560 INFO:teuthology.orchestra.run.smithi107.stdout:/tmp/tmp.8SmFFvND38 2024-09-13T16:12:43.561 DEBUG:teuthology.orchestra.run.smithi107:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.8SmFFvND38 2024-09-13T16:12:43.628 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0666 /tmp/tmp.8SmFFvND38 2024-09-13T16:12:43.774 DEBUG:teuthology.orchestra.remote:smithi107:/tmp/tmp.8SmFFvND38 is 6KB 2024-09-13T16:12:43.794 DEBUG:teuthology.orchestra.run.smithi107:> rm -fr /tmp/tmp.8SmFFvND38 2024-09-13T16:12:43.809 DEBUG:teuthology.orchestra.run.smithi107:> sudo /bin/ls /boot/loader/entries || true 2024-09-13T16:12:43.872 INFO:teuthology.orchestra.run.smithi107.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-13T16:12:43.872 INFO:teuthology.orchestra.run.smithi107.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-13T16:12:43.872 INFO:teuthology.orchestra.run.smithi107.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-13T16:12:43.872 INFO:teuthology.orchestra.run.smithi107.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-13T16:12:43.873 DEBUG:teuthology.orchestra.run.smithi107:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-13T16:12:44.028 DEBUG:teuthology.orchestra.run.smithi107:> sudo shutdown -r now 2024-09-13T16:13:05.337 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-13T16:13:05.338 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-13T16:13:05.338 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:13:07.133 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-09-13T16:13:07.133 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-13T16:13:07.133 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:13:14.033 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-13T16:13:14.034 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-13T16:13:14.034 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:13:41.825 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2024-09-13T16:13:50.834 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-13T16:13:50.835 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:13:53.917 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2024-09-13T16:14:05.377 DEBUG:teuthology.orchestra.remote:timed out 2024-09-13T16:14:05.918 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-13T16:14:05.919 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:14:14.053 DEBUG:teuthology.orchestra.remote:timed out 2024-09-13T16:14:14.378 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-13T16:14:14.379 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:14:14.756 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-09-13T16:14:15.143 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-13T16:14:15.143 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-13T16:14:15.144 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-09-13T16:14:15.159 INFO:teuthology.orchestra.run.smithi016.stdout:5.14.0-505.el9.x86_64 2024-09-13T16:14:15.159 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-13T16:14:15.159 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T16:14:15.159 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-13T16:14:16.160 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-13T16:14:16.161 DEBUG:teuthology.orchestra.run.smithi016:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T16:14:16.278 INFO:teuthology.orchestra.run.smithi016.stdout:ttyS1 2024-09-13T16:14:16.304 DEBUG:teuthology.parallel:result is None 2024-09-13T16:14:23.054 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-13T16:14:23.055 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:14:24.317 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2024-09-13T16:14:30.600 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-09-13T16:14:30.977 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-13T16:14:30.977 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-13T16:14:30.977 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2024-09-13T16:14:30.992 INFO:teuthology.orchestra.run.smithi107.stdout:5.14.0-505.el9.x86_64 2024-09-13T16:14:30.992 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-13T16:14:30.992 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T16:14:30.992 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-13T16:14:31.993 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-13T16:14:31.993 DEBUG:teuthology.orchestra.run.smithi107:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T16:14:32.082 INFO:teuthology.orchestra.run.smithi107.stdout:ttyS1 2024-09-13T16:14:32.114 DEBUG:teuthology.parallel:result is None 2024-09-13T16:14:39.318 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-13T16:14:39.318 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:14:39.751 DEBUG:teuthology.orchestra.run.smithi111:> true 2024-09-13T16:14:40.159 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-13T16:14:40.159 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-505.el9.x86_64"... 2024-09-13T16:14:40.159 DEBUG:teuthology.orchestra.run.smithi111:> uname -r 2024-09-13T16:14:40.174 INFO:teuthology.orchestra.run.smithi111.stdout:5.14.0-505.el9.x86_64 2024-09-13T16:14:40.174 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-13T16:14:40.174 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T16:14:40.174 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-13T16:14:41.175 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-09-13T16:14:41.175 DEBUG:teuthology.orchestra.run.smithi111:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T16:14:41.238 INFO:teuthology.orchestra.run.smithi111.stdout:ttyS1 2024-09-13T16:14:41.262 DEBUG:teuthology.parallel:result is None 2024-09-13T16:14:41.263 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-13T16:14:41.271 INFO:teuthology.task.internal:Creating test directory... 2024-09-13T16:14:41.271 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T16:14:41.274 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T16:14:41.278 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T16:14:41.295 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-13T16:14:41.301 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-13T16:14:41.311 INFO:teuthology.task.internal:Creating archive directory... 2024-09-13T16:14:41.311 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T16:14:41.334 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T16:14:41.337 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T16:14:41.448 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-13T16:14:41.454 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-13T16:14:41.455 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T16:14:41.457 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T16:14:41.459 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T16:14:41.487 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T16:14:41.488 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T16:14:41.491 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T16:14:41.499 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T16:14:41.500 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T16:14:41.502 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T16:14:41.503 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-13T16:14:41.510 INFO:teuthology.task.internal:Configuring sudo... 2024-09-13T16:14:41.510 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T16:14:41.543 DEBUG:teuthology.orchestra.run.smithi107:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T16:14:41.546 DEBUG:teuthology.orchestra.run.smithi111:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T16:14:41.571 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-13T16:14:41.585 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-13T16:14:41.586 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T16:14:41.613 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T16:14:41.616 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T16:14:41.631 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T16:14:41.723 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T16:14:41.759 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T16:14:41.851 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T16:14:41.886 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-13T16:14:41.886 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T16:14:41.953 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T16:14:42.005 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T16:14:42.049 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T16:14:42.137 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T16:14:42.175 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-13T16:14:42.175 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T16:14:42.241 DEBUG:teuthology.orchestra.run.smithi111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T16:14:42.293 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T16:14:42.344 DEBUG:teuthology.orchestra.run.smithi111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T16:14:42.396 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T16:14:42.427 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T16:14:42.427 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T16:14:42.492 DEBUG:teuthology.orchestra.run.smithi016:> sudo service rsyslog restart 2024-09-13T16:14:42.494 DEBUG:teuthology.orchestra.run.smithi107:> sudo service rsyslog restart 2024-09-13T16:14:42.496 DEBUG:teuthology.orchestra.run.smithi111:> sudo service rsyslog restart 2024-09-13T16:14:42.542 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T16:14:42.553 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T16:14:42.592 INFO:teuthology.orchestra.run.smithi111.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T16:14:42.894 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-13T16:14:42.902 INFO:teuthology.task.internal:Starting timer... 2024-09-13T16:14:42.902 INFO:teuthology.run_tasks:Running task pcp... 2024-09-13T16:14:42.947 INFO:teuthology.run_tasks:Running task selinux... 2024-09-13T16:14:42.955 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-13T16:14:42.956 DEBUG:teuthology.orchestra.run.smithi016:> sudo service auditd rotate 2024-09-13T16:14:43.018 INFO:teuthology.orchestra.run.smithi016.stdout:Rotating logs: 2024-09-13T16:14:43.020 DEBUG:teuthology.orchestra.run.smithi107:> sudo service auditd rotate 2024-09-13T16:14:43.115 INFO:teuthology.orchestra.run.smithi107.stdout:Rotating logs: 2024-09-13T16:14:43.117 DEBUG:teuthology.orchestra.run.smithi111:> sudo service auditd rotate 2024-09-13T16:14:43.161 INFO:teuthology.orchestra.run.smithi111.stdout:Rotating logs: 2024-09-13T16:14:43.163 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-13T16:14:43.163 DEBUG:teuthology.orchestra.run.smithi016:> /usr/sbin/getenforce 2024-09-13T16:14:43.188 INFO:teuthology.orchestra.run.smithi016.stdout:Permissive 2024-09-13T16:14:43.188 DEBUG:teuthology.orchestra.run.smithi107:> /usr/sbin/getenforce 2024-09-13T16:14:43.216 INFO:teuthology.orchestra.run.smithi107.stdout:Permissive 2024-09-13T16:14:43.216 DEBUG:teuthology.orchestra.run.smithi111:> /usr/sbin/getenforce 2024-09-13T16:14:43.241 INFO:teuthology.orchestra.run.smithi111.stdout:Permissive 2024-09-13T16:14:43.242 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi016.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi107.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi111.front.sepia.ceph.com': 'permissive'} 2024-09-13T16:14:43.242 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-13T16:14:43.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:14:43.271 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-13T16:14:43.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:14:43.299 DEBUG:teuthology.orchestra.run.smithi111:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-13T16:14:43.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:14:43.325 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-13T16:14:43.326 DEBUG:teuthology.orchestra.run.smithi016:> sudo /usr/sbin/setenforce permissive 2024-09-13T16:14:43.361 DEBUG:teuthology.orchestra.run.smithi107:> sudo /usr/sbin/setenforce permissive 2024-09-13T16:14:43.394 DEBUG:teuthology.orchestra.run.smithi111:> sudo /usr/sbin/setenforce permissive 2024-09-13T16:14:43.430 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-13T16:14:43.445 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-13T16:14:43.448 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-13T16:14:43.569 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-13T16:14:43.580 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-13T16:14:43.581 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi016.front.sepia.ceph.com,smithi107.front.sepia.ceph.com,smithi111.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-13T16:18:31.801 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi016.front.sepia.ceph.com'), Remote(name='ubuntu@smithi107.front.sepia.ceph.com'), Remote(name='ubuntu@smithi111.front.sepia.ceph.com')] 2024-09-13T16:18:31.802 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-13T16:18:31.804 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:18:31.894 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-09-13T16:18:31.975 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-13T16:18:31.975 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-13T16:18:31.976 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:18:32.080 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-09-13T16:18:32.162 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-13T16:18:32.162 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-13T16:18:32.163 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:18:32.247 DEBUG:teuthology.orchestra.run.smithi111:> true 2024-09-13T16:18:32.325 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-13T16:18:32.326 INFO:teuthology.run_tasks:Running task clock... 2024-09-13T16:18:32.337 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-13T16:18:32.337 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T16:18:32.337 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T16:18:32.340 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T16:18:32.340 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T16:18:32.342 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T16:18:32.343 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T16:18:32.373 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-13T16:18:32.374 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-13T16:18:32.390 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-13T16:18:32.390 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-13T16:18:32.395 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-13T16:18:32.410 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-13T16:18:32.415 INFO:teuthology.orchestra.run.smithi107.stderr:sudo: ntpd: command not found 2024-09-13T16:18:32.418 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: ntpd: command not found 2024-09-13T16:18:32.427 INFO:teuthology.orchestra.run.smithi107.stdout:506 Cannot talk to daemon 2024-09-13T16:18:32.430 INFO:teuthology.orchestra.run.smithi016.stdout:506 Cannot talk to daemon 2024-09-13T16:18:32.434 INFO:teuthology.orchestra.run.smithi111.stderr:sudo: ntpd: command not found 2024-09-13T16:18:32.442 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-13T16:18:32.445 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-13T16:18:32.446 INFO:teuthology.orchestra.run.smithi111.stdout:506 Cannot talk to daemon 2024-09-13T16:18:32.456 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-13T16:18:32.460 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-13T16:18:32.461 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-13T16:18:32.474 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-13T16:18:32.512 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-09-13T16:18:32.516 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T16:18:32.516 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-09-13T16:18:32.516 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T16:18:32.516 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T16:18:32.516 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T16:18:32.516 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T16:18:32.517 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-09-13T16:18:32.521 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T16:18:32.521 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-09-13T16:18:32.521 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T16:18:32.521 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T16:18:32.521 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T16:18:32.521 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T16:18:32.533 INFO:teuthology.orchestra.run.smithi111.stderr:bash: line 1: ntpq: command not found 2024-09-13T16:18:32.536 INFO:teuthology.orchestra.run.smithi111.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T16:18:32.537 INFO:teuthology.orchestra.run.smithi111.stdout:=============================================================================== 2024-09-13T16:18:32.537 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T16:18:32.537 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T16:18:32.537 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T16:18:32.537 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T16:18:32.538 INFO:teuthology.run_tasks:Running task install... 2024-09-13T16:18:32.547 DEBUG:teuthology.task.install:project ceph 2024-09-13T16:18:32.548 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '50bbe7f395997dcd871813949c49addbbf630361'}} 2024-09-13T16:18:32.548 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '50bbe7f395997dcd871813949c49addbbf630361'} 2024-09-13T16:18:32.548 INFO:teuthology.task.install:Using flavor: default 2024-09-13T16:18:32.553 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-13T16:18:32.553 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-09-13T16:18:32.554 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '50bbe7f395997dcd871813949c49addbbf630361', 'tag': None, 'wait_for_package': False} 2024-09-13T16:18:32.555 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T16:18:32.557 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '50bbe7f395997dcd871813949c49addbbf630361', 'tag': None, 'wait_for_package': False} 2024-09-13T16:18:32.557 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T16:18:32.558 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '50bbe7f395997dcd871813949c49addbbf630361', 'tag': None, 'wait_for_package': False} 2024-09-13T16:18:32.559 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T16:18:32.713 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/centos/9/flavors/default/ 2024-09-13T16:18:32.713 INFO:teuthology.task.install.rpm:Package version is 18.2.4-850.g50bbe7f3 2024-09-13T16:18:32.725 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/centos/9/flavors/default/ 2024-09-13T16:18:32.725 INFO:teuthology.task.install.rpm:Package version is 18.2.4-850.g50bbe7f3 2024-09-13T16:18:32.726 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/centos/9/flavors/default/ 2024-09-13T16:18:32.726 INFO:teuthology.task.install.rpm:Package version is 18.2.4-850.g50bbe7f3 2024-09-13T16:18:32.835 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-13T16:18:32.835 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-13T16:18:32.835 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-13T16:18:32.844 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-13T16:18:32.845 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T16:18:32.845 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-13T16:18:32.853 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-13T16:18:32.853 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-13T16:18:32.853 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-13T16:18:32.861 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-13T16:18:32.861 DEBUG:teuthology.orchestra.run.smithi107:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/50bbe7f395997dcd871813949c49addbbf630361/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-13T16:18:32.870 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-13T16:18:32.871 DEBUG:teuthology.orchestra.run.smithi111:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/50bbe7f395997dcd871813949c49addbbf630361/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-13T16:18:32.881 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-13T16:18:32.882 DEBUG:teuthology.orchestra.run.smithi016:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/50bbe7f395997dcd871813949c49addbbf630361/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-13T16:18:32.927 DEBUG:teuthology.orchestra.run.smithi107:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-13T16:18:32.936 DEBUG:teuthology.orchestra.run.smithi111:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-13T16:18:32.950 DEBUG:teuthology.orchestra.run.smithi016:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-13T16:18:33.002 DEBUG:teuthology.orchestra.run.smithi107:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-13T16:18:33.011 DEBUG:teuthology.orchestra.run.smithi111:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-13T16:18:33.028 DEBUG:teuthology.orchestra.run.smithi016:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-13T16:18:33.066 INFO:teuthology.orchestra.run.smithi107.stdout:check_obsoletes = 1 2024-09-13T16:18:33.067 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum clean all 2024-09-13T16:18:33.075 INFO:teuthology.orchestra.run.smithi111.stdout:check_obsoletes = 1 2024-09-13T16:18:33.077 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum clean all 2024-09-13T16:18:33.094 INFO:teuthology.orchestra.run.smithi016.stdout:check_obsoletes = 1 2024-09-13T16:18:33.095 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum clean all 2024-09-13T16:18:33.292 INFO:teuthology.orchestra.run.smithi107.stdout:45 files removed 2024-09-13T16:18:33.319 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-13T16:18:33.335 INFO:teuthology.orchestra.run.smithi016.stdout:45 files removed 2024-09-13T16:18:33.337 INFO:teuthology.orchestra.run.smithi111.stdout:45 files removed 2024-09-13T16:18:33.363 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-13T16:18:33.364 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-13T16:18:33.869 INFO:teuthology.orchestra.run.smithi107.stdout:ceph packages for x86_64 293 kB/s | 77 kB 00:00 2024-09-13T16:18:33.916 INFO:teuthology.orchestra.run.smithi111.stdout:ceph packages for x86_64 289 kB/s | 77 kB 00:00 2024-09-13T16:18:33.926 INFO:teuthology.orchestra.run.smithi016.stdout:ceph packages for x86_64 293 kB/s | 77 kB 00:00 2024-09-13T16:18:34.144 INFO:teuthology.orchestra.run.smithi107.stdout:ceph noarch packages 50 kB/s | 11 kB 00:00 2024-09-13T16:18:34.208 INFO:teuthology.orchestra.run.smithi111.stdout:ceph noarch packages 46 kB/s | 11 kB 00:00 2024-09-13T16:18:34.210 INFO:teuthology.orchestra.run.smithi016.stdout:ceph noarch packages 48 kB/s | 11 kB 00:00 2024-09-13T16:18:34.420 INFO:teuthology.orchestra.run.smithi107.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-13T16:18:34.483 INFO:teuthology.orchestra.run.smithi111.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-13T16:18:34.485 INFO:teuthology.orchestra.run.smithi016.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-13T16:18:34.985 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.2 MB 00:00 2024-09-13T16:18:36.241 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.2 MB 00:01 2024-09-13T16:18:36.345 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.2 MB 00:01 2024-09-13T16:18:41.485 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - AppStream 4.9 MB/s | 20 MB 00:04 2024-09-13T16:18:41.870 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 4.6 MB/s | 20 MB 00:04 2024-09-13T16:18:44.088 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 2.6 MB/s | 20 MB 00:07 2024-09-13T16:18:45.702 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-09-13T16:18:46.070 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-09-13T16:18:47.519 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 19 kB 00:00 2024-09-13T16:18:47.621 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 19 kB 00:00 2024-09-13T16:18:48.596 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-13T16:18:48.948 INFO:teuthology.orchestra.run.smithi111.stdout:Extra Packages for Enterprise Linux 17 MB/s | 23 MB 00:01 2024-09-13T16:18:49.041 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 17 MB/s | 23 MB 00:01 2024-09-13T16:18:50.055 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 94 kB/s | 19 kB 00:00 2024-09-13T16:18:51.217 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-13T16:18:54.413 INFO:teuthology.orchestra.run.smithi111.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-13T16:18:54.572 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 45 kB/s | 1.7 kB 00:00 2024-09-13T16:18:56.708 INFO:teuthology.orchestra.run.smithi111.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-13T16:18:56.708 INFO:teuthology.orchestra.run.smithi111.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-13T16:18:56.721 INFO:teuthology.orchestra.run.smithi111.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-13T16:18:56.722 INFO:teuthology.orchestra.run.smithi111.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-13T16:18:56.723 INFO:teuthology.orchestra.run.smithi111.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-13T16:18:56.724 INFO:teuthology.orchestra.run.smithi111.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-13T16:18:56.726 INFO:teuthology.orchestra.run.smithi111.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-13T16:18:56.728 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-13T16:18:56.729 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-13T16:18:56.788 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-13T16:18:56.805 INFO:teuthology.orchestra.run.smithi111.stdout:====================================================================================== 2024-09-13T16:18:56.805 INFO:teuthology.orchestra.run.smithi111.stdout: Package Arch Version Repository Size 2024-09-13T16:18:56.805 INFO:teuthology.orchestra.run.smithi111.stdout:====================================================================================== 2024-09-13T16:18:56.805 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2024-09-13T16:18:56.805 INFO:teuthology.orchestra.run.smithi111.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-13T16:18:56.805 INFO:teuthology.orchestra.run.smithi111.stdout: ceph x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 6.5 k 2024-09-13T16:18:56.806 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-base x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 5.1 M 2024-09-13T16:18:56.806 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-fuse x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 852 k 2024-09-13T16:18:56.806 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 143 k 2024-09-13T16:18:56.806 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 1.5 M 2024-09-13T16:18:56.806 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-cephadm noarch 2:18.2.4-850.g50bbe7f3.el9 ceph-noarch 139 k 2024-09-13T16:18:56.806 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-dashboard noarch 2:18.2.4-850.g50bbe7f3.el9 ceph-noarch 3.5 M 2024-09-13T16:18:56.806 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-850.g50bbe7f3.el9 ceph-noarch 7.4 M 2024-09-13T16:18:56.806 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-rook noarch 2:18.2.4-850.g50bbe7f3.el9 ceph-noarch 49 k 2024-09-13T16:18:56.806 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-radosgw x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 7.8 M 2024-09-13T16:18:56.806 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-test x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 40 M 2024-09-13T16:18:56.806 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-volume noarch 2:18.2.4-850.g50bbe7f3.el9 ceph-noarch 264 k 2024-09-13T16:18:56.806 INFO:teuthology.orchestra.run.smithi111.stdout: cephadm noarch 2:18.2.4-850.g50bbe7f3.el9 ceph-noarch 226 k 2024-09-13T16:18:56.807 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-mirror x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 224 k 2024-09-13T16:18:56.807 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-top noarch 2:18.2.4-850.g50bbe7f3.el9 ceph-noarch 18 k 2024-09-13T16:18:56.807 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-13T16:18:56.807 INFO:teuthology.orchestra.run.smithi111.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-13T16:18:56.807 INFO:teuthology.orchestra.run.smithi111.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-13T16:18:56.807 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs-devel x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 31 k 2024-09-13T16:18:56.807 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs2 x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 712 k 2024-09-13T16:18:56.807 INFO:teuthology.orchestra.run.smithi111.stdout: librados-devel x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 127 k 2024-09-13T16:18:56.807 INFO:teuthology.orchestra.run.smithi111.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-13T16:18:56.807 INFO:teuthology.orchestra.run.smithi111.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-13T16:18:56.807 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-13T16:18:56.807 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-13T16:18:56.808 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-13T16:18:56.808 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-13T16:18:56.808 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cephfs x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 162 k 2024-09-13T16:18:56.808 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rados x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 319 k 2024-09-13T16:18:56.808 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rbd x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 301 k 2024-09-13T16:18:56.808 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rgw x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 99 k 2024-09-13T16:18:56.808 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-fuse x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 87 k 2024-09-13T16:18:56.808 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-mirror x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 3.0 M 2024-09-13T16:18:56.808 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-nbd x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 172 k 2024-09-13T16:18:56.808 INFO:teuthology.orchestra.run.smithi111.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-13T16:18:56.808 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrading: 2024-09-13T16:18:56.808 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-13T16:18:56.809 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-13T16:18:56.809 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-13T16:18:56.809 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-13T16:18:56.809 INFO:teuthology.orchestra.run.smithi111.stdout: librados2 x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 3.3 M 2024-09-13T16:18:56.809 INFO:teuthology.orchestra.run.smithi111.stdout: librbd1 x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 3.0 M 2024-09-13T16:18:56.809 INFO:teuthology.orchestra.run.smithi111.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-13T16:18:56.809 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-13T16:18:56.809 INFO:teuthology.orchestra.run.smithi111.stdout:Installing dependencies: 2024-09-13T16:18:56.809 INFO:teuthology.orchestra.run.smithi111.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-13T16:18:56.809 INFO:teuthology.orchestra.run.smithi111.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-13T16:18:56.810 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-common x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 18 M 2024-09-13T16:18:56.810 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-grafana-dashboards noarch 2:18.2.4-850.g50bbe7f3.el9 ceph-noarch 24 k 2024-09-13T16:18:56.810 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mds x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 2.1 M 2024-09-13T16:18:56.810 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-modules-core noarch 2:18.2.4-850.g50bbe7f3.el9 ceph-noarch 246 k 2024-09-13T16:18:56.810 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mon x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 4.7 M 2024-09-13T16:18:56.810 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-osd x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 17 M 2024-09-13T16:18:56.810 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-prometheus-alerts noarch 2:18.2.4-850.g50bbe7f3.el9 ceph-noarch 16 k 2024-09-13T16:18:56.810 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-selinux x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 25 k 2024-09-13T16:18:56.810 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-13T16:18:56.810 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-13T16:18:56.810 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-13T16:18:56.811 INFO:teuthology.orchestra.run.smithi111.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-13T16:18:56.811 INFO:teuthology.orchestra.run.smithi111.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-13T16:18:56.811 INFO:teuthology.orchestra.run.smithi111.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-13T16:18:56.811 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-13T16:18:56.811 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-13T16:18:56.811 INFO:teuthology.orchestra.run.smithi111.stdout: libcephsqlite x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 166 k 2024-09-13T16:18:56.811 INFO:teuthology.orchestra.run.smithi111.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-13T16:18:56.811 INFO:teuthology.orchestra.run.smithi111.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-13T16:18:56.811 INFO:teuthology.orchestra.run.smithi111.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-13T16:18:56.811 INFO:teuthology.orchestra.run.smithi111.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-13T16:18:56.811 INFO:teuthology.orchestra.run.smithi111.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-13T16:18:56.811 INFO:teuthology.orchestra.run.smithi111.stdout: libradosstriper1 x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 477 k 2024-09-13T16:18:56.811 INFO:teuthology.orchestra.run.smithi111.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-13T16:18:56.811 INFO:teuthology.orchestra.run.smithi111.stdout: librgw2 x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 4.5 M 2024-09-13T16:18:56.812 INFO:teuthology.orchestra.run.smithi111.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-13T16:18:56.812 INFO:teuthology.orchestra.run.smithi111.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-13T16:18:56.812 INFO:teuthology.orchestra.run.smithi111.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-13T16:18:56.812 INFO:teuthology.orchestra.run.smithi111.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-13T16:18:56.812 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-13T16:18:56.812 INFO:teuthology.orchestra.run.smithi111.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-13T16:18:56.812 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-13T16:18:56.812 INFO:teuthology.orchestra.run.smithi111.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-13T16:18:56.812 INFO:teuthology.orchestra.run.smithi111.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-13T16:18:56.812 INFO:teuthology.orchestra.run.smithi111.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-13T16:18:56.812 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-13T16:18:56.812 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-argparse x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 45 k 2024-09-13T16:18:56.813 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-common x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 129 k 2024-09-13T16:18:56.813 INFO:teuthology.orchestra.run.smithi111.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-13T16:18:56.813 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-13T16:18:56.813 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-13T16:18:56.813 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-13T16:18:56.813 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-13T16:18:56.813 INFO:teuthology.orchestra.run.smithi111.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-13T16:18:56.813 INFO:teuthology.orchestra.run.smithi111.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-13T16:18:56.813 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-13T16:18:56.813 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-13T16:18:56.813 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-13T16:18:56.813 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-13T16:18:56.813 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-13T16:18:56.814 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-13T16:18:56.814 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-13T16:18:56.814 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-13T16:18:56.814 INFO:teuthology.orchestra.run.smithi111.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-13T16:18:56.814 INFO:teuthology.orchestra.run.smithi111.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-13T16:18:56.814 INFO:teuthology.orchestra.run.smithi111.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-13T16:18:56.814 INFO:teuthology.orchestra.run.smithi111.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-13T16:18:56.814 INFO:teuthology.orchestra.run.smithi111.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-13T16:18:56.814 INFO:teuthology.orchestra.run.smithi111.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-13T16:18:56.814 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-13T16:18:56.814 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-13T16:18:56.814 INFO:teuthology.orchestra.run.smithi111.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-13T16:18:56.814 INFO:teuthology.orchestra.run.smithi111.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-13T16:18:56.814 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-13T16:18:56.815 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-13T16:18:56.815 INFO:teuthology.orchestra.run.smithi111.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-13T16:18:56.815 INFO:teuthology.orchestra.run.smithi111.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-13T16:18:56.815 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-13T16:18:56.815 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-13T16:18:56.815 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-13T16:18:56.815 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-13T16:18:56.815 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-13T16:18:56.815 INFO:teuthology.orchestra.run.smithi111.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-13T16:18:56.815 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-13T16:18:56.815 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-13T16:18:56.815 INFO:teuthology.orchestra.run.smithi111.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-13T16:18:56.815 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-13T16:18:56.815 INFO:teuthology.orchestra.run.smithi111.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-13T16:18:56.816 INFO:teuthology.orchestra.run.smithi111.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-13T16:18:56.816 INFO:teuthology.orchestra.run.smithi111.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-13T16:18:56.816 INFO:teuthology.orchestra.run.smithi111.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-13T16:18:56.816 INFO:teuthology.orchestra.run.smithi111.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-13T16:18:56.816 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-13T16:18:56.816 INFO:teuthology.orchestra.run.smithi111.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-13T16:18:56.816 INFO:teuthology.orchestra.run.smithi111.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-13T16:18:56.816 INFO:teuthology.orchestra.run.smithi111.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-13T16:18:56.816 INFO:teuthology.orchestra.run.smithi111.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-13T16:18:56.816 INFO:teuthology.orchestra.run.smithi111.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-13T16:18:56.816 INFO:teuthology.orchestra.run.smithi111.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-13T16:18:56.816 INFO:teuthology.orchestra.run.smithi111.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-13T16:18:56.816 INFO:teuthology.orchestra.run.smithi111.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-13T16:18:56.816 INFO:teuthology.orchestra.run.smithi111.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-13T16:18:56.816 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T16:18:56.817 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2024-09-13T16:18:56.817 INFO:teuthology.orchestra.run.smithi111.stdout:====================================================================================== 2024-09-13T16:18:56.817 INFO:teuthology.orchestra.run.smithi111.stdout:Install 130 Packages 2024-09-13T16:18:56.817 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrade 8 Packages 2024-09-13T16:18:56.817 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T16:18:56.817 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 203 M 2024-09-13T16:18:56.817 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2024-09-13T16:18:56.856 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-13T16:18:56.857 INFO:teuthology.orchestra.run.smithi107.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-13T16:18:56.857 INFO:teuthology.orchestra.run.smithi107.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-13T16:18:56.867 INFO:teuthology.orchestra.run.smithi107.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-13T16:18:56.868 INFO:teuthology.orchestra.run.smithi107.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-13T16:18:56.869 INFO:teuthology.orchestra.run.smithi107.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-13T16:18:56.870 INFO:teuthology.orchestra.run.smithi107.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-13T16:18:56.871 INFO:teuthology.orchestra.run.smithi107.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-13T16:18:56.874 INFO:teuthology.orchestra.run.smithi107.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-13T16:18:56.874 INFO:teuthology.orchestra.run.smithi107.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-13T16:18:56.933 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-13T16:18:56.946 INFO:teuthology.orchestra.run.smithi107.stdout:====================================================================================== 2024-09-13T16:18:56.946 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-13T16:18:56.946 INFO:teuthology.orchestra.run.smithi107.stdout:====================================================================================== 2024-09-13T16:18:56.946 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-13T16:18:56.946 INFO:teuthology.orchestra.run.smithi107.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-13T16:18:56.947 INFO:teuthology.orchestra.run.smithi107.stdout: ceph x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 6.5 k 2024-09-13T16:18:56.947 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 5.1 M 2024-09-13T16:18:56.947 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 852 k 2024-09-13T16:18:56.947 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 143 k 2024-09-13T16:18:56.947 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 1.5 M 2024-09-13T16:18:56.947 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm noarch 2:18.2.4-850.g50bbe7f3.el9 ceph-noarch 139 k 2024-09-13T16:18:56.947 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard noarch 2:18.2.4-850.g50bbe7f3.el9 ceph-noarch 3.5 M 2024-09-13T16:18:56.948 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-850.g50bbe7f3.el9 ceph-noarch 7.4 M 2024-09-13T16:18:56.948 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook noarch 2:18.2.4-850.g50bbe7f3.el9 ceph-noarch 49 k 2024-09-13T16:18:56.948 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 7.8 M 2024-09-13T16:18:56.948 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 40 M 2024-09-13T16:18:56.948 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume noarch 2:18.2.4-850.g50bbe7f3.el9 ceph-noarch 264 k 2024-09-13T16:18:56.948 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm noarch 2:18.2.4-850.g50bbe7f3.el9 ceph-noarch 226 k 2024-09-13T16:18:56.948 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-mirror x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 224 k 2024-09-13T16:18:56.948 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-top noarch 2:18.2.4-850.g50bbe7f3.el9 ceph-noarch 18 k 2024-09-13T16:18:56.948 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-13T16:18:56.948 INFO:teuthology.orchestra.run.smithi107.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-13T16:18:56.948 INFO:teuthology.orchestra.run.smithi107.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-13T16:18:56.948 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 31 k 2024-09-13T16:18:56.949 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2 x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 712 k 2024-09-13T16:18:56.949 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 127 k 2024-09-13T16:18:56.949 INFO:teuthology.orchestra.run.smithi107.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-13T16:18:56.949 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-13T16:18:56.949 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-13T16:18:56.949 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-13T16:18:56.949 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-13T16:18:56.949 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-13T16:18:56.949 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 162 k 2024-09-13T16:18:56.949 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 319 k 2024-09-13T16:18:56.949 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 301 k 2024-09-13T16:18:56.949 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 99 k 2024-09-13T16:18:56.949 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 87 k 2024-09-13T16:18:56.950 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 3.0 M 2024-09-13T16:18:56.950 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 172 k 2024-09-13T16:18:56.950 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-13T16:18:56.950 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrading: 2024-09-13T16:18:56.950 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-13T16:18:56.950 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-13T16:18:56.950 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-13T16:18:56.950 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-13T16:18:56.950 INFO:teuthology.orchestra.run.smithi107.stdout: librados2 x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 3.3 M 2024-09-13T16:18:56.950 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1 x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 3.0 M 2024-09-13T16:18:56.950 INFO:teuthology.orchestra.run.smithi107.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-13T16:18:56.950 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-13T16:18:56.950 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-13T16:18:56.951 INFO:teuthology.orchestra.run.smithi107.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-13T16:18:56.951 INFO:teuthology.orchestra.run.smithi107.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-13T16:18:56.951 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 18 M 2024-09-13T16:18:56.951 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards noarch 2:18.2.4-850.g50bbe7f3.el9 ceph-noarch 24 k 2024-09-13T16:18:56.951 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 2.1 M 2024-09-13T16:18:56.951 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core noarch 2:18.2.4-850.g50bbe7f3.el9 ceph-noarch 246 k 2024-09-13T16:18:56.951 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 4.7 M 2024-09-13T16:18:56.951 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 17 M 2024-09-13T16:18:56.951 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts noarch 2:18.2.4-850.g50bbe7f3.el9 ceph-noarch 16 k 2024-09-13T16:18:56.951 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 25 k 2024-09-13T16:18:56.951 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-13T16:18:56.951 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-13T16:18:56.951 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-13T16:18:56.952 INFO:teuthology.orchestra.run.smithi107.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-13T16:18:56.952 INFO:teuthology.orchestra.run.smithi107.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-13T16:18:56.952 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-13T16:18:56.952 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-13T16:18:56.952 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-13T16:18:56.952 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 166 k 2024-09-13T16:18:56.952 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-13T16:18:56.952 INFO:teuthology.orchestra.run.smithi107.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-13T16:18:56.952 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-13T16:18:56.952 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-13T16:18:56.952 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-13T16:18:56.952 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1 x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 477 k 2024-09-13T16:18:56.952 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-13T16:18:56.952 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2 x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 4.5 M 2024-09-13T16:18:56.953 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-13T16:18:56.953 INFO:teuthology.orchestra.run.smithi107.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-13T16:18:56.953 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-13T16:18:56.953 INFO:teuthology.orchestra.run.smithi107.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-13T16:18:56.953 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-13T16:18:56.953 INFO:teuthology.orchestra.run.smithi107.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-13T16:18:56.953 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-13T16:18:56.953 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-13T16:18:56.953 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-13T16:18:56.953 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-13T16:18:56.953 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-13T16:18:56.953 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 45 k 2024-09-13T16:18:56.953 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 129 k 2024-09-13T16:18:56.954 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-13T16:18:56.954 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-13T16:18:56.954 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-13T16:18:56.954 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-13T16:18:56.954 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-13T16:18:56.954 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-13T16:18:56.954 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-13T16:18:56.954 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-13T16:18:56.954 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-13T16:18:56.954 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-13T16:18:56.954 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-13T16:18:56.954 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-13T16:18:56.955 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-13T16:18:56.955 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-13T16:18:56.955 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-13T16:18:56.955 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-13T16:18:56.955 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-13T16:18:56.955 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-13T16:18:56.955 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-13T16:18:56.955 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-13T16:18:56.955 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-13T16:18:56.955 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-13T16:18:56.955 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-13T16:18:56.955 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-13T16:18:56.955 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-13T16:18:56.956 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-13T16:18:56.956 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-13T16:18:56.956 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-13T16:18:56.956 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-13T16:18:56.956 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-13T16:18:56.956 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-13T16:18:56.956 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-13T16:18:56.956 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-13T16:18:56.956 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-13T16:18:56.956 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-13T16:18:56.956 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-13T16:18:56.956 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-13T16:18:56.956 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-13T16:18:56.957 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-13T16:18:56.957 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-13T16:18:56.957 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-13T16:18:56.957 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-13T16:18:56.957 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-13T16:18:56.957 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-13T16:18:56.957 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-13T16:18:56.957 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-13T16:18:56.957 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-13T16:18:56.957 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-13T16:18:56.957 INFO:teuthology.orchestra.run.smithi107.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-13T16:18:56.957 INFO:teuthology.orchestra.run.smithi107.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-13T16:18:56.957 INFO:teuthology.orchestra.run.smithi107.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-13T16:18:56.958 INFO:teuthology.orchestra.run.smithi107.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-13T16:18:56.958 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-13T16:18:56.958 INFO:teuthology.orchestra.run.smithi107.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-13T16:18:56.958 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-13T16:18:56.958 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-13T16:18:56.958 INFO:teuthology.orchestra.run.smithi107.stdout:====================================================================================== 2024-09-13T16:18:56.958 INFO:teuthology.orchestra.run.smithi107.stdout:Install 130 Packages 2024-09-13T16:18:56.958 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrade 8 Packages 2024-09-13T16:18:56.958 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-13T16:18:56.959 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 203 M 2024-09-13T16:18:56.959 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-13T16:18:57.280 INFO:teuthology.orchestra.run.smithi111.stdout:(1/138): ceph-18.2.4-850.g50bbe7f3.el9.x86_64.r 50 kB/s | 6.5 kB 00:00 2024-09-13T16:18:57.489 INFO:teuthology.orchestra.run.smithi111.stdout:(2/138): ceph-fuse-18.2.4-850.g50bbe7f3.el9.x86 4.0 MB/s | 852 kB 00:00 2024-09-13T16:18:57.564 INFO:teuthology.orchestra.run.smithi111.stdout:(3/138): ceph-immutable-object-cache-18.2.4-850 1.9 MB/s | 143 kB 00:00 2024-09-13T16:18:57.656 INFO:teuthology.orchestra.run.smithi111.stdout:(4/138): ceph-base-18.2.4-850.g50bbe7f3.el9.x86 10 MB/s | 5.1 MB 00:00 2024-09-13T16:18:57.723 INFO:teuthology.orchestra.run.smithi111.stdout:(5/138): ceph-mds-18.2.4-850.g50bbe7f3.el9.x86_ 13 MB/s | 2.1 MB 00:00 2024-09-13T16:18:57.799 INFO:teuthology.orchestra.run.smithi111.stdout:(6/138): ceph-mgr-18.2.4-850.g50bbe7f3.el9.x86_ 10 MB/s | 1.5 MB 00:00 2024-09-13T16:18:57.981 INFO:teuthology.orchestra.run.smithi107.stdout:(1/138): ceph-18.2.4-850.g50bbe7f3.el9.x86_64.r 29 kB/s | 6.5 kB 00:00 2024-09-13T16:18:58.074 INFO:teuthology.orchestra.run.smithi111.stdout:(7/138): ceph-common-18.2.4-850.g50bbe7f3.el9.x 20 MB/s | 18 MB 00:00 2024-09-13T16:18:58.157 INFO:teuthology.orchestra.run.smithi111.stdout:(8/138): ceph-mon-18.2.4-850.g50bbe7f3.el9.x86_ 11 MB/s | 4.7 MB 00:00 2024-09-13T16:18:58.248 INFO:teuthology.orchestra.run.smithi107.stdout:(2/138): ceph-fuse-18.2.4-850.g50bbe7f3.el9.x86 3.1 MB/s | 852 kB 00:00 2024-09-13T16:18:58.257 INFO:teuthology.orchestra.run.smithi111.stdout:(9/138): ceph-selinux-18.2.4-850.g50bbe7f3.el9. 252 kB/s | 25 kB 00:00 2024-09-13T16:18:58.515 INFO:teuthology.orchestra.run.smithi107.stdout:(3/138): ceph-base-18.2.4-850.g50bbe7f3.el9.x86 6.7 MB/s | 5.1 MB 00:00 2024-09-13T16:18:58.599 INFO:teuthology.orchestra.run.smithi111.stdout:(10/138): ceph-radosgw-18.2.4-850.g50bbe7f3.el9 15 MB/s | 7.8 MB 00:00 2024-09-13T16:18:58.700 INFO:teuthology.orchestra.run.smithi111.stdout:(11/138): cephfs-mirror-18.2.4-850.g50bbe7f3.el 2.2 MB/s | 224 kB 00:00 2024-09-13T16:18:58.740 INFO:teuthology.orchestra.run.smithi107.stdout:(4/138): ceph-mds-18.2.4-850.g50bbe7f3.el9.x86_ 9.3 MB/s | 2.1 MB 00:00 2024-09-13T16:18:58.765 INFO:teuthology.orchestra.run.smithi107.stdout:(5/138): ceph-immutable-object-cache-18.2.4-850 276 kB/s | 143 kB 00:00 2024-09-13T16:18:58.900 INFO:teuthology.orchestra.run.smithi111.stdout:(12/138): ceph-osd-18.2.4-850.g50bbe7f3.el9.x86 15 MB/s | 17 MB 00:01 2024-09-13T16:18:58.907 INFO:teuthology.orchestra.run.smithi107.stdout:(6/138): ceph-mgr-18.2.4-850.g50bbe7f3.el9.x86_ 8.7 MB/s | 1.5 MB 00:00 2024-09-13T16:18:59.025 INFO:teuthology.orchestra.run.smithi111.stdout:(13/138): libcephfs-devel-18.2.4-850.g50bbe7f3. 95 kB/s | 31 kB 00:00 2024-09-13T16:18:59.059 INFO:teuthology.orchestra.run.smithi111.stdout:(14/138): libcephfs2-18.2.4-850.g50bbe7f3.el9.x 4.4 MB/s | 712 kB 00:00 2024-09-13T16:18:59.084 INFO:teuthology.orchestra.run.smithi111.stdout:(15/138): libcephsqlite-18.2.4-850.g50bbe7f3.el 2.8 MB/s | 166 kB 00:00 2024-09-13T16:18:59.091 INFO:teuthology.orchestra.run.smithi107.stdout:(7/138): ceph-mon-18.2.4-850.g50bbe7f3.el9.x86_ 14 MB/s | 4.7 MB 00:00 2024-09-13T16:18:59.109 INFO:teuthology.orchestra.run.smithi111.stdout:(16/138): librados-devel-18.2.4-850.g50bbe7f3.e 2.5 MB/s | 127 kB 00:00 2024-09-13T16:18:59.134 INFO:teuthology.orchestra.run.smithi111.stdout:(17/138): libradosstriper1-18.2.4-850.g50bbe7f3 9.3 MB/s | 477 kB 00:00 2024-09-13T16:18:59.210 INFO:teuthology.orchestra.run.smithi111.stdout:(18/138): python3-ceph-argparse-18.2.4-850.g50b 601 kB/s | 45 kB 00:00 2024-09-13T16:18:59.302 INFO:teuthology.orchestra.run.smithi111.stdout:(19/138): python3-ceph-common-18.2.4-850.g50bbe 1.4 MB/s | 129 kB 00:00 2024-09-13T16:18:59.303 INFO:teuthology.orchestra.run.smithi016.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-13T16:18:59.304 INFO:teuthology.orchestra.run.smithi016.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-13T16:18:59.317 INFO:teuthology.orchestra.run.smithi016.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-13T16:18:59.318 INFO:teuthology.orchestra.run.smithi016.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-13T16:18:59.319 INFO:teuthology.orchestra.run.smithi016.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-13T16:18:59.320 INFO:teuthology.orchestra.run.smithi016.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-13T16:18:59.322 INFO:teuthology.orchestra.run.smithi016.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-13T16:18:59.325 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-13T16:18:59.326 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-13T16:18:59.393 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-09-13T16:18:59.394 INFO:teuthology.orchestra.run.smithi111.stdout:(20/138): librgw2-18.2.4-850.g50bbe7f3.el9.x86_ 16 MB/s | 4.5 MB 00:00 2024-09-13T16:18:59.406 INFO:teuthology.orchestra.run.smithi016.stdout:====================================================================================== 2024-09-13T16:18:59.406 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-09-13T16:18:59.406 INFO:teuthology.orchestra.run.smithi016.stdout:====================================================================================== 2024-09-13T16:18:59.406 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-09-13T16:18:59.407 INFO:teuthology.orchestra.run.smithi016.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-13T16:18:59.407 INFO:teuthology.orchestra.run.smithi016.stdout: ceph x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 6.5 k 2024-09-13T16:18:59.407 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 5.1 M 2024-09-13T16:18:59.407 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 852 k 2024-09-13T16:18:59.407 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 143 k 2024-09-13T16:18:59.407 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 1.5 M 2024-09-13T16:18:59.407 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm noarch 2:18.2.4-850.g50bbe7f3.el9 ceph-noarch 139 k 2024-09-13T16:18:59.407 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard noarch 2:18.2.4-850.g50bbe7f3.el9 ceph-noarch 3.5 M 2024-09-13T16:18:59.407 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-850.g50bbe7f3.el9 ceph-noarch 7.4 M 2024-09-13T16:18:59.407 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook noarch 2:18.2.4-850.g50bbe7f3.el9 ceph-noarch 49 k 2024-09-13T16:18:59.407 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 7.8 M 2024-09-13T16:18:59.408 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 40 M 2024-09-13T16:18:59.408 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume noarch 2:18.2.4-850.g50bbe7f3.el9 ceph-noarch 264 k 2024-09-13T16:18:59.408 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm noarch 2:18.2.4-850.g50bbe7f3.el9 ceph-noarch 226 k 2024-09-13T16:18:59.408 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 224 k 2024-09-13T16:18:59.408 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top noarch 2:18.2.4-850.g50bbe7f3.el9 ceph-noarch 18 k 2024-09-13T16:18:59.408 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-13T16:18:59.408 INFO:teuthology.orchestra.run.smithi016.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-13T16:18:59.408 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-13T16:18:59.408 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 31 k 2024-09-13T16:18:59.408 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2 x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 712 k 2024-09-13T16:18:59.408 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 127 k 2024-09-13T16:18:59.409 INFO:teuthology.orchestra.run.smithi016.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-13T16:18:59.409 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-13T16:18:59.409 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-13T16:18:59.409 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-13T16:18:59.409 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-13T16:18:59.409 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-13T16:18:59.409 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 162 k 2024-09-13T16:18:59.409 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 319 k 2024-09-13T16:18:59.409 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 301 k 2024-09-13T16:18:59.409 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 99 k 2024-09-13T16:18:59.409 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 87 k 2024-09-13T16:18:59.409 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 3.0 M 2024-09-13T16:18:59.410 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 172 k 2024-09-13T16:18:59.410 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-13T16:18:59.410 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2024-09-13T16:18:59.410 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-13T16:18:59.410 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-13T16:18:59.410 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-13T16:18:59.410 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-13T16:18:59.410 INFO:teuthology.orchestra.run.smithi016.stdout: librados2 x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 3.3 M 2024-09-13T16:18:59.410 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1 x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 3.0 M 2024-09-13T16:18:59.410 INFO:teuthology.orchestra.run.smithi016.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-13T16:18:59.410 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-13T16:18:59.410 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-09-13T16:18:59.410 INFO:teuthology.orchestra.run.smithi016.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-13T16:18:59.411 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-13T16:18:59.411 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 18 M 2024-09-13T16:18:59.411 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards noarch 2:18.2.4-850.g50bbe7f3.el9 ceph-noarch 24 k 2024-09-13T16:18:59.411 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 2.1 M 2024-09-13T16:18:59.411 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core noarch 2:18.2.4-850.g50bbe7f3.el9 ceph-noarch 246 k 2024-09-13T16:18:59.411 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 4.7 M 2024-09-13T16:18:59.411 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 17 M 2024-09-13T16:18:59.411 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts noarch 2:18.2.4-850.g50bbe7f3.el9 ceph-noarch 16 k 2024-09-13T16:18:59.412 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 25 k 2024-09-13T16:18:59.412 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-13T16:18:59.412 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-13T16:18:59.412 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-13T16:18:59.412 INFO:teuthology.orchestra.run.smithi016.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-13T16:18:59.412 INFO:teuthology.orchestra.run.smithi016.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-13T16:18:59.412 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-13T16:18:59.412 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-13T16:18:59.412 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-13T16:18:59.412 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 166 k 2024-09-13T16:18:59.413 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-13T16:18:59.413 INFO:teuthology.orchestra.run.smithi016.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-13T16:18:59.413 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-13T16:18:59.413 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-13T16:18:59.413 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-13T16:18:59.413 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 477 k 2024-09-13T16:18:59.413 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-13T16:18:59.413 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2 x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 4.5 M 2024-09-13T16:18:59.413 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-13T16:18:59.413 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-13T16:18:59.413 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-13T16:18:59.413 INFO:teuthology.orchestra.run.smithi016.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-13T16:18:59.414 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-13T16:18:59.414 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-13T16:18:59.414 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-13T16:18:59.414 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-13T16:18:59.414 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-13T16:18:59.414 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-13T16:18:59.414 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-13T16:18:59.414 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 45 k 2024-09-13T16:18:59.414 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 129 k 2024-09-13T16:18:59.414 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-13T16:18:59.414 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-13T16:18:59.414 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-13T16:18:59.414 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-13T16:18:59.414 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-13T16:18:59.415 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-13T16:18:59.415 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-13T16:18:59.415 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-13T16:18:59.415 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-13T16:18:59.415 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-13T16:18:59.415 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-13T16:18:59.415 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-13T16:18:59.415 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-13T16:18:59.415 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-13T16:18:59.415 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-13T16:18:59.415 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-13T16:18:59.415 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-13T16:18:59.416 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-13T16:18:59.416 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-13T16:18:59.416 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-13T16:18:59.416 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-13T16:18:59.416 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-13T16:18:59.416 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-13T16:18:59.416 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-13T16:18:59.416 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-13T16:18:59.416 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-13T16:18:59.416 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-13T16:18:59.416 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-13T16:18:59.416 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-13T16:18:59.417 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-13T16:18:59.417 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-13T16:18:59.417 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-13T16:18:59.417 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-13T16:18:59.417 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-13T16:18:59.417 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-13T16:18:59.417 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-13T16:18:59.417 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-13T16:18:59.417 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-13T16:18:59.417 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-13T16:18:59.417 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-13T16:18:59.418 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-13T16:18:59.418 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-13T16:18:59.418 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-13T16:18:59.418 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-13T16:18:59.418 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-13T16:18:59.418 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-13T16:18:59.418 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-13T16:18:59.418 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-13T16:18:59.418 INFO:teuthology.orchestra.run.smithi016.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-13T16:18:59.419 INFO:teuthology.orchestra.run.smithi016.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-13T16:18:59.419 INFO:teuthology.orchestra.run.smithi016.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-13T16:18:59.419 INFO:teuthology.orchestra.run.smithi016.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-13T16:18:59.419 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-13T16:18:59.419 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-13T16:18:59.419 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:18:59.419 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-09-13T16:18:59.419 INFO:teuthology.orchestra.run.smithi016.stdout:====================================================================================== 2024-09-13T16:18:59.419 INFO:teuthology.orchestra.run.smithi016.stdout:Install 130 Packages 2024-09-13T16:18:59.419 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 8 Packages 2024-09-13T16:18:59.419 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:18:59.420 INFO:teuthology.orchestra.run.smithi111.stdout:(21/138): python3-cephfs-18.2.4-850.g50bbe7f3.e 1.4 MB/s | 162 kB 00:00 2024-09-13T16:18:59.420 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 203 M 2024-09-13T16:18:59.420 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-09-13T16:18:59.461 INFO:teuthology.orchestra.run.smithi111.stdout:(22/138): python3-rados-18.2.4-850.g50bbe7f3.el 4.7 MB/s | 319 kB 00:00 2024-09-13T16:18:59.495 INFO:teuthology.orchestra.run.smithi111.stdout:(23/138): python3-rbd-18.2.4-850.g50bbe7f3.el9. 3.9 MB/s | 301 kB 00:00 2024-09-13T16:18:59.529 INFO:teuthology.orchestra.run.smithi111.stdout:(24/138): python3-rgw-18.2.4-850.g50bbe7f3.el9. 1.4 MB/s | 99 kB 00:00 2024-09-13T16:18:59.562 INFO:teuthology.orchestra.run.smithi111.stdout:(25/138): rbd-fuse-18.2.4-850.g50bbe7f3.el9.x86 1.3 MB/s | 87 kB 00:00 2024-09-13T16:18:59.654 INFO:teuthology.orchestra.run.smithi111.stdout:(26/138): rbd-nbd-18.2.4-850.g50bbe7f3.el9.x86_ 1.8 MB/s | 172 kB 00:00 2024-09-13T16:18:59.763 INFO:teuthology.orchestra.run.smithi111.stdout:(27/138): rbd-mirror-18.2.4-850.g50bbe7f3.el9.x 13 MB/s | 3.0 MB 00:00 2024-09-13T16:18:59.779 INFO:teuthology.orchestra.run.smithi111.stdout:(28/138): ceph-grafana-dashboards-18.2.4-850.g5 195 kB/s | 24 kB 00:00 2024-09-13T16:18:59.808 INFO:teuthology.orchestra.run.smithi107.stdout:(8/138): ceph-radosgw-18.2.4-850.g50bbe7f3.el9. 11 MB/s | 7.8 MB 00:00 2024-09-13T16:18:59.821 INFO:teuthology.orchestra.run.smithi111.stdout:(29/138): ceph-mgr-cephadm-18.2.4-850.g50bbe7f3 2.3 MB/s | 139 kB 00:00 2024-09-13T16:18:59.875 INFO:teuthology.orchestra.run.smithi107.stdout:(9/138): ceph-selinux-18.2.4-850.g50bbe7f3.el9. 377 kB/s | 25 kB 00:00 2024-09-13T16:19:00.055 INFO:teuthology.orchestra.run.smithi111.stdout:(30/138): ceph-mgr-dashboard-18.2.4-850.g50bbe7 13 MB/s | 3.5 MB 00:00 2024-09-13T16:19:00.139 INFO:teuthology.orchestra.run.smithi111.stdout:(31/138): ceph-mgr-modules-core-18.2.4-850.g50b 2.9 MB/s | 246 kB 00:00 2024-09-13T16:19:00.206 INFO:teuthology.orchestra.run.smithi111.stdout:(32/138): ceph-mgr-rook-18.2.4-850.g50bbe7f3.el 742 kB/s | 49 kB 00:00 2024-09-13T16:19:00.233 INFO:teuthology.orchestra.run.smithi016.stdout:(1/138): ceph-18.2.4-850.g50bbe7f3.el9.x86_64.r 31 kB/s | 6.5 kB 00:00 2024-09-13T16:19:00.273 INFO:teuthology.orchestra.run.smithi111.stdout:(33/138): ceph-prometheus-alerts-18.2.4-850.g50 247 kB/s | 16 kB 00:00 2024-09-13T16:19:00.365 INFO:teuthology.orchestra.run.smithi111.stdout:(34/138): ceph-mgr-diskprediction-local-18.2.4- 14 MB/s | 7.4 MB 00:00 2024-09-13T16:19:00.383 INFO:teuthology.orchestra.run.smithi107.stdout:(10/138): ceph-common-18.2.4-850.g50bbe7f3.el9. 7.0 MB/s | 18 MB 00:02 2024-09-13T16:19:00.390 INFO:teuthology.orchestra.run.smithi111.stdout:(35/138): ceph-volume-18.2.4-850.g50bbe7f3.el9. 2.2 MB/s | 264 kB 00:00 2024-09-13T16:19:00.424 INFO:teuthology.orchestra.run.smithi111.stdout:(36/138): cephadm-18.2.4-850.g50bbe7f3.el9.noar 3.8 MB/s | 226 kB 00:00 2024-09-13T16:19:00.459 INFO:teuthology.orchestra.run.smithi107.stdout:(11/138): cephfs-mirror-18.2.4-850.g50bbe7f3.el 2.9 MB/s | 224 kB 00:00 2024-09-13T16:19:00.474 INFO:teuthology.orchestra.run.smithi111.stdout:(37/138): cephfs-top-18.2.4-850.g50bbe7f3.el9.n 218 kB/s | 18 kB 00:00 2024-09-13T16:19:00.542 INFO:teuthology.orchestra.run.smithi107.stdout:(12/138): libcephfs-devel-18.2.4-850.g50bbe7f3. 374 kB/s | 31 kB 00:00 2024-09-13T16:19:00.549 INFO:teuthology.orchestra.run.smithi016.stdout:(2/138): ceph-fuse-18.2.4-850.g50bbe7f3.el9.x86 2.6 MB/s | 852 kB 00:00 2024-09-13T16:19:00.566 INFO:teuthology.orchestra.run.smithi111.stdout:(38/138): fuse-2.9.9-16.el9.x86_64.rpm 562 kB/s | 80 kB 00:00 2024-09-13T16:19:00.633 INFO:teuthology.orchestra.run.smithi016.stdout:(3/138): ceph-immutable-object-cache-18.2.4-850 1.7 MB/s | 143 kB 00:00 2024-09-13T16:19:00.658 INFO:teuthology.orchestra.run.smithi111.stdout:(39/138): libgfortran-11.5.0-2.el9.x86_64.rpm 4.3 MB/s | 801 kB 00:00 2024-09-13T16:19:00.683 INFO:teuthology.orchestra.run.smithi111.stdout:(40/138): libquadmath-11.5.0-2.el9.x86_64.rpm 1.6 MB/s | 191 kB 00:00 2024-09-13T16:19:00.742 INFO:teuthology.orchestra.run.smithi107.stdout:(13/138): libcephfs2-18.2.4-850.g50bbe7f3.el9.x 3.5 MB/s | 712 kB 00:00 2024-09-13T16:19:00.758 INFO:teuthology.orchestra.run.smithi111.stdout:(41/138): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-09-13T16:19:00.817 INFO:teuthology.orchestra.run.smithi111.stdout:(42/138): python3-cryptography-36.0.1-4.el9.x86 9.2 MB/s | 1.2 MB 00:00 2024-09-13T16:19:00.842 INFO:teuthology.orchestra.run.smithi111.stdout:(43/138): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-09-13T16:19:00.843 INFO:teuthology.orchestra.run.smithi107.stdout:(14/138): libcephsqlite-18.2.4-850.g50bbe7f3.el 1.6 MB/s | 166 kB 00:00 2024-09-13T16:19:00.909 INFO:teuthology.orchestra.run.smithi111.stdout:(44/138): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-09-13T16:19:00.935 INFO:teuthology.orchestra.run.smithi111.stdout:(45/138): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-09-13T16:19:00.943 INFO:teuthology.orchestra.run.smithi107.stdout:(15/138): librados-devel-18.2.4-850.g50bbe7f3.e 1.2 MB/s | 127 kB 00:00 2024-09-13T16:19:01.000 INFO:teuthology.orchestra.run.smithi016.stdout:(4/138): ceph-mds-18.2.4-850.g50bbe7f3.el9.x86_ 5.7 MB/s | 2.1 MB 00:00 2024-09-13T16:19:01.001 INFO:teuthology.orchestra.run.smithi111.stdout:(46/138): python3-urllib3-1.26.5-6.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-09-13T16:19:01.110 INFO:teuthology.orchestra.run.smithi107.stdout:(16/138): libradosstriper1-18.2.4-850.g50bbe7f3 2.8 MB/s | 477 kB 00:00 2024-09-13T16:19:01.236 INFO:teuthology.orchestra.run.smithi016.stdout:(5/138): ceph-mgr-18.2.4-850.g50bbe7f3.el9.x86_ 6.2 MB/s | 1.5 MB 00:00 2024-09-13T16:19:01.394 INFO:teuthology.orchestra.run.smithi111.stdout:(47/138): ceph-test-18.2.4-850.g50bbe7f3.el9.x8 13 MB/s | 40 MB 00:03 2024-09-13T16:19:01.410 INFO:teuthology.orchestra.run.smithi107.stdout:(17/138): ceph-osd-18.2.4-850.g50bbe7f3.el9.x86 6.6 MB/s | 17 MB 00:02 2024-09-13T16:19:01.419 INFO:teuthology.orchestra.run.smithi016.stdout:(6/138): ceph-common-18.2.4-850.g50bbe7f3.el9.x 13 MB/s | 18 MB 00:01 2024-09-13T16:19:01.460 INFO:teuthology.orchestra.run.smithi107.stdout:(18/138): python3-ceph-argparse-18.2.4-850.g50b 903 kB/s | 45 kB 00:00 2024-09-13T16:19:01.543 INFO:teuthology.orchestra.run.smithi107.stdout:(19/138): python3-ceph-common-18.2.4-850.g50bbe 1.5 MB/s | 129 kB 00:00 2024-09-13T16:19:01.636 INFO:teuthology.orchestra.run.smithi107.stdout:(20/138): python3-cephfs-18.2.4-850.g50bbe7f3.e 1.7 MB/s | 162 kB 00:00 2024-09-13T16:19:01.695 INFO:teuthology.orchestra.run.smithi016.stdout:(7/138): ceph-mon-18.2.4-850.g50bbe7f3.el9.x86_ 10 MB/s | 4.7 MB 00:00 2024-09-13T16:19:01.702 INFO:teuthology.orchestra.run.smithi107.stdout:(21/138): python3-rados-18.2.4-850.g50bbe7f3.el 4.7 MB/s | 319 kB 00:00 2024-09-13T16:19:01.803 INFO:teuthology.orchestra.run.smithi107.stdout:(22/138): python3-rbd-18.2.4-850.g50bbe7f3.el9. 2.9 MB/s | 301 kB 00:00 2024-09-13T16:19:01.895 INFO:teuthology.orchestra.run.smithi107.stdout:(23/138): python3-rgw-18.2.4-850.g50bbe7f3.el9. 1.1 MB/s | 99 kB 00:00 2024-09-13T16:19:01.978 INFO:teuthology.orchestra.run.smithi107.stdout:(24/138): librgw2-18.2.4-850.g50bbe7f3.el9.x86_ 5.1 MB/s | 4.5 MB 00:00 2024-09-13T16:19:02.003 INFO:teuthology.orchestra.run.smithi107.stdout:(25/138): rbd-fuse-18.2.4-850.g50bbe7f3.el9.x86 798 kB/s | 87 kB 00:00 2024-09-13T16:19:02.054 INFO:teuthology.orchestra.run.smithi111.stdout:(48/138): boost-program-options-1.75.0-8.el9.x8 162 kB/s | 107 kB 00:00 2024-09-13T16:19:02.096 INFO:teuthology.orchestra.run.smithi107.stdout:(26/138): rbd-nbd-18.2.4-850.g50bbe7f3.el9.x86_ 1.8 MB/s | 172 kB 00:00 2024-09-13T16:19:02.165 INFO:teuthology.orchestra.run.smithi107.stdout:(27/138): ceph-grafana-dashboards-18.2.4-850.g5 488 kB/s | 24 kB 00:00 2024-09-13T16:19:02.196 INFO:teuthology.orchestra.run.smithi107.stdout:(28/138): ceph-mgr-cephadm-18.2.4-850.g50bbe7f3 2.7 MB/s | 139 kB 00:00 2024-09-13T16:19:02.204 INFO:teuthology.orchestra.run.smithi111.stdout:(49/138): automake-1.16.2-8.el9.noarch.rpm 543 kB/s | 689 kB 00:01 2024-09-13T16:19:02.228 INFO:teuthology.orchestra.run.smithi016.stdout:(8/138): ceph-osd-18.2.4-850.g50bbe7f3.el9.x86_ 20 MB/s | 17 MB 00:00 2024-09-13T16:19:02.238 INFO:teuthology.orchestra.run.smithi111.stdout:(50/138): bison-3.7.4-5.el9.x86_64.rpm 763 kB/s | 943 kB 00:01 2024-09-13T16:19:02.303 INFO:teuthology.orchestra.run.smithi016.stdout:(9/138): ceph-base-18.2.4-850.g50bbe7f3.el9.x86 2.2 MB/s | 5.1 MB 00:02 2024-09-13T16:19:02.328 INFO:teuthology.orchestra.run.smithi016.stdout:(10/138): ceph-selinux-18.2.4-850.g50bbe7f3.el9 251 kB/s | 25 kB 00:00 2024-09-13T16:19:02.371 INFO:teuthology.orchestra.run.smithi107.stdout:(29/138): rbd-mirror-18.2.4-850.g50bbe7f3.el9.x 7.6 MB/s | 3.0 MB 00:00 2024-09-13T16:19:02.429 INFO:teuthology.orchestra.run.smithi016.stdout:(11/138): ceph-radosgw-18.2.4-850.g50bbe7f3.el9 11 MB/s | 7.8 MB 00:00 2024-09-13T16:19:02.454 INFO:teuthology.orchestra.run.smithi016.stdout:(12/138): cephfs-mirror-18.2.4-850.g50bbe7f3.el 1.7 MB/s | 224 kB 00:00 2024-09-13T16:19:02.479 INFO:teuthology.orchestra.run.smithi016.stdout:(13/138): libcephfs-devel-18.2.4-850.g50bbe7f3. 621 kB/s | 31 kB 00:00 2024-09-13T16:19:02.522 INFO:teuthology.orchestra.run.smithi016.stdout:(14/138): libcephfs2-18.2.4-850.g50bbe7f3.el9.x 10 MB/s | 712 kB 00:00 2024-09-13T16:19:02.538 INFO:teuthology.orchestra.run.smithi107.stdout:(30/138): ceph-mgr-dashboard-18.2.4-850.g50bbe7 10 MB/s | 3.5 MB 00:00 2024-09-13T16:19:02.546 INFO:teuthology.orchestra.run.smithi016.stdout:(15/138): libcephsqlite-18.2.4-850.g50bbe7f3.el 2.4 MB/s | 166 kB 00:00 2024-09-13T16:19:02.572 INFO:teuthology.orchestra.run.smithi016.stdout:(16/138): librados-devel-18.2.4-850.g50bbe7f3.e 2.5 MB/s | 127 kB 00:00 2024-09-13T16:19:02.588 INFO:teuthology.orchestra.run.smithi111.stdout:(51/138): elfutils-libelf-devel-0.191-4.el9.x86 41 kB/s | 22 kB 00:00 2024-09-13T16:19:02.597 INFO:teuthology.orchestra.run.smithi107.stdout:(31/138): ceph-mgr-modules-core-18.2.4-850.g50b 4.1 MB/s | 246 kB 00:00 2024-09-13T16:19:02.622 INFO:teuthology.orchestra.run.smithi016.stdout:(17/138): libradosstriper1-18.2.4-850.g50bbe7f3 6.2 MB/s | 477 kB 00:00 2024-09-13T16:19:02.689 INFO:teuthology.orchestra.run.smithi107.stdout:(32/138): ceph-mgr-rook-18.2.4-850.g50bbe7f3.el 539 kB/s | 49 kB 00:00 2024-09-13T16:19:02.714 INFO:teuthology.orchestra.run.smithi016.stdout:(18/138): python3-ceph-argparse-18.2.4-850.g50b 491 kB/s | 45 kB 00:00 2024-09-13T16:19:02.756 INFO:teuthology.orchestra.run.smithi107.stdout:(33/138): ceph-prometheus-alerts-18.2.4-850.g50 246 kB/s | 16 kB 00:00 2024-09-13T16:19:02.780 INFO:teuthology.orchestra.run.smithi111.stdout:(52/138): flexiblas-3.0.4-8.el9.x86_64.rpm 60 kB/s | 33 kB 00:00 2024-09-13T16:19:02.781 INFO:teuthology.orchestra.run.smithi016.stdout:(19/138): librgw2-18.2.4-850.g50bbe7f3.el9.x86_ 21 MB/s | 4.5 MB 00:00 2024-09-13T16:19:02.798 INFO:teuthology.orchestra.run.smithi016.stdout:(20/138): python3-ceph-common-18.2.4-850.g50bbe 1.5 MB/s | 129 kB 00:00 2024-09-13T16:19:02.806 INFO:teuthology.orchestra.run.smithi107.stdout:(34/138): ceph-volume-18.2.4-850.g50bbe7f3.el9. 5.2 MB/s | 264 kB 00:00 2024-09-13T16:19:02.832 INFO:teuthology.orchestra.run.smithi016.stdout:(21/138): python3-cephfs-18.2.4-850.g50bbe7f3.e 3.1 MB/s | 162 kB 00:00 2024-09-13T16:19:02.865 INFO:teuthology.orchestra.run.smithi016.stdout:(22/138): python3-rados-18.2.4-850.g50bbe7f3.el 4.7 MB/s | 319 kB 00:00 2024-09-13T16:19:02.891 INFO:teuthology.orchestra.run.smithi016.stdout:(23/138): python3-rbd-18.2.4-850.g50bbe7f3.el9. 5.0 MB/s | 301 kB 00:00 2024-09-13T16:19:02.916 INFO:teuthology.orchestra.run.smithi016.stdout:(24/138): python3-rgw-18.2.4-850.g50bbe7f3.el9. 1.9 MB/s | 99 kB 00:00 2024-09-13T16:19:02.941 INFO:teuthology.orchestra.run.smithi016.stdout:(25/138): rbd-fuse-18.2.4-850.g50bbe7f3.el9.x86 1.7 MB/s | 87 kB 00:00 2024-09-13T16:19:02.997 INFO:teuthology.orchestra.run.smithi111.stdout:(53/138): flex-2.6.4-9.el9.x86_64.rpm 396 kB/s | 313 kB 00:00 2024-09-13T16:19:03.008 INFO:teuthology.orchestra.run.smithi016.stdout:(26/138): rbd-nbd-18.2.4-850.g50bbe7f3.el9.x86_ 2.5 MB/s | 172 kB 00:00 2024-09-13T16:19:03.092 INFO:teuthology.orchestra.run.smithi016.stdout:(27/138): rbd-mirror-18.2.4-850.g50bbe7f3.el9.x 17 MB/s | 3.0 MB 00:00 2024-09-13T16:19:03.117 INFO:teuthology.orchestra.run.smithi016.stdout:(28/138): ceph-grafana-dashboards-18.2.4-850.g5 225 kB/s | 24 kB 00:00 2024-09-13T16:19:03.142 INFO:teuthology.orchestra.run.smithi016.stdout:(29/138): ceph-mgr-cephadm-18.2.4-850.g50bbe7f3 2.7 MB/s | 139 kB 00:00 2024-09-13T16:19:03.156 INFO:teuthology.orchestra.run.smithi107.stdout:(35/138): ceph-test-18.2.4-850.g50bbe7f3.el9.x8 12 MB/s | 40 MB 00:03 2024-09-13T16:19:03.181 INFO:teuthology.orchestra.run.smithi107.stdout:(36/138): cephadm-18.2.4-850.g50bbe7f3.el9.noar 603 kB/s | 226 kB 00:00 2024-09-13T16:19:03.255 INFO:teuthology.orchestra.run.smithi111.stdout:(54/138): flexiblas-openblas-openmp-3.0.4-8.el9 38 kB/s | 18 kB 00:00 2024-09-13T16:19:03.256 INFO:teuthology.orchestra.run.smithi107.stdout:(37/138): cephfs-top-18.2.4-850.g50bbe7f3.el9.n 183 kB/s | 18 kB 00:00 2024-09-13T16:19:03.343 INFO:teuthology.orchestra.run.smithi016.stdout:(30/138): ceph-mgr-dashboard-18.2.4-850.g50bbe7 16 MB/s | 3.5 MB 00:00 2024-09-13T16:19:03.443 INFO:teuthology.orchestra.run.smithi016.stdout:(31/138): ceph-mgr-modules-core-18.2.4-850.g50b 2.4 MB/s | 246 kB 00:00 2024-09-13T16:19:03.473 INFO:teuthology.orchestra.run.smithi111.stdout:(55/138): libaio-devel-0.3.111-13.el9.x86_64.rp 24 kB/s | 11 kB 00:00 2024-09-13T16:19:03.498 INFO:teuthology.orchestra.run.smithi107.stdout:(38/138): ceph-mgr-diskprediction-local-18.2.4- 6.6 MB/s | 7.4 MB 00:01 2024-09-13T16:19:03.510 INFO:teuthology.orchestra.run.smithi016.stdout:(32/138): ceph-mgr-rook-18.2.4-850.g50bbe7f3.el 742 kB/s | 49 kB 00:00 2024-09-13T16:19:03.523 INFO:teuthology.orchestra.run.smithi107.stdout:(39/138): fuse-2.9.9-16.el9.x86_64.rpm 233 kB/s | 80 kB 00:00 2024-09-13T16:19:03.602 INFO:teuthology.orchestra.run.smithi016.stdout:(33/138): ceph-mgr-diskprediction-local-18.2.4- 16 MB/s | 7.4 MB 00:00 2024-09-13T16:19:03.627 INFO:teuthology.orchestra.run.smithi016.stdout:(34/138): ceph-prometheus-alerts-18.2.4-850.g50 140 kB/s | 16 kB 00:00 2024-09-13T16:19:03.661 INFO:teuthology.orchestra.run.smithi016.stdout:(35/138): ceph-volume-18.2.4-850.g50bbe7f3.el9. 4.4 MB/s | 264 kB 00:00 2024-09-13T16:19:03.686 INFO:teuthology.orchestra.run.smithi016.stdout:(36/138): cephadm-18.2.4-850.g50bbe7f3.el9.noar 3.8 MB/s | 226 kB 00:00 2024-09-13T16:19:03.712 INFO:teuthology.orchestra.run.smithi016.stdout:(37/138): cephfs-top-18.2.4-850.g50bbe7f3.el9.n 362 kB/s | 18 kB 00:00 2024-09-13T16:19:03.756 INFO:teuthology.orchestra.run.smithi111.stdout:(56/138): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.6 MB/s | 3.0 MB 00:01 2024-09-13T16:19:03.795 INFO:teuthology.orchestra.run.smithi016.stdout:(38/138): fuse-2.9.9-16.el9.x86_64.rpm 736 kB/s | 80 kB 00:00 2024-09-13T16:19:03.832 INFO:teuthology.orchestra.run.smithi107.stdout:(40/138): python3-cffi-1.14.5-5.el9.x86_64.rpm 821 kB/s | 253 kB 00:00 2024-09-13T16:19:03.854 INFO:teuthology.orchestra.run.smithi016.stdout:(39/138): libquadmath-11.5.0-2.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-13T16:19:03.858 INFO:teuthology.orchestra.run.smithi107.stdout:(41/138): libquadmath-11.5.0-2.el9.x86_64.rpm 532 kB/s | 191 kB 00:00 2024-09-13T16:19:03.896 INFO:teuthology.orchestra.run.smithi016.stdout:(40/138): libgfortran-11.5.0-2.el9.x86_64.rpm 4.3 MB/s | 801 kB 00:00 2024-09-13T16:19:03.938 INFO:teuthology.orchestra.run.smithi016.stdout:(41/138): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-09-13T16:19:03.985 INFO:teuthology.orchestra.run.smithi107.stdout:(42/138): libgfortran-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 801 kB 00:00 2024-09-13T16:19:03.988 INFO:teuthology.orchestra.run.smithi016.stdout:(42/138): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-13T16:19:03.998 INFO:teuthology.orchestra.run.smithi111.stdout:(57/138): libpmemobj-1.12.1-1.el9.x86_64.rpm 216 kB/s | 160 kB 00:00 2024-09-13T16:19:04.008 INFO:teuthology.orchestra.run.smithi107.stdout:(43/138): python3-ply-3.11-14.el9.noarch.rpm 710 kB/s | 106 kB 00:00 2024-09-13T16:19:04.030 INFO:teuthology.orchestra.run.smithi016.stdout:(43/138): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-09-13T16:19:04.040 INFO:teuthology.orchestra.run.smithi111.stdout:(58/138): librabbitmq-0.11.0-7.el9.x86_64.rpm 81 kB/s | 45 kB 00:00 2024-09-13T16:19:04.117 INFO:teuthology.orchestra.run.smithi107.stdout:(44/138): python3-pycparser-2.20-6.el9.noarch.r 1.0 MB/s | 135 kB 00:00 2024-09-13T16:19:04.142 INFO:teuthology.orchestra.run.smithi107.stdout:(45/138): python3-requests-2.25.1-8.el9.noarch. 935 kB/s | 125 kB 00:00 2024-09-13T16:19:04.251 INFO:teuthology.orchestra.run.smithi107.stdout:(46/138): python3-urllib3-1.26.5-6.el9.noarch.r 1.6 MB/s | 215 kB 00:00 2024-09-13T16:19:04.406 INFO:teuthology.orchestra.run.smithi016.stdout:(44/138): ceph-test-18.2.4-850.g50bbe7f3.el9.x8 19 MB/s | 40 MB 00:02 2024-09-13T16:19:04.426 INFO:teuthology.orchestra.run.smithi107.stdout:(47/138): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-09-13T16:19:04.431 INFO:teuthology.orchestra.run.smithi016.stdout:(45/138): python3-requests-2.25.1-8.el9.noarch. 312 kB/s | 125 kB 00:00 2024-09-13T16:19:04.460 INFO:teuthology.orchestra.run.smithi107.stdout:(48/138): python3-cryptography-36.0.1-4.el9.x86 2.0 MB/s | 1.2 MB 00:00 2024-09-13T16:19:04.473 INFO:teuthology.orchestra.run.smithi016.stdout:(46/138): python3-cryptography-36.0.1-4.el9.x86 2.1 MB/s | 1.2 MB 00:00 2024-09-13T16:19:04.485 INFO:teuthology.orchestra.run.smithi107.stdout:(49/138): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-09-13T16:19:04.548 INFO:teuthology.orchestra.run.smithi016.stdout:(47/138): python3-urllib3-1.26.5-6.el9.noarch.r 1.5 MB/s | 215 kB 00:00 2024-09-13T16:19:04.568 INFO:teuthology.orchestra.run.smithi107.stdout:(50/138): elfutils-libelf-devel-0.191-4.el9.x86 200 kB/s | 22 kB 00:00 2024-09-13T16:19:04.573 INFO:teuthology.orchestra.run.smithi111.stdout:(59/138): libuuid-devel-2.37.4-20.el9.x86_64.rp 46 kB/s | 24 kB 00:00 2024-09-13T16:19:04.594 INFO:teuthology.orchestra.run.smithi107.stdout:(51/138): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-09-13T16:19:04.627 INFO:teuthology.orchestra.run.smithi107.stdout:(52/138): bison-3.7.4-5.el9.x86_64.rpm 2.4 MB/s | 943 kB 00:00 2024-09-13T16:19:04.640 INFO:teuthology.orchestra.run.smithi111.stdout:(60/138): librdkafka-1.6.1-102.el9.x86_64.rpm 750 kB/s | 662 kB 00:00 2024-09-13T16:19:04.677 INFO:teuthology.orchestra.run.smithi107.stdout:(53/138): flexiblas-3.0.4-8.el9.x86_64.rpm 301 kB/s | 33 kB 00:00 2024-09-13T16:19:04.702 INFO:teuthology.orchestra.run.smithi107.stdout:(54/138): flexiblas-openblas-openmp-3.0.4-8.el9 242 kB/s | 18 kB 00:00 2024-09-13T16:19:04.728 INFO:teuthology.orchestra.run.smithi107.stdout:(55/138): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-13T16:19:04.753 INFO:teuthology.orchestra.run.smithi107.stdout:(56/138): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-13T16:19:04.787 INFO:teuthology.orchestra.run.smithi107.stdout:(57/138): librabbitmq-0.11.0-7.el9.x86_64.rpm 772 kB/s | 45 kB 00:00 2024-09-13T16:19:04.903 INFO:teuthology.orchestra.run.smithi107.stdout:(58/138): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-09-13T16:19:04.932 INFO:teuthology.orchestra.run.smithi111.stdout:(61/138): libtool-2.4.6-46.el9.x86_64.rpm 627 kB/s | 581 kB 00:00 2024-09-13T16:19:04.970 INFO:teuthology.orchestra.run.smithi107.stdout:(59/138): libuuid-devel-2.37.4-20.el9.x86_64.rp 367 kB/s | 24 kB 00:00 2024-09-13T16:19:04.996 INFO:teuthology.orchestra.run.smithi107.stdout:(60/138): libtool-2.4.6-46.el9.x86_64.rpm 2.7 MB/s | 581 kB 00:00 2024-09-13T16:19:05.021 INFO:teuthology.orchestra.run.smithi107.stdout:(61/138): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-13T16:19:05.071 INFO:teuthology.orchestra.run.smithi107.stdout:(62/138): openblas-0.3.26-2.el9.x86_64.rpm 772 kB/s | 38 kB 00:00 2024-09-13T16:19:05.090 INFO:teuthology.orchestra.run.smithi016.stdout:(48/138): boost-program-options-1.75.0-8.el9.x8 197 kB/s | 107 kB 00:00 2024-09-13T16:19:05.096 INFO:teuthology.orchestra.run.smithi107.stdout:(63/138): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-09-13T16:19:05.174 INFO:teuthology.orchestra.run.smithi016.stdout:(49/138): automake-1.16.2-8.el9.noarch.rpm 928 kB/s | 689 kB 00:00 2024-09-13T16:19:05.183 INFO:teuthology.orchestra.run.smithi111.stdout:(62/138): libzstd-devel-1.5.1-2.el9.x86_64.rpm 78 kB/s | 47 kB 00:00 2024-09-13T16:19:05.197 INFO:teuthology.orchestra.run.smithi107.stdout:(64/138): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.0 MB/s | 3.0 MB 00:00 2024-09-13T16:19:05.215 INFO:teuthology.orchestra.run.smithi016.stdout:(50/138): elfutils-libelf-devel-0.191-4.el9.x86 173 kB/s | 22 kB 00:00 2024-09-13T16:19:05.299 INFO:teuthology.orchestra.run.smithi016.stdout:(51/138): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-09-13T16:19:05.333 INFO:teuthology.orchestra.run.smithi016.stdout:(52/138): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-09-13T16:19:05.349 INFO:teuthology.orchestra.run.smithi016.stdout:(53/138): flexiblas-3.0.4-8.el9.x86_64.rpm 244 kB/s | 33 kB 00:00 2024-09-13T16:19:05.380 INFO:teuthology.orchestra.run.smithi107.stdout:(65/138): postgresql-13.16-1.el9.x86_64.rpm 8.7 MB/s | 1.6 MB 00:00 2024-09-13T16:19:05.416 INFO:teuthology.orchestra.run.smithi111.stdout:(63/138): lttng-ust-2.12.0-6.el9.x86_64.rpm 377 kB/s | 292 kB 00:00 2024-09-13T16:19:05.433 INFO:teuthology.orchestra.run.smithi016.stdout:(54/138): flexiblas-openblas-openmp-3.0.4-8.el9 182 kB/s | 18 kB 00:00 2024-09-13T16:19:05.458 INFO:teuthology.orchestra.run.smithi016.stdout:(55/138): libaio-devel-0.3.111-13.el9.x86_64.rp 102 kB/s | 11 kB 00:00 2024-09-13T16:19:05.491 INFO:teuthology.orchestra.run.smithi111.stdout:(64/138): openblas-0.3.26-2.el9.x86_64.rpm 69 kB/s | 38 kB 00:00 2024-09-13T16:19:05.505 INFO:teuthology.orchestra.run.smithi107.stdout:(66/138): postgresql-contrib-13.16-1.el9.x86_64 7.1 MB/s | 910 kB 00:00 2024-09-13T16:19:05.551 INFO:teuthology.orchestra.run.smithi016.stdout:(56/138): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-09-13T16:19:05.564 INFO:teuthology.orchestra.run.smithi107.stdout:(67/138): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-13T16:19:05.575 INFO:teuthology.orchestra.run.smithi016.stdout:(57/138): librabbitmq-0.11.0-7.el9.x86_64.rpm 387 kB/s | 45 kB 00:00 2024-09-13T16:19:05.718 INFO:teuthology.orchestra.run.smithi016.stdout:(58/138): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.1 MB/s | 3.0 MB 00:00 2024-09-13T16:19:05.781 INFO:teuthology.orchestra.run.smithi107.stdout:(68/138): openblas-openmp-0.3.26-2.el9.x86_64.r 7.0 MB/s | 4.9 MB 00:00 2024-09-13T16:19:05.826 INFO:teuthology.orchestra.run.smithi016.stdout:(59/138): libuuid-devel-2.37.4-20.el9.x86_64.rp 226 kB/s | 24 kB 00:00 2024-09-13T16:19:05.839 INFO:teuthology.orchestra.run.smithi107.stdout:(69/138): openssl-devel-3.2.2-6.el9.x86_64.rpm 6.0 MB/s | 4.4 MB 00:00 2024-09-13T16:19:05.860 INFO:teuthology.orchestra.run.smithi016.stdout:(60/138): librdkafka-1.6.1-102.el9.x86_64.rpm 2.1 MB/s | 662 kB 00:00 2024-09-13T16:19:05.906 INFO:teuthology.orchestra.run.smithi107.stdout:(70/138): python3-devel-3.9.19-8.el9.x86_64.rpm 3.6 MB/s | 245 kB 00:00 2024-09-13T16:19:05.935 INFO:teuthology.orchestra.run.smithi016.stdout:(61/138): libzstd-devel-1.5.1-2.el9.x86_64.rpm 435 kB/s | 47 kB 00:00 2024-09-13T16:19:05.966 INFO:teuthology.orchestra.run.smithi107.stdout:(71/138): python3-jinja2-2.11.3-6.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-09-13T16:19:05.977 INFO:teuthology.orchestra.run.smithi016.stdout:(62/138): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-13T16:19:06.032 INFO:teuthology.orchestra.run.smithi107.stdout:(72/138): postgresql-server-13.16-1.el9.x86_64. 12 MB/s | 5.8 MB 00:00 2024-09-13T16:19:06.052 INFO:teuthology.orchestra.run.smithi016.stdout:(63/138): libtool-2.4.6-46.el9.x86_64.rpm 1.2 MB/s | 581 kB 00:00 2024-09-13T16:19:06.057 INFO:teuthology.orchestra.run.smithi107.stdout:(73/138): python3-jsonpatch-1.21-16.el9.noarch. 278 kB/s | 26 kB 00:00 2024-09-13T16:19:06.078 INFO:teuthology.orchestra.run.smithi016.stdout:(64/138): openblas-0.3.26-2.el9.x86_64.rpm 271 kB/s | 38 kB 00:00 2024-09-13T16:19:06.082 INFO:teuthology.orchestra.run.smithi107.stdout:(74/138): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-09-13T16:19:06.116 INFO:teuthology.orchestra.run.smithi107.stdout:(75/138): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-13T16:19:06.133 INFO:teuthology.orchestra.run.smithi107.stdout:(76/138): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-09-13T16:19:06.192 INFO:teuthology.orchestra.run.smithi107.stdout:(77/138): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-13T16:19:06.300 INFO:teuthology.orchestra.run.smithi111.stdout:(65/138): openblas-openmp-0.3.26-2.el9.x86_64.r 4.4 MB/s | 4.9 MB 00:01 2024-09-13T16:19:06.301 INFO:teuthology.orchestra.run.smithi107.stdout:(78/138): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-09-13T16:19:06.326 INFO:teuthology.orchestra.run.smithi107.stdout:(79/138): python3-oauthlib-3.1.1-5.el9.noarch.r 1.6 MB/s | 222 kB 00:00 2024-09-13T16:19:06.351 INFO:teuthology.orchestra.run.smithi107.stdout:(80/138): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-13T16:19:06.376 INFO:teuthology.orchestra.run.smithi107.stdout:(81/138): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-09-13T16:19:06.410 INFO:teuthology.orchestra.run.smithi107.stdout:(82/138): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-09-13T16:19:06.435 INFO:teuthology.orchestra.run.smithi107.stdout:(83/138): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-09-13T16:19:06.469 INFO:teuthology.orchestra.run.smithi107.stdout:(84/138): python3-pytz-2021.1-5.el9.noarch.rpm 871 kB/s | 51 kB 00:00 2024-09-13T16:19:06.483 INFO:teuthology.orchestra.run.smithi111.stdout:(66/138): postgresql-13.16-1.el9.x86_64.rpm 1.6 MB/s | 1.6 MB 00:00 2024-09-13T16:19:06.494 INFO:teuthology.orchestra.run.smithi107.stdout:(85/138): python3-requests-oauthlib-1.3.0-12.el 918 kB/s | 54 kB 00:00 2024-09-13T16:19:06.544 INFO:teuthology.orchestra.run.smithi107.stdout:(86/138): python3-toml-0.10.2-6.el9.noarch.rpm 835 kB/s | 42 kB 00:00 2024-09-13T16:19:06.619 INFO:teuthology.orchestra.run.smithi107.stdout:(87/138): python3-numpy-1.20.1-5.el9.x86_64.rpm 10 MB/s | 5.1 MB 00:00 2024-09-13T16:19:06.645 INFO:teuthology.orchestra.run.smithi107.stdout:(88/138): socat-1.7.4.1-6.el9.x86_64.rpm 3.0 MB/s | 304 kB 00:00 2024-09-13T16:19:06.650 INFO:teuthology.orchestra.run.smithi111.stdout:(67/138): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.6 MB/s | 4.4 MB 00:01 2024-09-13T16:19:06.670 INFO:teuthology.orchestra.run.smithi107.stdout:(89/138): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-13T16:19:06.703 INFO:teuthology.orchestra.run.smithi107.stdout:(90/138): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 1.6 MB/s | 96 kB 00:00 2024-09-13T16:19:06.729 INFO:teuthology.orchestra.run.smithi107.stdout:(91/138): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-13T16:19:06.762 INFO:teuthology.orchestra.run.smithi107.stdout:(92/138): zlib-devel-1.2.11-41.el9.x86_64.rpm 765 kB/s | 45 kB 00:00 2024-09-13T16:19:06.796 INFO:teuthology.orchestra.run.smithi107.stdout:(93/138): fmt-8.1.1-5.el9.x86_64.rpm 1.6 MB/s | 111 kB 00:00 2024-09-13T16:19:06.847 INFO:teuthology.orchestra.run.smithi107.stdout:(94/138): gperftools-libs-2.9.1-3.el9.x86_64.rp 3.7 MB/s | 308 kB 00:00 2024-09-13T16:19:06.880 INFO:teuthology.orchestra.run.smithi107.stdout:(95/138): libarrow-doc-9.0.0-11.el9.noarch.rpm 784 kB/s | 26 kB 00:00 2024-09-13T16:19:06.905 INFO:teuthology.orchestra.run.smithi107.stdout:(96/138): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-13T16:19:06.930 INFO:teuthology.orchestra.run.smithi107.stdout:(97/138): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-13T16:19:07.022 INFO:teuthology.orchestra.run.smithi107.stdout:(98/138): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-09-13T16:19:07.089 INFO:teuthology.orchestra.run.smithi107.stdout:(99/138): python3-asyncssh-2.13.2-5.el9.noarch. 8.1 MB/s | 548 kB 00:00 2024-09-13T16:19:07.114 INFO:teuthology.orchestra.run.smithi107.stdout:(100/138): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-13T16:19:07.209 INFO:teuthology.orchestra.run.smithi111.stdout:(68/138): postgresql-private-libs-13.16-1.el9.x 189 kB/s | 137 kB 00:00 2024-09-13T16:19:07.260 INFO:teuthology.orchestra.run.smithi111.stdout:(69/138): postgresql-contrib-13.16-1.el9.x86_64 949 kB/s | 910 kB 00:00 2024-09-13T16:19:07.273 INFO:teuthology.orchestra.run.smithi107.stdout:(101/138): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-09-13T16:19:07.298 INFO:teuthology.orchestra.run.smithi107.stdout:(102/138): python3-cachetools-4.2.4-1.el9.noarc 175 kB/s | 32 kB 00:00 2024-09-13T16:19:07.334 INFO:teuthology.orchestra.run.smithi107.stdout:(103/138): python3-cheroot-8.6.0-4.el9.noarch.r 5.9 MB/s | 172 kB 00:00 2024-09-13T16:19:07.393 INFO:teuthology.orchestra.run.smithi107.stdout:(104/138): libarrow-9.0.0-11.el9.x86_64.rpm 7.4 MB/s | 4.4 MB 00:00 2024-09-13T16:19:07.410 INFO:teuthology.orchestra.run.smithi107.stdout:(105/138): python3-certifi-2023.05.07-4.el9.noa 104 kB/s | 14 kB 00:00 2024-09-13T16:19:07.435 INFO:teuthology.orchestra.run.smithi107.stdout:(106/138): python3-jaraco-8.2.1-3.el9.noarch.rp 433 kB/s | 11 kB 00:00 2024-09-13T16:19:07.460 INFO:teuthology.orchestra.run.smithi107.stdout:(107/138): python3-google-auth-2.34.0-1.el9.noa 3.2 MB/s | 220 kB 00:00 2024-09-13T16:19:07.485 INFO:teuthology.orchestra.run.smithi107.stdout:(108/138): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-09-13T16:19:07.510 INFO:teuthology.orchestra.run.smithi107.stdout:(109/138): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-13T16:19:07.536 INFO:teuthology.orchestra.run.smithi107.stdout:(110/138): python3-cherrypy-18.6.1-2.el9.noarch 1.7 MB/s | 358 kB 00:00 2024-09-13T16:19:07.561 INFO:teuthology.orchestra.run.smithi107.stdout:(111/138): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-09-13T16:19:07.587 INFO:teuthology.orchestra.run.smithi107.stdout:(112/138): python3-jaraco-text-3.2.0-6.el9.noar 260 kB/s | 20 kB 00:00 2024-09-13T16:19:07.612 INFO:teuthology.orchestra.run.smithi107.stdout:(113/138): python3-logutils-0.3.5-21.el9.noarch 919 kB/s | 46 kB 00:00 2024-09-13T16:19:07.646 INFO:teuthology.orchestra.run.smithi107.stdout:(114/138): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-09-13T16:19:07.670 INFO:teuthology.orchestra.run.smithi107.stdout:(115/138): python3-natsort-7.1.1-5.el9.noarch.r 987 kB/s | 58 kB 00:00 2024-09-13T16:19:07.696 INFO:teuthology.orchestra.run.smithi107.stdout:(116/138): python3-portend-3.1.0-2.el9.noarch.r 657 kB/s | 16 kB 00:00 2024-09-13T16:19:07.703 INFO:teuthology.orchestra.run.smithi016.stdout:(65/138): openssl-devel-3.2.2-6.el9.x86_64.rpm 2.7 MB/s | 4.4 MB 00:01 2024-09-13T16:19:07.721 INFO:teuthology.orchestra.run.smithi107.stdout:(117/138): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-13T16:19:07.746 INFO:teuthology.orchestra.run.smithi107.stdout:(118/138): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-13T16:19:07.772 INFO:teuthology.orchestra.run.smithi107.stdout:(119/138): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-13T16:19:07.797 INFO:teuthology.orchestra.run.smithi107.stdout:(120/138): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-13T16:19:07.801 INFO:teuthology.orchestra.run.smithi111.stdout:(70/138): postgresql-server-13.16-1.el9.x86_64. 5.0 MB/s | 5.8 MB 00:01 2024-09-13T16:19:07.822 INFO:teuthology.orchestra.run.smithi107.stdout:(121/138): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-13T16:19:07.849 INFO:teuthology.orchestra.run.smithi107.stdout:(122/138): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-09-13T16:19:07.881 INFO:teuthology.orchestra.run.smithi107.stdout:(123/138): python3-kubernetes-26.1.0-2.el9.noar 3.0 MB/s | 1.0 MB 00:00 2024-09-13T16:19:07.907 INFO:teuthology.orchestra.run.smithi107.stdout:(124/138): python3-typing-extensions-4.12.2-2.e 888 kB/s | 74 kB 00:00 2024-09-13T16:19:07.932 INFO:teuthology.orchestra.run.smithi107.stdout:(125/138): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-13T16:19:07.957 INFO:teuthology.orchestra.run.smithi107.stdout:(126/138): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-13T16:19:07.982 INFO:teuthology.orchestra.run.smithi107.stdout:(127/138): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-09-13T16:19:07.995 INFO:teuthology.orchestra.run.smithi016.stdout:(66/138): postgresql-contrib-13.16-1.el9.x86_64 3.1 MB/s | 910 kB 00:00 2024-09-13T16:19:08.007 INFO:teuthology.orchestra.run.smithi107.stdout:(128/138): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-13T16:19:08.034 INFO:teuthology.orchestra.run.smithi111.stdout:(71/138): python3-devel-3.9.19-8.el9.x86_64.rpm 316 kB/s | 245 kB 00:00 2024-09-13T16:19:08.041 INFO:teuthology.orchestra.run.smithi107.stdout:(129/138): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-13T16:19:08.062 INFO:teuthology.orchestra.run.smithi016.stdout:(67/138): openblas-openmp-0.3.26-2.el9.x86_64.r 2.4 MB/s | 4.9 MB 00:02 2024-09-13T16:19:08.113 INFO:teuthology.orchestra.run.smithi016.stdout:(68/138): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-13T16:19:08.116 INFO:teuthology.orchestra.run.smithi107.stdout:(130/138): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-13T16:19:08.292 INFO:teuthology.orchestra.run.smithi107.stdout:(131/138): librbd1-18.2.4-850.g50bbe7f3.el9.x86 12 MB/s | 3.0 MB 00:00 2024-09-13T16:19:08.335 INFO:teuthology.orchestra.run.smithi111.stdout:(72/138): python3-babel-2.9.1-2.el9.noarch.rpm 5.3 MB/s | 6.0 MB 00:01 2024-09-13T16:19:08.401 INFO:teuthology.orchestra.run.smithi107.stdout:(132/138): librados2-18.2.4-850.g50bbe7f3.el9.x 8.4 MB/s | 3.3 MB 00:00 2024-09-13T16:19:08.529 INFO:teuthology.orchestra.run.smithi016.stdout:(69/138): postgresql-server-13.16-1.el9.x86_64. 12 MB/s | 5.8 MB 00:00 2024-09-13T16:19:08.559 INFO:teuthology.orchestra.run.smithi107.stdout:(133/138): NetworkManager-team-1.49.90-1.el9.x8 257 kB/s | 41 kB 00:00 2024-09-13T16:19:08.563 INFO:teuthology.orchestra.run.smithi016.stdout:(70/138): postgresql-13.16-1.el9.x86_64.rpm 654 kB/s | 1.6 MB 00:02 2024-09-13T16:19:08.635 INFO:teuthology.orchestra.run.smithi111.stdout:(73/138): python3-jinja2-2.11.3-6.el9.noarch.rp 298 kB/s | 248 kB 00:00 2024-09-13T16:19:08.638 INFO:teuthology.orchestra.run.smithi016.stdout:(71/138): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-13T16:19:08.680 INFO:teuthology.orchestra.run.smithi016.stdout:(72/138): python3-jinja2-2.11.3-6.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-09-13T16:19:08.702 INFO:teuthology.orchestra.run.smithi111.stdout:(74/138): python3-jsonpatch-1.21-16.el9.noarch. 39 kB/s | 26 kB 00:00 2024-09-13T16:19:08.747 INFO:teuthology.orchestra.run.smithi016.stdout:(73/138): python3-jsonpatch-1.21-16.el9.noarch. 236 kB/s | 26 kB 00:00 2024-09-13T16:19:08.789 INFO:teuthology.orchestra.run.smithi016.stdout:(74/138): python3-jsonpointer-2.0-4.el9.noarch. 176 kB/s | 19 kB 00:00 2024-09-13T16:19:08.809 INFO:teuthology.orchestra.run.smithi107.stdout:(134/138): NetworkManager-tui-1.49.90-1.el9.x86 996 kB/s | 249 kB 00:00 2024-09-13T16:19:08.852 INFO:teuthology.orchestra.run.smithi111.stdout:(75/138): python3-jsonpointer-2.0-4.el9.noarch. 38 kB/s | 19 kB 00:00 2024-09-13T16:19:08.856 INFO:teuthology.orchestra.run.smithi016.stdout:(75/138): python3-mako-1.1.4-6.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-09-13T16:19:08.898 INFO:teuthology.orchestra.run.smithi016.stdout:(76/138): python3-markupsafe-1.1.1-12.el9.x86_6 319 kB/s | 35 kB 00:00 2024-09-13T16:19:09.031 INFO:teuthology.orchestra.run.smithi016.stdout:(77/138): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.8 MB/s | 250 kB 00:00 2024-09-13T16:19:09.173 INFO:teuthology.orchestra.run.smithi016.stdout:(78/138): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-09-13T16:19:09.198 INFO:teuthology.orchestra.run.smithi016.stdout:(79/138): python3-oauthlib-3.1.1-5.el9.noarch.r 1.3 MB/s | 222 kB 00:00 2024-09-13T16:19:09.269 INFO:teuthology.orchestra.run.smithi111.stdout:(76/138): python3-markupsafe-1.1.1-12.el9.x86_6 61 kB/s | 35 kB 00:00 2024-09-13T16:19:09.282 INFO:teuthology.orchestra.run.smithi016.stdout:(80/138): python3-babel-2.9.1-2.el9.noarch.rpm 5.1 MB/s | 6.0 MB 00:01 2024-09-13T16:19:09.307 INFO:teuthology.orchestra.run.smithi016.stdout:(81/138): python3-packaging-20.9-5.el9.noarch.r 575 kB/s | 77 kB 00:00 2024-09-13T16:19:09.332 INFO:teuthology.orchestra.run.smithi016.stdout:(82/138): python3-prettytable-0.7.2-27.el9.noar 316 kB/s | 42 kB 00:00 2024-09-13T16:19:09.360 INFO:teuthology.orchestra.run.smithi107.stdout:(135/138): NetworkManager-libnm-1.49.90-1.el9.x 1.7 MB/s | 1.8 MB 00:01 2024-09-13T16:19:09.361 INFO:teuthology.orchestra.run.smithi111.stdout:(77/138): python3-mako-1.1.4-6.el9.noarch.rpm 237 kB/s | 172 kB 00:00 2024-09-13T16:19:09.391 INFO:teuthology.orchestra.run.smithi016.stdout:(83/138): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-13T16:19:09.410 INFO:teuthology.orchestra.run.smithi107.stdout:(136/138): NetworkManager-1.49.90-1.el9.x86_64. 1.8 MB/s | 2.3 MB 00:01 2024-09-13T16:19:09.425 INFO:teuthology.orchestra.run.smithi016.stdout:(84/138): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-09-13T16:19:09.450 INFO:teuthology.orchestra.run.smithi016.stdout:(85/138): python3-pytz-2021.1-5.el9.noarch.rpm 435 kB/s | 51 kB 00:00 2024-09-13T16:19:09.494 INFO:teuthology.orchestra.run.smithi107.stdout:(137/138): openssl-3.2.2-6.el9.x86_64.rpm 2.0 MB/s | 1.4 MB 00:00 2024-09-13T16:19:09.500 INFO:teuthology.orchestra.run.smithi016.stdout:(86/138): python3-requests-oauthlib-1.3.0-12.el 493 kB/s | 54 kB 00:00 2024-09-13T16:19:09.576 INFO:teuthology.orchestra.run.smithi016.stdout:(87/138): python3-toml-0.10.2-6.el9.noarch.rpm 332 kB/s | 42 kB 00:00 2024-09-13T16:19:09.617 INFO:teuthology.orchestra.run.smithi016.stdout:(88/138): socat-1.7.4.1-6.el9.x86_64.rpm 2.5 MB/s | 304 kB 00:00 2024-09-13T16:19:09.685 INFO:teuthology.orchestra.run.smithi016.stdout:(89/138): uuid-1.6.2-55.el9.x86_64.rpm 532 kB/s | 58 kB 00:00 2024-09-13T16:19:09.727 INFO:teuthology.orchestra.run.smithi016.stdout:(90/138): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 880 kB/s | 96 kB 00:00 2024-09-13T16:19:09.794 INFO:teuthology.orchestra.run.smithi016.stdout:(91/138): xmlstarlet-1.6.1-20.el9.x86_64.rpm 585 kB/s | 64 kB 00:00 2024-09-13T16:19:09.827 INFO:teuthology.orchestra.run.smithi016.stdout:(92/138): zlib-devel-1.2.11-41.el9.x86_64.rpm 447 kB/s | 45 kB 00:00 2024-09-13T16:19:09.861 INFO:teuthology.orchestra.run.smithi016.stdout:(93/138): fmt-8.1.1-5.el9.x86_64.rpm 1.6 MB/s | 111 kB 00:00 2024-09-13T16:19:09.911 INFO:teuthology.orchestra.run.smithi016.stdout:(94/138): gperftools-libs-2.9.1-3.el9.x86_64.rp 3.6 MB/s | 308 kB 00:00 2024-09-13T16:19:09.928 INFO:teuthology.orchestra.run.smithi016.stdout:(95/138): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.5 MB/s | 26 kB 00:00 2024-09-13T16:19:09.953 INFO:teuthology.orchestra.run.smithi016.stdout:(96/138): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-13T16:19:09.987 INFO:teuthology.orchestra.run.smithi016.stdout:(97/138): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-13T16:19:10.053 INFO:teuthology.orchestra.run.smithi111.stdout:(78/138): python3-numpy-f2py-1.20.1-5.el9.x86_6 320 kB/s | 250 kB 00:00 2024-09-13T16:19:10.128 INFO:teuthology.orchestra.run.smithi111.stdout:(79/138): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.0 MB/s | 5.1 MB 00:01 2024-09-13T16:19:10.146 INFO:teuthology.orchestra.run.smithi016.stdout:(98/138): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-09-13T16:19:10.187 INFO:teuthology.orchestra.run.smithi111.stdout:(80/138): python3-oauthlib-3.1.1-5.el9.noarch.r 269 kB/s | 222 kB 00:00 2024-09-13T16:19:10.229 INFO:teuthology.orchestra.run.smithi016.stdout:(99/138): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.4 MB/s | 839 kB 00:00 2024-09-13T16:19:10.263 INFO:teuthology.orchestra.run.smithi016.stdout:(100/138): python3-asyncssh-2.13.2-5.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-09-13T16:19:10.297 INFO:teuthology.orchestra.run.smithi016.stdout:(101/138): python3-bcrypt-3.2.2-1.el9.x86_64.rp 650 kB/s | 43 kB 00:00 2024-09-13T16:19:10.322 INFO:teuthology.orchestra.run.smithi016.stdout:(102/138): python3-cachetools-4.2.4-1.el9.noarc 550 kB/s | 32 kB 00:00 2024-09-13T16:19:10.335 INFO:teuthology.orchestra.run.smithi107.stdout:(138/138): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.5 MB/s | 2.4 MB 00:00 2024-09-13T16:19:10.341 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-13T16:19:10.341 INFO:teuthology.orchestra.run.smithi107.stdout:Total 15 MB/s | 203 MB 00:13 2024-09-13T16:19:10.347 INFO:teuthology.orchestra.run.smithi016.stdout:(103/138): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-13T16:19:10.372 INFO:teuthology.orchestra.run.smithi016.stdout:(104/138): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-13T16:19:10.415 INFO:teuthology.orchestra.run.smithi016.stdout:(105/138): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-13T16:19:10.439 INFO:teuthology.orchestra.run.smithi016.stdout:(106/138): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-13T16:19:10.465 INFO:teuthology.orchestra.run.smithi016.stdout:(107/138): python3-google-auth-2.34.0-1.el9.noa 2.3 MB/s | 220 kB 00:00 2024-09-13T16:19:10.490 INFO:teuthology.orchestra.run.smithi016.stdout:(108/138): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-09-13T16:19:10.516 INFO:teuthology.orchestra.run.smithi016.stdout:(109/138): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-09-13T16:19:10.533 INFO:teuthology.orchestra.run.smithi016.stdout:(110/138): python3-jaraco-functools-3.5.0-2.el9 462 kB/s | 19 kB 00:00 2024-09-13T16:19:10.558 INFO:teuthology.orchestra.run.smithi016.stdout:(111/138): python3-jaraco-text-3.2.0-6.el9.noar 468 kB/s | 20 kB 00:00 2024-09-13T16:19:10.633 INFO:teuthology.orchestra.run.smithi016.stdout:(112/138): libarrow-9.0.0-11.el9.x86_64.rpm 5.7 MB/s | 4.4 MB 00:00 2024-09-13T16:19:10.658 INFO:teuthology.orchestra.run.smithi016.stdout:(113/138): python3-logutils-0.3.5-21.el9.noarch 462 kB/s | 46 kB 00:00 2024-09-13T16:19:10.684 INFO:teuthology.orchestra.run.smithi016.stdout:(114/138): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-09-13T16:19:10.695 INFO:teuthology.orchestra.run.smithi111.stdout:(81/138): python3-packaging-20.9-5.el9.noarch.r 120 kB/s | 77 kB 00:00 2024-09-13T16:19:10.712 INFO:teuthology.orchestra.run.smithi111.stdout:(82/138): python3-prettytable-0.7.2-27.el9.noar 72 kB/s | 42 kB 00:00 2024-09-13T16:19:10.717 INFO:teuthology.orchestra.run.smithi016.stdout:(115/138): python3-natsort-7.1.1-5.el9.noarch.r 985 kB/s | 58 kB 00:00 2024-09-13T16:19:10.734 INFO:teuthology.orchestra.run.smithi016.stdout:(116/138): python3-portend-3.1.0-2.el9.noarch.r 987 kB/s | 16 kB 00:00 2024-09-13T16:19:10.759 INFO:teuthology.orchestra.run.smithi016.stdout:(117/138): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-13T16:19:10.776 INFO:teuthology.orchestra.run.smithi016.stdout:(118/138): python3-repoze-lru-0.7-16.el9.noarch 1.8 MB/s | 31 kB 00:00 2024-09-13T16:19:10.802 INFO:teuthology.orchestra.run.smithi016.stdout:(119/138): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-09-13T16:19:10.827 INFO:teuthology.orchestra.run.smithi016.stdout:(120/138): python3-routes-2.5.1-5.el9.noarch.rp 3.6 MB/s | 188 kB 00:00 2024-09-13T16:19:10.853 INFO:teuthology.orchestra.run.smithi016.stdout:(121/138): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-13T16:19:10.878 INFO:teuthology.orchestra.run.smithi016.stdout:(122/138): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-09-13T16:19:10.903 INFO:teuthology.orchestra.run.smithi016.stdout:(123/138): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-13T16:19:10.937 INFO:teuthology.orchestra.run.smithi016.stdout:(124/138): python3-kubernetes-26.1.0-2.el9.noar 2.5 MB/s | 1.0 MB 00:00 2024-09-13T16:19:10.954 INFO:teuthology.orchestra.run.smithi111.stdout:(83/138): python3-pyasn1-0.4.8-6.el9.noarch.rpm 207 kB/s | 159 kB 00:00 2024-09-13T16:19:10.962 INFO:teuthology.orchestra.run.smithi016.stdout:(125/138): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-09-13T16:19:10.987 INFO:teuthology.orchestra.run.smithi016.stdout:(126/138): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-13T16:19:11.012 INFO:teuthology.orchestra.run.smithi016.stdout:(127/138): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-13T16:19:11.038 INFO:teuthology.orchestra.run.smithi016.stdout:(128/138): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-13T16:19:11.063 INFO:teuthology.orchestra.run.smithi016.stdout:(129/138): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-13T16:19:11.155 INFO:teuthology.orchestra.run.smithi016.stdout:(130/138): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-13T16:19:11.322 INFO:teuthology.orchestra.run.smithi016.stdout:(131/138): librbd1-18.2.4-850.g50bbe7f3.el9.x86 12 MB/s | 3.0 MB 00:00 2024-09-13T16:19:11.371 INFO:teuthology.orchestra.run.smithi111.stdout:(84/138): python3-pytz-2021.1-5.el9.noarch.rpm 77 kB/s | 51 kB 00:00 2024-09-13T16:19:11.372 INFO:teuthology.orchestra.run.smithi016.stdout:(132/138): librados2-18.2.4-850.g50bbe7f3.el9.x 9.9 MB/s | 3.3 MB 00:00 2024-09-13T16:19:11.455 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-13T16:19:11.456 INFO:teuthology.orchestra.run.smithi016.stdout:(133/138): NetworkManager-team-1.49.90-1.el9.x8 489 kB/s | 41 kB 00:00 2024-09-13T16:19:11.463 INFO:teuthology.orchestra.run.smithi111.stdout:(85/138): python3-pyasn1-modules-0.4.8-6.el9.no 364 kB/s | 279 kB 00:00 2024-09-13T16:19:11.540 INFO:teuthology.orchestra.run.smithi016.stdout:(134/138): NetworkManager-1.49.90-1.el9.x86_64. 6.0 MB/s | 2.3 MB 00:00 2024-09-13T16:19:11.555 INFO:teuthology.orchestra.run.smithi111.stdout:(86/138): python3-requests-oauthlib-1.3.0-12.el 90 kB/s | 54 kB 00:00 2024-09-13T16:19:11.573 INFO:teuthology.orchestra.run.smithi016.stdout:(135/138): NetworkManager-tui-1.49.90-1.el9.x86 2.1 MB/s | 249 kB 00:00 2024-09-13T16:19:11.599 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-13T16:19:11.599 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-13T16:19:11.615 INFO:teuthology.orchestra.run.smithi016.stdout:(136/138): openssl-3.2.2-6.el9.x86_64.rpm 18 MB/s | 1.4 MB 00:00 2024-09-13T16:19:11.832 INFO:teuthology.orchestra.run.smithi016.stdout:(137/138): NetworkManager-libnm-1.49.90-1.el9.x 3.5 MB/s | 1.8 MB 00:00 2024-09-13T16:19:11.882 INFO:teuthology.orchestra.run.smithi016.stdout:(138/138): openssl-libs-3.2.2-6.el9.x86_64.rpm 7.7 MB/s | 2.4 MB 00:00 2024-09-13T16:19:11.888 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-09-13T16:19:11.888 INFO:teuthology.orchestra.run.smithi016.stdout:Total 16 MB/s | 203 MB 00:12 2024-09-13T16:19:12.022 INFO:teuthology.orchestra.run.smithi111.stdout:(87/138): python3-toml-0.10.2-6.el9.noarch.rpm 75 kB/s | 42 kB 00:00 2024-09-13T16:19:12.339 INFO:teuthology.orchestra.run.smithi111.stdout:(88/138): socat-1.7.4.1-6.el9.x86_64.rpm 388 kB/s | 304 kB 00:00 2024-09-13T16:19:12.661 INFO:teuthology.orchestra.run.smithi111.stdout:(89/138): uuid-1.6.2-55.el9.x86_64.rpm 91 kB/s | 58 kB 00:00 2024-09-13T16:19:12.933 INFO:teuthology.orchestra.run.smithi111.stdout:(90/138): python3-scipy-1.6.2-8.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-09-13T16:19:13.030 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-09-13T16:19:13.056 INFO:teuthology.orchestra.run.smithi111.stdout:(91/138): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 134 kB/s | 96 kB 00:00 2024-09-13T16:19:13.107 INFO:teuthology.orchestra.run.smithi111.stdout:(92/138): fmt-8.1.1-5.el9.x86_64.rpm 2.2 MB/s | 111 kB 00:00 2024-09-13T16:19:13.140 INFO:teuthology.orchestra.run.smithi111.stdout:(93/138): gperftools-libs-2.9.1-3.el9.x86_64.rp 9.0 MB/s | 308 kB 00:00 2024-09-13T16:19:13.192 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-09-13T16:19:13.192 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-09-13T16:19:13.432 INFO:teuthology.orchestra.run.smithi111.stdout:(94/138): libarrow-9.0.0-11.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-09-13T16:19:13.449 INFO:teuthology.orchestra.run.smithi111.stdout:(95/138): xmlstarlet-1.6.1-20.el9.x86_64.rpm 80 kB/s | 64 kB 00:00 2024-09-13T16:19:13.466 INFO:teuthology.orchestra.run.smithi111.stdout:(96/138): libarrow-doc-9.0.0-11.el9.noarch.rpm 804 kB/s | 26 kB 00:00 2024-09-13T16:19:13.499 INFO:teuthology.orchestra.run.smithi111.stdout:(97/138): libunwind-1.6.2-1.el9.x86_64.rpm 2.5 MB/s | 67 kB 00:00 2024-09-13T16:19:13.516 INFO:teuthology.orchestra.run.smithi111.stdout:(98/138): zlib-devel-1.2.11-41.el9.x86_64.rpm 77 kB/s | 45 kB 00:00 2024-09-13T16:19:13.535 INFO:teuthology.orchestra.run.smithi111.stdout:(99/138): liboath-2.6.7-2.el9.x86_64.rpm 578 kB/s | 49 kB 00:00 2024-09-13T16:19:13.558 INFO:teuthology.orchestra.run.smithi111.stdout:(100/138): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.4 MB/s | 43 kB 00:00 2024-09-13T16:19:13.575 INFO:teuthology.orchestra.run.smithi111.stdout:(101/138): python3-cachetools-4.2.4-1.el9.noarc 2.1 MB/s | 32 kB 00:00 2024-09-13T16:19:13.592 INFO:teuthology.orchestra.run.smithi111.stdout:(102/138): python3-certifi-2023.05.07-4.el9.noa 845 kB/s | 14 kB 00:00 2024-09-13T16:19:13.617 INFO:teuthology.orchestra.run.smithi111.stdout:(103/138): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.1 MB/s | 839 kB 00:00 2024-09-13T16:19:13.618 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-13T16:19:13.618 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-13T16:19:13.643 INFO:teuthology.orchestra.run.smithi111.stdout:(104/138): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-09-13T16:19:13.694 INFO:teuthology.orchestra.run.smithi111.stdout:(105/138): python3-asyncssh-2.13.2-5.el9.noarch 3.0 MB/s | 548 kB 00:00 2024-09-13T16:19:13.718 INFO:teuthology.orchestra.run.smithi111.stdout:(106/138): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-09-13T16:19:13.744 INFO:teuthology.orchestra.run.smithi111.stdout:(107/138): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-09-13T16:19:13.769 INFO:teuthology.orchestra.run.smithi111.stdout:(108/138): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-09-13T16:19:13.794 INFO:teuthology.orchestra.run.smithi111.stdout:(109/138): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-13T16:19:13.820 INFO:teuthology.orchestra.run.smithi111.stdout:(110/138): python3-jaraco-functools-3.5.0-2.el9 385 kB/s | 19 kB 00:00 2024-09-13T16:19:13.845 INFO:teuthology.orchestra.run.smithi111.stdout:(111/138): python3-jaraco-text-3.2.0-6.el9.noar 389 kB/s | 20 kB 00:00 2024-09-13T16:19:13.870 INFO:teuthology.orchestra.run.smithi111.stdout:(112/138): python3-google-auth-2.34.0-1.el9.noa 969 kB/s | 220 kB 00:00 2024-09-13T16:19:13.895 INFO:teuthology.orchestra.run.smithi111.stdout:(113/138): python3-logutils-0.3.5-21.el9.noarch 926 kB/s | 46 kB 00:00 2024-09-13T16:19:13.921 INFO:teuthology.orchestra.run.smithi111.stdout:(114/138): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-09-13T16:19:13.946 INFO:teuthology.orchestra.run.smithi111.stdout:(115/138): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-13T16:19:13.971 INFO:teuthology.orchestra.run.smithi111.stdout:(116/138): python3-portend-3.1.0-2.el9.noarch.r 662 kB/s | 16 kB 00:00 2024-09-13T16:19:13.997 INFO:teuthology.orchestra.run.smithi111.stdout:(117/138): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-13T16:19:14.022 INFO:teuthology.orchestra.run.smithi111.stdout:(118/138): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-13T16:19:14.055 INFO:teuthology.orchestra.run.smithi111.stdout:(119/138): python3-repoze-lru-0.7-16.el9.noarch 527 kB/s | 31 kB 00:00 2024-09-13T16:19:14.081 INFO:teuthology.orchestra.run.smithi111.stdout:(120/138): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-09-13T16:19:14.106 INFO:teuthology.orchestra.run.smithi111.stdout:(121/138): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-13T16:19:14.131 INFO:teuthology.orchestra.run.smithi111.stdout:(122/138): python3-tempora-5.0.0-2.el9.noarch.r 715 kB/s | 36 kB 00:00 2024-09-13T16:19:14.156 INFO:teuthology.orchestra.run.smithi111.stdout:(123/138): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-13T16:19:14.190 INFO:teuthology.orchestra.run.smithi111.stdout:(124/138): python3-kubernetes-26.1.0-2.el9.noar 2.8 MB/s | 1.0 MB 00:00 2024-09-13T16:19:14.215 INFO:teuthology.orchestra.run.smithi111.stdout:(125/138): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-09-13T16:19:14.241 INFO:teuthology.orchestra.run.smithi111.stdout:(126/138): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-13T16:19:14.257 INFO:teuthology.orchestra.run.smithi111.stdout:(127/138): python3-zc-lockfile-2.0-10.el9.noarc 476 kB/s | 20 kB 00:00 2024-09-13T16:19:14.283 INFO:teuthology.orchestra.run.smithi111.stdout:(128/138): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-13T16:19:14.308 INFO:teuthology.orchestra.run.smithi111.stdout:(129/138): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-09-13T16:19:14.392 INFO:teuthology.orchestra.run.smithi111.stdout:(130/138): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-13T16:19:14.534 INFO:teuthology.orchestra.run.smithi111.stdout:(131/138): NetworkManager-1.49.90-1.el9.x86_64. 16 MB/s | 2.3 MB 00:00 2024-09-13T16:19:14.592 INFO:teuthology.orchestra.run.smithi111.stdout:(132/138): librados2-18.2.4-850.g50bbe7f3.el9.x 11 MB/s | 3.3 MB 00:00 2024-09-13T16:19:14.651 INFO:teuthology.orchestra.run.smithi111.stdout:(133/138): librbd1-18.2.4-850.g50bbe7f3.el9.x86 8.8 MB/s | 3.0 MB 00:00 2024-09-13T16:19:14.667 INFO:teuthology.orchestra.run.smithi111.stdout:(134/138): NetworkManager-team-1.49.90-1.el9.x8 542 kB/s | 41 kB 00:00 2024-09-13T16:19:14.710 INFO:teuthology.orchestra.run.smithi111.stdout:(135/138): NetworkManager-libnm-1.49.90-1.el9.x 10 MB/s | 1.8 MB 00:00 2024-09-13T16:19:14.760 INFO:teuthology.orchestra.run.smithi111.stdout:(136/138): openssl-3.2.2-6.el9.x86_64.rpm 15 MB/s | 1.4 MB 00:00 2024-09-13T16:19:14.818 INFO:teuthology.orchestra.run.smithi111.stdout:(137/138): NetworkManager-tui-1.49.90-1.el9.x86 1.5 MB/s | 249 kB 00:00 2024-09-13T16:19:14.860 INFO:teuthology.orchestra.run.smithi111.stdout:(138/138): openssl-libs-3.2.2-6.el9.x86_64.rpm 16 MB/s | 2.4 MB 00:00 2024-09-13T16:19:14.865 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-09-13T16:19:14.866 INFO:teuthology.orchestra.run.smithi111.stdout:Total 11 MB/s | 203 MB 00:18 2024-09-13T16:19:15.330 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-09-13T16:19:15.330 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-09-13T16:19:15.935 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2024-09-13T16:19:16.078 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2024-09-13T16:19:16.078 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2024-09-13T16:19:18.083 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2024-09-13T16:19:18.084 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2024-09-13T16:19:19.374 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-13T16:19:19.514 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/146 2024-09-13T16:19:19.560 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/146 2024-09-13T16:19:19.614 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/146 2024-09-13T16:19:19.647 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/146 2024-09-13T16:19:19.673 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/146 2024-09-13T16:19:19.727 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/146 2024-09-13T16:19:20.046 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/146 2024-09-13T16:19:20.187 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librados2-2:18.2.4-850.g50bbe7f3.el9.x86_64 8/146 2024-09-13T16:19:20.309 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:18.2.4-850.g50bbe7f3.el9.x86_64 8/146 2024-09-13T16:19:20.338 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs2-2:18.2.4-850.g50bbe7f3.el9.x86_64 9/146 2024-09-13T16:19:20.420 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephfs2-2:18.2.4-850.g50bbe7f3.el9.x86_64 9/146 2024-09-13T16:19:20.459 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rados-2:18.2.4-850.g50bbe7f3.el9.x86_64 10/146 2024-09-13T16:19:20.626 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/146 2024-09-13T16:19:20.682 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/146 2024-09-13T16:19:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephsqlite-2:18.2.4-850.g50bbe7f3.el9.x86_64 13/146 2024-09-13T16:19:20.802 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephsqlite-2:18.2.4-850.g50bbe7f3.el9.x86_64 13/146 2024-09-13T16:19:20.826 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libradosstriper1-2:18.2.4-850.g50bbe7f3.el9.x86_ 14/146 2024-09-13T16:19:21.060 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libradosstriper1-2:18.2.4-850.g50bbe7f3.el9.x86_ 14/146 2024-09-13T16:19:21.156 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-09-13T16:19:21.173 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 15/146 2024-09-13T16:19:21.244 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/146 2024-09-13T16:19:21.284 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/146 2024-09-13T16:19:21.295 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T16:19:21.343 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/146 2024-09-13T16:19:21.358 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T16:19:21.379 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/146 2024-09-13T16:19:21.404 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/146 2024-09-13T16:19:21.457 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/146 2024-09-13T16:19:21.647 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T16:19:21.688 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/146 2024-09-13T16:19:21.722 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/146 2024-09-13T16:19:21.796 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/146 2024-09-13T16:19:21.830 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 20/146 2024-09-13T16:19:21.857 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 21/146 2024-09-13T16:19:21.884 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/146 2024-09-13T16:19:21.927 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 22/146 2024-09-13T16:19:21.980 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 23/146 2024-09-13T16:19:21.996 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librados2-2:18.2.4-850.g50bbe7f3.el9.x86_64 8/146 2024-09-13T16:19:22.011 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 24/146 2024-09-13T16:19:22.057 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 25/146 2024-09-13T16:19:22.118 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:18.2.4-850.g50bbe7f3.el9.x86_64 8/146 2024-09-13T16:19:22.129 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 26/146 2024-09-13T16:19:22.147 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs2-2:18.2.4-850.g50bbe7f3.el9.x86_64 9/146 2024-09-13T16:19:22.165 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-common-2:18.2.4-850.g50bbe7f3.el9.x 27/146 2024-09-13T16:19:22.209 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-argparse-2:18.2.4-850.g50bbe7f3.el9 28/146 2024-09-13T16:19:22.223 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs2-2:18.2.4-850.g50bbe7f3.el9.x86_64 9/146 2024-09-13T16:19:22.252 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rados-2:18.2.4-850.g50bbe7f3.el9.x86_64 10/146 2024-09-13T16:19:22.283 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cephfs-2:18.2.4-850.g50bbe7f3.el9.x86_64 29/146 2024-09-13T16:19:22.418 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/146 2024-09-13T16:19:22.432 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/146 2024-09-13T16:19:22.485 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/146 2024-09-13T16:19:22.492 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/146 2024-09-13T16:19:22.513 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephsqlite-2:18.2.4-850.g50bbe7f3.el9.x86_64 13/146 2024-09-13T16:19:22.608 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephsqlite-2:18.2.4-850.g50bbe7f3.el9.x86_64 13/146 2024-09-13T16:19:22.631 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libradosstriper1-2:18.2.4-850.g50bbe7f3.el9.x86_ 14/146 2024-09-13T16:19:22.831 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libradosstriper1-2:18.2.4-850.g50bbe7f3.el9.x86_ 14/146 2024-09-13T16:19:22.961 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 15/146 2024-09-13T16:19:23.018 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/146 2024-09-13T16:19:23.067 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T16:19:23.105 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/146 2024-09-13T16:19:23.138 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T16:19:23.153 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 34/146 2024-09-13T16:19:23.182 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librados-devel-2:18.2.4-850.g50bbe7f3.el9.x86_64 35/146 2024-09-13T16:19:23.224 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/146 2024-09-13T16:19:23.398 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 37/146 2024-09-13T16:19:23.441 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 38/146 2024-09-13T16:19:23.445 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T16:19:23.484 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/146 2024-09-13T16:19:23.519 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/146 2024-09-13T16:19:23.547 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : re2-1:20211101-20.el9.x86_64 39/146 2024-09-13T16:19:23.589 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 40/146 2024-09-13T16:19:23.595 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/146 2024-09-13T16:19:23.635 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 20/146 2024-09-13T16:19:23.641 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 41/146 2024-09-13T16:19:23.662 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 21/146 2024-09-13T16:19:23.713 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 42/146 2024-09-13T16:19:23.733 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 22/146 2024-09-13T16:19:23.745 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 43/146 2024-09-13T16:19:23.773 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 44/146 2024-09-13T16:19:23.778 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2024-09-13T16:19:23.794 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 23/146 2024-09-13T16:19:23.819 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 45/146 2024-09-13T16:19:23.833 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 24/146 2024-09-13T16:19:23.865 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 46/146 2024-09-13T16:19:23.873 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/146 2024-09-13T16:19:23.876 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 25/146 2024-09-13T16:19:23.907 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 47/146 2024-09-13T16:19:23.920 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/146 2024-09-13T16:19:23.953 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 26/146 2024-09-13T16:19:23.984 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/146 2024-09-13T16:19:23.987 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-common-2:18.2.4-850.g50bbe7f3.el9.x 27/146 2024-09-13T16:19:24.031 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/146 2024-09-13T16:19:24.031 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-argparse-2:18.2.4-850.g50bbe7f3.el9 28/146 2024-09-13T16:19:24.041 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 48/146 2024-09-13T16:19:24.057 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/146 2024-09-13T16:19:24.084 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 49/146 2024-09-13T16:19:24.105 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/146 2024-09-13T16:19:24.108 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cephfs-2:18.2.4-850.g50bbe7f3.el9.x86_64 29/146 2024-09-13T16:19:24.126 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 50/146 2024-09-13T16:19:24.227 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 51/146 2024-09-13T16:19:24.251 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/146 2024-09-13T16:19:24.259 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 52/146 2024-09-13T16:19:24.300 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 53/146 2024-09-13T16:19:24.308 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/146 2024-09-13T16:19:24.333 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 54/146 2024-09-13T16:19:24.360 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 55/146 2024-09-13T16:19:24.393 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : fmt-8.1.1-5.el9.x86_64 56/146 2024-09-13T16:19:24.419 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/146 2024-09-13T16:19:24.422 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 57/146 2024-09-13T16:19:24.465 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 58/146 2024-09-13T16:19:24.501 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : uuid-1.6.2-55.el9.x86_64 59/146 2024-09-13T16:19:24.516 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : librados2-2:18.2.4-850.g50bbe7f3.el9.x86_64 8/146 2024-09-13T16:19:24.531 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 60/146 2024-09-13T16:19:24.565 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 61/146 2024-09-13T16:19:24.598 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 62/146 2024-09-13T16:19:24.618 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librados2-2:18.2.4-850.g50bbe7f3.el9.x86_64 8/146 2024-09-13T16:19:24.640 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 63/146 2024-09-13T16:19:24.651 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephfs2-2:18.2.4-850.g50bbe7f3.el9.x86_64 9/146 2024-09-13T16:19:24.674 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 64/146 2024-09-13T16:19:24.709 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 65/146 2024-09-13T16:19:24.726 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libcephfs2-2:18.2.4-850.g50bbe7f3.el9.x86_64 9/146 2024-09-13T16:19:24.757 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 66/146 2024-09-13T16:19:24.761 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rados-2:18.2.4-850.g50bbe7f3.el9.x86_64 10/146 2024-09-13T16:19:24.863 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-packaging-20.9-5.el9.noarch 67/146 2024-09-13T16:19:24.871 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/146 2024-09-13T16:19:24.911 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 68/146 2024-09-13T16:19:24.936 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/146 2024-09-13T16:19:24.964 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/146 2024-09-13T16:19:24.986 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/146 2024-09-13T16:19:25.009 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 34/146 2024-09-13T16:19:25.013 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephsqlite-2:18.2.4-850.g50bbe7f3.el9.x86_64 13/146 2024-09-13T16:19:25.055 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librados-devel-2:18.2.4-850.g50bbe7f3.el9.x86_64 35/146 2024-09-13T16:19:25.105 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/146 2024-09-13T16:19:25.115 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libcephsqlite-2:18.2.4-850.g50bbe7f3.el9.x86_64 13/146 2024-09-13T16:19:25.139 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libradosstriper1-2:18.2.4-850.g50bbe7f3.el9.x86_ 14/146 2024-09-13T16:19:25.268 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 37/146 2024-09-13T16:19:25.322 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 38/146 2024-09-13T16:19:25.354 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libradosstriper1-2:18.2.4-850.g50bbe7f3.el9.x86_ 14/146 2024-09-13T16:19:25.450 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : re2-1:20211101-20.el9.x86_64 39/146 2024-09-13T16:19:25.477 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 15/146 2024-09-13T16:19:25.492 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 40/146 2024-09-13T16:19:25.548 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 41/146 2024-09-13T16:19:25.601 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T16:19:25.623 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 42/146 2024-09-13T16:19:25.629 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 69/146 2024-09-13T16:19:25.662 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T16:19:25.668 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 43/146 2024-09-13T16:19:25.684 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 70/146 2024-09-13T16:19:25.705 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 44/146 2024-09-13T16:19:25.717 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 71/146 2024-09-13T16:19:25.753 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 45/146 2024-09-13T16:19:25.797 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 46/146 2024-09-13T16:19:25.815 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 72/146 2024-09-13T16:19:25.839 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 47/146 2024-09-13T16:19:25.851 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 73/146 2024-09-13T16:19:25.950 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T16:19:25.982 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 48/146 2024-09-13T16:19:25.992 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/146 2024-09-13T16:19:26.024 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 49/146 2024-09-13T16:19:26.026 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/146 2024-09-13T16:19:26.037 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-0.3.26-2.el9.x86_64 74/146 2024-09-13T16:19:26.066 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 75/146 2024-09-13T16:19:26.066 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 50/146 2024-09-13T16:19:26.100 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/146 2024-09-13T16:19:26.134 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 20/146 2024-09-13T16:19:26.154 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 76/146 2024-09-13T16:19:26.161 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 21/146 2024-09-13T16:19:26.172 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 51/146 2024-09-13T16:19:26.208 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 52/146 2024-09-13T16:19:26.231 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 22/146 2024-09-13T16:19:26.248 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 53/146 2024-09-13T16:19:26.281 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 54/146 2024-09-13T16:19:26.283 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 23/146 2024-09-13T16:19:26.309 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 55/146 2024-09-13T16:19:26.315 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 24/146 2024-09-13T16:19:26.342 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fmt-8.1.1-5.el9.x86_64 56/146 2024-09-13T16:19:26.370 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 57/146 2024-09-13T16:19:26.376 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 25/146 2024-09-13T16:19:26.405 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 58/146 2024-09-13T16:19:26.450 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : uuid-1.6.2-55.el9.x86_64 59/146 2024-09-13T16:19:26.457 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 26/146 2024-09-13T16:19:26.480 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 60/146 2024-09-13T16:19:26.494 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ceph-common-2:18.2.4-850.g50bbe7f3.el9.x 27/146 2024-09-13T16:19:26.522 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 61/146 2024-09-13T16:19:26.537 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ceph-argparse-2:18.2.4-850.g50bbe7f3.el9 28/146 2024-09-13T16:19:26.556 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 62/146 2024-09-13T16:19:26.598 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 63/146 2024-09-13T16:19:26.612 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cephfs-2:18.2.4-850.g50bbe7f3.el9.x86_64 29/146 2024-09-13T16:19:26.632 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 64/146 2024-09-13T16:19:26.667 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 65/146 2024-09-13T16:19:26.716 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 66/146 2024-09-13T16:19:26.750 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/146 2024-09-13T16:19:26.805 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/146 2024-09-13T16:19:26.815 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 77/146 2024-09-13T16:19:26.826 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-packaging-20.9-5.el9.noarch 67/146 2024-09-13T16:19:26.887 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 68/146 2024-09-13T16:19:26.934 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 78/146 2024-09-13T16:19:27.332 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/146 2024-09-13T16:19:27.418 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/146 2024-09-13T16:19:27.456 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 34/146 2024-09-13T16:19:27.494 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librados-devel-2:18.2.4-850.g50bbe7f3.el9.x86_64 35/146 2024-09-13T16:19:27.536 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/146 2024-09-13T16:19:27.650 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 69/146 2024-09-13T16:19:27.701 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 37/146 2024-09-13T16:19:27.707 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 70/146 2024-09-13T16:19:27.741 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 71/146 2024-09-13T16:19:27.774 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 38/146 2024-09-13T16:19:27.844 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 72/146 2024-09-13T16:19:27.875 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 73/146 2024-09-13T16:19:27.877 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : re2-1:20211101-20.el9.x86_64 39/146 2024-09-13T16:19:27.923 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 40/146 2024-09-13T16:19:27.978 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 41/146 2024-09-13T16:19:28.050 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 42/146 2024-09-13T16:19:28.066 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-0.3.26-2.el9.x86_64 74/146 2024-09-13T16:19:28.082 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 43/146 2024-09-13T16:19:28.098 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 75/146 2024-09-13T16:19:28.126 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 44/146 2024-09-13T16:19:28.167 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 79/146 2024-09-13T16:19:28.172 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 45/146 2024-09-13T16:19:28.178 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 76/146 2024-09-13T16:19:28.218 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 46/146 2024-09-13T16:19:28.258 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 80/146 2024-09-13T16:19:28.260 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 47/146 2024-09-13T16:19:28.286 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 81/146 2024-09-13T16:19:28.319 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 82/146 2024-09-13T16:19:28.395 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 48/146 2024-09-13T16:19:28.445 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 49/146 2024-09-13T16:19:28.487 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 50/146 2024-09-13T16:19:28.589 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 51/146 2024-09-13T16:19:28.621 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 52/146 2024-09-13T16:19:28.621 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 83/146 2024-09-13T16:19:28.662 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 53/146 2024-09-13T16:19:28.665 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librbd1-2:18.2.4-850.g50bbe7f3.el9.x86_64 84/146 2024-09-13T16:19:28.694 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 54/146 2024-09-13T16:19:28.721 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 55/146 2024-09-13T16:19:28.739 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:18.2.4-850.g50bbe7f3.el9.x86_64 84/146 2024-09-13T16:19:28.754 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : fmt-8.1.1-5.el9.x86_64 56/146 2024-09-13T16:19:28.770 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rbd-2:18.2.4-850.g50bbe7f3.el9.x86_64 85/146 2024-09-13T16:19:28.783 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 57/146 2024-09-13T16:19:28.806 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 86/146 2024-09-13T16:19:28.818 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 58/146 2024-09-13T16:19:28.854 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : uuid-1.6.2-55.el9.x86_64 59/146 2024-09-13T16:19:28.857 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 77/146 2024-09-13T16:19:28.884 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 60/146 2024-09-13T16:19:28.918 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 61/146 2024-09-13T16:19:28.952 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 62/146 2024-09-13T16:19:28.983 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 78/146 2024-09-13T16:19:28.993 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 63/146 2024-09-13T16:19:29.103 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 64/146 2024-09-13T16:19:29.215 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 87/146 2024-09-13T16:19:29.243 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librgw2-2:18.2.4-850.g50bbe7f3.el9.x86_64 88/146 2024-09-13T16:19:29.281 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 65/146 2024-09-13T16:19:29.315 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librgw2-2:18.2.4-850.g50bbe7f3.el9.x86_64 88/146 2024-09-13T16:19:29.329 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 66/146 2024-09-13T16:19:29.349 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rgw-2:18.2.4-850.g50bbe7f3.el9.x86_64 89/146 2024-09-13T16:19:29.440 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-packaging-20.9-5.el9.noarch 67/146 2024-09-13T16:19:29.498 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 68/146 2024-09-13T16:19:30.207 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 69/146 2024-09-13T16:19:30.261 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 70/146 2024-09-13T16:19:30.294 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 71/146 2024-09-13T16:19:30.324 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 79/146 2024-09-13T16:19:30.393 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 72/146 2024-09-13T16:19:30.423 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 80/146 2024-09-13T16:19:30.428 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 73/146 2024-09-13T16:19:30.451 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 81/146 2024-09-13T16:19:30.484 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 82/146 2024-09-13T16:19:30.612 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openblas-0.3.26-2.el9.x86_64 74/146 2024-09-13T16:19:30.643 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 75/146 2024-09-13T16:19:30.725 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 76/146 2024-09-13T16:19:30.799 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 83/146 2024-09-13T16:19:30.839 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librbd1-2:18.2.4-850.g50bbe7f3.el9.x86_64 84/146 2024-09-13T16:19:30.932 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:18.2.4-850.g50bbe7f3.el9.x86_64 84/146 2024-09-13T16:19:30.960 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rbd-2:18.2.4-850.g50bbe7f3.el9.x86_64 85/146 2024-09-13T16:19:31.005 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 86/146 2024-09-13T16:19:31.325 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:18.2.4-850.g50bbe7f3.el9.x86_64 90/146 2024-09-13T16:19:31.363 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-common-2:18.2.4-850.g50bbe7f3.el9.x86_64 90/146 2024-09-13T16:19:31.385 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 77/146 2024-09-13T16:19:31.431 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 87/146 2024-09-13T16:19:31.458 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librgw2-2:18.2.4-850.g50bbe7f3.el9.x86_64 88/146 2024-09-13T16:19:31.510 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 78/146 2024-09-13T16:19:31.530 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librgw2-2:18.2.4-850.g50bbe7f3.el9.x86_64 88/146 2024-09-13T16:19:31.560 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rgw-2:18.2.4-850.g50bbe7f3.el9.x86_64 89/146 2024-09-13T16:19:31.883 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:18.2.4-850.g50bbe7f3.el9.x86_64 90/146 2024-09-13T16:19:31.916 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-base-2:18.2.4-850.g50bbe7f3.el9.x86_64 91/146 2024-09-13T16:19:31.997 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-base-2:18.2.4-850.g50bbe7f3.el9.x86_64 91/146 2024-09-13T16:19:31.997 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-13T16:19:31.997 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-13T16:19:31.997 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-13T16:19:32.034 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-selinux-2:18.2.4-850.g50bbe7f3.el9.x86_64 92/146 2024-09-13T16:19:32.764 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 79/146 2024-09-13T16:19:32.841 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 80/146 2024-09-13T16:19:32.869 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 81/146 2024-09-13T16:19:32.902 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 82/146 2024-09-13T16:19:33.220 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 83/146 2024-09-13T16:19:33.257 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : librbd1-2:18.2.4-850.g50bbe7f3.el9.x86_64 84/146 2024-09-13T16:19:33.347 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librbd1-2:18.2.4-850.g50bbe7f3.el9.x86_64 84/146 2024-09-13T16:19:33.378 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rbd-2:18.2.4-850.g50bbe7f3.el9.x86_64 85/146 2024-09-13T16:19:33.415 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 86/146 2024-09-13T16:19:33.723 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:18.2.4-850.g50bbe7f3.el9.x86_64 90/146 2024-09-13T16:19:33.752 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-common-2:18.2.4-850.g50bbe7f3.el9.x86_64 90/146 2024-09-13T16:19:33.816 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 87/146 2024-09-13T16:19:33.843 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librgw2-2:18.2.4-850.g50bbe7f3.el9.x86_64 88/146 2024-09-13T16:19:33.898 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librgw2-2:18.2.4-850.g50bbe7f3.el9.x86_64 88/146 2024-09-13T16:19:33.920 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rgw-2:18.2.4-850.g50bbe7f3.el9.x86_64 89/146 2024-09-13T16:19:34.309 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:18.2.4-850.g50bbe7f3.el9.x86_64 90/146 2024-09-13T16:19:34.347 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-base-2:18.2.4-850.g50bbe7f3.el9.x86_64 91/146 2024-09-13T16:19:34.452 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-base-2:18.2.4-850.g50bbe7f3.el9.x86_64 91/146 2024-09-13T16:19:34.452 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-13T16:19:34.452 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-13T16:19:34.453 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:19:34.490 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-selinux-2:18.2.4-850.g50bbe7f3.el9.x86_64 92/146 2024-09-13T16:19:35.955 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-common-2:18.2.4-850.g50bbe7f3.el9.x86_64 90/146 2024-09-13T16:19:35.992 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-common-2:18.2.4-850.g50bbe7f3.el9.x86_64 90/146 2024-09-13T16:19:36.530 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-common-2:18.2.4-850.g50bbe7f3.el9.x86_64 90/146 2024-09-13T16:19:36.569 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-base-2:18.2.4-850.g50bbe7f3.el9.x86_64 91/146 2024-09-13T16:19:36.675 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-base-2:18.2.4-850.g50bbe7f3.el9.x86_64 91/146 2024-09-13T16:19:36.675 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-13T16:19:36.675 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-13T16:19:36.676 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T16:19:36.713 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-selinux-2:18.2.4-850.g50bbe7f3.el9.x86_64 92/146 2024-09-13T16:19:43.877 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-selinux-2:18.2.4-850.g50bbe7f3.el9.x86_64 92/146 2024-09-13T16:19:43.877 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /sys 2024-09-13T16:19:43.877 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /proc 2024-09-13T16:19:43.877 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /mnt 2024-09-13T16:19:43.877 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /var/tmp 2024-09-13T16:19:43.877 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /home 2024-09-13T16:19:43.877 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /root 2024-09-13T16:19:43.877 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /tmp 2024-09-13T16:19:43.878 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-13T16:19:44.047 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-volume-2:18.2.4-850.g50bbe7f3.el9.noarch 93/146 2024-09-13T16:19:44.063 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-volume-2:18.2.4-850.g50bbe7f3.el9.noarch 93/146 2024-09-13T16:19:44.063 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T16:19:44.063 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-13T16:19:44.063 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-13T16:19:45.534 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-osd-2:18.2.4-850.g50bbe7f3.el9.x86_64 94/146 2024-09-13T16:19:45.561 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-osd-2:18.2.4-850.g50bbe7f3.el9.x86_64 94/146 2024-09-13T16:19:45.561 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T16:19:45.561 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-13T16:19:45.561 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T16:19:45.561 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T16:19:45.561 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-13T16:19:45.784 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mds-2:18.2.4-850.g50bbe7f3.el9.x86_64 95/146 2024-09-13T16:19:45.808 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mds-2:18.2.4-850.g50bbe7f3.el9.x86_64 95/146 2024-09-13T16:19:45.808 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T16:19:45.808 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-13T16:19:45.808 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T16:19:45.808 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T16:19:45.808 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-13T16:19:46.252 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mon-2:18.2.4-850.g50bbe7f3.el9.x86_64 96/146 2024-09-13T16:19:46.275 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mon-2:18.2.4-850.g50bbe7f3.el9.x86_64 96/146 2024-09-13T16:19:46.275 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T16:19:46.275 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-13T16:19:46.275 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T16:19:46.276 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T16:19:46.276 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-13T16:19:46.373 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : automake-1.16.2-8.el9.noarch 97/146 2024-09-13T16:19:46.429 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ply-3.11-14.el9.noarch 98/146 2024-09-13T16:19:46.485 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 99/146 2024-09-13T16:19:46.669 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 100/146 2024-09-13T16:19:46.719 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 101/146 2024-09-13T16:19:46.759 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-selinux-2:18.2.4-850.g50bbe7f3.el9.x86_64 92/146 2024-09-13T16:19:46.759 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /sys 2024-09-13T16:19:46.760 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /proc 2024-09-13T16:19:46.760 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /mnt 2024-09-13T16:19:46.760 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /var/tmp 2024-09-13T16:19:46.760 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /home 2024-09-13T16:19:46.760 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /root 2024-09-13T16:19:46.760 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /tmp 2024-09-13T16:19:46.760 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:19:46.795 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 102/146 2024-09-13T16:19:46.887 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 103/146 2024-09-13T16:19:46.928 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-volume-2:18.2.4-850.g50bbe7f3.el9.noarch 93/146 2024-09-13T16:19:46.945 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-volume-2:18.2.4-850.g50bbe7f3.el9.noarch 93/146 2024-09-13T16:19:46.945 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T16:19:46.945 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-13T16:19:46.945 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:19:47.050 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/146 2024-09-13T16:19:47.088 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 105/146 2024-09-13T16:19:47.120 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 106/146 2024-09-13T16:19:47.147 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : fuse-2.9.9-16.el9.x86_64 107/146 2024-09-13T16:19:47.528 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephadm-2:18.2.4-850.g50bbe7f3.el9.noarch 108/146 2024-09-13T16:19:47.563 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephadm-2:18.2.4-850.g50bbe7f3.el9.noarch 108/146 2024-09-13T16:19:47.592 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-850.g50bbe7f3.el 109/146 2024-09-13T16:19:47.655 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-850.g50bbe7f3.e 110/146 2024-09-13T16:19:47.678 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-850.g50bbe7f3.el9.noar 111/146 2024-09-13T16:19:48.031 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-selinux-2:18.2.4-850.g50bbe7f3.el9.x86_64 92/146 2024-09-13T16:19:48.032 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /sys 2024-09-13T16:19:48.032 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /proc 2024-09-13T16:19:48.032 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /mnt 2024-09-13T16:19:48.032 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /var/tmp 2024-09-13T16:19:48.032 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /home 2024-09-13T16:19:48.032 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /root 2024-09-13T16:19:48.032 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /tmp 2024-09-13T16:19:48.032 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T16:19:48.197 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-volume-2:18.2.4-850.g50bbe7f3.el9.noarch 93/146 2024-09-13T16:19:48.212 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-volume-2:18.2.4-850.g50bbe7f3.el9.noarch 93/146 2024-09-13T16:19:48.213 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T16:19:48.213 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-13T16:19:48.213 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T16:19:48.531 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-osd-2:18.2.4-850.g50bbe7f3.el9.x86_64 94/146 2024-09-13T16:19:48.560 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-osd-2:18.2.4-850.g50bbe7f3.el9.x86_64 94/146 2024-09-13T16:19:48.560 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T16:19:48.560 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-13T16:19:48.561 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T16:19:48.561 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T16:19:48.561 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:19:48.704 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-850.g50bbe7f3.el9.noar 111/146 2024-09-13T16:19:48.738 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-850.g50bbe7f3.el9.no 112/146 2024-09-13T16:19:48.774 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mds-2:18.2.4-850.g50bbe7f3.el9.x86_64 95/146 2024-09-13T16:19:48.799 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mds-2:18.2.4-850.g50bbe7f3.el9.x86_64 95/146 2024-09-13T16:19:48.799 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T16:19:48.799 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-13T16:19:48.799 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T16:19:48.799 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T16:19:48.799 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:19:49.267 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mon-2:18.2.4-850.g50bbe7f3.el9.x86_64 96/146 2024-09-13T16:19:49.293 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mon-2:18.2.4-850.g50bbe7f3.el9.x86_64 96/146 2024-09-13T16:19:49.293 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T16:19:49.293 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-13T16:19:49.293 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T16:19:49.293 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T16:19:49.294 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:19:49.398 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : automake-1.16.2-8.el9.noarch 97/146 2024-09-13T16:19:49.456 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ply-3.11-14.el9.noarch 98/146 2024-09-13T16:19:49.520 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 99/146 2024-09-13T16:19:49.675 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-osd-2:18.2.4-850.g50bbe7f3.el9.x86_64 94/146 2024-09-13T16:19:49.697 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-850.g50bbe7f3.el9.no 112/146 2024-09-13T16:19:49.701 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-osd-2:18.2.4-850.g50bbe7f3.el9.x86_64 94/146 2024-09-13T16:19:49.701 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T16:19:49.701 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-13T16:19:49.701 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T16:19:49.701 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T16:19:49.702 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T16:19:49.722 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 100/146 2024-09-13T16:19:49.730 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-850.g50bb 113/146 2024-09-13T16:19:49.785 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 101/146 2024-09-13T16:19:49.831 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-850.g50bb 113/146 2024-09-13T16:19:49.873 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 102/146 2024-09-13T16:19:49.909 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mds-2:18.2.4-850.g50bbe7f3.el9.x86_64 95/146 2024-09-13T16:19:49.932 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mds-2:18.2.4-850.g50bbe7f3.el9.x86_64 95/146 2024-09-13T16:19:49.932 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T16:19:49.932 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-13T16:19:49.932 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T16:19:49.932 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T16:19:49.932 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T16:19:49.971 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 103/146 2024-09-13T16:19:49.993 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-850.g50bbe7f3.el9 114/146 2024-09-13T16:19:50.024 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-2:18.2.4-850.g50bbe7f3.el9.x86_64 115/146 2024-09-13T16:19:50.048 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-2:18.2.4-850.g50bbe7f3.el9.x86_64 115/146 2024-09-13T16:19:50.048 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T16:19:50.048 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-13T16:19:50.048 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T16:19:50.048 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T16:19:50.048 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-13T16:19:50.099 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-rook-2:18.2.4-850.g50bbe7f3.el9.noarch 116/146 2024-09-13T16:19:50.111 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-850.g50bbe7f3.el9.noarch 116/146 2024-09-13T16:19:50.113 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/146 2024-09-13T16:19:50.170 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 105/146 2024-09-13T16:19:50.201 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 106/146 2024-09-13T16:19:50.210 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-2:18.2.4-850.g50bbe7f3.el9.x86_64 117/146 2024-09-13T16:19:50.228 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fuse-2.9.9-16.el9.x86_64 107/146 2024-09-13T16:19:50.268 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-fuse-2:18.2.4-850.g50bbe7f3.el9.x86_64 118/146 2024-09-13T16:19:50.385 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mon-2:18.2.4-850.g50bbe7f3.el9.x86_64 96/146 2024-09-13T16:19:50.408 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mon-2:18.2.4-850.g50bbe7f3.el9.x86_64 96/146 2024-09-13T16:19:50.408 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T16:19:50.408 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-13T16:19:50.408 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T16:19:50.408 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T16:19:50.408 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T16:19:50.515 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : automake-1.16.2-8.el9.noarch 97/146 2024-09-13T16:19:50.563 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ply-3.11-14.el9.noarch 98/146 2024-09-13T16:19:50.618 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 99/146 2024-09-13T16:19:50.686 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephadm-2:18.2.4-850.g50bbe7f3.el9.noarch 108/146 2024-09-13T16:19:50.718 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephadm-2:18.2.4-850.g50bbe7f3.el9.noarch 108/146 2024-09-13T16:19:50.756 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-850.g50bbe7f3.el 109/146 2024-09-13T16:19:50.810 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 100/146 2024-09-13T16:19:50.823 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-850.g50bbe7f3.e 110/146 2024-09-13T16:19:50.849 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-850.g50bbe7f3.el9.noar 111/146 2024-09-13T16:19:50.861 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 101/146 2024-09-13T16:19:50.938 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 102/146 2024-09-13T16:19:50.995 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libtool-2.4.6-46.el9.x86_64 119/146 2024-09-13T16:19:51.026 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 103/146 2024-09-13T16:19:51.027 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-radosgw-2:18.2.4-850.g50bbe7f3.el9.x86_64 120/146 2024-09-13T16:19:51.052 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-850.g50bbe7f3.el9.x86_64 120/146 2024-09-13T16:19:51.052 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T16:19:51.052 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-13T16:19:51.052 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T16:19:51.052 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T16:19:51.052 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-13T16:19:51.094 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-850.g50bbe7 121/146 2024-09-13T16:19:51.118 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-850.g50bbe7 121/146 2024-09-13T16:19:51.118 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T16:19:51.118 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-13T16:19:51.118 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-13T16:19:51.162 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-mirror-2:18.2.4-850.g50bbe7f3.el9.x86_64 122/146 2024-09-13T16:19:51.165 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/146 2024-09-13T16:19:51.185 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephfs-mirror-2:18.2.4-850.g50bbe7f3.el9.x86_64 122/146 2024-09-13T16:19:51.185 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T16:19:51.185 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-13T16:19:51.185 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-13T16:19:51.186 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-13T16:19:51.186 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-13T16:19:51.221 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 105/146 2024-09-13T16:19:51.259 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 106/146 2024-09-13T16:19:51.286 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : fuse-2.9.9-16.el9.x86_64 107/146 2024-09-13T16:19:51.488 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-mirror-2:18.2.4-850.g50bbe7f3.el9.x86_64 123/146 2024-09-13T16:19:51.511 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-mirror-2:18.2.4-850.g50bbe7f3.el9.x86_64 123/146 2024-09-13T16:19:51.512 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T16:19:51.512 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-13T16:19:51.512 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T16:19:51.512 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T16:19:51.512 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-13T16:19:51.719 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: cephadm-2:18.2.4-850.g50bbe7f3.el9.noarch 108/146 2024-09-13T16:19:51.745 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephadm-2:18.2.4-850.g50bbe7f3.el9.noarch 108/146 2024-09-13T16:19:51.774 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-850.g50bbe7f3.el 109/146 2024-09-13T16:19:51.837 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-850.g50bbe7f3.e 110/146 2024-09-13T16:19:51.860 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-850.g50bbe7f3.el9.noar 111/146 2024-09-13T16:19:51.947 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-850.g50bbe7f3.el9.noar 111/146 2024-09-13T16:19:51.985 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-850.g50bbe7f3.el9.no 112/146 2024-09-13T16:19:52.896 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-850.g50bbe7f3.el9.noar 111/146 2024-09-13T16:19:52.945 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-850.g50bbe7f3.el9.no 112/146 2024-09-13T16:19:52.999 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-850.g50bbe7f3.el9.no 112/146 2024-09-13T16:19:53.035 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-850.g50bb 113/146 2024-09-13T16:19:53.144 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-850.g50bb 113/146 2024-09-13T16:19:53.307 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-850.g50bbe7f3.el9 114/146 2024-09-13T16:19:53.337 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-2:18.2.4-850.g50bbe7f3.el9.x86_64 115/146 2024-09-13T16:19:53.363 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-2:18.2.4-850.g50bbe7f3.el9.x86_64 115/146 2024-09-13T16:19:53.363 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T16:19:53.363 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-13T16:19:53.363 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T16:19:53.363 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T16:19:53.363 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:19:53.413 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-rook-2:18.2.4-850.g50bbe7f3.el9.noarch 116/146 2024-09-13T16:19:53.425 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-850.g50bbe7f3.el9.noarch 116/146 2024-09-13T16:19:53.527 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-2:18.2.4-850.g50bbe7f3.el9.x86_64 117/146 2024-09-13T16:19:53.584 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-fuse-2:18.2.4-850.g50bbe7f3.el9.x86_64 118/146 2024-09-13T16:19:53.910 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-850.g50bbe7f3.el9.no 112/146 2024-09-13T16:19:53.945 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-850.g50bb 113/146 2024-09-13T16:19:54.046 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-850.g50bb 113/146 2024-09-13T16:19:54.202 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-850.g50bbe7f3.el9 114/146 2024-09-13T16:19:54.231 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-2:18.2.4-850.g50bbe7f3.el9.x86_64 115/146 2024-09-13T16:19:54.254 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-2:18.2.4-850.g50bbe7f3.el9.x86_64 115/146 2024-09-13T16:19:54.254 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T16:19:54.254 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-13T16:19:54.254 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T16:19:54.254 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T16:19:54.255 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T16:19:54.298 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-rook-2:18.2.4-850.g50bbe7f3.el9.noarch 116/146 2024-09-13T16:19:54.309 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-850.g50bbe7f3.el9.noarch 116/146 2024-09-13T16:19:54.343 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libtool-2.4.6-46.el9.x86_64 119/146 2024-09-13T16:19:54.382 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-radosgw-2:18.2.4-850.g50bbe7f3.el9.x86_64 120/146 2024-09-13T16:19:54.408 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-850.g50bbe7f3.el9.x86_64 120/146 2024-09-13T16:19:54.408 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T16:19:54.408 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-13T16:19:54.408 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T16:19:54.408 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T16:19:54.408 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:19:54.409 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-2:18.2.4-850.g50bbe7f3.el9.x86_64 117/146 2024-09-13T16:19:54.449 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-850.g50bbe7 121/146 2024-09-13T16:19:54.457 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-fuse-2:18.2.4-850.g50bbe7f3.el9.x86_64 118/146 2024-09-13T16:19:54.473 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-850.g50bbe7 121/146 2024-09-13T16:19:54.473 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T16:19:54.473 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-13T16:19:54.473 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:19:54.525 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-mirror-2:18.2.4-850.g50bbe7f3.el9.x86_64 122/146 2024-09-13T16:19:54.550 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephfs-mirror-2:18.2.4-850.g50bbe7f3.el9.x86_64 122/146 2024-09-13T16:19:54.550 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T16:19:54.550 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-13T16:19:54.550 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-13T16:19:54.550 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-13T16:19:54.550 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:19:54.868 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-mirror-2:18.2.4-850.g50bbe7f3.el9.x86_64 123/146 2024-09-13T16:19:54.892 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-mirror-2:18.2.4-850.g50bbe7f3.el9.x86_64 123/146 2024-09-13T16:19:54.893 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T16:19:54.893 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-13T16:19:54.893 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T16:19:54.893 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T16:19:54.893 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:19:55.158 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-test-2:18.2.4-850.g50bbe7f3.el9.x86_64 124/146 2024-09-13T16:19:55.175 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libtool-2.4.6-46.el9.x86_64 119/146 2024-09-13T16:19:55.200 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-radosgw-2:18.2.4-850.g50bbe7f3.el9.x86_64 120/146 2024-09-13T16:19:55.207 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-fuse-2:18.2.4-850.g50bbe7f3.el9.x86_64 125/146 2024-09-13T16:19:55.223 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-850.g50bbe7f3.el9.x86_64 120/146 2024-09-13T16:19:55.224 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T16:19:55.224 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-13T16:19:55.224 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T16:19:55.224 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T16:19:55.224 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T16:19:55.247 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-nbd-2:18.2.4-850.g50bbe7f3.el9.x86_64 126/146 2024-09-13T16:19:55.268 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-850.g50bbe7 121/146 2024-09-13T16:19:55.280 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 127/146 2024-09-13T16:19:55.290 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-850.g50bbe7 121/146 2024-09-13T16:19:55.290 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T16:19:55.290 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-13T16:19:55.290 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T16:19:55.335 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-mirror-2:18.2.4-850.g50bbe7f3.el9.x86_64 122/146 2024-09-13T16:19:55.358 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: cephfs-mirror-2:18.2.4-850.g50bbe7f3.el9.x86_64 122/146 2024-09-13T16:19:55.358 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T16:19:55.358 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-13T16:19:55.358 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-13T16:19:55.358 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-13T16:19:55.358 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T16:19:55.460 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 128/146 2024-09-13T16:19:55.504 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 129/146 2024-09-13T16:19:55.535 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs-devel-2:18.2.4-850.g50bbe7f3.el9.x86_6 130/146 2024-09-13T16:19:55.564 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 131/146 2024-09-13T16:19:55.610 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 132/146 2024-09-13T16:19:55.661 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-mirror-2:18.2.4-850.g50bbe7f3.el9.x86_64 123/146 2024-09-13T16:19:55.684 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: rbd-mirror-2:18.2.4-850.g50bbe7f3.el9.x86_64 123/146 2024-09-13T16:19:55.684 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T16:19:55.685 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-13T16:19:55.685 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T16:19:55.685 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T16:19:55.685 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T16:19:56.174 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T16:19:56.208 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T16:19:56.229 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T16:19:57.043 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-top-2:18.2.4-850.g50bbe7f3.el9.noarch 134/146 2024-09-13T16:19:57.131 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 135/146 2024-09-13T16:19:57.180 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 136/146 2024-09-13T16:19:57.261 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flex-2.6.4-9.el9.x86_64 137/146 2024-09-13T16:19:57.301 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : bison-3.7.4-5.el9.x86_64 138/146 2024-09-13T16:19:57.327 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 139/146 2024-09-13T16:19:57.352 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 140/146 2024-09-13T16:19:57.353 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 141/146 2024-09-13T16:19:57.437 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 141/146 2024-09-13T16:19:57.437 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 142/146 2024-09-13T16:19:57.521 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 142/146 2024-09-13T16:19:57.546 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 143/146 2024-09-13T16:19:57.552 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T16:19:57.552 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T16:19:57.731 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T16:19:57.731 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 145/146 2024-09-13T16:19:57.816 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 145/146 2024-09-13T16:19:57.880 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T16:19:58.772 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-test-2:18.2.4-850.g50bbe7f3.el9.x86_64 124/146 2024-09-13T16:19:58.813 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-fuse-2:18.2.4-850.g50bbe7f3.el9.x86_64 125/146 2024-09-13T16:19:58.854 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-nbd-2:18.2.4-850.g50bbe7f3.el9.x86_64 126/146 2024-09-13T16:19:58.894 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 127/146 2024-09-13T16:19:59.084 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 128/146 2024-09-13T16:19:59.126 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 129/146 2024-09-13T16:19:59.156 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs-devel-2:18.2.4-850.g50bbe7f3.el9.x86_6 130/146 2024-09-13T16:19:59.186 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 131/146 2024-09-13T16:19:59.223 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 132/146 2024-09-13T16:19:59.357 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-test-2:18.2.4-850.g50bbe7f3.el9.x86_64 124/146 2024-09-13T16:19:59.397 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-fuse-2:18.2.4-850.g50bbe7f3.el9.x86_64 125/146 2024-09-13T16:19:59.467 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-nbd-2:18.2.4-850.g50bbe7f3.el9.x86_64 126/146 2024-09-13T16:19:59.503 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 127/146 2024-09-13T16:19:59.687 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 128/146 2024-09-13T16:19:59.735 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 129/146 2024-09-13T16:19:59.765 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephfs-devel-2:18.2.4-850.g50bbe7f3.el9.x86_6 130/146 2024-09-13T16:19:59.795 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 131/146 2024-09-13T16:19:59.841 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 132/146 2024-09-13T16:19:59.882 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T16:19:59.913 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T16:19:59.935 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T16:20:00.433 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T16:20:00.472 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T16:20:00.493 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T16:20:00.832 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-top-2:18.2.4-850.g50bbe7f3.el9.noarch 134/146 2024-09-13T16:20:00.919 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 135/146 2024-09-13T16:20:00.968 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 136/146 2024-09-13T16:20:01.056 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flex-2.6.4-9.el9.x86_64 137/146 2024-09-13T16:20:01.089 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : bison-3.7.4-5.el9.x86_64 138/146 2024-09-13T16:20:01.115 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 139/146 2024-09-13T16:20:01.141 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 140/146 2024-09-13T16:20:01.141 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 141/146 2024-09-13T16:20:01.224 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 141/146 2024-09-13T16:20:01.225 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 142/146 2024-09-13T16:20:01.309 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 142/146 2024-09-13T16:20:01.321 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-top-2:18.2.4-850.g50bbe7f3.el9.noarch 134/146 2024-09-13T16:20:01.334 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 143/146 2024-09-13T16:20:01.340 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T16:20:01.340 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T16:20:01.419 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 135/146 2024-09-13T16:20:01.471 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 136/146 2024-09-13T16:20:01.527 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T16:20:01.528 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 145/146 2024-09-13T16:20:01.567 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flex-2.6.4-9.el9.x86_64 137/146 2024-09-13T16:20:01.597 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : bison-3.7.4-5.el9.x86_64 138/146 2024-09-13T16:20:01.612 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 145/146 2024-09-13T16:20:01.622 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 139/146 2024-09-13T16:20:01.656 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 140/146 2024-09-13T16:20:01.656 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 141/146 2024-09-13T16:20:01.679 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T16:20:01.749 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 141/146 2024-09-13T16:20:01.749 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 142/146 2024-09-13T16:20:01.833 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 142/146 2024-09-13T16:20:01.858 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 143/146 2024-09-13T16:20:01.863 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T16:20:01.863 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T16:20:02.060 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T16:20:02.060 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 145/146 2024-09-13T16:20:02.128 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 145/146 2024-09-13T16:20:02.191 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T16:20:02.981 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T16:20:02.981 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-2:18.2.4-850.g50bbe7f3.el9.x86_64 1/146 2024-09-13T16:20:02.981 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-base-2:18.2.4-850.g50bbe7f3.el9.x86_64 2/146 2024-09-13T16:20:02.981 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-common-2:18.2.4-850.g50bbe7f3.el9.x86_64 3/146 2024-09-13T16:20:02.981 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-fuse-2:18.2.4-850.g50bbe7f3.el9.x86_64 4/146 2024-09-13T16:20:02.981 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-850.g50bbe7 5/146 2024-09-13T16:20:02.981 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mds-2:18.2.4-850.g50bbe7f3.el9.x86_64 6/146 2024-09-13T16:20:02.981 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-2:18.2.4-850.g50bbe7f3.el9.x86_64 7/146 2024-09-13T16:20:02.981 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mon-2:18.2.4-850.g50bbe7f3.el9.x86_64 8/146 2024-09-13T16:20:02.981 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-osd-2:18.2.4-850.g50bbe7f3.el9.x86_64 9/146 2024-09-13T16:20:02.981 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-radosgw-2:18.2.4-850.g50bbe7f3.el9.x86_64 10/146 2024-09-13T16:20:02.981 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-selinux-2:18.2.4-850.g50bbe7f3.el9.x86_64 11/146 2024-09-13T16:20:02.982 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-test-2:18.2.4-850.g50bbe7f3.el9.x86_64 12/146 2024-09-13T16:20:02.982 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-mirror-2:18.2.4-850.g50bbe7f3.el9.x86_64 13/146 2024-09-13T16:20:02.982 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs-devel-2:18.2.4-850.g50bbe7f3.el9.x86_6 14/146 2024-09-13T16:20:02.982 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs2-2:18.2.4-850.g50bbe7f3.el9.x86_64 15/146 2024-09-13T16:20:02.982 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephsqlite-2:18.2.4-850.g50bbe7f3.el9.x86_64 16/146 2024-09-13T16:20:02.982 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados-devel-2:18.2.4-850.g50bbe7f3.el9.x86_64 17/146 2024-09-13T16:20:02.982 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libradosstriper1-2:18.2.4-850.g50bbe7f3.el9.x86_ 18/146 2024-09-13T16:20:02.982 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librgw2-2:18.2.4-850.g50bbe7f3.el9.x86_64 19/146 2024-09-13T16:20:02.982 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-argparse-2:18.2.4-850.g50bbe7f3.el9 20/146 2024-09-13T16:20:02.982 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-common-2:18.2.4-850.g50bbe7f3.el9.x 21/146 2024-09-13T16:20:02.982 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cephfs-2:18.2.4-850.g50bbe7f3.el9.x86_64 22/146 2024-09-13T16:20:02.982 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rados-2:18.2.4-850.g50bbe7f3.el9.x86_64 23/146 2024-09-13T16:20:02.982 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rbd-2:18.2.4-850.g50bbe7f3.el9.x86_64 24/146 2024-09-13T16:20:02.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rgw-2:18.2.4-850.g50bbe7f3.el9.x86_64 25/146 2024-09-13T16:20:02.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-fuse-2:18.2.4-850.g50bbe7f3.el9.x86_64 26/146 2024-09-13T16:20:02.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-mirror-2:18.2.4-850.g50bbe7f3.el9.x86_64 27/146 2024-09-13T16:20:02.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-nbd-2:18.2.4-850.g50bbe7f3.el9.x86_64 28/146 2024-09-13T16:20:02.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-850.g50bbe7f3.e 29/146 2024-09-13T16:20:02.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-850.g50bbe7f3.el9.noar 30/146 2024-09-13T16:20:02.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-850.g50bbe7f3.el9.no 31/146 2024-09-13T16:20:02.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-850.g50bb 32/146 2024-09-13T16:20:02.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-850.g50bbe7f3.el9 33/146 2024-09-13T16:20:02.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-rook-2:18.2.4-850.g50bbe7f3.el9.noarch 34/146 2024-09-13T16:20:02.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-850.g50bbe7f3.el 35/146 2024-09-13T16:20:02.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-volume-2:18.2.4-850.g50bbe7f3.el9.noarch 36/146 2024-09-13T16:20:02.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephadm-2:18.2.4-850.g50bbe7f3.el9.noarch 37/146 2024-09-13T16:20:02.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-top-2:18.2.4-850.g50bbe7f3.el9.noarch 38/146 2024-09-13T16:20:02.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 39/146 2024-09-13T16:20:02.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 40/146 2024-09-13T16:20:02.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 41/146 2024-09-13T16:20:02.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 42/146 2024-09-13T16:20:02.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 43/146 2024-09-13T16:20:02.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ply-3.11-14.el9.noarch 44/146 2024-09-13T16:20:02.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 45/146 2024-09-13T16:20:02.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 46/146 2024-09-13T16:20:02.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 47/146 2024-09-13T16:20:02.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : automake-1.16.2-8.el9.noarch 48/146 2024-09-13T16:20:02.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : bison-3.7.4-5.el9.x86_64 49/146 2024-09-13T16:20:02.985 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 50/146 2024-09-13T16:20:02.985 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 51/146 2024-09-13T16:20:02.985 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flex-2.6.4-9.el9.x86_64 52/146 2024-09-13T16:20:02.985 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 53/146 2024-09-13T16:20:02.985 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 54/146 2024-09-13T16:20:02.985 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 55/146 2024-09-13T16:20:02.985 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 56/146 2024-09-13T16:20:02.985 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 57/146 2024-09-13T16:20:02.985 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 58/146 2024-09-13T16:20:02.985 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 59/146 2024-09-13T16:20:02.986 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 60/146 2024-09-13T16:20:02.986 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 61/146 2024-09-13T16:20:02.986 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 62/146 2024-09-13T16:20:02.986 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 63/146 2024-09-13T16:20:02.986 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 64/146 2024-09-13T16:20:02.986 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 65/146 2024-09-13T16:20:02.986 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 66/146 2024-09-13T16:20:02.986 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-13.16-1.el9.x86_64 67/146 2024-09-13T16:20:02.986 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 68/146 2024-09-13T16:20:02.986 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 69/146 2024-09-13T16:20:02.986 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 70/146 2024-09-13T16:20:02.986 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 71/146 2024-09-13T16:20:02.987 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 72/146 2024-09-13T16:20:02.987 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 73/146 2024-09-13T16:20:02.987 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 74/146 2024-09-13T16:20:02.987 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 75/146 2024-09-13T16:20:02.987 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 76/146 2024-09-13T16:20:02.987 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 77/146 2024-09-13T16:20:02.987 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 78/146 2024-09-13T16:20:02.987 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 79/146 2024-09-13T16:20:02.987 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 80/146 2024-09-13T16:20:02.987 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 81/146 2024-09-13T16:20:02.987 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 82/146 2024-09-13T16:20:02.987 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 83/146 2024-09-13T16:20:02.988 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 84/146 2024-09-13T16:20:02.988 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 85/146 2024-09-13T16:20:02.988 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/146 2024-09-13T16:20:02.988 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 87/146 2024-09-13T16:20:02.988 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 88/146 2024-09-13T16:20:02.988 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 89/146 2024-09-13T16:20:02.988 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 90/146 2024-09-13T16:20:02.988 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 91/146 2024-09-13T16:20:02.988 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 92/146 2024-09-13T16:20:02.988 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 93/146 2024-09-13T16:20:02.988 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 94/146 2024-09-13T16:20:02.988 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 95/146 2024-09-13T16:20:02.989 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 96/146 2024-09-13T16:20:02.989 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 97/146 2024-09-13T16:20:02.989 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/146 2024-09-13T16:20:02.989 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/146 2024-09-13T16:20:02.989 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 100/146 2024-09-13T16:20:02.989 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 101/146 2024-09-13T16:20:02.989 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/146 2024-09-13T16:20:02.989 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/146 2024-09-13T16:20:02.989 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 104/146 2024-09-13T16:20:02.989 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/146 2024-09-13T16:20:02.989 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/146 2024-09-13T16:20:02.989 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 107/146 2024-09-13T16:20:02.989 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/146 2024-09-13T16:20:02.990 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/146 2024-09-13T16:20:02.990 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/146 2024-09-13T16:20:02.990 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/146 2024-09-13T16:20:02.990 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/146 2024-09-13T16:20:02.990 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 113/146 2024-09-13T16:20:02.990 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 114/146 2024-09-13T16:20:02.990 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 115/146 2024-09-13T16:20:02.990 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 116/146 2024-09-13T16:20:02.990 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 117/146 2024-09-13T16:20:02.990 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 118/146 2024-09-13T16:20:02.990 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/146 2024-09-13T16:20:02.990 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 120/146 2024-09-13T16:20:02.991 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 121/146 2024-09-13T16:20:02.991 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 122/146 2024-09-13T16:20:02.991 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 123/146 2024-09-13T16:20:02.991 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 124/146 2024-09-13T16:20:02.991 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 125/146 2024-09-13T16:20:02.991 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 126/146 2024-09-13T16:20:02.991 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 127/146 2024-09-13T16:20:02.991 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 128/146 2024-09-13T16:20:02.991 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : re2-1:20211101-20.el9.x86_64 129/146 2024-09-13T16:20:02.991 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 130/146 2024-09-13T16:20:02.991 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:18.2.4-850.g50bbe7f3.el9.x86_64 131/146 2024-09-13T16:20:02.991 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 132/146 2024-09-13T16:20:02.992 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:18.2.4-850.g50bbe7f3.el9.x86_64 133/146 2024-09-13T16:20:02.992 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 134/146 2024-09-13T16:20:02.992 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 135/146 2024-09-13T16:20:02.992 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 136/146 2024-09-13T16:20:02.992 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 137/146 2024-09-13T16:20:02.992 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 138/146 2024-09-13T16:20:02.992 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 139/146 2024-09-13T16:20:02.992 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 140/146 2024-09-13T16:20:02.992 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 141/146 2024-09-13T16:20:02.992 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 142/146 2024-09-13T16:20:02.992 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 143/146 2024-09-13T16:20:02.992 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 144/146 2024-09-13T16:20:02.992 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 145/146 2024-09-13T16:20:05.575 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T16:20:05.575 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-13T16:20:05.575 INFO:teuthology.orchestra.run.smithi107.stdout:Upgraded: 2024-09-13T16:20:05.575 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-13T16:20:05.575 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-13T16:20:05.575 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-13T16:20:05.576 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-13T16:20:05.576 INFO:teuthology.orchestra.run.smithi107.stdout: librados2-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:05.576 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:05.576 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-13T16:20:05.576 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-13T16:20:05.576 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-13T16:20:05.576 INFO:teuthology.orchestra.run.smithi107.stdout: automake-1.16.2-8.el9.noarch 2024-09-13T16:20:05.576 INFO:teuthology.orchestra.run.smithi107.stdout: bison-3.7.4-5.el9.x86_64 2024-09-13T16:20:05.576 INFO:teuthology.orchestra.run.smithi107.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-13T16:20:05.576 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:05.576 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:05.576 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:05.577 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:05.577 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards-2:18.2.4-850.g50bbe7f3.el9.noarch 2024-09-13T16:20:05.577 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:05.577 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:05.577 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:05.577 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm-2:18.2.4-850.g50bbe7f3.el9.noarch 2024-09-13T16:20:05.577 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard-2:18.2.4-850.g50bbe7f3.el9.noarch 2024-09-13T16:20:05.577 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local-2:18.2.4-850.g50bbe7f3.el9.noarch 2024-09-13T16:20:05.577 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core-2:18.2.4-850.g50bbe7f3.el9.noarch 2024-09-13T16:20:05.577 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook-2:18.2.4-850.g50bbe7f3.el9.noarch 2024-09-13T16:20:05.577 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:05.577 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:05.578 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts-2:18.2.4-850.g50bbe7f3.el9.noarch 2024-09-13T16:20:05.578 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:05.578 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:05.578 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:05.578 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume-2:18.2.4-850.g50bbe7f3.el9.noarch 2024-09-13T16:20:05.578 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm-2:18.2.4-850.g50bbe7f3.el9.noarch 2024-09-13T16:20:05.578 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-mirror-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:05.578 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-top-2:18.2.4-850.g50bbe7f3.el9.noarch 2024-09-13T16:20:05.578 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-13T16:20:05.578 INFO:teuthology.orchestra.run.smithi107.stdout: flex-2.6.4-9.el9.x86_64 2024-09-13T16:20:05.578 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-13T16:20:05.578 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-13T16:20:05.579 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-13T16:20:05.579 INFO:teuthology.orchestra.run.smithi107.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-13T16:20:05.579 INFO:teuthology.orchestra.run.smithi107.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-13T16:20:05.579 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-13T16:20:05.579 INFO:teuthology.orchestra.run.smithi107.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-13T16:20:05.579 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-13T16:20:05.579 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-13T16:20:05.579 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:05.579 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:05.579 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:05.579 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-13T16:20:05.579 INFO:teuthology.orchestra.run.smithi107.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-13T16:20:05.579 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-13T16:20:05.580 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-13T16:20:05.580 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-13T16:20:05.580 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:05.580 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:05.580 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-13T16:20:05.580 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:05.580 INFO:teuthology.orchestra.run.smithi107.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-13T16:20:05.580 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-13T16:20:05.580 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-13T16:20:05.580 INFO:teuthology.orchestra.run.smithi107.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-13T16:20:05.580 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-13T16:20:05.580 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-13T16:20:05.580 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-13T16:20:05.581 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-13T16:20:05.581 INFO:teuthology.orchestra.run.smithi107.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-13T16:20:05.581 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-13.16-1.el9.x86_64 2024-09-13T16:20:05.581 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-13T16:20:05.581 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-13T16:20:05.581 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-13T16:20:05.581 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-13T16:20:05.581 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-13T16:20:05.581 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-13T16:20:05.581 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-13T16:20:05.581 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:05.581 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:05.581 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:05.581 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-13T16:20:05.581 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-13T16:20:05.582 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-13T16:20:05.582 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-13T16:20:05.582 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-13T16:20:05.582 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-13T16:20:05.582 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-13T16:20:05.582 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-13T16:20:05.582 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-13T16:20:05.582 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-13T16:20:05.582 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-13T16:20:05.582 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-13T16:20:05.582 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-13T16:20:05.582 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-13T16:20:05.582 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-13T16:20:05.583 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-13T16:20:05.583 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-13T16:20:05.583 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-13T16:20:05.583 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-13T16:20:05.583 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-13T16:20:05.583 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-13T16:20:05.583 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-13T16:20:05.583 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-13T16:20:05.583 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-13T16:20:05.583 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-13T16:20:05.583 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-13T16:20:05.583 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply-3.11-14.el9.noarch 2024-09-13T16:20:05.583 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-13T16:20:05.583 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-13T16:20:05.584 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-13T16:20:05.584 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-13T16:20:05.584 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-13T16:20:05.584 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-13T16:20:05.584 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-13T16:20:05.584 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:05.584 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:05.584 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-13T16:20:05.584 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-13T16:20:05.584 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-13T16:20:05.584 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:05.584 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-13T16:20:05.584 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-13T16:20:05.584 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-13T16:20:05.585 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-13T16:20:05.585 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-13T16:20:05.585 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-13T16:20:05.585 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-13T16:20:05.585 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-13T16:20:05.585 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-13T16:20:05.585 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-13T16:20:05.585 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-13T16:20:05.585 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:05.585 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:05.585 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:05.585 INFO:teuthology.orchestra.run.smithi107.stdout: re2-1:20211101-20.el9.x86_64 2024-09-13T16:20:05.585 INFO:teuthology.orchestra.run.smithi107.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-13T16:20:05.585 INFO:teuthology.orchestra.run.smithi107.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-13T16:20:05.586 INFO:teuthology.orchestra.run.smithi107.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-13T16:20:05.586 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-13T16:20:05.586 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-13T16:20:05.586 INFO:teuthology.orchestra.run.smithi107.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-13T16:20:05.586 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-13T16:20:05.586 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-13T16:20:05.957 DEBUG:teuthology.parallel:result is None 2024-09-13T16:20:06.751 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T16:20:06.752 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-2:18.2.4-850.g50bbe7f3.el9.x86_64 1/146 2024-09-13T16:20:06.752 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-base-2:18.2.4-850.g50bbe7f3.el9.x86_64 2/146 2024-09-13T16:20:06.752 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-common-2:18.2.4-850.g50bbe7f3.el9.x86_64 3/146 2024-09-13T16:20:06.752 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-fuse-2:18.2.4-850.g50bbe7f3.el9.x86_64 4/146 2024-09-13T16:20:06.752 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-850.g50bbe7 5/146 2024-09-13T16:20:06.752 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mds-2:18.2.4-850.g50bbe7f3.el9.x86_64 6/146 2024-09-13T16:20:06.752 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-2:18.2.4-850.g50bbe7f3.el9.x86_64 7/146 2024-09-13T16:20:06.752 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mon-2:18.2.4-850.g50bbe7f3.el9.x86_64 8/146 2024-09-13T16:20:06.752 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-osd-2:18.2.4-850.g50bbe7f3.el9.x86_64 9/146 2024-09-13T16:20:06.752 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-radosgw-2:18.2.4-850.g50bbe7f3.el9.x86_64 10/146 2024-09-13T16:20:06.752 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-selinux-2:18.2.4-850.g50bbe7f3.el9.x86_64 11/146 2024-09-13T16:20:06.752 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-test-2:18.2.4-850.g50bbe7f3.el9.x86_64 12/146 2024-09-13T16:20:06.753 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-mirror-2:18.2.4-850.g50bbe7f3.el9.x86_64 13/146 2024-09-13T16:20:06.753 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs-devel-2:18.2.4-850.g50bbe7f3.el9.x86_6 14/146 2024-09-13T16:20:06.753 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs2-2:18.2.4-850.g50bbe7f3.el9.x86_64 15/146 2024-09-13T16:20:06.753 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephsqlite-2:18.2.4-850.g50bbe7f3.el9.x86_64 16/146 2024-09-13T16:20:06.753 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados-devel-2:18.2.4-850.g50bbe7f3.el9.x86_64 17/146 2024-09-13T16:20:06.753 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libradosstriper1-2:18.2.4-850.g50bbe7f3.el9.x86_ 18/146 2024-09-13T16:20:06.753 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librgw2-2:18.2.4-850.g50bbe7f3.el9.x86_64 19/146 2024-09-13T16:20:06.753 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-argparse-2:18.2.4-850.g50bbe7f3.el9 20/146 2024-09-13T16:20:06.753 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-common-2:18.2.4-850.g50bbe7f3.el9.x 21/146 2024-09-13T16:20:06.753 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cephfs-2:18.2.4-850.g50bbe7f3.el9.x86_64 22/146 2024-09-13T16:20:06.753 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rados-2:18.2.4-850.g50bbe7f3.el9.x86_64 23/146 2024-09-13T16:20:06.753 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rbd-2:18.2.4-850.g50bbe7f3.el9.x86_64 24/146 2024-09-13T16:20:06.753 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rgw-2:18.2.4-850.g50bbe7f3.el9.x86_64 25/146 2024-09-13T16:20:06.754 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-fuse-2:18.2.4-850.g50bbe7f3.el9.x86_64 26/146 2024-09-13T16:20:06.754 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-mirror-2:18.2.4-850.g50bbe7f3.el9.x86_64 27/146 2024-09-13T16:20:06.754 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-nbd-2:18.2.4-850.g50bbe7f3.el9.x86_64 28/146 2024-09-13T16:20:06.754 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-850.g50bbe7f3.e 29/146 2024-09-13T16:20:06.764 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-850.g50bbe7f3.el9.noar 30/146 2024-09-13T16:20:06.764 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-850.g50bbe7f3.el9.no 31/146 2024-09-13T16:20:06.764 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-850.g50bb 32/146 2024-09-13T16:20:06.764 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-850.g50bbe7f3.el9 33/146 2024-09-13T16:20:06.765 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-rook-2:18.2.4-850.g50bbe7f3.el9.noarch 34/146 2024-09-13T16:20:06.765 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-850.g50bbe7f3.el 35/146 2024-09-13T16:20:06.765 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-volume-2:18.2.4-850.g50bbe7f3.el9.noarch 36/146 2024-09-13T16:20:06.765 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephadm-2:18.2.4-850.g50bbe7f3.el9.noarch 37/146 2024-09-13T16:20:06.765 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-top-2:18.2.4-850.g50bbe7f3.el9.noarch 38/146 2024-09-13T16:20:06.765 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 39/146 2024-09-13T16:20:06.765 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 40/146 2024-09-13T16:20:06.765 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 41/146 2024-09-13T16:20:06.765 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 42/146 2024-09-13T16:20:06.765 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 43/146 2024-09-13T16:20:06.765 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ply-3.11-14.el9.noarch 44/146 2024-09-13T16:20:06.765 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 45/146 2024-09-13T16:20:06.766 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 46/146 2024-09-13T16:20:06.766 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 47/146 2024-09-13T16:20:06.766 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : automake-1.16.2-8.el9.noarch 48/146 2024-09-13T16:20:06.766 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : bison-3.7.4-5.el9.x86_64 49/146 2024-09-13T16:20:06.766 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 50/146 2024-09-13T16:20:06.766 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 51/146 2024-09-13T16:20:06.766 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flex-2.6.4-9.el9.x86_64 52/146 2024-09-13T16:20:06.766 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 53/146 2024-09-13T16:20:06.766 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 54/146 2024-09-13T16:20:06.766 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 55/146 2024-09-13T16:20:06.766 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 56/146 2024-09-13T16:20:06.766 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 57/146 2024-09-13T16:20:06.766 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 58/146 2024-09-13T16:20:06.767 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 59/146 2024-09-13T16:20:06.767 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 60/146 2024-09-13T16:20:06.767 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 61/146 2024-09-13T16:20:06.767 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 62/146 2024-09-13T16:20:06.767 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 63/146 2024-09-13T16:20:06.767 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 64/146 2024-09-13T16:20:06.767 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 65/146 2024-09-13T16:20:06.767 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 66/146 2024-09-13T16:20:06.767 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-13.16-1.el9.x86_64 67/146 2024-09-13T16:20:06.767 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 68/146 2024-09-13T16:20:06.767 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 69/146 2024-09-13T16:20:06.767 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 70/146 2024-09-13T16:20:06.767 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 71/146 2024-09-13T16:20:06.768 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 72/146 2024-09-13T16:20:06.768 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 73/146 2024-09-13T16:20:06.768 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 74/146 2024-09-13T16:20:06.768 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 75/146 2024-09-13T16:20:06.768 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 76/146 2024-09-13T16:20:06.768 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 77/146 2024-09-13T16:20:06.768 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 78/146 2024-09-13T16:20:06.768 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 79/146 2024-09-13T16:20:06.768 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 80/146 2024-09-13T16:20:06.768 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 81/146 2024-09-13T16:20:06.768 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 82/146 2024-09-13T16:20:06.768 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 83/146 2024-09-13T16:20:06.768 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 84/146 2024-09-13T16:20:06.769 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 85/146 2024-09-13T16:20:06.769 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/146 2024-09-13T16:20:06.769 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 87/146 2024-09-13T16:20:06.769 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 88/146 2024-09-13T16:20:06.769 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 89/146 2024-09-13T16:20:06.769 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 90/146 2024-09-13T16:20:06.769 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 91/146 2024-09-13T16:20:06.769 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 92/146 2024-09-13T16:20:06.769 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 93/146 2024-09-13T16:20:06.769 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 94/146 2024-09-13T16:20:06.769 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 95/146 2024-09-13T16:20:06.769 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 96/146 2024-09-13T16:20:06.770 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 97/146 2024-09-13T16:20:06.770 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/146 2024-09-13T16:20:06.770 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/146 2024-09-13T16:20:06.770 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 100/146 2024-09-13T16:20:06.770 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 101/146 2024-09-13T16:20:06.770 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/146 2024-09-13T16:20:06.770 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/146 2024-09-13T16:20:06.770 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 104/146 2024-09-13T16:20:06.770 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/146 2024-09-13T16:20:06.770 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/146 2024-09-13T16:20:06.770 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 107/146 2024-09-13T16:20:06.770 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/146 2024-09-13T16:20:06.770 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/146 2024-09-13T16:20:06.770 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/146 2024-09-13T16:20:06.771 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/146 2024-09-13T16:20:06.771 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/146 2024-09-13T16:20:06.771 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 113/146 2024-09-13T16:20:06.771 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 114/146 2024-09-13T16:20:06.771 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 115/146 2024-09-13T16:20:06.771 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 116/146 2024-09-13T16:20:06.771 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 117/146 2024-09-13T16:20:06.771 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 118/146 2024-09-13T16:20:06.771 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/146 2024-09-13T16:20:06.771 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 120/146 2024-09-13T16:20:06.771 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 121/146 2024-09-13T16:20:06.771 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 122/146 2024-09-13T16:20:06.772 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 123/146 2024-09-13T16:20:06.772 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 124/146 2024-09-13T16:20:06.772 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 125/146 2024-09-13T16:20:06.772 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 126/146 2024-09-13T16:20:06.772 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 127/146 2024-09-13T16:20:06.772 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 128/146 2024-09-13T16:20:06.772 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : re2-1:20211101-20.el9.x86_64 129/146 2024-09-13T16:20:06.772 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 130/146 2024-09-13T16:20:06.772 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:18.2.4-850.g50bbe7f3.el9.x86_64 131/146 2024-09-13T16:20:06.772 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 132/146 2024-09-13T16:20:06.772 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:18.2.4-850.g50bbe7f3.el9.x86_64 133/146 2024-09-13T16:20:06.772 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 134/146 2024-09-13T16:20:06.772 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 135/146 2024-09-13T16:20:06.772 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 136/146 2024-09-13T16:20:06.772 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 137/146 2024-09-13T16:20:06.773 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 138/146 2024-09-13T16:20:06.773 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 139/146 2024-09-13T16:20:06.773 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 140/146 2024-09-13T16:20:06.773 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 141/146 2024-09-13T16:20:06.773 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 142/146 2024-09-13T16:20:06.773 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 143/146 2024-09-13T16:20:06.773 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 144/146 2024-09-13T16:20:06.773 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 145/146 2024-09-13T16:20:07.495 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T16:20:07.495 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-2:18.2.4-850.g50bbe7f3.el9.x86_64 1/146 2024-09-13T16:20:07.495 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-base-2:18.2.4-850.g50bbe7f3.el9.x86_64 2/146 2024-09-13T16:20:07.495 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-common-2:18.2.4-850.g50bbe7f3.el9.x86_64 3/146 2024-09-13T16:20:07.495 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-fuse-2:18.2.4-850.g50bbe7f3.el9.x86_64 4/146 2024-09-13T16:20:07.495 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-850.g50bbe7 5/146 2024-09-13T16:20:07.496 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mds-2:18.2.4-850.g50bbe7f3.el9.x86_64 6/146 2024-09-13T16:20:07.496 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-2:18.2.4-850.g50bbe7f3.el9.x86_64 7/146 2024-09-13T16:20:07.496 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mon-2:18.2.4-850.g50bbe7f3.el9.x86_64 8/146 2024-09-13T16:20:07.496 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-osd-2:18.2.4-850.g50bbe7f3.el9.x86_64 9/146 2024-09-13T16:20:07.496 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-radosgw-2:18.2.4-850.g50bbe7f3.el9.x86_64 10/146 2024-09-13T16:20:07.496 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-selinux-2:18.2.4-850.g50bbe7f3.el9.x86_64 11/146 2024-09-13T16:20:07.496 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-test-2:18.2.4-850.g50bbe7f3.el9.x86_64 12/146 2024-09-13T16:20:07.496 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-mirror-2:18.2.4-850.g50bbe7f3.el9.x86_64 13/146 2024-09-13T16:20:07.496 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephfs-devel-2:18.2.4-850.g50bbe7f3.el9.x86_6 14/146 2024-09-13T16:20:07.496 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephfs2-2:18.2.4-850.g50bbe7f3.el9.x86_64 15/146 2024-09-13T16:20:07.496 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephsqlite-2:18.2.4-850.g50bbe7f3.el9.x86_64 16/146 2024-09-13T16:20:07.496 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados-devel-2:18.2.4-850.g50bbe7f3.el9.x86_64 17/146 2024-09-13T16:20:07.496 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libradosstriper1-2:18.2.4-850.g50bbe7f3.el9.x86_ 18/146 2024-09-13T16:20:07.497 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librgw2-2:18.2.4-850.g50bbe7f3.el9.x86_64 19/146 2024-09-13T16:20:07.497 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ceph-argparse-2:18.2.4-850.g50bbe7f3.el9 20/146 2024-09-13T16:20:07.497 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ceph-common-2:18.2.4-850.g50bbe7f3.el9.x 21/146 2024-09-13T16:20:07.497 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cephfs-2:18.2.4-850.g50bbe7f3.el9.x86_64 22/146 2024-09-13T16:20:07.497 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rados-2:18.2.4-850.g50bbe7f3.el9.x86_64 23/146 2024-09-13T16:20:07.497 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rbd-2:18.2.4-850.g50bbe7f3.el9.x86_64 24/146 2024-09-13T16:20:07.497 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rgw-2:18.2.4-850.g50bbe7f3.el9.x86_64 25/146 2024-09-13T16:20:07.497 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-fuse-2:18.2.4-850.g50bbe7f3.el9.x86_64 26/146 2024-09-13T16:20:07.497 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-mirror-2:18.2.4-850.g50bbe7f3.el9.x86_64 27/146 2024-09-13T16:20:07.497 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-nbd-2:18.2.4-850.g50bbe7f3.el9.x86_64 28/146 2024-09-13T16:20:07.497 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-850.g50bbe7f3.e 29/146 2024-09-13T16:20:07.497 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-850.g50bbe7f3.el9.noar 30/146 2024-09-13T16:20:07.498 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-850.g50bbe7f3.el9.no 31/146 2024-09-13T16:20:07.498 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-850.g50bb 32/146 2024-09-13T16:20:07.498 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-850.g50bbe7f3.el9 33/146 2024-09-13T16:20:07.498 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-rook-2:18.2.4-850.g50bbe7f3.el9.noarch 34/146 2024-09-13T16:20:07.498 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-850.g50bbe7f3.el 35/146 2024-09-13T16:20:07.498 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-volume-2:18.2.4-850.g50bbe7f3.el9.noarch 36/146 2024-09-13T16:20:07.498 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephadm-2:18.2.4-850.g50bbe7f3.el9.noarch 37/146 2024-09-13T16:20:07.498 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-top-2:18.2.4-850.g50bbe7f3.el9.noarch 38/146 2024-09-13T16:20:07.498 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 39/146 2024-09-13T16:20:07.498 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 40/146 2024-09-13T16:20:07.498 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 41/146 2024-09-13T16:20:07.498 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 42/146 2024-09-13T16:20:07.498 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 43/146 2024-09-13T16:20:07.499 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ply-3.11-14.el9.noarch 44/146 2024-09-13T16:20:07.499 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 45/146 2024-09-13T16:20:07.499 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 46/146 2024-09-13T16:20:07.499 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 47/146 2024-09-13T16:20:07.499 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : automake-1.16.2-8.el9.noarch 48/146 2024-09-13T16:20:07.499 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : bison-3.7.4-5.el9.x86_64 49/146 2024-09-13T16:20:07.499 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 50/146 2024-09-13T16:20:07.499 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 51/146 2024-09-13T16:20:07.499 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flex-2.6.4-9.el9.x86_64 52/146 2024-09-13T16:20:07.499 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 53/146 2024-09-13T16:20:07.499 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 54/146 2024-09-13T16:20:07.499 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 55/146 2024-09-13T16:20:07.499 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 56/146 2024-09-13T16:20:07.500 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 57/146 2024-09-13T16:20:07.500 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 58/146 2024-09-13T16:20:07.500 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 59/146 2024-09-13T16:20:07.500 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 60/146 2024-09-13T16:20:07.500 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 61/146 2024-09-13T16:20:07.500 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 62/146 2024-09-13T16:20:07.500 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 63/146 2024-09-13T16:20:07.500 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 64/146 2024-09-13T16:20:07.500 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 65/146 2024-09-13T16:20:07.500 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 66/146 2024-09-13T16:20:07.500 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-13.16-1.el9.x86_64 67/146 2024-09-13T16:20:07.500 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 68/146 2024-09-13T16:20:07.500 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 69/146 2024-09-13T16:20:07.501 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 70/146 2024-09-13T16:20:07.501 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 71/146 2024-09-13T16:20:07.501 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 72/146 2024-09-13T16:20:07.501 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 73/146 2024-09-13T16:20:07.501 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 74/146 2024-09-13T16:20:07.501 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 75/146 2024-09-13T16:20:07.501 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 76/146 2024-09-13T16:20:07.501 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 77/146 2024-09-13T16:20:07.501 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 78/146 2024-09-13T16:20:07.501 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 79/146 2024-09-13T16:20:07.501 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 80/146 2024-09-13T16:20:07.501 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 81/146 2024-09-13T16:20:07.501 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 82/146 2024-09-13T16:20:07.501 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 83/146 2024-09-13T16:20:07.502 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 84/146 2024-09-13T16:20:07.502 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 85/146 2024-09-13T16:20:07.502 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/146 2024-09-13T16:20:07.502 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 87/146 2024-09-13T16:20:07.502 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 88/146 2024-09-13T16:20:07.502 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 89/146 2024-09-13T16:20:07.502 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 90/146 2024-09-13T16:20:07.502 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 91/146 2024-09-13T16:20:07.502 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 92/146 2024-09-13T16:20:07.502 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 93/146 2024-09-13T16:20:07.502 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 94/146 2024-09-13T16:20:07.502 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 95/146 2024-09-13T16:20:07.502 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 96/146 2024-09-13T16:20:07.502 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 97/146 2024-09-13T16:20:07.503 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/146 2024-09-13T16:20:07.503 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/146 2024-09-13T16:20:07.503 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 100/146 2024-09-13T16:20:07.503 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 101/146 2024-09-13T16:20:07.503 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/146 2024-09-13T16:20:07.503 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/146 2024-09-13T16:20:07.503 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 104/146 2024-09-13T16:20:07.503 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/146 2024-09-13T16:20:07.503 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/146 2024-09-13T16:20:07.503 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 107/146 2024-09-13T16:20:07.503 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/146 2024-09-13T16:20:07.503 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/146 2024-09-13T16:20:07.504 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/146 2024-09-13T16:20:07.504 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/146 2024-09-13T16:20:07.504 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/146 2024-09-13T16:20:07.504 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 113/146 2024-09-13T16:20:07.504 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 114/146 2024-09-13T16:20:07.504 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 115/146 2024-09-13T16:20:07.504 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 116/146 2024-09-13T16:20:07.504 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 117/146 2024-09-13T16:20:07.504 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 118/146 2024-09-13T16:20:07.504 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/146 2024-09-13T16:20:07.504 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 120/146 2024-09-13T16:20:07.504 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 121/146 2024-09-13T16:20:07.504 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 122/146 2024-09-13T16:20:07.504 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 123/146 2024-09-13T16:20:07.505 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 124/146 2024-09-13T16:20:07.505 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 125/146 2024-09-13T16:20:07.505 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 126/146 2024-09-13T16:20:07.505 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 127/146 2024-09-13T16:20:07.505 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 128/146 2024-09-13T16:20:07.505 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : re2-1:20211101-20.el9.x86_64 129/146 2024-09-13T16:20:07.505 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 130/146 2024-09-13T16:20:07.505 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados2-2:18.2.4-850.g50bbe7f3.el9.x86_64 131/146 2024-09-13T16:20:07.505 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 132/146 2024-09-13T16:20:07.505 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librbd1-2:18.2.4-850.g50bbe7f3.el9.x86_64 133/146 2024-09-13T16:20:07.505 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 134/146 2024-09-13T16:20:07.505 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 135/146 2024-09-13T16:20:07.505 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 136/146 2024-09-13T16:20:07.505 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 137/146 2024-09-13T16:20:07.505 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 138/146 2024-09-13T16:20:07.506 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 139/146 2024-09-13T16:20:07.506 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 140/146 2024-09-13T16:20:07.506 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 141/146 2024-09-13T16:20:07.506 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 142/146 2024-09-13T16:20:07.506 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 143/146 2024-09-13T16:20:07.506 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 144/146 2024-09-13T16:20:07.506 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 145/146 2024-09-13T16:20:09.277 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T16:20:09.277 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:20:09.278 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2024-09-13T16:20:09.278 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-13T16:20:09.278 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-13T16:20:09.278 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-13T16:20:09.278 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-13T16:20:09.278 INFO:teuthology.orchestra.run.smithi016.stdout: librados2-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.278 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.278 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-13T16:20:09.278 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-13T16:20:09.278 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-09-13T16:20:09.278 INFO:teuthology.orchestra.run.smithi016.stdout: automake-1.16.2-8.el9.noarch 2024-09-13T16:20:09.279 INFO:teuthology.orchestra.run.smithi016.stdout: bison-3.7.4-5.el9.x86_64 2024-09-13T16:20:09.279 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-13T16:20:09.279 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.279 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.279 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.279 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.279 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards-2:18.2.4-850.g50bbe7f3.el9.noarch 2024-09-13T16:20:09.279 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.279 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.280 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.280 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm-2:18.2.4-850.g50bbe7f3.el9.noarch 2024-09-13T16:20:09.280 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard-2:18.2.4-850.g50bbe7f3.el9.noarch 2024-09-13T16:20:09.280 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local-2:18.2.4-850.g50bbe7f3.el9.noarch 2024-09-13T16:20:09.280 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core-2:18.2.4-850.g50bbe7f3.el9.noarch 2024-09-13T16:20:09.280 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook-2:18.2.4-850.g50bbe7f3.el9.noarch 2024-09-13T16:20:09.280 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.280 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.280 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts-2:18.2.4-850.g50bbe7f3.el9.noarch 2024-09-13T16:20:09.280 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.280 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.281 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.281 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume-2:18.2.4-850.g50bbe7f3.el9.noarch 2024-09-13T16:20:09.281 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm-2:18.2.4-850.g50bbe7f3.el9.noarch 2024-09-13T16:20:09.281 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.281 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top-2:18.2.4-850.g50bbe7f3.el9.noarch 2024-09-13T16:20:09.281 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-13T16:20:09.281 INFO:teuthology.orchestra.run.smithi016.stdout: flex-2.6.4-9.el9.x86_64 2024-09-13T16:20:09.281 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-13T16:20:09.281 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-13T16:20:09.281 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-13T16:20:09.281 INFO:teuthology.orchestra.run.smithi016.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-13T16:20:09.282 INFO:teuthology.orchestra.run.smithi016.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-13T16:20:09.282 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-13T16:20:09.282 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-13T16:20:09.282 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-13T16:20:09.282 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-13T16:20:09.282 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.282 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.282 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.283 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-13T16:20:09.283 INFO:teuthology.orchestra.run.smithi016.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-13T16:20:09.283 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-13T16:20:09.283 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-13T16:20:09.283 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-13T16:20:09.283 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.283 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.283 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-13T16:20:09.283 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.283 INFO:teuthology.orchestra.run.smithi016.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-13T16:20:09.283 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-13T16:20:09.283 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-13T16:20:09.284 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-13T16:20:09.284 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-13T16:20:09.284 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-13T16:20:09.284 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-13T16:20:09.284 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-13T16:20:09.284 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-13T16:20:09.284 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-13.16-1.el9.x86_64 2024-09-13T16:20:09.284 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-13T16:20:09.284 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-13T16:20:09.284 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-13T16:20:09.285 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-13T16:20:09.285 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-13T16:20:09.285 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-13T16:20:09.285 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-13T16:20:09.285 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.285 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.285 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.285 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-13T16:20:09.285 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-13T16:20:09.285 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-13T16:20:09.285 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-13T16:20:09.285 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-13T16:20:09.286 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-13T16:20:09.286 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-13T16:20:09.286 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-13T16:20:09.286 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-13T16:20:09.286 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-13T16:20:09.286 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-13T16:20:09.286 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-13T16:20:09.286 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-13T16:20:09.286 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-13T16:20:09.286 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-13T16:20:09.286 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-13T16:20:09.286 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-13T16:20:09.287 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-13T16:20:09.287 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-13T16:20:09.287 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-13T16:20:09.287 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-13T16:20:09.287 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-13T16:20:09.287 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-13T16:20:09.287 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-13T16:20:09.287 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-13T16:20:09.287 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-13T16:20:09.287 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply-3.11-14.el9.noarch 2024-09-13T16:20:09.287 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-13T16:20:09.287 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-13T16:20:09.288 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-13T16:20:09.288 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-13T16:20:09.288 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-13T16:20:09.288 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-13T16:20:09.288 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-13T16:20:09.288 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.288 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.288 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-13T16:20:09.288 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-13T16:20:09.288 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-13T16:20:09.288 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.288 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-13T16:20:09.288 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-13T16:20:09.289 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-13T16:20:09.289 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-13T16:20:09.289 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-13T16:20:09.289 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-13T16:20:09.289 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-13T16:20:09.289 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-13T16:20:09.289 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-13T16:20:09.289 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-13T16:20:09.289 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-13T16:20:09.289 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.289 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.289 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.290 INFO:teuthology.orchestra.run.smithi016.stdout: re2-1:20211101-20.el9.x86_64 2024-09-13T16:20:09.290 INFO:teuthology.orchestra.run.smithi016.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-13T16:20:09.290 INFO:teuthology.orchestra.run.smithi016.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-13T16:20:09.290 INFO:teuthology.orchestra.run.smithi016.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-13T16:20:09.290 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-13T16:20:09.290 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-13T16:20:09.290 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-13T16:20:09.291 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:20:09.291 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-09-13T16:20:09.655 DEBUG:teuthology.parallel:result is None 2024-09-13T16:20:09.818 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T16:20:09.819 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T16:20:09.819 INFO:teuthology.orchestra.run.smithi111.stdout:Upgraded: 2024-09-13T16:20:09.819 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-13T16:20:09.819 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-13T16:20:09.819 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-13T16:20:09.819 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-13T16:20:09.819 INFO:teuthology.orchestra.run.smithi111.stdout: librados2-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.819 INFO:teuthology.orchestra.run.smithi111.stdout: librbd1-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.819 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-13T16:20:09.819 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-13T16:20:09.819 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2024-09-13T16:20:09.819 INFO:teuthology.orchestra.run.smithi111.stdout: automake-1.16.2-8.el9.noarch 2024-09-13T16:20:09.820 INFO:teuthology.orchestra.run.smithi111.stdout: bison-3.7.4-5.el9.x86_64 2024-09-13T16:20:09.820 INFO:teuthology.orchestra.run.smithi111.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-13T16:20:09.820 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.820 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-base-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.820 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-common-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.820 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-fuse-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.820 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-grafana-dashboards-2:18.2.4-850.g50bbe7f3.el9.noarch 2024-09-13T16:20:09.820 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-immutable-object-cache-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.820 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mds-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.820 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.820 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-cephadm-2:18.2.4-850.g50bbe7f3.el9.noarch 2024-09-13T16:20:09.820 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-dashboard-2:18.2.4-850.g50bbe7f3.el9.noarch 2024-09-13T16:20:09.821 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-diskprediction-local-2:18.2.4-850.g50bbe7f3.el9.noarch 2024-09-13T16:20:09.821 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-modules-core-2:18.2.4-850.g50bbe7f3.el9.noarch 2024-09-13T16:20:09.821 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-rook-2:18.2.4-850.g50bbe7f3.el9.noarch 2024-09-13T16:20:09.821 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mon-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.821 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-osd-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.821 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-prometheus-alerts-2:18.2.4-850.g50bbe7f3.el9.noarch 2024-09-13T16:20:09.821 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-radosgw-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.821 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-selinux-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.821 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-test-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.821 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-volume-2:18.2.4-850.g50bbe7f3.el9.noarch 2024-09-13T16:20:09.821 INFO:teuthology.orchestra.run.smithi111.stdout: cephadm-2:18.2.4-850.g50bbe7f3.el9.noarch 2024-09-13T16:20:09.821 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-mirror-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.822 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-top-2:18.2.4-850.g50bbe7f3.el9.noarch 2024-09-13T16:20:09.822 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-13T16:20:09.822 INFO:teuthology.orchestra.run.smithi111.stdout: flex-2.6.4-9.el9.x86_64 2024-09-13T16:20:09.822 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-13T16:20:09.822 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-13T16:20:09.822 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-13T16:20:09.822 INFO:teuthology.orchestra.run.smithi111.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-13T16:20:09.822 INFO:teuthology.orchestra.run.smithi111.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-13T16:20:09.822 INFO:teuthology.orchestra.run.smithi111.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-13T16:20:09.822 INFO:teuthology.orchestra.run.smithi111.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-13T16:20:09.822 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-13T16:20:09.823 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-13T16:20:09.823 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs-devel-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.823 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs2-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.823 INFO:teuthology.orchestra.run.smithi111.stdout: libcephsqlite-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.823 INFO:teuthology.orchestra.run.smithi111.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-13T16:20:09.823 INFO:teuthology.orchestra.run.smithi111.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-13T16:20:09.823 INFO:teuthology.orchestra.run.smithi111.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-13T16:20:09.823 INFO:teuthology.orchestra.run.smithi111.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-13T16:20:09.823 INFO:teuthology.orchestra.run.smithi111.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-13T16:20:09.823 INFO:teuthology.orchestra.run.smithi111.stdout: librados-devel-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.823 INFO:teuthology.orchestra.run.smithi111.stdout: libradosstriper1-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.823 INFO:teuthology.orchestra.run.smithi111.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-13T16:20:09.824 INFO:teuthology.orchestra.run.smithi111.stdout: librgw2-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.824 INFO:teuthology.orchestra.run.smithi111.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-13T16:20:09.824 INFO:teuthology.orchestra.run.smithi111.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-13T16:20:09.824 INFO:teuthology.orchestra.run.smithi111.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-13T16:20:09.824 INFO:teuthology.orchestra.run.smithi111.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-13T16:20:09.824 INFO:teuthology.orchestra.run.smithi111.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-13T16:20:09.824 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-13T16:20:09.824 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-13T16:20:09.824 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-13T16:20:09.824 INFO:teuthology.orchestra.run.smithi111.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-13T16:20:09.824 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-13.16-1.el9.x86_64 2024-09-13T16:20:09.824 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-13T16:20:09.825 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-13T16:20:09.825 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-13T16:20:09.825 INFO:teuthology.orchestra.run.smithi111.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-13T16:20:09.825 INFO:teuthology.orchestra.run.smithi111.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-13T16:20:09.825 INFO:teuthology.orchestra.run.smithi111.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-13T16:20:09.825 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-13T16:20:09.825 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-argparse-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.825 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-common-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.825 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cephfs-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.825 INFO:teuthology.orchestra.run.smithi111.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-13T16:20:09.825 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-13T16:20:09.825 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-13T16:20:09.826 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-13T16:20:09.826 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-13T16:20:09.826 INFO:teuthology.orchestra.run.smithi111.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-13T16:20:09.826 INFO:teuthology.orchestra.run.smithi111.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-13T16:20:09.826 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-13T16:20:09.826 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-13T16:20:09.826 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-13T16:20:09.826 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-13T16:20:09.826 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-13T16:20:09.826 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-13T16:20:09.826 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-13T16:20:09.826 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-13T16:20:09.826 INFO:teuthology.orchestra.run.smithi111.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-13T16:20:09.827 INFO:teuthology.orchestra.run.smithi111.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-13T16:20:09.827 INFO:teuthology.orchestra.run.smithi111.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-13T16:20:09.827 INFO:teuthology.orchestra.run.smithi111.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-13T16:20:09.827 INFO:teuthology.orchestra.run.smithi111.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-13T16:20:09.827 INFO:teuthology.orchestra.run.smithi111.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-13T16:20:09.827 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-13T16:20:09.827 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-13T16:20:09.827 INFO:teuthology.orchestra.run.smithi111.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-13T16:20:09.827 INFO:teuthology.orchestra.run.smithi111.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-13T16:20:09.827 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-13T16:20:09.827 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ply-3.11-14.el9.noarch 2024-09-13T16:20:09.827 INFO:teuthology.orchestra.run.smithi111.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-13T16:20:09.827 INFO:teuthology.orchestra.run.smithi111.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-13T16:20:09.828 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-13T16:20:09.828 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-13T16:20:09.828 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-13T16:20:09.828 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-13T16:20:09.828 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-13T16:20:09.828 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rados-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.828 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rbd-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.828 INFO:teuthology.orchestra.run.smithi111.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-13T16:20:09.828 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-13T16:20:09.828 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-13T16:20:09.828 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rgw-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.828 INFO:teuthology.orchestra.run.smithi111.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-13T16:20:09.828 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-13T16:20:09.829 INFO:teuthology.orchestra.run.smithi111.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-13T16:20:09.829 INFO:teuthology.orchestra.run.smithi111.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-13T16:20:09.829 INFO:teuthology.orchestra.run.smithi111.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-13T16:20:09.829 INFO:teuthology.orchestra.run.smithi111.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-13T16:20:09.829 INFO:teuthology.orchestra.run.smithi111.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-13T16:20:09.829 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-13T16:20:09.829 INFO:teuthology.orchestra.run.smithi111.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-13T16:20:09.829 INFO:teuthology.orchestra.run.smithi111.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-13T16:20:09.829 INFO:teuthology.orchestra.run.smithi111.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-13T16:20:09.829 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-fuse-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.829 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-mirror-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.829 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-nbd-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T16:20:09.830 INFO:teuthology.orchestra.run.smithi111.stdout: re2-1:20211101-20.el9.x86_64 2024-09-13T16:20:09.830 INFO:teuthology.orchestra.run.smithi111.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-13T16:20:09.830 INFO:teuthology.orchestra.run.smithi111.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-13T16:20:09.830 INFO:teuthology.orchestra.run.smithi111.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-13T16:20:09.830 INFO:teuthology.orchestra.run.smithi111.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-13T16:20:09.830 INFO:teuthology.orchestra.run.smithi111.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-13T16:20:09.830 INFO:teuthology.orchestra.run.smithi111.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-13T16:20:09.830 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T16:20:09.830 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-13T16:20:10.225 DEBUG:teuthology.parallel:result is None 2024-09-13T16:20:10.225 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T16:20:10.381 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-13T16:20:10.408 INFO:teuthology.orchestra.run.smithi016.stdout:18.2.4-850.g50bbe7f3.el9 2024-09-13T16:20:10.408 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-850.g50bbe7f3.el9 2024-09-13T16:20:10.409 INFO:teuthology.task.install:The correct ceph version 18.2.4-850.g50bbe7f3 is installed. 2024-09-13T16:20:10.410 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T16:20:10.561 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-13T16:20:10.587 INFO:teuthology.orchestra.run.smithi107.stdout:18.2.4-850.g50bbe7f3.el9 2024-09-13T16:20:10.588 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-850.g50bbe7f3.el9 2024-09-13T16:20:10.588 INFO:teuthology.task.install:The correct ceph version 18.2.4-850.g50bbe7f3 is installed. 2024-09-13T16:20:10.590 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T16:20:10.730 DEBUG:teuthology.orchestra.run.smithi111:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-13T16:20:10.756 INFO:teuthology.orchestra.run.smithi111.stdout:18.2.4-850.g50bbe7f3.el9 2024-09-13T16:20:10.757 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-850.g50bbe7f3.el9 2024-09-13T16:20:10.757 INFO:teuthology.task.install:The correct ceph version 18.2.4-850.g50bbe7f3 is installed. 2024-09-13T16:20:10.758 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-13T16:20:10.759 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-13T16:20:10.759 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T16:20:10.792 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-13T16:20:10.792 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T16:20:10.824 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T16:20:10.824 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T16:20:10.856 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-13T16:20:10.856 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-13T16:20:10.856 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T16:20:10.889 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T16:20:10.961 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-13T16:20:10.961 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T16:20:10.991 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T16:20:11.061 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T16:20:11.061 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T16:20:11.093 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T16:20:11.162 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-13T16:20:11.163 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-13T16:20:11.163 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T16:20:11.193 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T16:20:11.263 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-13T16:20:11.264 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T16:20:11.293 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T16:20:11.362 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T16:20:11.362 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T16:20:11.392 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T16:20:11.461 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-13T16:20:11.461 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-13T16:20:11.461 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T16:20:11.492 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T16:20:11.561 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-13T16:20:11.561 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T16:20:11.590 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T16:20:11.659 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T16:20:11.659 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T16:20:11.689 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T16:20:11.757 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-13T16:20:11.845 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '50bbe7f395997dcd871813949c49addbbf630361', 'subvols': {'create': 2}} 2024-09-13T16:20:11.846 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T16:20:11.846 INFO:tasks.cephadm:Cluster fsid is 0d241f6a-71ec-11ef-bceb-c7b262605968 2024-09-13T16:20:11.846 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-13T16:20:11.847 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.16', 'mon.b': '172.21.15.107', 'mon.c': '172.21.15.111'} 2024-09-13T16:20:11.847 INFO:tasks.cephadm:First mon is mon.a on smithi016 2024-09-13T16:20:11.847 INFO:tasks.cephadm:First mgr is x 2024-09-13T16:20:11.847 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-13T16:20:11.847 DEBUG:teuthology.orchestra.run.smithi016:> sudo hostname $(hostname -s) 2024-09-13T16:20:11.878 DEBUG:teuthology.orchestra.run.smithi107:> sudo hostname $(hostname -s) 2024-09-13T16:20:11.909 DEBUG:teuthology.orchestra.run.smithi111:> sudo hostname $(hostname -s) 2024-09-13T16:20:11.937 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-13T16:20:11.937 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T16:20:12.084 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '50bbe7f395997dcd871813949c49addbbf630361', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82946/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.4-850-g50bbe7f3', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.4-850.g50bbe7f3'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-10 12:31:04.094992', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240910.095512-reef', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-13T16:20:12.229 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-testing-20240910.095512-reef, sha1 50bbe7f395997dcd871813949c49addbbf630361 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T16:20:12.231 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/centos/9/x86_64/flavors/default/cephadm 2024-09-13T16:20:12.231 DEBUG:teuthology.orchestra.run.smithi016:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-13T16:20:12.576 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r--. 1 ubuntu ubuntu 217224 Sep 13 16:20 /home/ubuntu/cephtest/cephadm 2024-09-13T16:20:12.577 DEBUG:teuthology.orchestra.run.smithi107:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-13T16:20:12.916 INFO:teuthology.orchestra.run.smithi107.stdout:-rw-r--r--. 1 ubuntu ubuntu 217224 Sep 13 16:20 /home/ubuntu/cephtest/cephadm 2024-09-13T16:20:12.916 DEBUG:teuthology.orchestra.run.smithi111:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-13T16:20:13.255 INFO:teuthology.orchestra.run.smithi111.stdout:-rw-r--r--. 1 ubuntu ubuntu 217224 Sep 13 16:20 /home/ubuntu/cephtest/cephadm 2024-09-13T16:20:13.256 DEBUG:teuthology.orchestra.run.smithi016:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-13T16:20:13.276 DEBUG:teuthology.orchestra.run.smithi107:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-13T16:20:13.296 DEBUG:teuthology.orchestra.run.smithi111:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-13T16:20:13.332 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 on all hosts... 2024-09-13T16:20:13.332 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 pull 2024-09-13T16:20:13.338 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 pull 2024-09-13T16:20:13.341 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 pull 2024-09-13T16:20:14.904 INFO:teuthology.orchestra.run.smithi016.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361... 2024-09-13T16:20:14.915 INFO:teuthology.orchestra.run.smithi107.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361... 2024-09-13T16:20:15.009 INFO:teuthology.orchestra.run.smithi111.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361... 2024-09-13T16:20:45.065 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-13T16:20:45.065 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_version": "ceph version 18.2.4-850-g50bbe7f3 (50bbe7f395997dcd871813949c49addbbf630361) reef (stable)", 2024-09-13T16:20:45.065 INFO:teuthology.orchestra.run.smithi111.stdout: "image_id": "152d8bc6b19e354515e8ba109d4619445de0e09b4d4a747c2f16eb5c0edb670c", 2024-09-13T16:20:45.065 INFO:teuthology.orchestra.run.smithi111.stdout: "repo_digests": [ 2024-09-13T16:20:45.065 INFO:teuthology.orchestra.run.smithi111.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a" 2024-09-13T16:20:45.065 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-13T16:20:45.065 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-13T16:21:01.734 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-13T16:21:01.734 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 18.2.4-850-g50bbe7f3 (50bbe7f395997dcd871813949c49addbbf630361) reef (stable)", 2024-09-13T16:21:01.734 INFO:teuthology.orchestra.run.smithi016.stdout: "image_id": "152d8bc6b19e354515e8ba109d4619445de0e09b4d4a747c2f16eb5c0edb670c", 2024-09-13T16:21:01.734 INFO:teuthology.orchestra.run.smithi016.stdout: "repo_digests": [ 2024-09-13T16:21:01.734 INFO:teuthology.orchestra.run.smithi016.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a" 2024-09-13T16:21:01.734 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-13T16:21:01.735 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-13T16:21:02.283 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-13T16:21:02.283 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 18.2.4-850-g50bbe7f3 (50bbe7f395997dcd871813949c49addbbf630361) reef (stable)", 2024-09-13T16:21:02.283 INFO:teuthology.orchestra.run.smithi107.stdout: "image_id": "152d8bc6b19e354515e8ba109d4619445de0e09b4d4a747c2f16eb5c0edb670c", 2024-09-13T16:21:02.283 INFO:teuthology.orchestra.run.smithi107.stdout: "repo_digests": [ 2024-09-13T16:21:02.283 INFO:teuthology.orchestra.run.smithi107.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a" 2024-09-13T16:21:02.283 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-13T16:21:02.283 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-13T16:21:02.303 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph 2024-09-13T16:21:02.338 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /etc/ceph 2024-09-13T16:21:02.367 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /etc/ceph 2024-09-13T16:21:02.399 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 777 /etc/ceph 2024-09-13T16:21:02.429 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 777 /etc/ceph 2024-09-13T16:21:02.455 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 777 /etc/ceph 2024-09-13T16:21:02.485 INFO:tasks.cephadm:Writing seed config... 2024-09-13T16:21:02.486 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-13T16:21:02.486 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-13T16:21:02.486 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-13T16:21:02.486 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-13T16:21:02.486 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-13T16:21:02.487 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-13T16:21:02.487 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-13T16:21:02.487 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-13T16:21:02.487 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-13T16:21:02.487 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-13T16:21:02.487 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-13T16:21:02.487 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-13T16:21:02.487 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-13T16:21:02.487 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-13T16:21:02.487 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-13T16:21:02.487 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-13T16:21:02.487 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-13T16:21:02.487 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-13T16:21:02.487 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-13T16:21:02.487 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-13T16:21:02.487 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-13T16:21:02.488 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-13T16:21:02.488 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-13T16:21:02.488 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-13T16:21:02.488 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-13T16:21:02.488 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-13T16:21:02.488 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-13T16:21:02.488 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-13T16:21:02.488 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-13T16:21:02.488 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-13T16:21:02.488 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-13T16:21:02.488 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-13T16:21:02.488 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-13T16:21:02.488 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-13T16:21:02.488 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-13T16:21:02.489 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-13T16:21:02.489 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-13T16:21:02.489 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-13T16:21:02.489 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-13T16:21:02.489 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-13T16:21:02.489 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-13T16:21:02.489 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-13T16:21:02.489 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-13T16:21:02.489 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-13T16:21:02.489 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-13T16:21:02.490 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-13T16:21:02.490 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-13T16:21:02.509 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 0d241f6a-71ec-11ef-bceb-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-13T16:21:02.509 DEBUG:teuthology.orchestra.run.smithi016:mon.a> sudo journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mon.a.service 2024-09-13T16:21:02.551 DEBUG:teuthology.orchestra.run.smithi016:mgr.x> sudo journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mgr.x.service 2024-09-13T16:21:02.595 INFO:tasks.cephadm:Bootstrapping... 2024-09-13T16:21:02.595 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 -v bootstrap --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.16 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-13T16:21:02.760 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-09-13T16:21:02.760 INFO:teuthology.orchestra.run.smithi016.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361', '-v', 'bootstrap', '--fsid', '0d241f6a-71ec-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.16', '--skip-admin-label'] 2024-09-13T16:21:02.783 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout 5.2.2 2024-09-13T16:21:02.784 INFO:teuthology.orchestra.run.smithi016.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-13T16:21:02.784 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying podman|docker is present... 2024-09-13T16:21:02.807 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout 5.2.2 2024-09-13T16:21:02.808 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying lvm2 is present... 2024-09-13T16:21:02.808 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying time synchronization is in place... 2024-09-13T16:21:02.816 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-13T16:21:02.816 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-13T16:21:02.822 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-13T16:21:02.822 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-13T16:21:02.829 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout enabled 2024-09-13T16:21:02.835 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout active 2024-09-13T16:21:02.835 INFO:teuthology.orchestra.run.smithi016.stdout:Unit chronyd.service is enabled and running 2024-09-13T16:21:02.835 INFO:teuthology.orchestra.run.smithi016.stdout:Repeating the final host check... 2024-09-13T16:21:02.857 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout 5.2.2 2024-09-13T16:21:02.858 INFO:teuthology.orchestra.run.smithi016.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-13T16:21:02.858 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl is present 2024-09-13T16:21:02.858 INFO:teuthology.orchestra.run.smithi016.stdout:lvcreate is present 2024-09-13T16:21:02.864 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-13T16:21:02.864 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-13T16:21:02.871 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-13T16:21:02.871 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-13T16:21:02.877 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout enabled 2024-09-13T16:21:02.885 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout active 2024-09-13T16:21:02.885 INFO:teuthology.orchestra.run.smithi016.stdout:Unit chronyd.service is enabled and running 2024-09-13T16:21:02.885 INFO:teuthology.orchestra.run.smithi016.stdout:Host looks OK 2024-09-13T16:21:02.885 INFO:teuthology.orchestra.run.smithi016.stdout:Cluster fsid: 0d241f6a-71ec-11ef-bceb-c7b262605968 2024-09-13T16:21:02.885 INFO:teuthology.orchestra.run.smithi016.stdout:Acquiring lock 139899912298064 on /run/cephadm/0d241f6a-71ec-11ef-bceb-c7b262605968.lock 2024-09-13T16:21:02.885 INFO:teuthology.orchestra.run.smithi016.stdout:Lock 139899912298064 acquired on /run/cephadm/0d241f6a-71ec-11ef-bceb-c7b262605968.lock 2024-09-13T16:21:02.885 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying IP 172.21.15.16 port 3300 ... 2024-09-13T16:21:02.885 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying IP 172.21.15.16 port 6789 ... 2024-09-13T16:21:02.885 INFO:teuthology.orchestra.run.smithi016.stdout:Base mon IP(s) is [172.21.15.16:3300, 172.21.15.16:6789], mon addrv is [v2:172.21.15.16:3300,v1:172.21.15.16:6789] 2024-09-13T16:21:02.888 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.16 metric 100 2024-09-13T16:21:02.889 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.16 metric 100 2024-09-13T16:21:02.891 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-13T16:21:02.891 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-13T16:21:02.891 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1792sec hoplimit 64 pref medium 2024-09-13T16:21:02.894 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-13T16:21:02.894 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-13T16:21:02.894 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-13T16:21:02.894 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-13T16:21:02.894 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-13T16:21:02.894 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-13T16:21:02.895 INFO:teuthology.orchestra.run.smithi016.stdout:Mon IP `172.21.15.16` is in CIDR network `172.21.0.0/20` 2024-09-13T16:21:02.895 INFO:teuthology.orchestra.run.smithi016.stdout:Mon IP `172.21.15.16` is in CIDR network `172.21.0.0/20` 2024-09-13T16:21:02.895 INFO:teuthology.orchestra.run.smithi016.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-13T16:21:02.895 INFO:teuthology.orchestra.run.smithi016.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-13T16:21:02.896 INFO:teuthology.orchestra.run.smithi016.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361... 2024-09-13T16:21:06.191 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout 152d8bc6b19e354515e8ba109d4619445de0e09b4d4a747c2f16eb5c0edb670c 2024-09-13T16:21:06.192 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361... 2024-09-13T16:21:06.192 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Getting image source signatures 2024-09-13T16:21:06.192 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying blob sha256:3bf8bde7788a2a6694fefa78ba3da98c997a778335fd0748e145012f747287ae 2024-09-13T16:21:06.192 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-13T16:21:06.192 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying config sha256:152d8bc6b19e354515e8ba109d4619445de0e09b4d4a747c2f16eb5c0edb670c 2024-09-13T16:21:06.192 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-13T16:21:08.922 INFO:teuthology.orchestra.run.smithi016.stdout:ceph: stdout ceph version 18.2.4-850-g50bbe7f3 (50bbe7f395997dcd871813949c49addbbf630361) reef (stable) 2024-09-13T16:21:08.922 INFO:teuthology.orchestra.run.smithi016.stdout:Ceph version: ceph version 18.2.4-850-g50bbe7f3 (50bbe7f395997dcd871813949c49addbbf630361) reef (stable) 2024-09-13T16:21:08.922 INFO:teuthology.orchestra.run.smithi016.stdout:Extracting ceph user uid/gid from container image... 2024-09-13T16:21:09.927 INFO:teuthology.orchestra.run.smithi016.stdout:stat: stdout 167 167 2024-09-13T16:21:09.927 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial keys... 2024-09-13T16:21:10.824 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQB2ZuRmeBIUDhAAj3znS3vXJJGL7exDCCqATQ== 2024-09-13T16:21:11.905 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQB3ZuRmGW1qBxAAib3x4DH3a2go7SEfDKBL9Q== 2024-09-13T16:21:12.776 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQB4ZuRmTZI6DhAARt5r3xexxqcUrZJBzfyZlg== 2024-09-13T16:21:12.777 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial monmap... 2024-09-13T16:21:15.593 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-13T16:21:15.593 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-09-13T16:21:15.593 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 0d241f6a-71ec-11ef-bceb-c7b262605968 2024-09-13T16:21:15.593 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-13T16:21:15.593 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool for a [v2:172.21.15.16:3300,v1:172.21.15.16:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-13T16:21:15.594 INFO:teuthology.orchestra.run.smithi016.stdout:setting min_mon_release = pacific 2024-09-13T16:21:15.594 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: set fsid to 0d241f6a-71ec-11ef-bceb-c7b262605968 2024-09-13T16:21:15.594 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-13T16:21:15.594 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:21:15.594 INFO:teuthology.orchestra.run.smithi016.stdout:Creating mon... 2024-09-13T16:21:16.521 INFO:teuthology.orchestra.run.smithi016.stdout:create mon.a on 2024-09-13T16:21:16.744 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-13T16:21:16.881 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-13T16:21:17.085 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-0d241f6a-71ec-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-0d241f6a-71ec-11ef-bceb-c7b262605968.target. 2024-09-13T16:21:17.085 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-0d241f6a-71ec-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-0d241f6a-71ec-11ef-bceb-c7b262605968.target. 2024-09-13T16:21:17.303 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mon.a 2024-09-13T16:21:17.303 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-13T16:21:17.472 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0d241f6a-71ec-11ef-bceb-c7b262605968.target.wants/ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@.service. 2024-09-13T16:21:18.030 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-13T16:21:18.030 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-09-13T16:21:18.038 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-13T16:21:18.038 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-13T16:21:18.038 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-09-13T16:21:18.038 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to enable service . firewalld.service is not available 2024-09-13T16:21:18.039 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mon to start... 2024-09-13T16:21:18.039 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mon... 2024-09-13T16:21:19.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:18 smithi016 ceph-mon[30810]: from='client.? 172.21.15.16:0/607938704' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-13T16:21:19.150 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout cluster: 2024-09-13T16:21:19.150 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout id: 0d241f6a-71ec-11ef-bceb-c7b262605968 2024-09-13T16:21:19.151 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-13T16:21:19.151 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-13T16:21:19.151 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout services: 2024-09-13T16:21:19.151 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.573309s) 2024-09-13T16:21:19.151 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-13T16:21:19.151 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-13T16:21:19.151 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-13T16:21:19.151 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout data: 2024-09-13T16:21:19.151 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-13T16:21:19.151 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-13T16:21:19.151 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-13T16:21:19.151 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout pgs: 2024-09-13T16:21:19.152 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-13T16:21:19.152 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.464+0000 7f59f9047640 1 Processor -- start 2024-09-13T16:21:19.152 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.464+0000 7f59f9047640 1 -- start start 2024-09-13T16:21:19.152 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.464+0000 7f59f9047640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59f4105fa0 0x7f59f4108390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:19.152 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.464+0000 7f59f9047640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f4079e40 con 0x7f59f4105fa0 2024-09-13T16:21:19.152 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.464+0000 7f59f2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59f4105fa0 0x7f59f4108390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:19.152 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.464+0000 7f59f2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59f4105fa0 0x7f59f4108390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55000/0 (socket says 172.21.15.16:55000) 2024-09-13T16:21:19.152 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.464+0000 7f59f2d9d640 1 -- 172.21.15.16:0/2119832264 learned_addr learned my addr 172.21.15.16:0/2119832264 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:19.152 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.465+0000 7f59f2d9d640 1 -- 172.21.15.16:0/2119832264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f41088d0 con 0x7f59f4105fa0 2024-09-13T16:21:19.152 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.468+0000 7f59f2d9d640 1 --2- 172.21.15.16:0/2119832264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59f4105fa0 0x7f59f4108390 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f59e4009820 tx=0x7f59e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2dff524ef58181 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:19.152 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.468+0000 7f59f1d9b640 1 -- 172.21.15.16:0/2119832264 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59e402ae70 con 0x7f59f4105fa0 2024-09-13T16:21:19.153 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.468+0000 7f59f1d9b640 1 -- 172.21.15.16:0/2119832264 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e4032440 con 0x7f59f4105fa0 2024-09-13T16:21:19.153 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.468+0000 7f59f1d9b640 1 -- 172.21.15.16:0/2119832264 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59e4032700 con 0x7f59f4105fa0 2024-09-13T16:21:19.153 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.468+0000 7f59f9047640 1 -- 172.21.15.16:0/2119832264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59f4105fa0 msgr2=0x7f59f4108390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:19.153 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.468+0000 7f59f9047640 1 --2- 172.21.15.16:0/2119832264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59f4105fa0 0x7f59f4108390 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f59e4009820 tx=0x7f59e402a270 comp rx=0 tx=0).stop 2024-09-13T16:21:19.153 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.468+0000 7f59f9047640 1 -- 172.21.15.16:0/2119832264 shutdown_connections 2024-09-13T16:21:19.153 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.468+0000 7f59f9047640 1 --2- 172.21.15.16:0/2119832264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59f4105fa0 0x7f59f4108390 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:19.153 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.468+0000 7f59f9047640 1 -- 172.21.15.16:0/2119832264 >> 172.21.15.16:0/2119832264 conn(0x7f59f40ffbe0 msgr2=0x7f59f4102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:19.153 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.468+0000 7f59f9047640 1 -- 172.21.15.16:0/2119832264 shutdown_connections 2024-09-13T16:21:19.153 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.469+0000 7f59f9047640 1 -- 172.21.15.16:0/2119832264 wait complete. 2024-09-13T16:21:19.153 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.469+0000 7f59f9047640 1 Processor -- start 2024-09-13T16:21:19.153 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.469+0000 7f59f9047640 1 -- start start 2024-09-13T16:21:19.153 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.470+0000 7f59f9047640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59f4105fa0 0x7f59f4105820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:19.154 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.470+0000 7f59f9047640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f4105d60 con 0x7f59f4105fa0 2024-09-13T16:21:19.154 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.470+0000 7f59f2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59f4105fa0 0x7f59f4105820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:19.154 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.470+0000 7f59f2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59f4105fa0 0x7f59f4105820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55016/0 (socket says 172.21.15.16:55016) 2024-09-13T16:21:19.154 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.470+0000 7f59f2d9d640 1 -- 172.21.15.16:0/607938704 learned_addr learned my addr 172.21.15.16:0/607938704 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:19.154 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.470+0000 7f59f2d9d640 1 -- 172.21.15.16:0/607938704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e40094d0 con 0x7f59f4105fa0 2024-09-13T16:21:19.154 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.470+0000 7f59f2d9d640 1 --2- 172.21.15.16:0/607938704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59f4105fa0 0x7f59f4105820 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f59e40090b0 tx=0x7f59e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:19.154 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.471+0000 7f59dbfff640 1 -- 172.21.15.16:0/607938704 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59e40308d0 con 0x7f59f4105fa0 2024-09-13T16:21:19.154 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.471+0000 7f59dbfff640 1 -- 172.21.15.16:0/607938704 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e4030a30 con 0x7f59f4105fa0 2024-09-13T16:21:19.154 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.471+0000 7f59f9047640 1 -- 172.21.15.16:0/607938704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59f4103f50 con 0x7f59f4105fa0 2024-09-13T16:21:19.154 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.471+0000 7f59f9047640 1 -- 172.21.15.16:0/607938704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f41043f0 con 0x7f59f4105fa0 2024-09-13T16:21:19.154 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.471+0000 7f59dbfff640 1 -- 172.21.15.16:0/607938704 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59e4030cf0 con 0x7f59f4105fa0 2024-09-13T16:21:19.154 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.472+0000 7f59dbfff640 1 -- 172.21.15.16:0/607938704 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f59e4039040 con 0x7f59f4105fa0 2024-09-13T16:21:19.154 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.472+0000 7f59dbfff640 1 -- 172.21.15.16:0/607938704 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f59e4046910 con 0x7f59f4105fa0 2024-09-13T16:21:19.155 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.472+0000 7f59f9047640 1 -- 172.21.15.16:0/607938704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59f4106400 con 0x7f59f4105fa0 2024-09-13T16:21:19.155 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.474+0000 7f59dbfff640 1 -- 172.21.15.16:0/607938704 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f59e4037070 con 0x7f59f4105fa0 2024-09-13T16:21:19.155 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.542+0000 7f59f9047640 1 -- 172.21.15.16:0/607938704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f59f4108bf0 con 0x7f59f4105fa0 2024-09-13T16:21:19.155 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.542+0000 7f59dbfff640 1 -- 172.21.15.16:0/607938704 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f59e4053070 con 0x7f59f4105fa0 2024-09-13T16:21:19.155 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.543+0000 7f59f9047640 1 -- 172.21.15.16:0/607938704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59f4105fa0 msgr2=0x7f59f4105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:19.155 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.543+0000 7f59f9047640 1 --2- 172.21.15.16:0/607938704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59f4105fa0 0x7f59f4105820 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f59e40090b0 tx=0x7f59e40046c0 comp rx=0 tx=0).stop 2024-09-13T16:21:19.155 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.543+0000 7f59f9047640 1 -- 172.21.15.16:0/607938704 shutdown_connections 2024-09-13T16:21:19.155 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.543+0000 7f59f9047640 1 --2- 172.21.15.16:0/607938704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59f4105fa0 0x7f59f4105820 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:19.155 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.543+0000 7f59f9047640 1 -- 172.21.15.16:0/607938704 >> 172.21.15.16:0/607938704 conn(0x7f59f40ffbe0 msgr2=0x7f59f4100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:19.155 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.544+0000 7f59f9047640 1 -- 172.21.15.16:0/607938704 shutdown_connections 2024-09-13T16:21:19.155 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:18.544+0000 7f59f9047640 1 -- 172.21.15.16:0/607938704 wait complete. 2024-09-13T16:21:19.155 INFO:teuthology.orchestra.run.smithi016.stdout:mon is available 2024-09-13T16:21:19.155 INFO:teuthology.orchestra.run.smithi016.stdout:Assimilating anything we can from ceph.conf... 2024-09-13T16:21:20.252 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:19 smithi016 ceph-mon[30810]: from='client.? 172.21.15.16:0/4179032979' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-13T16:21:20.252 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:19 smithi016 ceph-mon[30810]: from='client.? 172.21.15.16:0/4179032979' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-13T16:21:20.260 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-13T16:21:20.260 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [global] 2024-09-13T16:21:20.260 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout fsid = 0d241f6a-71ec-11ef-bceb-c7b262605968 2024-09-13T16:21:20.261 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.16:3300,v1:172.21.15.16:6789] 2024-09-13T16:21:20.261 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-13T16:21:20.261 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-13T16:21:20.261 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-13T16:21:20.261 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-13T16:21:20.261 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-13T16:21:20.261 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-13T16:21:20.261 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-13T16:21:20.261 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-13T16:21:20.261 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [osd] 2024-09-13T16:21:20.261 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-13T16:21:20.261 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-13T16:21:20.261 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-13T16:21:20.261 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-13T16:21:20.262 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-13T16:21:20.262 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.611+0000 7f2bce328640 1 Processor -- start 2024-09-13T16:21:20.262 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.611+0000 7f2bce328640 1 -- start start 2024-09-13T16:21:20.262 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.611+0000 7f2bce328640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bc8104430 0x7f2bc8104830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:20.262 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.611+0000 7f2bce328640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc8104e00 con 0x7f2bc8104430 2024-09-13T16:21:20.262 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.611+0000 7f2bc7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bc8104430 0x7f2bc8104830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:20.262 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.611+0000 7f2bc7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bc8104430 0x7f2bc8104830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55028/0 (socket says 172.21.15.16:55028) 2024-09-13T16:21:20.262 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.611+0000 7f2bc7fff640 1 -- 172.21.15.16:0/3307860080 learned_addr learned my addr 172.21.15.16:0/3307860080 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:20.262 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.612+0000 7f2bc7fff640 1 -- 172.21.15.16:0/3307860080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc8105630 con 0x7f2bc8104430 2024-09-13T16:21:20.262 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.612+0000 7f2bc7fff640 1 --2- 172.21.15.16:0/3307860080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bc8104430 0x7f2bc8104830 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8009820 tx=0x7f2bb802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c6818e6bafd23a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:20.262 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.613+0000 7f2bc77fe640 1 -- 172.21.15.16:0/3307860080 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2bb802ae70 con 0x7f2bc8104430 2024-09-13T16:21:20.262 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.613+0000 7f2bc77fe640 1 -- 172.21.15.16:0/3307860080 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb8032440 con 0x7f2bc8104430 2024-09-13T16:21:20.262 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.613+0000 7f2bc77fe640 1 -- 172.21.15.16:0/3307860080 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2bb8032700 con 0x7f2bc8104430 2024-09-13T16:21:20.263 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.613+0000 7f2bce328640 1 -- 172.21.15.16:0/3307860080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bc8104430 msgr2=0x7f2bc8104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:20.263 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.613+0000 7f2bce328640 1 --2- 172.21.15.16:0/3307860080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bc8104430 0x7f2bc8104830 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8009820 tx=0x7f2bb802a270 comp rx=0 tx=0).stop 2024-09-13T16:21:20.263 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.613+0000 7f2bce328640 1 -- 172.21.15.16:0/3307860080 shutdown_connections 2024-09-13T16:21:20.263 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.613+0000 7f2bce328640 1 --2- 172.21.15.16:0/3307860080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bc8104430 0x7f2bc8104830 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:20.263 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.613+0000 7f2bce328640 1 -- 172.21.15.16:0/3307860080 >> 172.21.15.16:0/3307860080 conn(0x7f2bc80ffbe0 msgr2=0x7f2bc8102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:20.263 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.613+0000 7f2bce328640 1 -- 172.21.15.16:0/3307860080 shutdown_connections 2024-09-13T16:21:20.263 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.613+0000 7f2bce328640 1 -- 172.21.15.16:0/3307860080 wait complete. 2024-09-13T16:21:20.263 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.614+0000 7f2bce328640 1 Processor -- start 2024-09-13T16:21:20.263 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.614+0000 7f2bce328640 1 -- start start 2024-09-13T16:21:20.263 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.614+0000 7f2bce328640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bc8104430 0x7f2bc8198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:20.263 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.614+0000 7f2bce328640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc8198f70 con 0x7f2bc8104430 2024-09-13T16:21:20.263 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.614+0000 7f2bc7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bc8104430 0x7f2bc8198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:20.263 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.614+0000 7f2bc7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bc8104430 0x7f2bc8198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55042/0 (socket says 172.21.15.16:55042) 2024-09-13T16:21:20.264 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.614+0000 7f2bc7fff640 1 -- 172.21.15.16:0/4179032979 learned_addr learned my addr 172.21.15.16:0/4179032979 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:20.264 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.615+0000 7f2bc7fff640 1 -- 172.21.15.16:0/4179032979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb80094d0 con 0x7f2bc8104430 2024-09-13T16:21:20.264 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.615+0000 7f2bc7fff640 1 --2- 172.21.15.16:0/4179032979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bc8104430 0x7f2bc8198a30 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80090b0 tx=0x7f2bb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:20.264 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.615+0000 7f2bc5ffb640 1 -- 172.21.15.16:0/4179032979 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2bb80308d0 con 0x7f2bc8104430 2024-09-13T16:21:20.264 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.615+0000 7f2bc5ffb640 1 -- 172.21.15.16:0/4179032979 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb8030a30 con 0x7f2bc8104430 2024-09-13T16:21:20.264 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.615+0000 7f2bce328640 1 -- 172.21.15.16:0/4179032979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc8199170 con 0x7f2bc8104430 2024-09-13T16:21:20.264 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.615+0000 7f2bc5ffb640 1 -- 172.21.15.16:0/4179032979 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2bb8030cf0 con 0x7f2bc8104430 2024-09-13T16:21:20.264 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.615+0000 7f2bce328640 1 -- 172.21.15.16:0/4179032979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc8199610 con 0x7f2bc8104430 2024-09-13T16:21:20.264 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.615+0000 7f2bc5ffb640 1 -- 172.21.15.16:0/4179032979 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f2bb8039040 con 0x7f2bc8104430 2024-09-13T16:21:20.264 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.615+0000 7f2bc5ffb640 1 -- 172.21.15.16:0/4179032979 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2bb80468c0 con 0x7f2bc8104430 2024-09-13T16:21:20.264 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.616+0000 7f2bce328640 1 -- 172.21.15.16:0/4179032979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc8192740 con 0x7f2bc8104430 2024-09-13T16:21:20.264 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.618+0000 7f2bc5ffb640 1 -- 172.21.15.16:0/4179032979 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f2bb8037070 con 0x7f2bc8104430 2024-09-13T16:21:20.264 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.676+0000 7f2bce328640 1 -- 172.21.15.16:0/4179032979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f2bc8105a10 con 0x7f2bc8104430 2024-09-13T16:21:20.265 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.680+0000 7f2bc5ffb640 1 -- 172.21.15.16:0/4179032979 <== mon.0 v2:172.21.15.16:3300/0 7 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f2bb804f8a0 con 0x7f2bc8104430 2024-09-13T16:21:20.265 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.680+0000 7f2bc5ffb640 1 -- 172.21.15.16:0/4179032979 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f2bb8046b20 con 0x7f2bc8104430 2024-09-13T16:21:20.265 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.681+0000 7f2bce328640 1 -- 172.21.15.16:0/4179032979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bc8104430 msgr2=0x7f2bc8198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:20.265 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.681+0000 7f2bce328640 1 --2- 172.21.15.16:0/4179032979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bc8104430 0x7f2bc8198a30 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80090b0 tx=0x7f2bb80046c0 comp rx=0 tx=0).stop 2024-09-13T16:21:20.265 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.681+0000 7f2bce328640 1 -- 172.21.15.16:0/4179032979 shutdown_connections 2024-09-13T16:21:20.265 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.681+0000 7f2bce328640 1 --2- 172.21.15.16:0/4179032979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bc8104430 0x7f2bc8198a30 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:20.265 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.681+0000 7f2bce328640 1 -- 172.21.15.16:0/4179032979 >> 172.21.15.16:0/4179032979 conn(0x7f2bc80ffbe0 msgr2=0x7f2bc818f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:20.265 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.681+0000 7f2bce328640 1 -- 172.21.15.16:0/4179032979 shutdown_connections 2024-09-13T16:21:20.265 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:19.681+0000 7f2bce328640 1 -- 172.21.15.16:0/4179032979 wait complete. 2024-09-13T16:21:20.265 INFO:teuthology.orchestra.run.smithi016.stdout:Generating new minimal ceph.conf... 2024-09-13T16:21:21.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:20 smithi016 ceph-mon[30810]: from='client.? 172.21.15.16:0/76542919' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:21:21.459 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.677+0000 7f0224586640 1 Processor -- start 2024-09-13T16:21:21.460 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.677+0000 7f0224586640 1 -- start start 2024-09-13T16:21:21.460 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.677+0000 7f0224586640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f021c104430 0x7f021c104830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:21.460 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.677+0000 7f0224586640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021c104e00 con 0x7f021c104430 2024-09-13T16:21:21.460 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.677+0000 7f0222322640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f021c104430 0x7f021c104830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:21.460 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.677+0000 7f0222322640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f021c104430 0x7f021c104830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55046/0 (socket says 172.21.15.16:55046) 2024-09-13T16:21:21.460 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.677+0000 7f0222322640 1 -- 172.21.15.16:0/3099009793 learned_addr learned my addr 172.21.15.16:0/3099009793 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:21.460 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.677+0000 7f0222322640 1 -- 172.21.15.16:0/3099009793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f021c105630 con 0x7f021c104430 2024-09-13T16:21:21.460 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.678+0000 7f0222322640 1 --2- 172.21.15.16:0/3099009793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f021c104430 0x7f021c104830 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0218009820 tx=0x7f021802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9687dead0a3b4af5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:21.460 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.678+0000 7f0221320640 1 -- 172.21.15.16:0/3099009793 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f021802ae70 con 0x7f021c104430 2024-09-13T16:21:21.460 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.678+0000 7f0221320640 1 -- 172.21.15.16:0/3099009793 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f02180328b0 con 0x7f021c104430 2024-09-13T16:21:21.460 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.678+0000 7f0221320640 1 -- 172.21.15.16:0/3099009793 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0218030850 con 0x7f021c104430 2024-09-13T16:21:21.461 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.679+0000 7f0224586640 1 -- 172.21.15.16:0/3099009793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f021c104430 msgr2=0x7f021c104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:21.461 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.679+0000 7f0224586640 1 --2- 172.21.15.16:0/3099009793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f021c104430 0x7f021c104830 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0218009820 tx=0x7f021802a270 comp rx=0 tx=0).stop 2024-09-13T16:21:21.461 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.679+0000 7f0224586640 1 -- 172.21.15.16:0/3099009793 shutdown_connections 2024-09-13T16:21:21.461 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.679+0000 7f0224586640 1 --2- 172.21.15.16:0/3099009793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f021c104430 0x7f021c104830 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:21.461 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.679+0000 7f0224586640 1 -- 172.21.15.16:0/3099009793 >> 172.21.15.16:0/3099009793 conn(0x7f021c0ffbe0 msgr2=0x7f021c102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:21.461 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.679+0000 7f0224586640 1 -- 172.21.15.16:0/3099009793 shutdown_connections 2024-09-13T16:21:21.461 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.679+0000 7f0224586640 1 -- 172.21.15.16:0/3099009793 wait complete. 2024-09-13T16:21:21.461 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.679+0000 7f0224586640 1 Processor -- start 2024-09-13T16:21:21.461 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.680+0000 7f0224586640 1 -- start start 2024-09-13T16:21:21.461 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.680+0000 7f0224586640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f021c104430 0x7f021c199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:21.461 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.680+0000 7f0224586640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021c1998c0 con 0x7f021c104430 2024-09-13T16:21:21.461 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.680+0000 7f0222322640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f021c104430 0x7f021c199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:21.461 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.680+0000 7f0222322640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f021c104430 0x7f021c199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55056/0 (socket says 172.21.15.16:55056) 2024-09-13T16:21:21.461 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.680+0000 7f0222322640 1 -- 172.21.15.16:0/76542919 learned_addr learned my addr 172.21.15.16:0/76542919 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:21.462 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.680+0000 7f0222322640 1 -- 172.21.15.16:0/76542919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02180094d0 con 0x7f021c104430 2024-09-13T16:21:21.462 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.680+0000 7f0222322640 1 --2- 172.21.15.16:0/76542919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f021c104430 0x7f021c199380 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f021802a790 tx=0x7f0218030db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:21.462 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.681+0000 7f02137fe640 1 -- 172.21.15.16:0/76542919 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0218032d60 con 0x7f021c104430 2024-09-13T16:21:21.462 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.681+0000 7f02137fe640 1 -- 172.21.15.16:0/76542919 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f021803b8a0 con 0x7f021c104430 2024-09-13T16:21:21.462 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.681+0000 7f0224586640 1 -- 172.21.15.16:0/76542919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f021c1064b0 con 0x7f021c104430 2024-09-13T16:21:21.462 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.681+0000 7f02137fe640 1 -- 172.21.15.16:0/76542919 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f021803a640 con 0x7f021c104430 2024-09-13T16:21:21.462 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.681+0000 7f0224586640 1 -- 172.21.15.16:0/76542919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f021c199dd0 con 0x7f021c104430 2024-09-13T16:21:21.462 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.681+0000 7f02137fe640 1 -- 172.21.15.16:0/76542919 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f0218039040 con 0x7f021c104430 2024-09-13T16:21:21.462 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.681+0000 7f02137fe640 1 -- 172.21.15.16:0/76542919 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0218044b80 con 0x7f021c104430 2024-09-13T16:21:21.462 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.682+0000 7f0224586640 1 -- 172.21.15.16:0/76542919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f021c1048b0 con 0x7f021c104430 2024-09-13T16:21:21.462 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.684+0000 7f02137fe640 1 -- 172.21.15.16:0/76542919 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f021803f020 con 0x7f021c104430 2024-09-13T16:21:21.462 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.742+0000 7f0224586640 1 -- 172.21.15.16:0/76542919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f021c193630 con 0x7f021c104430 2024-09-13T16:21:21.462 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.743+0000 7f02137fe640 1 -- 172.21.15.16:0/76542919 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f0218037070 con 0x7f021c104430 2024-09-13T16:21:21.463 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.744+0000 7f0224586640 1 -- 172.21.15.16:0/76542919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f021c104430 msgr2=0x7f021c199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:21.463 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.744+0000 7f0224586640 1 --2- 172.21.15.16:0/76542919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f021c104430 0x7f021c199380 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f021802a790 tx=0x7f0218030db0 comp rx=0 tx=0).stop 2024-09-13T16:21:21.463 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.744+0000 7f0224586640 1 -- 172.21.15.16:0/76542919 shutdown_connections 2024-09-13T16:21:21.463 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.744+0000 7f0224586640 1 --2- 172.21.15.16:0/76542919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f021c104430 0x7f021c199380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:21.463 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.744+0000 7f0224586640 1 -- 172.21.15.16:0/76542919 >> 172.21.15.16:0/76542919 conn(0x7f021c0ffbe0 msgr2=0x7f021c100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:21.463 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.744+0000 7f0224586640 1 -- 172.21.15.16:0/76542919 shutdown_connections 2024-09-13T16:21:21.463 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:20.744+0000 7f0224586640 1 -- 172.21.15.16:0/76542919 wait complete. 2024-09-13T16:21:21.463 INFO:teuthology.orchestra.run.smithi016.stdout:Restarting the monitor... 2024-09-13T16:21:21.739 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:21 smithi016 systemd[1]: Stopping Ceph mon.a for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:21:22.022 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:21 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mon-a[30806]: 2024-09-13T16:21:21.736+0000 7f1a9c4ae640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T16:21:22.022 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:21 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mon-a[30806]: 2024-09-13T16:21:21.736+0000 7f1a9c4ae640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-13T16:21:22.022 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:21 smithi016 podman[31063]: 2024-09-13 16:21:21.748044632 +0000 UTC m=+0.114138293 container died b8ba1ccb564284268a0e67d4cad807f35dc27d82bf6ec3871517e3305abd63e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-13T16:21:22.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:22 smithi016 podman[31063]: 2024-09-13 16:21:22.111993387 +0000 UTC m=+0.478087049 container cleanup b8ba1ccb564284268a0e67d4cad807f35dc27d82bf6ec3871517e3305abd63e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mon-a, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-13T16:21:22.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:22 smithi016 bash[31063]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mon-a 2024-09-13T16:21:22.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:22 smithi016 podman[31075]: 2024-09-13 16:21:22.538139634 +0000 UTC m=+0.782424954 container remove b8ba1ccb564284268a0e67d4cad807f35dc27d82bf6ec3871517e3305abd63e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f) 2024-09-13T16:21:22.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:22 smithi016 systemd[1]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-13T16:21:22.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:22 smithi016 systemd[1]: Stopped Ceph mon.a for 0d241f6a-71ec-11ef-bceb-c7b262605968. 2024-09-13T16:21:22.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:22 smithi016 systemd[1]: Starting Ceph mon.a for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:21:23.158 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:22 smithi016 podman[31167]: 2024-09-13 16:21:22.805546814 +0000 UTC m=+0.016208607 image pull 152d8bc6b19e354515e8ba109d4619445de0e09b4d4a747c2f16eb5c0edb670c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T16:21:23.158 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:22 smithi016 podman[31167]: 2024-09-13 16:21:22.947300473 +0000 UTC m=+0.157962313 container create 0d55658237b815e9fb2a4784256f6757d6b6b8a3169ad78d6339521adbeea852 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-13T16:21:23.158 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 podman[31167]: 2024-09-13 16:21:23.15764554 +0000 UTC m=+0.368307332 container init 0d55658237b815e9fb2a4784256f6757d6b6b8a3169ad78d6339521adbeea852 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T16:21:23.255 INFO:teuthology.orchestra.run.smithi016.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-13T16:21:23.485 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 podman[31167]: 2024-09-13 16:21:23.160963863 +0000 UTC m=+0.371625645 container start 0d55658237b815e9fb2a4784256f6757d6b6b8a3169ad78d6339521adbeea852 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-13T16:21:23.486 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: set uid:gid to 167:167 (ceph:ceph) 2024-09-13T16:21:23.486 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: ceph version 18.2.4-850-g50bbe7f3 (50bbe7f395997dcd871813949c49addbbf630361) reef (stable), process ceph-mon, pid 2 2024-09-13T16:21:23.486 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: pidfile_write: ignore empty --pid-file 2024-09-13T16:21:23.486 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: load: jerasure load: lrc 2024-09-13T16:21:23.486 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: RocksDB version: 7.9.2 2024-09-13T16:21:23.486 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Git sha 0 2024-09-13T16:21:23.486 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Compile date 2024-09-10 11:17:42 2024-09-13T16:21:23.486 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: DB SUMMARY 2024-09-13T16:21:23.486 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: DB Session ID: UDBI98CO5VUG9JKWSVT2 2024-09-13T16:21:23.486 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: CURRENT file: CURRENT 2024-09-13T16:21:23.486 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: IDENTITY file: IDENTITY 2024-09-13T16:21:23.486 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-13T16:21:23.487 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-13T16:21:23.487 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 99331 ; 2024-09-13T16:21:23.487 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.error_if_exists: 0 2024-09-13T16:21:23.487 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.create_if_missing: 0 2024-09-13T16:21:23.487 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.paranoid_checks: 1 2024-09-13T16:21:23.487 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-13T16:21:23.487 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-13T16:21:23.487 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-13T16:21:23.487 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.env: 0x555ce234abc0 2024-09-13T16:21:23.487 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.fs: PosixFileSystem 2024-09-13T16:21:23.487 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.info_log: 0x555ce43b4be0 2024-09-13T16:21:23.487 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.max_file_opening_threads: 16 2024-09-13T16:21:23.488 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.statistics: (nil) 2024-09-13T16:21:23.488 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.use_fsync: 0 2024-09-13T16:21:23.488 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.max_log_file_size: 0 2024-09-13T16:21:23.488 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-13T16:21:23.488 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-13T16:21:23.488 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.keep_log_file_num: 1000 2024-09-13T16:21:23.488 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.recycle_log_file_num: 0 2024-09-13T16:21:23.488 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.allow_fallocate: 1 2024-09-13T16:21:23.489 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.allow_mmap_reads: 0 2024-09-13T16:21:23.489 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.allow_mmap_writes: 0 2024-09-13T16:21:23.489 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.use_direct_reads: 0 2024-09-13T16:21:23.490 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-13T16:21:23.490 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.create_missing_column_families: 0 2024-09-13T16:21:23.490 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.db_log_dir: 2024-09-13T16:21:23.490 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.wal_dir: 2024-09-13T16:21:23.490 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-13T16:21:23.490 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-13T16:21:23.490 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-13T16:21:23.490 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-13T16:21:23.490 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-13T16:21:23.490 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-13T16:21:23.490 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.advise_random_on_open: 1 2024-09-13T16:21:23.490 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.db_write_buffer_size: 0 2024-09-13T16:21:23.490 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.write_buffer_manager: 0x555ce43c2320 2024-09-13T16:21:23.490 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-13T16:21:23.491 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-13T16:21:23.491 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-13T16:21:23.491 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.rate_limiter: (nil) 2024-09-13T16:21:23.491 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-13T16:21:23.491 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.wal_recovery_mode: 2 2024-09-13T16:21:23.491 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.enable_thread_tracking: 0 2024-09-13T16:21:23.491 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.enable_pipelined_write: 0 2024-09-13T16:21:23.491 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.unordered_write: 0 2024-09-13T16:21:23.491 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-13T16:21:23.491 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-13T16:21:23.491 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-13T16:21:23.491 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-13T16:21:23.491 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.row_cache: None 2024-09-13T16:21:23.492 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.wal_filter: None 2024-09-13T16:21:23.492 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-13T16:21:23.492 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.allow_ingest_behind: 0 2024-09-13T16:21:23.492 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.two_write_queues: 0 2024-09-13T16:21:23.492 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.manual_wal_flush: 0 2024-09-13T16:21:23.492 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.wal_compression: 0 2024-09-13T16:21:23.492 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.atomic_flush: 0 2024-09-13T16:21:23.492 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-13T16:21:23.492 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-13T16:21:23.492 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-13T16:21:23.492 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.log_readahead_size: 0 2024-09-13T16:21:23.492 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-13T16:21:23.492 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.best_efforts_recovery: 0 2024-09-13T16:21:23.492 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-13T16:21:23.493 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-13T16:21:23.493 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.allow_data_in_errors: 0 2024-09-13T16:21:23.493 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.db_host_id: __hostname__ 2024-09-13T16:21:23.493 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-13T16:21:23.493 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.max_background_jobs: 2 2024-09-13T16:21:23.493 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.max_background_compactions: -1 2024-09-13T16:21:23.493 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.max_subcompactions: 1 2024-09-13T16:21:23.493 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-13T16:21:23.493 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-13T16:21:23.493 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-13T16:21:23.493 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.max_total_wal_size: 0 2024-09-13T16:21:23.493 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-13T16:21:23.493 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-13T16:21:23.493 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-13T16:21:23.494 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-13T16:21:23.494 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.max_open_files: -1 2024-09-13T16:21:23.494 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.bytes_per_sync: 0 2024-09-13T16:21:23.494 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-13T16:21:23.494 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-13T16:21:23.494 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.compaction_readahead_size: 0 2024-09-13T16:21:23.494 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.max_background_flushes: -1 2024-09-13T16:21:23.494 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Compression algorithms supported: 2024-09-13T16:21:23.494 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: kZSTD supported: 0 2024-09-13T16:21:23.494 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: kXpressCompression supported: 0 2024-09-13T16:21:23.494 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: kBZip2Compression supported: 0 2024-09-13T16:21:23.494 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-13T16:21:23.494 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: kLZ4Compression supported: 1 2024-09-13T16:21:23.495 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: kZlibCompression supported: 1 2024-09-13T16:21:23.495 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: kLZ4HCCompression supported: 1 2024-09-13T16:21:23.495 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: kSnappyCompression supported: 1 2024-09-13T16:21:23.495 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-13T16:21:23.495 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-13T16:21:23.495 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-13T16:21:23.495 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-13T16:21:23.495 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-13T16:21:23.495 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.merge_operator: 2024-09-13T16:21:23.495 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.compaction_filter: None 2024-09-13T16:21:23.495 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.compaction_filter_factory: None 2024-09-13T16:21:23.495 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.sst_partitioner_factory: None 2024-09-13T16:21:23.495 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-13T16:21:23.495 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-13T16:21:23.496 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555ce43b47e0) 2024-09-13T16:21:23.496 INFO:journalctl@ceph.mon.a.smithi016.stdout: cache_index_and_filter_blocks: 1 2024-09-13T16:21:23.496 INFO:journalctl@ceph.mon.a.smithi016.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-13T16:21:23.496 INFO:journalctl@ceph.mon.a.smithi016.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-13T16:21:23.496 INFO:journalctl@ceph.mon.a.smithi016.stdout: pin_top_level_index_and_filter: 1 2024-09-13T16:21:23.496 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_type: 0 2024-09-13T16:21:23.496 INFO:journalctl@ceph.mon.a.smithi016.stdout: data_block_index_type: 0 2024-09-13T16:21:23.496 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_shortening: 1 2024-09-13T16:21:23.496 INFO:journalctl@ceph.mon.a.smithi016.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-13T16:21:23.496 INFO:journalctl@ceph.mon.a.smithi016.stdout: checksum: 4 2024-09-13T16:21:23.496 INFO:journalctl@ceph.mon.a.smithi016.stdout: no_block_cache: 0 2024-09-13T16:21:23.496 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache: 0x555ce43a9350 2024-09-13T16:21:23.497 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_name: BinnedLRUCache 2024-09-13T16:21:23.497 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_options: 2024-09-13T16:21:23.497 INFO:journalctl@ceph.mon.a.smithi016.stdout: capacity : 536870912 2024-09-13T16:21:23.497 INFO:journalctl@ceph.mon.a.smithi016.stdout: num_shard_bits : 4 2024-09-13T16:21:23.497 INFO:journalctl@ceph.mon.a.smithi016.stdout: strict_capacity_limit : 0 2024-09-13T16:21:23.497 INFO:journalctl@ceph.mon.a.smithi016.stdout: high_pri_pool_ratio: 0.000 2024-09-13T16:21:23.497 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_compressed: (nil) 2024-09-13T16:21:23.497 INFO:journalctl@ceph.mon.a.smithi016.stdout: persistent_cache: (nil) 2024-09-13T16:21:23.497 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_size: 4096 2024-09-13T16:21:23.497 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_size_deviation: 10 2024-09-13T16:21:23.497 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_restart_interval: 16 2024-09-13T16:21:23.497 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_block_restart_interval: 1 2024-09-13T16:21:23.497 INFO:journalctl@ceph.mon.a.smithi016.stdout: metadata_block_size: 4096 2024-09-13T16:21:23.498 INFO:journalctl@ceph.mon.a.smithi016.stdout: partition_filters: 0 2024-09-13T16:21:23.498 INFO:journalctl@ceph.mon.a.smithi016.stdout: use_delta_encoding: 1 2024-09-13T16:21:23.498 INFO:journalctl@ceph.mon.a.smithi016.stdout: filter_policy: bloomfilter 2024-09-13T16:21:23.498 INFO:journalctl@ceph.mon.a.smithi016.stdout: whole_key_filtering: 1 2024-09-13T16:21:23.498 INFO:journalctl@ceph.mon.a.smithi016.stdout: verify_compression: 0 2024-09-13T16:21:23.498 INFO:journalctl@ceph.mon.a.smithi016.stdout: read_amp_bytes_per_bit: 0 2024-09-13T16:21:23.498 INFO:journalctl@ceph.mon.a.smithi016.stdout: format_version: 5 2024-09-13T16:21:23.498 INFO:journalctl@ceph.mon.a.smithi016.stdout: enable_index_compression: 1 2024-09-13T16:21:23.498 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_align: 0 2024-09-13T16:21:23.498 INFO:journalctl@ceph.mon.a.smithi016.stdout: max_auto_readahead_size: 262144 2024-09-13T16:21:23.498 INFO:journalctl@ceph.mon.a.smithi016.stdout: prepopulate_block_cache: 0 2024-09-13T16:21:23.498 INFO:journalctl@ceph.mon.a.smithi016.stdout: initial_auto_readahead_size: 8192 2024-09-13T16:21:23.498 INFO:journalctl@ceph.mon.a.smithi016.stdout: num_file_reads_for_auto_readahead: 2 2024-09-13T16:21:23.498 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.write_buffer_size: 33554432 2024-09-13T16:21:23.499 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.max_write_buffer_number: 2 2024-09-13T16:21:23.499 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.compression: NoCompression 2024-09-13T16:21:23.499 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.bottommost_compression: Disabled 2024-09-13T16:21:23.499 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.prefix_extractor: nullptr 2024-09-13T16:21:23.499 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-13T16:21:23.499 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.num_levels: 7 2024-09-13T16:21:23.499 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-13T16:21:23.499 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-13T16:21:23.499 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-13T16:21:23.499 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-13T16:21:23.499 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-13T16:21:23.499 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-13T16:21:23.499 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-13T16:21:23.500 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-13T16:21:23.500 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-13T16:21:23.500 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-13T16:21:23.500 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-13T16:21:23.500 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-13T16:21:23.500 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-13T16:21:23.500 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.compression_opts.level: 32767 2024-09-13T16:21:23.500 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.compression_opts.strategy: 0 2024-09-13T16:21:23.500 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-13T16:21:23.500 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-13T16:21:23.500 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-13T16:21:23.500 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-13T16:21:23.500 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.compression_opts.enabled: false 2024-09-13T16:21:23.500 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-13T16:21:23.500 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-13T16:21:23.500 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-13T16:21:23.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-13T16:21:23.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.target_file_size_base: 67108864 2024-09-13T16:21:23.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-13T16:21:23.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-13T16:21:23.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-13T16:21:23.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-13T16:21:23.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-13T16:21:23.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-13T16:21:23.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-13T16:21:23.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-13T16:21:23.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-13T16:21:23.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-13T16:21:23.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-13T16:21:23.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-13T16:21:23.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-13T16:21:23.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-13T16:21:23.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.arena_block_size: 1048576 2024-09-13T16:21:23.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-13T16:21:23.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-13T16:21:23.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.disable_auto_compactions: 0 2024-09-13T16:21:23.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-13T16:21:23.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-13T16:21:23.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-13T16:21:23.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-13T16:21:23.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-13T16:21:23.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-13T16:21:23.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-13T16:21:23.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-13T16:21:23.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-13T16:21:23.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-13T16:21:23.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-13T16:21:23.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.inplace_update_support: 0 2024-09-13T16:21:23.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-13T16:21:23.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-13T16:21:23.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-13T16:21:23.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-13T16:21:23.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.bloom_locality: 0 2024-09-13T16:21:23.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.max_successive_merges: 0 2024-09-13T16:21:23.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-13T16:21:23.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.paranoid_file_checks: 0 2024-09-13T16:21:23.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.force_consistency_checks: 1 2024-09-13T16:21:23.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.report_bg_io_stats: 0 2024-09-13T16:21:23.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.ttl: 2592000 2024-09-13T16:21:23.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-13T16:21:23.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-13T16:21:23.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-13T16:21:23.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.enable_blob_files: false 2024-09-13T16:21:23.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.min_blob_size: 0 2024-09-13T16:21:23.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.blob_file_size: 268435456 2024-09-13T16:21:23.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-13T16:21:23.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-13T16:21:23.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-13T16:21:23.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-13T16:21:23.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-13T16:21:23.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.blob_file_starting_level: 0 2024-09-13T16:21:23.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-13T16:21:23.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-13T16:21:23.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-13T16:21:23.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9db69c75-b2ca-4c14-a7b4-4d62a2cd4731 2024-09-13T16:21:23.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726244483187369, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-13T16:21:23.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-13T16:21:23.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726244483188396, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94564, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 316, "table_properties": {"data_size": 92536, "index_size": 284, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13763, "raw_average_key_size": 50, "raw_value_size": 85029, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 275, "num_filter_entries": 275, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726244483, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9db69c75-b2ca-4c14-a7b4-4d62a2cd4731", "db_session_id": "UDBI98CO5VUG9JKWSVT2", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-13T16:21:23.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726244483188495, "job": 1, "event": "recovery_finished"} 2024-09-13T16:21:23.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-13T16:21:23.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-13T16:21:23.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x555ce43d4e00 2024-09-13T16:21:23.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: DB pointer 0x555ce44c8000 2024-09-13T16:21:23.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-13T16:21:23.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-13T16:21:23.505 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** DB Stats ** 2024-09-13T16:21:23.505 INFO:journalctl@ceph.mon.a.smithi016.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-13T16:21:23.505 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-13T16:21:23.505 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-13T16:21:23.506 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-13T16:21:23.506 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-13T16:21:23.506 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-13T16:21:23.506 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-13T16:21:23.506 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-13T16:21:23.506 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** Compaction Stats [default] ** 2024-09-13T16:21:23.506 INFO:journalctl@ceph.mon.a.smithi016.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-13T16:21:23.506 INFO:journalctl@ceph.mon.a.smithi016.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-13T16:21:23.506 INFO:journalctl@ceph.mon.a.smithi016.stdout: L0 2/0 94.21 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-13T16:21:23.506 INFO:journalctl@ceph.mon.a.smithi016.stdout: Sum 2/0 94.21 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-13T16:21:23.506 INFO:journalctl@ceph.mon.a.smithi016.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-13T16:21:23.506 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-13T16:21:23.506 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** Compaction Stats [default] ** 2024-09-13T16:21:23.506 INFO:journalctl@ceph.mon.a.smithi016.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-13T16:21:23.506 INFO:journalctl@ceph.mon.a.smithi016.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-13T16:21:23.507 INFO:journalctl@ceph.mon.a.smithi016.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 145.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-13T16:21:23.507 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-13T16:21:23.507 INFO:journalctl@ceph.mon.a.smithi016.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-13T16:21:23.507 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-13T16:21:23.507 INFO:journalctl@ceph.mon.a.smithi016.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-13T16:21:23.507 INFO:journalctl@ceph.mon.a.smithi016.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-13T16:21:23.507 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-13T16:21:23.507 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-13T16:21:23.507 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-13T16:21:23.507 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-13T16:21:23.507 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative compaction: 0.00 GB write, 27.26 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-13T16:21:23.507 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval compaction: 0.00 GB write, 27.26 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-13T16:21:23.507 INFO:journalctl@ceph.mon.a.smithi016.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-13T16:21:23.508 INFO:journalctl@ceph.mon.a.smithi016.stdout: Block cache BinnedLRUCache@0x555ce43a9350#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-13T16:21:23.508 INFO:journalctl@ceph.mon.a.smithi016.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-13T16:21:23.508 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-13T16:21:23.508 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-13T16:21:23.508 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: starting mon.a rank 0 at public addrs [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] at bind addrs [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 2024-09-13T16:21:23.508 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: mon.a@-1(???) e1 preinit fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 2024-09-13T16:21:23.508 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: mon.a@-1(???).mds e1 new map 2024-09-13T16:21:23.508 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: mon.a@-1(???).mds e1 print_map 2024-09-13T16:21:23.508 INFO:journalctl@ceph.mon.a.smithi016.stdout: e1 2024-09-13T16:21:23.508 INFO:journalctl@ceph.mon.a.smithi016.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-13T16:21:23.508 INFO:journalctl@ceph.mon.a.smithi016.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-13T16:21:23.508 INFO:journalctl@ceph.mon.a.smithi016.stdout: legacy client fscid: -1 2024-09-13T16:21:23.508 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-13T16:21:23.508 INFO:journalctl@ceph.mon.a.smithi016.stdout: No filesystems configured 2024-09-13T16:21:23.509 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-13T16:21:23.509 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-13T16:21:23.509 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-13T16:21:23.509 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-13T16:21:23.509 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-13T16:21:23.509 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-13T16:21:23.509 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: monmap e1: 1 mons at {a=[v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-13T16:21:23.509 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: fsmap 2024-09-13T16:21:23.509 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: osdmap e1: 0 total, 0 up, 0 in 2024-09-13T16:21:23.509 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 ceph-mon[31187]: mgrmap e1: no daemons active 2024-09-13T16:21:23.509 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 bash[31167]: 0d55658237b815e9fb2a4784256f6757d6b6b8a3169ad78d6339521adbeea852 2024-09-13T16:21:23.509 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:23 smithi016 systemd[1]: Started Ceph mon.a for 0d241f6a-71ec-11ef-bceb-c7b262605968. 2024-09-13T16:21:24.213 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.677+0000 7f68d92c5640 1 Processor -- start 2024-09-13T16:21:24.213 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.677+0000 7f68d92c5640 1 -- start start 2024-09-13T16:21:24.213 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.677+0000 7f68d92c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68d4103e00 0x7f68d4104200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:24.213 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.677+0000 7f68d92c5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d4079dc0 con 0x7f68d4103e00 2024-09-13T16:21:24.213 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.678+0000 7f68d2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68d4103e00 0x7f68d4104200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:24.213 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.678+0000 7f68d2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68d4103e00 0x7f68d4104200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42328/0 (socket says 172.21.15.16:42328) 2024-09-13T16:21:24.213 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.678+0000 7f68d2ffd640 1 -- 172.21.15.16:0/3929603608 learned_addr learned my addr 172.21.15.16:0/3929603608 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:24.213 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.678+0000 7f68d2ffd640 1 -- 172.21.15.16:0/3929603608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d4104740 con 0x7f68d4103e00 2024-09-13T16:21:24.213 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.680+0000 7f68d2ffd640 1 --2- 172.21.15.16:0/3929603608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68d4103e00 0x7f68d4104200 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f68cc009820 tx=0x7f68cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed184e07f8f86ba6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:24.213 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.680+0000 7f68d1ffb640 1 -- 172.21.15.16:0/3929603608 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f68cc02ae70 con 0x7f68d4103e00 2024-09-13T16:21:24.214 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.680+0000 7f68d1ffb640 1 -- 172.21.15.16:0/3929603608 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f68cc0328b0 con 0x7f68d4103e00 2024-09-13T16:21:24.214 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.680+0000 7f68d1ffb640 1 -- 172.21.15.16:0/3929603608 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f68cc030850 con 0x7f68d4103e00 2024-09-13T16:21:24.214 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.681+0000 7f68d92c5640 1 -- 172.21.15.16:0/3929603608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68d4103e00 msgr2=0x7f68d4104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:24.214 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.681+0000 7f68d92c5640 1 --2- 172.21.15.16:0/3929603608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68d4103e00 0x7f68d4104200 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f68cc009820 tx=0x7f68cc02a270 comp rx=0 tx=0).stop 2024-09-13T16:21:24.214 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.681+0000 7f68d92c5640 1 -- 172.21.15.16:0/3929603608 shutdown_connections 2024-09-13T16:21:24.214 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.681+0000 7f68d92c5640 1 --2- 172.21.15.16:0/3929603608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68d4103e00 0x7f68d4104200 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:24.214 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.681+0000 7f68d92c5640 1 -- 172.21.15.16:0/3929603608 >> 172.21.15.16:0/3929603608 conn(0x7f68d40ffbb0 msgr2=0x7f68d4101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:24.214 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.681+0000 7f68d92c5640 1 -- 172.21.15.16:0/3929603608 shutdown_connections 2024-09-13T16:21:24.214 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.681+0000 7f68d92c5640 1 -- 172.21.15.16:0/3929603608 wait complete. 2024-09-13T16:21:24.214 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.681+0000 7f68d92c5640 1 Processor -- start 2024-09-13T16:21:24.214 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.682+0000 7f68d92c5640 1 -- start start 2024-09-13T16:21:24.214 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.682+0000 7f68d92c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68d4103e00 0x7f68d419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:24.214 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.682+0000 7f68d92c5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d419b870 con 0x7f68d4103e00 2024-09-13T16:21:24.215 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.682+0000 7f68d2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68d4103e00 0x7f68d419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:24.215 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.682+0000 7f68d2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68d4103e00 0x7f68d419b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42340/0 (socket says 172.21.15.16:42340) 2024-09-13T16:21:24.215 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.682+0000 7f68d2ffd640 1 -- 172.21.15.16:0/498044182 learned_addr learned my addr 172.21.15.16:0/498044182 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:24.215 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.682+0000 7f68d2ffd640 1 -- 172.21.15.16:0/498044182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68cc0094d0 con 0x7f68d4103e00 2024-09-13T16:21:24.215 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.682+0000 7f68d2ffd640 1 --2- 172.21.15.16:0/498044182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68d4103e00 0x7f68d419b330 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f68cc030560 tx=0x7f68cc030d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:24.215 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.682+0000 7f68cbfff640 1 -- 172.21.15.16:0/498044182 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f68cc032d60 con 0x7f68d4103e00 2024-09-13T16:21:24.215 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.683+0000 7f68cbfff640 1 -- 172.21.15.16:0/498044182 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f68cc03b8a0 con 0x7f68d4103e00 2024-09-13T16:21:24.215 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.683+0000 7f68cbfff640 1 -- 172.21.15.16:0/498044182 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f68cc03a740 con 0x7f68d4103e00 2024-09-13T16:21:24.215 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.683+0000 7f68d92c5640 1 -- 172.21.15.16:0/498044182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68d419ba70 con 0x7f68d4103e00 2024-09-13T16:21:24.215 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.683+0000 7f68d92c5640 1 -- 172.21.15.16:0/498044182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68d419bf10 con 0x7f68d4103e00 2024-09-13T16:21:24.215 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.683+0000 7f68cbfff640 1 -- 172.21.15.16:0/498044182 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f68cc039040 con 0x7f68d4103e00 2024-09-13T16:21:24.215 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.683+0000 7f68cbfff640 1 -- 172.21.15.16:0/498044182 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f68cc044c50 con 0x7f68d4103e00 2024-09-13T16:21:24.215 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.684+0000 7f68d92c5640 1 -- 172.21.15.16:0/498044182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68d4105d90 con 0x7f68d4103e00 2024-09-13T16:21:24.216 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.686+0000 7f68cbfff640 1 -- 172.21.15.16:0/498044182 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f68cc03f020 con 0x7f68d4103e00 2024-09-13T16:21:24.216 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.747+0000 7f68d92c5640 1 -- 172.21.15.16:0/498044182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f68d41055c0 con 0x7f68d4103e00 2024-09-13T16:21:24.216 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.749+0000 7f68cbfff640 1 -- 172.21.15.16:0/498044182 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f68cc037070 con 0x7f68d4103e00 2024-09-13T16:21:24.216 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.750+0000 7f68d92c5640 1 -- 172.21.15.16:0/498044182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68d4103e00 msgr2=0x7f68d419b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:24.216 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.750+0000 7f68d92c5640 1 --2- 172.21.15.16:0/498044182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68d4103e00 0x7f68d419b330 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f68cc030560 tx=0x7f68cc030d90 comp rx=0 tx=0).stop 2024-09-13T16:21:24.216 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.750+0000 7f68d92c5640 1 -- 172.21.15.16:0/498044182 shutdown_connections 2024-09-13T16:21:24.216 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.750+0000 7f68d92c5640 1 --2- 172.21.15.16:0/498044182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68d4103e00 0x7f68d419b330 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:24.216 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.750+0000 7f68d92c5640 1 -- 172.21.15.16:0/498044182 >> 172.21.15.16:0/498044182 conn(0x7f68d40ffbb0 msgr2=0x7f68d418fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:24.216 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.750+0000 7f68d92c5640 1 -- 172.21.15.16:0/498044182 shutdown_connections 2024-09-13T16:21:24.216 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:23.750+0000 7f68d92c5640 1 -- 172.21.15.16:0/498044182 wait complete. 2024-09-13T16:21:24.216 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-13T16:21:24.231 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-13T16:21:24.231 INFO:teuthology.orchestra.run.smithi016.stdout:Creating mgr... 2024-09-13T16:21:24.231 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-13T16:21:24.231 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-13T16:21:24.481 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mgr.x 2024-09-13T16:21:24.481 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mgr.x.service: Unit ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mgr.x.service not loaded. 2024-09-13T16:21:24.633 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0d241f6a-71ec-11ef-bceb-c7b262605968.target.wants/ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@.service. 2024-09-13T16:21:24.750 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:24 smithi016 systemd[1]: Starting Ceph mgr.x for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:21:25.077 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:24 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/498044182' entity='client.admin' 2024-09-13T16:21:25.077 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:24 smithi016 podman[31402]: 2024-09-13 16:21:24.771551699 +0000 UTC m=+0.016417843 image pull 152d8bc6b19e354515e8ba109d4619445de0e09b4d4a747c2f16eb5c0edb670c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T16:21:25.077 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:24 smithi016 podman[31402]: 2024-09-13 16:21:24.882730688 +0000 UTC m=+0.127596823 container create afeb66054b3492280676fa41a48de05b46deecf1c352a19416e27a41d7a40d20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-13T16:21:25.077 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:25 smithi016 podman[31402]: 2024-09-13 16:21:25.076788303 +0000 UTC m=+0.321654445 container init afeb66054b3492280676fa41a48de05b46deecf1c352a19416e27a41d7a40d20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T16:21:25.192 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-13T16:21:25.192 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-09-13T16:21:25.199 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-13T16:21:25.199 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-13T16:21:25.199 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-09-13T16:21:25.200 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to enable service . firewalld.service is not available 2024-09-13T16:21:25.209 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-13T16:21:25.209 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-09-13T16:21:25.217 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-13T16:21:25.217 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-13T16:21:25.217 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-09-13T16:21:25.217 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-13T16:21:25.217 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr to start... 2024-09-13T16:21:25.217 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr... 2024-09-13T16:21:25.341 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:25 smithi016 podman[31402]: 2024-09-13 16:21:25.079700712 +0000 UTC m=+0.324566844 container start afeb66054b3492280676fa41a48de05b46deecf1c352a19416e27a41d7a40d20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T16:21:25.341 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:25 smithi016 bash[31402]: afeb66054b3492280676fa41a48de05b46deecf1c352a19416e27a41d7a40d20 2024-09-13T16:21:25.341 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:25 smithi016 systemd[1]: Started Ceph mgr.x for 0d241f6a-71ec-11ef-bceb-c7b262605968. 2024-09-13T16:21:25.341 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:25 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:25.202+0000 7f8390d24040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T16:21:25.341 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:25 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:25.265+0000 7f8390d24040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T16:21:25.604 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:25 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:25.422+0000 7f8390d24040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T16:21:25.604 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:25 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:25.493+0000 7f8390d24040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T16:21:26.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:25 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/676181692' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-13T16:21:26.059 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:25 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:25.646+0000 7f8390d24040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T16:21:26.059 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:25 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:25.798+0000 7f8390d24040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T16:21:26.354 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:26 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:26.126+0000 7f8390d24040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T16:21:26.354 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:26 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:26.189+0000 7f8390d24040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T16:21:26.479 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-13T16:21:26.481 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-13T16:21:26.481 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "0d241f6a-71ec-11ef-bceb-c7b262605968", 2024-09-13T16:21:26.482 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-09-13T16:21:26.482 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-13T16:21:26.482 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-13T16:21:26.482 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-13T16:21:26.482 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-13T16:21:26.482 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-13T16:21:26.482 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-13T16:21:26.482 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-09-13T16:21:26.482 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-13T16:21:26.482 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-13T16:21:26.482 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-09-13T16:21:26.482 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-13T16:21:26.482 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-13T16:21:26.482 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-13T16:21:26.483 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T16:21:26.483 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-13T16:21:26.483 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-13T16:21:26.483 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-13T16:21:26.483 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-13T16:21:26.483 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T16:21:26.483 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-13T16:21:26.483 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-13T16:21:26.483 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-13T16:21:26.490 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-13T16:21:26.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-13T16:21:26.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-13T16:21:26.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-13T16:21:26.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-13T16:21:26.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-13T16:21:26.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-13T16:21:26.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-13T16:21:26.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-13T16:21:26.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-13T16:21:26.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-13T16:21:26.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-13T16:21:26.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-13T16:21:26.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-13T16:21:26.492 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-13T16:21:26.492 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T16:21:26.492 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-13T16:21:26.492 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-13T16:21:26.492 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-13T16:21:26.492 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-13T16:21:26.492 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-13T16:21:26.492 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-13T16:21:26.492 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-13T16:21:26.492 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-13T16:21:26.492 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-13T16:21:26.492 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-09-13T16:21:26.492 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-13T16:21:26.492 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-13T16:21:26.493 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-13T16:21:26.493 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-13T16:21:26.493 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T16:21:26.493 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-09-13T16:21:17.972182+0000", 2024-09-13T16:21:26.493 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-13T16:21:26.493 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-13T16:21:26.493 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-13T16:21:26.493 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-13T16:21:26.493 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.693+0000 7f3c750a7640 1 Processor -- start 2024-09-13T16:21:26.493 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.693+0000 7f3c750a7640 1 -- start start 2024-09-13T16:21:26.493 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.693+0000 7f3c750a7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c70105fa0 0x7f3c70108390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:26.493 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.693+0000 7f3c750a7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c70079e40 con 0x7f3c70105fa0 2024-09-13T16:21:26.493 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.694+0000 7f3c6ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c70105fa0 0x7f3c70108390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:26.493 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.694+0000 7f3c6ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c70105fa0 0x7f3c70108390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42382/0 (socket says 172.21.15.16:42382) 2024-09-13T16:21:26.493 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.694+0000 7f3c6ed9d640 1 -- 172.21.15.16:0/1194880394 learned_addr learned my addr 172.21.15.16:0/1194880394 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:26.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.694+0000 7f3c6ed9d640 1 -- 172.21.15.16:0/1194880394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c701088d0 con 0x7f3c70105fa0 2024-09-13T16:21:26.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.695+0000 7f3c6ed9d640 1 --2- 172.21.15.16:0/1194880394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c70105fa0 0x7f3c70108390 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3c60009820 tx=0x7f3c6002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a28274fbbbac09d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:26.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.695+0000 7f3c6dd9b640 1 -- 172.21.15.16:0/1194880394 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c6002ae70 con 0x7f3c70105fa0 2024-09-13T16:21:26.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.695+0000 7f3c6dd9b640 1 -- 172.21.15.16:0/1194880394 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f3c600328b0 con 0x7f3c70105fa0 2024-09-13T16:21:26.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.695+0000 7f3c6dd9b640 1 -- 172.21.15.16:0/1194880394 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c60030850 con 0x7f3c70105fa0 2024-09-13T16:21:26.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.695+0000 7f3c750a7640 1 -- 172.21.15.16:0/1194880394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c70105fa0 msgr2=0x7f3c70108390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:26.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.695+0000 7f3c750a7640 1 --2- 172.21.15.16:0/1194880394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c70105fa0 0x7f3c70108390 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3c60009820 tx=0x7f3c6002a270 comp rx=0 tx=0).stop 2024-09-13T16:21:26.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.696+0000 7f3c750a7640 1 -- 172.21.15.16:0/1194880394 shutdown_connections 2024-09-13T16:21:26.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.696+0000 7f3c750a7640 1 --2- 172.21.15.16:0/1194880394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c70105fa0 0x7f3c70108390 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:26.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.696+0000 7f3c750a7640 1 -- 172.21.15.16:0/1194880394 >> 172.21.15.16:0/1194880394 conn(0x7f3c700ffbe0 msgr2=0x7f3c70102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:26.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.696+0000 7f3c750a7640 1 -- 172.21.15.16:0/1194880394 shutdown_connections 2024-09-13T16:21:26.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.696+0000 7f3c750a7640 1 -- 172.21.15.16:0/1194880394 wait complete. 2024-09-13T16:21:26.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.697+0000 7f3c750a7640 1 Processor -- start 2024-09-13T16:21:26.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.697+0000 7f3c750a7640 1 -- start start 2024-09-13T16:21:26.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.697+0000 7f3c750a7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c70105fa0 0x7f3c70199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:26.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.697+0000 7f3c750a7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c701996c0 con 0x7f3c70105fa0 2024-09-13T16:21:26.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.697+0000 7f3c6ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c70105fa0 0x7f3c70199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:26.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.697+0000 7f3c6ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c70105fa0 0x7f3c70199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42392/0 (socket says 172.21.15.16:42392) 2024-09-13T16:21:26.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.697+0000 7f3c6ed9d640 1 -- 172.21.15.16:0/676181692 learned_addr learned my addr 172.21.15.16:0/676181692 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:26.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.697+0000 7f3c6ed9d640 1 -- 172.21.15.16:0/676181692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c600094d0 con 0x7f3c70105fa0 2024-09-13T16:21:26.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.698+0000 7f3c6ed9d640 1 --2- 172.21.15.16:0/676181692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c70105fa0 0x7f3c70199180 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3c60030560 tx=0x7f3c60030d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:26.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.698+0000 7f3c57fff640 1 -- 172.21.15.16:0/676181692 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c60032d60 con 0x7f3c70105fa0 2024-09-13T16:21:26.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.698+0000 7f3c57fff640 1 -- 172.21.15.16:0/676181692 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f3c6003b8a0 con 0x7f3c70105fa0 2024-09-13T16:21:26.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.698+0000 7f3c57fff640 1 -- 172.21.15.16:0/676181692 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c6003a740 con 0x7f3c70105fa0 2024-09-13T16:21:26.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.698+0000 7f3c750a7640 1 -- 172.21.15.16:0/676181692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c701998c0 con 0x7f3c70105fa0 2024-09-13T16:21:26.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.698+0000 7f3c750a7640 1 -- 172.21.15.16:0/676181692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c70199d60 con 0x7f3c70105fa0 2024-09-13T16:21:26.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.698+0000 7f3c57fff640 1 -- 172.21.15.16:0/676181692 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f3c60039040 con 0x7f3c70105fa0 2024-09-13T16:21:26.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.698+0000 7f3c57fff640 1 -- 172.21.15.16:0/676181692 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3c60044bc0 con 0x7f3c70105fa0 2024-09-13T16:21:26.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.699+0000 7f3c750a7640 1 -- 172.21.15.16:0/676181692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c70109690 con 0x7f3c70105fa0 2024-09-13T16:21:26.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.701+0000 7f3c57fff640 1 -- 172.21.15.16:0/676181692 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f3c6003f020 con 0x7f3c70105fa0 2024-09-13T16:21:26.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.762+0000 7f3c750a7640 1 -- 172.21.15.16:0/676181692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f3c70109850 con 0x7f3c70105fa0 2024-09-13T16:21:26.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.763+0000 7f3c57fff640 1 -- 172.21.15.16:0/676181692 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f3c60037070 con 0x7f3c70105fa0 2024-09-13T16:21:26.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.764+0000 7f3c750a7640 1 -- 172.21.15.16:0/676181692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c70105fa0 msgr2=0x7f3c70199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:26.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.764+0000 7f3c750a7640 1 --2- 172.21.15.16:0/676181692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c70105fa0 0x7f3c70199180 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3c60030560 tx=0x7f3c60030d90 comp rx=0 tx=0).stop 2024-09-13T16:21:26.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.764+0000 7f3c750a7640 1 -- 172.21.15.16:0/676181692 shutdown_connections 2024-09-13T16:21:26.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.764+0000 7f3c750a7640 1 --2- 172.21.15.16:0/676181692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c70105fa0 0x7f3c70199180 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:26.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.764+0000 7f3c750a7640 1 -- 172.21.15.16:0/676181692 >> 172.21.15.16:0/676181692 conn(0x7f3c700ffbe0 msgr2=0x7f3c70100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:26.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.764+0000 7f3c750a7640 1 -- 172.21.15.16:0/676181692 shutdown_connections 2024-09-13T16:21:26.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:25.764+0000 7f3c750a7640 1 -- 172.21.15.16:0/676181692 wait complete. 2024-09-13T16:21:26.496 INFO:teuthology.orchestra.run.smithi016.stdout:mgr not available, waiting (1/15)... 2024-09-13T16:21:26.854 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:26 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:26.379+0000 7f8390d24040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T16:21:27.354 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:27 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:27.297+0000 7f8390d24040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T16:21:27.854 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:27 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:27.366+0000 7f8390d24040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T16:21:28.383 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:28 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:28.115+0000 7f8390d24040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-13T16:21:28.383 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:28 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:28.234+0000 7f8390d24040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T16:21:28.383 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:28 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:28.381+0000 7f8390d24040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T16:21:28.672 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:28 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:28.445+0000 7f8390d24040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T16:21:28.673 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:28 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:28.600+0000 7f8390d24040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T16:21:28.673 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:28 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:28.671+0000 7f8390d24040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T16:21:28.933 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:28 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:28.735+0000 7f8390d24040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T16:21:28.934 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:28 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:28.901+0000 7f8390d24040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T16:21:29.196 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:29 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/273024950' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-13T16:21:29.196 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:28 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:28.971+0000 7f8390d24040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T16:21:29.197 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:29 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:29.047+0000 7f8390d24040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T16:21:29.197 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:29 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:29.133+0000 7f8390d24040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T16:21:29.197 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:29 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:29.195+0000 7f8390d24040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T16:21:29.591 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:29 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:29.256+0000 7f8390d24040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T16:21:29.603 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-13T16:21:29.604 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-13T16:21:29.604 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "0d241f6a-71ec-11ef-bceb-c7b262605968", 2024-09-13T16:21:29.604 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-09-13T16:21:29.604 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-13T16:21:29.604 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-13T16:21:29.604 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-13T16:21:29.604 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-13T16:21:29.604 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-13T16:21:29.604 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-13T16:21:29.604 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-09-13T16:21:29.604 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-13T16:21:29.604 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-13T16:21:29.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-09-13T16:21:29.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-13T16:21:29.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-13T16:21:29.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-13T16:21:29.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T16:21:29.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-13T16:21:29.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-13T16:21:29.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-13T16:21:29.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-13T16:21:29.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T16:21:29.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-13T16:21:29.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-13T16:21:29.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-13T16:21:29.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-13T16:21:29.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-13T16:21:29.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-13T16:21:29.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-13T16:21:29.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-13T16:21:29.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-13T16:21:29.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-13T16:21:29.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-13T16:21:29.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-13T16:21:29.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-13T16:21:29.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-13T16:21:29.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-13T16:21:29.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-13T16:21:29.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-13T16:21:29.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-13T16:21:29.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T16:21:29.607 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-13T16:21:29.607 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-13T16:21:29.607 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-13T16:21:29.607 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-13T16:21:29.607 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-13T16:21:29.607 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-13T16:21:29.607 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-13T16:21:29.607 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-13T16:21:29.607 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-13T16:21:29.607 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-09-13T16:21:29.607 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-13T16:21:29.607 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-13T16:21:29.607 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-13T16:21:29.607 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-13T16:21:29.608 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T16:21:29.608 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-09-13T16:21:17.972182+0000", 2024-09-13T16:21:29.608 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-13T16:21:29.608 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-13T16:21:29.608 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-13T16:21:29.608 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-13T16:21:29.608 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.932+0000 7f2dfab69640 1 Processor -- start 2024-09-13T16:21:29.608 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.932+0000 7f2dfab69640 1 -- start start 2024-09-13T16:21:29.608 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.932+0000 7f2dfab69640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df4105fa0 0x7f2df4108390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:29.608 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.932+0000 7f2dfab69640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df4079e40 con 0x7f2df4105fa0 2024-09-13T16:21:29.608 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.933+0000 7f2df8905640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df4105fa0 0x7f2df4108390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:29.608 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.933+0000 7f2df8905640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df4105fa0 0x7f2df4108390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42400/0 (socket says 172.21.15.16:42400) 2024-09-13T16:21:29.608 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.933+0000 7f2df8905640 1 -- 172.21.15.16:0/535846119 learned_addr learned my addr 172.21.15.16:0/535846119 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:29.608 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.933+0000 7f2df8905640 1 -- 172.21.15.16:0/535846119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df41088d0 con 0x7f2df4105fa0 2024-09-13T16:21:29.609 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.934+0000 7f2df8905640 1 --2- 172.21.15.16:0/535846119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df4105fa0 0x7f2df4108390 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc009820 tx=0x7f2ddc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eda04c6fb79a502 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:29.609 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.934+0000 7f2df37fe640 1 -- 172.21.15.16:0/535846119 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ddc02ae70 con 0x7f2df4105fa0 2024-09-13T16:21:29.609 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.934+0000 7f2df37fe640 1 -- 172.21.15.16:0/535846119 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f2ddc0328b0 con 0x7f2df4105fa0 2024-09-13T16:21:29.609 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.934+0000 7f2df37fe640 1 -- 172.21.15.16:0/535846119 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ddc030850 con 0x7f2df4105fa0 2024-09-13T16:21:29.609 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.935+0000 7f2dfab69640 1 -- 172.21.15.16:0/535846119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df4105fa0 msgr2=0x7f2df4108390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:29.609 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.935+0000 7f2dfab69640 1 --2- 172.21.15.16:0/535846119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df4105fa0 0x7f2df4108390 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc009820 tx=0x7f2ddc02a270 comp rx=0 tx=0).stop 2024-09-13T16:21:29.609 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.935+0000 7f2dfab69640 1 -- 172.21.15.16:0/535846119 shutdown_connections 2024-09-13T16:21:29.609 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.935+0000 7f2dfab69640 1 --2- 172.21.15.16:0/535846119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df4105fa0 0x7f2df4108390 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:29.609 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.935+0000 7f2dfab69640 1 -- 172.21.15.16:0/535846119 >> 172.21.15.16:0/535846119 conn(0x7f2df40ffbe0 msgr2=0x7f2df4102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:29.609 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.935+0000 7f2dfab69640 1 -- 172.21.15.16:0/535846119 shutdown_connections 2024-09-13T16:21:29.609 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.935+0000 7f2dfab69640 1 -- 172.21.15.16:0/535846119 wait complete. 2024-09-13T16:21:29.609 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.935+0000 7f2dfab69640 1 Processor -- start 2024-09-13T16:21:29.609 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.935+0000 7f2dfab69640 1 -- start start 2024-09-13T16:21:29.610 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.936+0000 7f2dfab69640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df4105fa0 0x7f2df419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:29.610 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.936+0000 7f2dfab69640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df419b8c0 con 0x7f2df4105fa0 2024-09-13T16:21:29.610 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.936+0000 7f2df8905640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df4105fa0 0x7f2df419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:29.610 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.936+0000 7f2df8905640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df4105fa0 0x7f2df419b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42416/0 (socket says 172.21.15.16:42416) 2024-09-13T16:21:29.610 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.936+0000 7f2df8905640 1 -- 172.21.15.16:0/273024950 learned_addr learned my addr 172.21.15.16:0/273024950 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:29.610 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.936+0000 7f2df8905640 1 -- 172.21.15.16:0/273024950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ddc0094d0 con 0x7f2df4105fa0 2024-09-13T16:21:29.610 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.936+0000 7f2df8905640 1 --2- 172.21.15.16:0/273024950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df4105fa0 0x7f2df419b380 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc030560 tx=0x7f2ddc030d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:29.610 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.936+0000 7f2df1ffb640 1 -- 172.21.15.16:0/273024950 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ddc032d60 con 0x7f2df4105fa0 2024-09-13T16:21:29.610 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.936+0000 7f2df1ffb640 1 -- 172.21.15.16:0/273024950 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f2ddc03b8a0 con 0x7f2df4105fa0 2024-09-13T16:21:29.610 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.936+0000 7f2dfab69640 1 -- 172.21.15.16:0/273024950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df419bac0 con 0x7f2df4105fa0 2024-09-13T16:21:29.610 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.936+0000 7f2df1ffb640 1 -- 172.21.15.16:0/273024950 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ddc03a740 con 0x7f2df4105fa0 2024-09-13T16:21:29.610 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.937+0000 7f2dfab69640 1 -- 172.21.15.16:0/273024950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df419bf60 con 0x7f2df4105fa0 2024-09-13T16:21:29.611 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.937+0000 7f2df1ffb640 1 -- 172.21.15.16:0/273024950 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f2ddc039040 con 0x7f2df4105fa0 2024-09-13T16:21:29.611 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.937+0000 7f2df1ffb640 1 -- 172.21.15.16:0/273024950 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2ddc044bc0 con 0x7f2df4105fa0 2024-09-13T16:21:29.611 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.938+0000 7f2dfab69640 1 -- 172.21.15.16:0/273024950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2df4109690 con 0x7f2df4105fa0 2024-09-13T16:21:29.611 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:28.940+0000 7f2df1ffb640 1 -- 172.21.15.16:0/273024950 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f2ddc03f020 con 0x7f2df4105fa0 2024-09-13T16:21:29.611 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:29.001+0000 7f2dfab69640 1 -- 172.21.15.16:0/273024950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f2df4109850 con 0x7f2df4105fa0 2024-09-13T16:21:29.611 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:29.001+0000 7f2df1ffb640 1 -- 172.21.15.16:0/273024950 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f2ddc037070 con 0x7f2df4105fa0 2024-09-13T16:21:29.611 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:29.002+0000 7f2dfab69640 1 -- 172.21.15.16:0/273024950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df4105fa0 msgr2=0x7f2df419b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:29.611 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:29.002+0000 7f2dfab69640 1 --2- 172.21.15.16:0/273024950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df4105fa0 0x7f2df419b380 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc030560 tx=0x7f2ddc030d90 comp rx=0 tx=0).stop 2024-09-13T16:21:29.611 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:29.002+0000 7f2dfab69640 1 -- 172.21.15.16:0/273024950 shutdown_connections 2024-09-13T16:21:29.611 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:29.002+0000 7f2dfab69640 1 --2- 172.21.15.16:0/273024950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df4105fa0 0x7f2df419b380 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:29.611 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:29.002+0000 7f2dfab69640 1 -- 172.21.15.16:0/273024950 >> 172.21.15.16:0/273024950 conn(0x7f2df40ffbe0 msgr2=0x7f2df41006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:29.611 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:29.002+0000 7f2dfab69640 1 -- 172.21.15.16:0/273024950 shutdown_connections 2024-09-13T16:21:29.612 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:29.002+0000 7f2dfab69640 1 -- 172.21.15.16:0/273024950 wait complete. 2024-09-13T16:21:29.612 INFO:teuthology.orchestra.run.smithi016.stdout:mgr not available, waiting (2/15)... 2024-09-13T16:21:30.354 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:30 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:30.073+0000 7f8390d24040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T16:21:30.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:30 smithi016 ceph-mon[31187]: Activating manager daemon x 2024-09-13T16:21:30.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:30 smithi016 ceph-mon[31187]: mgrmap e2: x(active, starting, since 0.0030467s) 2024-09-13T16:21:30.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:30 smithi016 ceph-mon[31187]: from='mgr.14100 172.21.15.16:0/4091042352' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-13T16:21:30.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:30 smithi016 ceph-mon[31187]: from='mgr.14100 172.21.15.16:0/4091042352' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-13T16:21:30.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:30 smithi016 ceph-mon[31187]: from='mgr.14100 172.21.15.16:0/4091042352' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-13T16:21:30.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:30 smithi016 ceph-mon[31187]: from='mgr.14100 172.21.15.16:0/4091042352' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T16:21:30.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:30 smithi016 ceph-mon[31187]: from='mgr.14100 172.21.15.16:0/4091042352' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-13T16:21:30.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:30 smithi016 ceph-mon[31187]: Manager daemon x is now available 2024-09-13T16:21:30.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:30 smithi016 ceph-mon[31187]: from='mgr.14100 172.21.15.16:0/4091042352' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-13T16:21:30.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:30 smithi016 ceph-mon[31187]: from='mgr.14100 172.21.15.16:0/4091042352' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-13T16:21:30.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:30 smithi016 ceph-mon[31187]: from='mgr.14100 172.21.15.16:0/4091042352' entity='mgr.x' 2024-09-13T16:21:30.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:30 smithi016 ceph-mon[31187]: from='mgr.14100 172.21.15.16:0/4091042352' entity='mgr.x' 2024-09-13T16:21:30.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:30 smithi016 ceph-mon[31187]: from='mgr.14100 172.21.15.16:0/4091042352' entity='mgr.x' 2024-09-13T16:21:32.229 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:32 smithi016 ceph-mon[31187]: mgrmap e3: x(active, since 1.00671s) 2024-09-13T16:21:32.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-13T16:21:32.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-13T16:21:32.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "0d241f6a-71ec-11ef-bceb-c7b262605968", 2024-09-13T16:21:32.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-09-13T16:21:32.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-13T16:21:32.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-13T16:21:32.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-13T16:21:32.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-13T16:21:32.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-13T16:21:32.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-13T16:21:32.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-09-13T16:21:32.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-13T16:21:32.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-13T16:21:32.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-09-13T16:21:32.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-13T16:21:32.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-13T16:21:32.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-13T16:21:32.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T16:21:32.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-13T16:21:32.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-13T16:21:32.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-13T16:21:32.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-13T16:21:32.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T16:21:32.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-13T16:21:32.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-13T16:21:32.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-13T16:21:32.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-13T16:21:32.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-13T16:21:32.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-13T16:21:32.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-13T16:21:32.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-13T16:21:32.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-13T16:21:32.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-13T16:21:32.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-13T16:21:32.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-13T16:21:32.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-13T16:21:32.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-13T16:21:32.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-13T16:21:32.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-13T16:21:32.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-13T16:21:32.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-13T16:21:32.873 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T16:21:32.873 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-13T16:21:32.873 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-13T16:21:32.873 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-13T16:21:32.873 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-13T16:21:32.873 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-13T16:21:32.873 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-13T16:21:32.873 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-13T16:21:32.873 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-13T16:21:32.873 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-13T16:21:32.873 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-09-13T16:21:32.873 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-13T16:21:32.873 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-13T16:21:32.873 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-13T16:21:32.873 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-13T16:21:32.874 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T16:21:32.874 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-09-13T16:21:17.972182+0000", 2024-09-13T16:21:32.874 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-13T16:21:32.874 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-13T16:21:32.874 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-13T16:21:32.874 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-13T16:21:32.874 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.976+0000 7f8bd0ffc640 1 Processor -- start 2024-09-13T16:21:32.874 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.976+0000 7f8bd0ffc640 1 -- start start 2024-09-13T16:21:32.874 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.976+0000 7f8bd0ffc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bcc105540 0x7f8bcc105940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:32.874 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.976+0000 7f8bd0ffc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc105f10 con 0x7f8bcc105540 2024-09-13T16:21:32.874 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.977+0000 7f8bca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bcc105540 0x7f8bcc105940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:32.874 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.977+0000 7f8bca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bcc105540 0x7f8bcc105940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42518/0 (socket says 172.21.15.16:42518) 2024-09-13T16:21:32.874 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.977+0000 7f8bca59c640 1 -- 172.21.15.16:0/2327686855 learned_addr learned my addr 172.21.15.16:0/2327686855 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:32.874 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.977+0000 7f8bca59c640 1 -- 172.21.15.16:0/2327686855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bcc106740 con 0x7f8bcc105540 2024-09-13T16:21:32.875 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.978+0000 7f8bca59c640 1 --2- 172.21.15.16:0/2327686855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bcc105540 0x7f8bcc105940 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009820 tx=0x7f8bbc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=561dc1df1ed08664 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:32.875 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.978+0000 7f8bc9d9b640 1 -- 172.21.15.16:0/2327686855 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8bbc02ae70 con 0x7f8bcc105540 2024-09-13T16:21:32.875 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.978+0000 7f8bc9d9b640 1 -- 172.21.15.16:0/2327686855 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f8bbc0328b0 con 0x7f8bcc105540 2024-09-13T16:21:32.875 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.978+0000 7f8bc9d9b640 1 -- 172.21.15.16:0/2327686855 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8bbc030850 con 0x7f8bcc105540 2024-09-13T16:21:32.875 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.978+0000 7f8bd0ffc640 1 -- 172.21.15.16:0/2327686855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bcc105540 msgr2=0x7f8bcc105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:32.875 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.978+0000 7f8bd0ffc640 1 --2- 172.21.15.16:0/2327686855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bcc105540 0x7f8bcc105940 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009820 tx=0x7f8bbc02a270 comp rx=0 tx=0).stop 2024-09-13T16:21:32.875 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.978+0000 7f8bd0ffc640 1 -- 172.21.15.16:0/2327686855 shutdown_connections 2024-09-13T16:21:32.875 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.978+0000 7f8bd0ffc640 1 --2- 172.21.15.16:0/2327686855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bcc105540 0x7f8bcc105940 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:32.875 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.978+0000 7f8bd0ffc640 1 -- 172.21.15.16:0/2327686855 >> 172.21.15.16:0/2327686855 conn(0x7f8bcc100cf0 msgr2=0x7f8bcc103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:32.875 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.979+0000 7f8bd0ffc640 1 -- 172.21.15.16:0/2327686855 shutdown_connections 2024-09-13T16:21:32.875 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.979+0000 7f8bd0ffc640 1 -- 172.21.15.16:0/2327686855 wait complete. 2024-09-13T16:21:32.875 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.979+0000 7f8bd0ffc640 1 Processor -- start 2024-09-13T16:21:32.875 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.979+0000 7f8bd0ffc640 1 -- start start 2024-09-13T16:21:32.875 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.980+0000 7f8bd0ffc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bcc105540 0x7f8bcc199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:32.876 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.980+0000 7f8bd0ffc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc1998a0 con 0x7f8bcc105540 2024-09-13T16:21:32.876 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.980+0000 7f8bca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bcc105540 0x7f8bcc199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:32.876 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.980+0000 7f8bca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bcc105540 0x7f8bcc199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42520/0 (socket says 172.21.15.16:42520) 2024-09-13T16:21:32.876 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.980+0000 7f8bca59c640 1 -- 172.21.15.16:0/3666482070 learned_addr learned my addr 172.21.15.16:0/3666482070 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:32.876 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.980+0000 7f8bca59c640 1 -- 172.21.15.16:0/3666482070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bbc0094d0 con 0x7f8bcc105540 2024-09-13T16:21:32.876 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.980+0000 7f8bca59c640 1 --2- 172.21.15.16:0/3666482070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bcc105540 0x7f8bcc199360 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc02a790 tx=0x7f8bbc030db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:32.876 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.980+0000 7f8bc37fe640 1 -- 172.21.15.16:0/3666482070 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8bbc032d60 con 0x7f8bcc105540 2024-09-13T16:21:32.876 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.980+0000 7f8bc37fe640 1 -- 172.21.15.16:0/3666482070 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f8bbc03b8a0 con 0x7f8bcc105540 2024-09-13T16:21:32.876 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.980+0000 7f8bd0ffc640 1 -- 172.21.15.16:0/3666482070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bcc1075c0 con 0x7f8bcc105540 2024-09-13T16:21:32.876 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.980+0000 7f8bc37fe640 1 -- 172.21.15.16:0/3666482070 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8bbc03a640 con 0x7f8bcc105540 2024-09-13T16:21:32.876 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.980+0000 7f8bd0ffc640 1 -- 172.21.15.16:0/3666482070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bcc199db0 con 0x7f8bcc105540 2024-09-13T16:21:32.876 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.981+0000 7f8bc37fe640 1 -- 172.21.15.16:0/3666482070 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 3) v1 ==== 45424+0+0 (secure 0 0 0) 0x7f8bbc039040 con 0x7f8bcc105540 2024-09-13T16:21:32.876 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.981+0000 7f8bc37fe640 1 --2- 172.21.15.16:0/3666482070 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7f8bb8038500 0x7f8bb803a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:32.877 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.981+0000 7f8bd0ffc640 1 -- 172.21.15.16:0/3666482070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bcc1059c0 con 0x7f8bcc105540 2024-09-13T16:21:32.877 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.982+0000 7f8bc37fe640 1 -- 172.21.15.16:0/3666482070 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8bbc06b2f0 con 0x7f8bcc105540 2024-09-13T16:21:32.877 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.982+0000 7f8bc3fff640 1 --2- 172.21.15.16:0/3666482070 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7f8bb8038500 0x7f8bb803a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:32.877 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.982+0000 7f8bc3fff640 1 --2- 172.21.15.16:0/3666482070 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7f8bb8038500 0x7f8bb803a9c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40098c0 tx=0x7f8bb4006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:32.877 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:31.987+0000 7f8bc37fe640 1 -- 172.21.15.16:0/3666482070 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f8bbc03f080 con 0x7f8bcc105540 2024-09-13T16:21:32.877 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:32.082+0000 7f8bc37fe640 1 -- 172.21.15.16:0/3666482070 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 4) v1 ==== 45530+0+0 (secure 0 0 0) 0x7f8bbc03ba10 con 0x7f8bcc105540 2024-09-13T16:21:32.877 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:32.227+0000 7f8bd0ffc640 1 -- 172.21.15.16:0/3666482070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f8bcc109970 con 0x7f8bcc105540 2024-09-13T16:21:32.877 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:32.228+0000 7f8bc37fe640 1 -- 172.21.15.16:0/3666482070 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1237 (secure 0 0 0) 0x7f8bbc04b710 con 0x7f8bcc105540 2024-09-13T16:21:32.877 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:32.230+0000 7f8bd0ffc640 1 -- 172.21.15.16:0/3666482070 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7f8bb8038500 msgr2=0x7f8bb803a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:32.877 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:32.230+0000 7f8bd0ffc640 1 --2- 172.21.15.16:0/3666482070 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7f8bb8038500 0x7f8bb803a9c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40098c0 tx=0x7f8bb4006db0 comp rx=0 tx=0).stop 2024-09-13T16:21:32.877 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:32.230+0000 7f8bd0ffc640 1 -- 172.21.15.16:0/3666482070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bcc105540 msgr2=0x7f8bcc199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:32.877 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:32.230+0000 7f8bd0ffc640 1 --2- 172.21.15.16:0/3666482070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bcc105540 0x7f8bcc199360 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc02a790 tx=0x7f8bbc030db0 comp rx=0 tx=0).stop 2024-09-13T16:21:32.877 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:32.230+0000 7f8bd0ffc640 1 -- 172.21.15.16:0/3666482070 shutdown_connections 2024-09-13T16:21:32.878 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:32.230+0000 7f8bd0ffc640 1 --2- 172.21.15.16:0/3666482070 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7f8bb8038500 0x7f8bb803a9c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:32.878 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:32.230+0000 7f8bd0ffc640 1 --2- 172.21.15.16:0/3666482070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bcc105540 0x7f8bcc199360 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:32.878 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:32.230+0000 7f8bd0ffc640 1 -- 172.21.15.16:0/3666482070 >> 172.21.15.16:0/3666482070 conn(0x7f8bcc100cf0 msgr2=0x7f8bcc1017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:32.878 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:32.230+0000 7f8bd0ffc640 1 -- 172.21.15.16:0/3666482070 shutdown_connections 2024-09-13T16:21:32.878 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:32.230+0000 7f8bd0ffc640 1 -- 172.21.15.16:0/3666482070 wait complete. 2024-09-13T16:21:32.878 INFO:teuthology.orchestra.run.smithi016.stdout:mgr is available 2024-09-13T16:21:33.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:33 smithi016 ceph-mon[31187]: mgrmap e4: x(active, since 2s) 2024-09-13T16:21:33.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:33 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3666482070' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-13T16:21:34.112 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-13T16:21:34.112 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [global] 2024-09-13T16:21:34.113 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout fsid = 0d241f6a-71ec-11ef-bceb-c7b262605968 2024-09-13T16:21:34.113 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-13T16:21:34.113 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-13T16:21:34.113 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-13T16:21:34.113 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-13T16:21:34.113 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-13T16:21:34.113 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-13T16:21:34.113 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-13T16:21:34.113 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-13T16:21:34.113 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [osd] 2024-09-13T16:21:34.113 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-13T16:21:34.113 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-13T16:21:34.113 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-13T16:21:34.113 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-13T16:21:34.114 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-13T16:21:34.114 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.266+0000 7eff7e497640 1 Processor -- start 2024-09-13T16:21:34.114 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.267+0000 7eff7e497640 1 -- start start 2024-09-13T16:21:34.114 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.267+0000 7eff7e497640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff78105540 0x7eff78105940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:34.114 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.267+0000 7eff7e497640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff78105f10 con 0x7eff78105540 2024-09-13T16:21:34.114 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.267+0000 7eff77fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff78105540 0x7eff78105940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:34.114 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.267+0000 7eff77fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff78105540 0x7eff78105940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42536/0 (socket says 172.21.15.16:42536) 2024-09-13T16:21:34.114 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.267+0000 7eff77fff640 1 -- 172.21.15.16:0/371785937 learned_addr learned my addr 172.21.15.16:0/371785937 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:34.114 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.267+0000 7eff77fff640 1 -- 172.21.15.16:0/371785937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff78106740 con 0x7eff78105540 2024-09-13T16:21:34.114 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.268+0000 7eff77fff640 1 --2- 172.21.15.16:0/371785937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff78105540 0x7eff78105940 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7eff68009820 tx=0x7eff6802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31b441e31bd2eae6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:34.115 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.268+0000 7eff76ffd640 1 -- 172.21.15.16:0/371785937 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff6802ae70 con 0x7eff78105540 2024-09-13T16:21:34.115 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.268+0000 7eff76ffd640 1 -- 172.21.15.16:0/371785937 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7eff680328b0 con 0x7eff78105540 2024-09-13T16:21:34.115 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.268+0000 7eff76ffd640 1 -- 172.21.15.16:0/371785937 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff68030850 con 0x7eff78105540 2024-09-13T16:21:34.115 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.269+0000 7eff7e497640 1 -- 172.21.15.16:0/371785937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff78105540 msgr2=0x7eff78105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:34.115 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.269+0000 7eff7e497640 1 --2- 172.21.15.16:0/371785937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff78105540 0x7eff78105940 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7eff68009820 tx=0x7eff6802a270 comp rx=0 tx=0).stop 2024-09-13T16:21:34.115 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.269+0000 7eff7e497640 1 -- 172.21.15.16:0/371785937 shutdown_connections 2024-09-13T16:21:34.115 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.269+0000 7eff7e497640 1 --2- 172.21.15.16:0/371785937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff78105540 0x7eff78105940 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:34.115 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.269+0000 7eff7e497640 1 -- 172.21.15.16:0/371785937 >> 172.21.15.16:0/371785937 conn(0x7eff78100cf0 msgr2=0x7eff78103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:34.115 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.269+0000 7eff7e497640 1 -- 172.21.15.16:0/371785937 shutdown_connections 2024-09-13T16:21:34.115 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.269+0000 7eff7e497640 1 -- 172.21.15.16:0/371785937 wait complete. 2024-09-13T16:21:34.115 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.269+0000 7eff7e497640 1 Processor -- start 2024-09-13T16:21:34.115 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.270+0000 7eff7e497640 1 -- start start 2024-09-13T16:21:34.115 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.270+0000 7eff7e497640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff78105540 0x7eff78199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:34.115 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.270+0000 7eff7e497640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff781998a0 con 0x7eff78105540 2024-09-13T16:21:34.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.270+0000 7eff77fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff78105540 0x7eff78199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:34.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.270+0000 7eff77fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff78105540 0x7eff78199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42542/0 (socket says 172.21.15.16:42542) 2024-09-13T16:21:34.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.270+0000 7eff77fff640 1 -- 172.21.15.16:0/913174622 learned_addr learned my addr 172.21.15.16:0/913174622 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:34.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.270+0000 7eff77fff640 1 -- 172.21.15.16:0/913174622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff680094d0 con 0x7eff78105540 2024-09-13T16:21:34.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.270+0000 7eff77fff640 1 --2- 172.21.15.16:0/913174622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff78105540 0x7eff78199360 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7eff6802a790 tx=0x7eff68030db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:34.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.271+0000 7eff757fa640 1 -- 172.21.15.16:0/913174622 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff68032d60 con 0x7eff78105540 2024-09-13T16:21:34.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.271+0000 7eff757fa640 1 -- 172.21.15.16:0/913174622 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7eff6803b8a0 con 0x7eff78105540 2024-09-13T16:21:34.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.271+0000 7eff7e497640 1 -- 172.21.15.16:0/913174622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff781075c0 con 0x7eff78105540 2024-09-13T16:21:34.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.271+0000 7eff757fa640 1 -- 172.21.15.16:0/913174622 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff6803a640 con 0x7eff78105540 2024-09-13T16:21:34.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.271+0000 7eff7e497640 1 -- 172.21.15.16:0/913174622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff78199db0 con 0x7eff78105540 2024-09-13T16:21:34.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.272+0000 7eff757fa640 1 -- 172.21.15.16:0/913174622 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 45530+0+0 (secure 0 0 0) 0x7eff68039040 con 0x7eff78105540 2024-09-13T16:21:34.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.272+0000 7eff757fa640 1 --2- 172.21.15.16:0/913174622 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7eff60038620 0x7eff6003aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:34.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.272+0000 7eff7e497640 1 -- 172.21.15.16:0/913174622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff781059c0 con 0x7eff78105540 2024-09-13T16:21:34.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.272+0000 7eff757fa640 1 -- 172.21.15.16:0/913174622 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7eff6806b510 con 0x7eff78105540 2024-09-13T16:21:34.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.272+0000 7eff777fe640 1 --2- 172.21.15.16:0/913174622 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7eff60038620 0x7eff6003aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:34.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.273+0000 7eff777fe640 1 --2- 172.21.15.16:0/913174622 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7eff60038620 0x7eff6003aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7eff6c0098c0 tx=0x7eff6c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:34.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.277+0000 7eff757fa640 1 -- 172.21.15.16:0/913174622 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7eff6803b430 con 0x7eff78105540 2024-09-13T16:21:34.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.449+0000 7eff7e497640 1 -- 172.21.15.16:0/913174622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7eff78109970 con 0x7eff78105540 2024-09-13T16:21:34.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.450+0000 7eff757fa640 1 -- 172.21.15.16:0/913174622 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7eff6803f030 con 0x7eff78105540 2024-09-13T16:21:34.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.452+0000 7eff7e497640 1 -- 172.21.15.16:0/913174622 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7eff60038620 msgr2=0x7eff6003aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:34.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.452+0000 7eff7e497640 1 --2- 172.21.15.16:0/913174622 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7eff60038620 0x7eff6003aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7eff6c0098c0 tx=0x7eff6c006db0 comp rx=0 tx=0).stop 2024-09-13T16:21:34.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.452+0000 7eff7e497640 1 -- 172.21.15.16:0/913174622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff78105540 msgr2=0x7eff78199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:34.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.452+0000 7eff7e497640 1 --2- 172.21.15.16:0/913174622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff78105540 0x7eff78199360 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7eff6802a790 tx=0x7eff68030db0 comp rx=0 tx=0).stop 2024-09-13T16:21:34.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.452+0000 7eff7e497640 1 -- 172.21.15.16:0/913174622 shutdown_connections 2024-09-13T16:21:34.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.453+0000 7eff7e497640 1 --2- 172.21.15.16:0/913174622 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7eff60038620 0x7eff6003aae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:34.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.453+0000 7eff7e497640 1 --2- 172.21.15.16:0/913174622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff78105540 0x7eff78199360 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:34.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.453+0000 7eff7e497640 1 -- 172.21.15.16:0/913174622 >> 172.21.15.16:0/913174622 conn(0x7eff78100cf0 msgr2=0x7eff781017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:34.118 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.453+0000 7eff7e497640 1 -- 172.21.15.16:0/913174622 shutdown_connections 2024-09-13T16:21:34.118 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:33.453+0000 7eff7e497640 1 -- 172.21.15.16:0/913174622 wait complete. 2024-09-13T16:21:34.118 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling cephadm module... 2024-09-13T16:21:34.246 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:34 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/913174622' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-13T16:21:35.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:35 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/4279894501' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-13T16:21:35.354 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:35 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: ignoring --setuser ceph since I am not root 2024-09-13T16:21:35.354 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:35 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: ignoring --setgroup ceph since I am not root 2024-09-13T16:21:35.355 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:35 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:35.226+0000 7f8a53ddc040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T16:21:35.355 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:35 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:35.285+0000 7f8a53ddc040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T16:21:35.660 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:35 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:35.442+0000 7f8a53ddc040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T16:21:35.660 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:35 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:35.511+0000 7f8a53ddc040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T16:21:35.667 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.521+0000 7fa9e1e9b640 1 Processor -- start 2024-09-13T16:21:35.667 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.521+0000 7fa9e1e9b640 1 -- start start 2024-09-13T16:21:35.667 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.521+0000 7fa9e1e9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9dc103d40 0x7fa9dc104140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:35.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.521+0000 7fa9e1e9b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc079de0 con 0x7fa9dc103d40 2024-09-13T16:21:35.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.521+0000 7fa9db7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9dc103d40 0x7fa9dc104140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:35.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.521+0000 7fa9db7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9dc103d40 0x7fa9dc104140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59472/0 (socket says 172.21.15.16:59472) 2024-09-13T16:21:35.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.521+0000 7fa9db7fe640 1 -- 172.21.15.16:0/225585610 learned_addr learned my addr 172.21.15.16:0/225585610 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:35.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.521+0000 7fa9db7fe640 1 -- 172.21.15.16:0/225585610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9dc104680 con 0x7fa9dc103d40 2024-09-13T16:21:35.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.522+0000 7fa9db7fe640 1 --2- 172.21.15.16:0/225585610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9dc103d40 0x7fa9dc104140 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc009820 tx=0x7fa9cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d48cdad184d15e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:35.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.522+0000 7fa9da7fc640 1 -- 172.21.15.16:0/225585610 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9cc02ae70 con 0x7fa9dc103d40 2024-09-13T16:21:35.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.522+0000 7fa9da7fc640 1 -- 172.21.15.16:0/225585610 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fa9cc0328b0 con 0x7fa9dc103d40 2024-09-13T16:21:35.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.522+0000 7fa9da7fc640 1 -- 172.21.15.16:0/225585610 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9cc030850 con 0x7fa9dc103d40 2024-09-13T16:21:35.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.523+0000 7fa9e1e9b640 1 -- 172.21.15.16:0/225585610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9dc103d40 msgr2=0x7fa9dc104140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:35.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.523+0000 7fa9e1e9b640 1 --2- 172.21.15.16:0/225585610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9dc103d40 0x7fa9dc104140 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc009820 tx=0x7fa9cc02a270 comp rx=0 tx=0).stop 2024-09-13T16:21:35.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.523+0000 7fa9e1e9b640 1 -- 172.21.15.16:0/225585610 shutdown_connections 2024-09-13T16:21:35.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.523+0000 7fa9e1e9b640 1 --2- 172.21.15.16:0/225585610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9dc103d40 0x7fa9dc104140 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:35.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.523+0000 7fa9e1e9b640 1 -- 172.21.15.16:0/225585610 >> 172.21.15.16:0/225585610 conn(0x7fa9dc0ffb50 msgr2=0x7fa9dc101f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:35.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.523+0000 7fa9e1e9b640 1 -- 172.21.15.16:0/225585610 shutdown_connections 2024-09-13T16:21:35.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.523+0000 7fa9e1e9b640 1 -- 172.21.15.16:0/225585610 wait complete. 2024-09-13T16:21:35.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.523+0000 7fa9e1e9b640 1 Processor -- start 2024-09-13T16:21:35.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.524+0000 7fa9e1e9b640 1 -- start start 2024-09-13T16:21:35.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.524+0000 7fa9e1e9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9dc103d40 0x7fa9dc19b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:35.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.524+0000 7fa9e1e9b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc19b840 con 0x7fa9dc103d40 2024-09-13T16:21:35.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.524+0000 7fa9db7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9dc103d40 0x7fa9dc19b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:35.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.524+0000 7fa9db7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9dc103d40 0x7fa9dc19b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59484/0 (socket says 172.21.15.16:59484) 2024-09-13T16:21:35.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.524+0000 7fa9db7fe640 1 -- 172.21.15.16:0/4279894501 learned_addr learned my addr 172.21.15.16:0/4279894501 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:35.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.524+0000 7fa9db7fe640 1 -- 172.21.15.16:0/4279894501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9cc0094d0 con 0x7fa9dc103d40 2024-09-13T16:21:35.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.524+0000 7fa9db7fe640 1 --2- 172.21.15.16:0/4279894501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9dc103d40 0x7fa9dc19b300 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc030560 tx=0x7fa9cc030d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:35.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.525+0000 7fa9d8ff9640 1 -- 172.21.15.16:0/4279894501 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9cc032d60 con 0x7fa9dc103d40 2024-09-13T16:21:35.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.525+0000 7fa9d8ff9640 1 -- 172.21.15.16:0/4279894501 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fa9cc03b8a0 con 0x7fa9dc103d40 2024-09-13T16:21:35.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.525+0000 7fa9d8ff9640 1 -- 172.21.15.16:0/4279894501 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9cc03a740 con 0x7fa9dc103d40 2024-09-13T16:21:35.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.525+0000 7fa9e1e9b640 1 -- 172.21.15.16:0/4279894501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9dc19ba40 con 0x7fa9dc103d40 2024-09-13T16:21:35.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.525+0000 7fa9e1e9b640 1 -- 172.21.15.16:0/4279894501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9dc19bee0 con 0x7fa9dc103d40 2024-09-13T16:21:35.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.526+0000 7fa9d8ff9640 1 -- 172.21.15.16:0/4279894501 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 45530+0+0 (secure 0 0 0) 0x7fa9cc039040 con 0x7fa9dc103d40 2024-09-13T16:21:35.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.526+0000 7fa9e1e9b640 1 -- 172.21.15.16:0/4279894501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9dc195030 con 0x7fa9dc103d40 2024-09-13T16:21:35.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.526+0000 7fa9d8ff9640 1 --2- 172.21.15.16:0/4279894501 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7fa9c4038620 0x7fa9c403aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:35.670 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.526+0000 7fa9d8ff9640 1 -- 172.21.15.16:0/4279894501 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa9cc06b590 con 0x7fa9dc103d40 2024-09-13T16:21:35.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.526+0000 7fa9daffd640 1 --2- 172.21.15.16:0/4279894501 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7fa9c4038620 0x7fa9c403aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:35.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.526+0000 7fa9daffd640 1 --2- 172.21.15.16:0/4279894501 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7fa9c4038620 0x7fa9c403aae0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00098c0 tx=0x7fa9d0006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:35.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.531+0000 7fa9d8ff9640 1 -- 172.21.15.16:0/4279894501 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa9cc044400 con 0x7fa9dc103d40 2024-09-13T16:21:35.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:34.749+0000 7fa9e1e9b640 1 -- 172.21.15.16:0/4279894501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fa9dc105720 con 0x7fa9dc103d40 2024-09-13T16:21:35.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:35.086+0000 7fa9d8ff9640 1 -- 172.21.15.16:0/4279894501 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fa9cc03ba10 con 0x7fa9dc103d40 2024-09-13T16:21:35.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:35.088+0000 7fa9d8ff9640 1 -- 172.21.15.16:0/4279894501 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fa9cc06b2f0 con 0x7fa9dc103d40 2024-09-13T16:21:35.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:35.090+0000 7fa9e1e9b640 1 -- 172.21.15.16:0/4279894501 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7fa9c4038620 msgr2=0x7fa9c403aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:35.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:35.090+0000 7fa9e1e9b640 1 --2- 172.21.15.16:0/4279894501 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7fa9c4038620 0x7fa9c403aae0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00098c0 tx=0x7fa9d0006db0 comp rx=0 tx=0).stop 2024-09-13T16:21:35.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:35.090+0000 7fa9e1e9b640 1 -- 172.21.15.16:0/4279894501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9dc103d40 msgr2=0x7fa9dc19b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:35.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:35.090+0000 7fa9e1e9b640 1 --2- 172.21.15.16:0/4279894501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9dc103d40 0x7fa9dc19b300 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc030560 tx=0x7fa9cc030d90 comp rx=0 tx=0).stop 2024-09-13T16:21:35.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:35.090+0000 7fa9e1e9b640 1 -- 172.21.15.16:0/4279894501 shutdown_connections 2024-09-13T16:21:35.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:35.090+0000 7fa9e1e9b640 1 --2- 172.21.15.16:0/4279894501 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7fa9c4038620 0x7fa9c403aae0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:35.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:35.090+0000 7fa9e1e9b640 1 --2- 172.21.15.16:0/4279894501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9dc103d40 0x7fa9dc19b300 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:35.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:35.090+0000 7fa9e1e9b640 1 -- 172.21.15.16:0/4279894501 >> 172.21.15.16:0/4279894501 conn(0x7fa9dc0ffb50 msgr2=0x7fa9dc192200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:35.672 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:35.090+0000 7fa9e1e9b640 1 -- 172.21.15.16:0/4279894501 shutdown_connections 2024-09-13T16:21:35.672 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:35.090+0000 7fa9e1e9b640 1 -- 172.21.15.16:0/4279894501 wait complete. 2024-09-13T16:21:35.918 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:35 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:35.663+0000 7f8a53ddc040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T16:21:35.918 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:35 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:35.814+0000 7f8a53ddc040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T16:21:36.209 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:36 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/4279894501' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-13T16:21:36.245 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:36 smithi016 ceph-mon[31187]: mgrmap e5: x(active, since 5s) 2024-09-13T16:21:36.245 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:36 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:36.145+0000 7f8a53ddc040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T16:21:36.245 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:36 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:36.208+0000 7f8a53ddc040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T16:21:36.604 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:36 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:36.403+0000 7f8a53ddc040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T16:21:36.907 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-13T16:21:36.907 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-13T16:21:36.908 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-13T16:21:36.908 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-13T16:21:36.908 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-13T16:21:36.908 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-13T16:21:36.908 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.098+0000 7fb3fe876640 1 Processor -- start 2024-09-13T16:21:36.908 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.099+0000 7fb3fe876640 1 -- start start 2024-09-13T16:21:36.908 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.099+0000 7fb3fe876640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f8105fa0 0x7fb3f8108390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:36.908 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.099+0000 7fb3fe876640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f8079e40 con 0x7fb3f8105fa0 2024-09-13T16:21:36.908 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.099+0000 7fb3f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f8105fa0 0x7fb3f8108390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:36.908 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.099+0000 7fb3f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f8105fa0 0x7fb3f8108390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59516/0 (socket says 172.21.15.16:59516) 2024-09-13T16:21:36.908 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.099+0000 7fb3f7fff640 1 -- 172.21.15.16:0/433263106 learned_addr learned my addr 172.21.15.16:0/433263106 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:36.908 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.099+0000 7fb3f7fff640 1 -- 172.21.15.16:0/433263106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f81088d0 con 0x7fb3f8105fa0 2024-09-13T16:21:36.908 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.100+0000 7fb3f7fff640 1 --2- 172.21.15.16:0/433263106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f8105fa0 0x7fb3f8108390 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8009820 tx=0x7fb3e802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f687319471c30a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:36.909 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.100+0000 7fb3f77fe640 1 -- 172.21.15.16:0/433263106 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3e802ae70 con 0x7fb3f8105fa0 2024-09-13T16:21:36.909 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.100+0000 7fb3f77fe640 1 -- 172.21.15.16:0/433263106 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fb3e80328b0 con 0x7fb3f8105fa0 2024-09-13T16:21:36.909 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.100+0000 7fb3f77fe640 1 -- 172.21.15.16:0/433263106 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3e8030850 con 0x7fb3f8105fa0 2024-09-13T16:21:36.909 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.101+0000 7fb3fe876640 1 -- 172.21.15.16:0/433263106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f8105fa0 msgr2=0x7fb3f8108390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:36.909 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.101+0000 7fb3fe876640 1 --2- 172.21.15.16:0/433263106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f8105fa0 0x7fb3f8108390 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8009820 tx=0x7fb3e802a270 comp rx=0 tx=0).stop 2024-09-13T16:21:36.909 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.101+0000 7fb3fe876640 1 -- 172.21.15.16:0/433263106 shutdown_connections 2024-09-13T16:21:36.909 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.101+0000 7fb3fe876640 1 --2- 172.21.15.16:0/433263106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f8105fa0 0x7fb3f8108390 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:36.909 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.101+0000 7fb3fe876640 1 -- 172.21.15.16:0/433263106 >> 172.21.15.16:0/433263106 conn(0x7fb3f80ffbe0 msgr2=0x7fb3f8102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:36.909 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.101+0000 7fb3fe876640 1 -- 172.21.15.16:0/433263106 shutdown_connections 2024-09-13T16:21:36.909 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.101+0000 7fb3fe876640 1 -- 172.21.15.16:0/433263106 wait complete. 2024-09-13T16:21:36.909 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.102+0000 7fb3fe876640 1 Processor -- start 2024-09-13T16:21:36.909 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.102+0000 7fb3fe876640 1 -- start start 2024-09-13T16:21:36.909 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.102+0000 7fb3fe876640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f8105fa0 0x7fb3f8105820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:36.909 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.102+0000 7fb3fe876640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f8105d60 con 0x7fb3f8105fa0 2024-09-13T16:21:36.910 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.102+0000 7fb3f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f8105fa0 0x7fb3f8105820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:36.910 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.102+0000 7fb3f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f8105fa0 0x7fb3f8105820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59518/0 (socket says 172.21.15.16:59518) 2024-09-13T16:21:36.910 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.102+0000 7fb3f7fff640 1 -- 172.21.15.16:0/2632549371 learned_addr learned my addr 172.21.15.16:0/2632549371 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:36.910 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.102+0000 7fb3f7fff640 1 -- 172.21.15.16:0/2632549371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e80094d0 con 0x7fb3f8105fa0 2024-09-13T16:21:36.910 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.102+0000 7fb3f7fff640 1 --2- 172.21.15.16:0/2632549371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f8105fa0 0x7fb3f8105820 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8030560 tx=0x7fb3e8030d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:36.910 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.103+0000 7fb3f5ffb640 1 -- 172.21.15.16:0/2632549371 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3e8032d60 con 0x7fb3f8105fa0 2024-09-13T16:21:36.910 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.103+0000 7fb3f5ffb640 1 -- 172.21.15.16:0/2632549371 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fb3e803b8a0 con 0x7fb3f8105fa0 2024-09-13T16:21:36.910 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.103+0000 7fb3fe876640 1 -- 172.21.15.16:0/2632549371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f8103f50 con 0x7fb3f8105fa0 2024-09-13T16:21:36.910 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.103+0000 7fb3f5ffb640 1 -- 172.21.15.16:0/2632549371 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3e803a740 con 0x7fb3f8105fa0 2024-09-13T16:21:36.910 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.103+0000 7fb3fe876640 1 -- 172.21.15.16:0/2632549371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f81043f0 con 0x7fb3f8105fa0 2024-09-13T16:21:36.910 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.104+0000 7fb3f5ffb640 1 -- 172.21.15.16:0/2632549371 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fb3e8039040 con 0x7fb3f8105fa0 2024-09-13T16:21:36.910 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.104+0000 7fb3fe876640 1 -- 172.21.15.16:0/2632549371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f8106400 con 0x7fb3f8105fa0 2024-09-13T16:21:36.911 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.104+0000 7fb3f5ffb640 1 --2- 172.21.15.16:0/2632549371 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7fb3e4038620 0x7fb3e403aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:36.911 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.104+0000 7fb3f5ffb640 1 -- 172.21.15.16:0/2632549371 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb3e806b8a0 con 0x7fb3f8105fa0 2024-09-13T16:21:36.911 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.104+0000 7fb3effff640 1 -- 172.21.15.16:0/2632549371 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7fb3e4038620 msgr2=0x7fb3e403aae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1147765163 2024-09-13T16:21:36.911 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.104+0000 7fb3effff640 1 --2- 172.21.15.16:0/2632549371 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7fb3e4038620 0x7fb3e403aae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T16:21:36.911 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.109+0000 7fb3f5ffb640 1 -- 172.21.15.16:0/2632549371 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb3e804b6c0 con 0x7fb3f8105fa0 2024-09-13T16:21:36.911 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.305+0000 7fb3effff640 1 -- 172.21.15.16:0/2632549371 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7fb3e4038620 msgr2=0x7fb3e403aae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1147765163 2024-09-13T16:21:36.911 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.305+0000 7fb3effff640 1 --2- 172.21.15.16:0/2632549371 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7fb3e4038620 0x7fb3e403aae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T16:21:36.911 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.327+0000 7fb3fe876640 1 -- 172.21.15.16:0/2632549371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fb3f8109690 con 0x7fb3f8105fa0 2024-09-13T16:21:36.911 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.327+0000 7fb3f5ffb640 1 -- 172.21.15.16:0/2632549371 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7fb3e8037070 con 0x7fb3f8105fa0 2024-09-13T16:21:36.911 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.329+0000 7fb3fe876640 1 -- 172.21.15.16:0/2632549371 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7fb3e4038620 msgr2=0x7fb3e403aae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T16:21:36.911 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.329+0000 7fb3fe876640 1 --2- 172.21.15.16:0/2632549371 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7fb3e4038620 0x7fb3e403aae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:36.911 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.329+0000 7fb3fe876640 1 -- 172.21.15.16:0/2632549371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f8105fa0 msgr2=0x7fb3f8105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:36.911 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.329+0000 7fb3fe876640 1 --2- 172.21.15.16:0/2632549371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f8105fa0 0x7fb3f8105820 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8030560 tx=0x7fb3e8030d90 comp rx=0 tx=0).stop 2024-09-13T16:21:36.911 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.329+0000 7fb3fe876640 1 -- 172.21.15.16:0/2632549371 shutdown_connections 2024-09-13T16:21:36.912 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.329+0000 7fb3fe876640 1 --2- 172.21.15.16:0/2632549371 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7fb3e4038620 0x7fb3e403aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:36.912 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.329+0000 7fb3fe876640 1 --2- 172.21.15.16:0/2632549371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3f8105fa0 0x7fb3f8105820 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:36.912 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.330+0000 7fb3fe876640 1 -- 172.21.15.16:0/2632549371 >> 172.21.15.16:0/2632549371 conn(0x7fb3f80ffbe0 msgr2=0x7fb3f8100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:36.912 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.330+0000 7fb3fe876640 1 -- 172.21.15.16:0/2632549371 shutdown_connections 2024-09-13T16:21:36.912 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:36.330+0000 7fb3fe876640 1 -- 172.21.15.16:0/2632549371 wait complete. 2024-09-13T16:21:36.912 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for the mgr to restart... 2024-09-13T16:21:36.912 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr epoch 5... 2024-09-13T16:21:37.173 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:37 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2632549371' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-13T16:21:37.436 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:37 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:37.366+0000 7f8a53ddc040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T16:21:37.436 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:37 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:37.435+0000 7f8a53ddc040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T16:21:38.453 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:38 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:38.184+0000 7f8a53ddc040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-13T16:21:38.453 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:38 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:38.303+0000 7f8a53ddc040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T16:21:38.708 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:38 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:38.451+0000 7f8a53ddc040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T16:21:38.708 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:38 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:38.515+0000 7f8a53ddc040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T16:21:38.708 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:38 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:38.669+0000 7f8a53ddc040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T16:21:38.966 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:38 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:38.739+0000 7f8a53ddc040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T16:21:38.966 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:38 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:38.802+0000 7f8a53ddc040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T16:21:38.966 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:38 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:38.965+0000 7f8a53ddc040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T16:21:39.250 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:39 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:39.032+0000 7f8a53ddc040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T16:21:39.251 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:39 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:39.101+0000 7f8a53ddc040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T16:21:39.251 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:39 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:39.188+0000 7f8a53ddc040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T16:21:39.251 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:39 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:39.249+0000 7f8a53ddc040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T16:21:39.604 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:39 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:39.310+0000 7f8a53ddc040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T16:21:40.604 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:40 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:40.124+0000 7f8a53ddc040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T16:21:40.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:40 smithi016 ceph-mon[31187]: Active manager daemon x restarted 2024-09-13T16:21:40.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:40 smithi016 ceph-mon[31187]: Activating manager daemon x 2024-09-13T16:21:40.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:40 smithi016 ceph-mon[31187]: osdmap e2: 0 total, 0 up, 0 in 2024-09-13T16:21:40.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:40 smithi016 ceph-mon[31187]: mgrmap e6: x(active, starting, since 0.00354194s) 2024-09-13T16:21:40.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:40 smithi016 ceph-mon[31187]: from='mgr.14118 172.21.15.16:0/600389341' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T16:21:40.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:40 smithi016 ceph-mon[31187]: from='mgr.14118 172.21.15.16:0/600389341' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-13T16:21:40.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:40 smithi016 ceph-mon[31187]: from='mgr.14118 172.21.15.16:0/600389341' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-13T16:21:40.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:40 smithi016 ceph-mon[31187]: from='mgr.14118 172.21.15.16:0/600389341' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-13T16:21:40.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:40 smithi016 ceph-mon[31187]: from='mgr.14118 172.21.15.16:0/600389341' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-13T16:21:40.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:40 smithi016 ceph-mon[31187]: Manager daemon x is now available 2024-09-13T16:21:40.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:40 smithi016 ceph-mon[31187]: from='mgr.14118 172.21.15.16:0/600389341' entity='mgr.x' 2024-09-13T16:21:40.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:40 smithi016 ceph-mon[31187]: from='mgr.14118 172.21.15.16:0/600389341' entity='mgr.x' 2024-09-13T16:21:40.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:40 smithi016 ceph-mon[31187]: from='mgr.14118 172.21.15.16:0/600389341' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:21:40.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:40 smithi016 ceph-mon[31187]: from='mgr.14118 172.21.15.16:0/600389341' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:21:40.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:40 smithi016 ceph-mon[31187]: from='mgr.14118 172.21.15.16:0/600389341' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-13T16:21:40.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:40 smithi016 ceph-mon[31187]: from='mgr.14118 172.21.15.16:0/600389341' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-13T16:21:41.731 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-13T16:21:41.731 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-13T16:21:41.731 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-13T16:21:41.731 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-13T16:21:41.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.298+0000 7eff0e71b640 1 Processor -- start 2024-09-13T16:21:41.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.298+0000 7eff0e71b640 1 -- start start 2024-09-13T16:21:41.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.298+0000 7eff0e71b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff08104430 0x7eff08104830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:41.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.298+0000 7eff0e71b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff08104e00 con 0x7eff08104430 2024-09-13T16:21:41.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.298+0000 7eff07fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff08104430 0x7eff08104830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:41.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.298+0000 7eff07fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff08104430 0x7eff08104830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59524/0 (socket says 172.21.15.16:59524) 2024-09-13T16:21:41.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.298+0000 7eff07fff640 1 -- 172.21.15.16:0/3552187559 learned_addr learned my addr 172.21.15.16:0/3552187559 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:41.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.299+0000 7eff07fff640 1 -- 172.21.15.16:0/3552187559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff08105630 con 0x7eff08104430 2024-09-13T16:21:41.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.299+0000 7eff07fff640 1 --2- 172.21.15.16:0/3552187559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff08104430 0x7eff08104830 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efef8009820 tx=0x7efef802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=151329fd6607330c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:41.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.300+0000 7eff06ffd640 1 -- 172.21.15.16:0/3552187559 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efef802ae70 con 0x7eff08104430 2024-09-13T16:21:41.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.300+0000 7eff06ffd640 1 -- 172.21.15.16:0/3552187559 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7efef80328b0 con 0x7eff08104430 2024-09-13T16:21:41.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.300+0000 7eff06ffd640 1 -- 172.21.15.16:0/3552187559 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efef8030850 con 0x7eff08104430 2024-09-13T16:21:41.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.300+0000 7eff0e71b640 1 -- 172.21.15.16:0/3552187559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff08104430 msgr2=0x7eff08104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:41.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.300+0000 7eff0e71b640 1 --2- 172.21.15.16:0/3552187559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff08104430 0x7eff08104830 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efef8009820 tx=0x7efef802a270 comp rx=0 tx=0).stop 2024-09-13T16:21:41.733 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.301+0000 7eff0e71b640 1 -- 172.21.15.16:0/3552187559 shutdown_connections 2024-09-13T16:21:41.733 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.301+0000 7eff0e71b640 1 --2- 172.21.15.16:0/3552187559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff08104430 0x7eff08104830 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:41.733 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.301+0000 7eff0e71b640 1 -- 172.21.15.16:0/3552187559 >> 172.21.15.16:0/3552187559 conn(0x7eff080ffbe0 msgr2=0x7eff08102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:41.733 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.301+0000 7eff0e71b640 1 -- 172.21.15.16:0/3552187559 shutdown_connections 2024-09-13T16:21:41.733 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.301+0000 7eff0e71b640 1 -- 172.21.15.16:0/3552187559 wait complete. 2024-09-13T16:21:41.733 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.301+0000 7eff0e71b640 1 Processor -- start 2024-09-13T16:21:41.733 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.302+0000 7eff0e71b640 1 -- start start 2024-09-13T16:21:41.733 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.302+0000 7eff0e71b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff08104430 0x7eff08079a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:41.733 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.302+0000 7eff0e71b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0819c060 con 0x7eff08104430 2024-09-13T16:21:41.733 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.302+0000 7eff07fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff08104430 0x7eff08079a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:41.733 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.302+0000 7eff07fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff08104430 0x7eff08079a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59540/0 (socket says 172.21.15.16:59540) 2024-09-13T16:21:41.733 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.302+0000 7eff07fff640 1 -- 172.21.15.16:0/986404480 learned_addr learned my addr 172.21.15.16:0/986404480 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:41.733 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.302+0000 7eff07fff640 1 -- 172.21.15.16:0/986404480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef80094d0 con 0x7eff08104430 2024-09-13T16:21:41.733 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.302+0000 7eff07fff640 1 --2- 172.21.15.16:0/986404480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff08104430 0x7eff08079a20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efef8030560 tx=0x7efef8030d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:41.734 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.303+0000 7eff057fa640 1 -- 172.21.15.16:0/986404480 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efef8032d60 con 0x7eff08104430 2024-09-13T16:21:41.734 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.303+0000 7eff057fa640 1 -- 172.21.15.16:0/986404480 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7efef803b8a0 con 0x7eff08104430 2024-09-13T16:21:41.734 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.303+0000 7eff0e71b640 1 -- 172.21.15.16:0/986404480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff081064b0 con 0x7eff08104430 2024-09-13T16:21:41.734 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.303+0000 7eff057fa640 1 -- 172.21.15.16:0/986404480 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efef803a680 con 0x7eff08104430 2024-09-13T16:21:41.734 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.303+0000 7eff0e71b640 1 -- 172.21.15.16:0/986404480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff0819c3f0 con 0x7eff08104430 2024-09-13T16:21:41.734 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.304+0000 7eff057fa640 1 -- 172.21.15.16:0/986404480 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7efef8039040 con 0x7eff08104430 2024-09-13T16:21:41.734 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.304+0000 7eff057fa640 1 --2- 172.21.15.16:0/986404480 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7efef0038670 0x7efef003ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:41.734 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.304+0000 7eff057fa640 1 -- 172.21.15.16:0/986404480 --> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7efef003b240 con 0x7efef0038670 2024-09-13T16:21:41.734 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.304+0000 7eff057fa640 1 -- 172.21.15.16:0/986404480 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7efef806b5d0 con 0x7eff08104430 2024-09-13T16:21:41.734 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.304+0000 7eff077fe640 1 -- 172.21.15.16:0/986404480 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7efef0038670 msgr2=0x7efef003ab30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1147765163 2024-09-13T16:21:41.735 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.304+0000 7eff077fe640 1 --2- 172.21.15.16:0/986404480 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7efef0038670 0x7efef003ab30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T16:21:41.735 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.505+0000 7eff077fe640 1 -- 172.21.15.16:0/986404480 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7efef0038670 msgr2=0x7efef003ab30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1147765163 2024-09-13T16:21:41.735 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.505+0000 7eff077fe640 1 --2- 172.21.15.16:0/986404480 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7efef0038670 0x7efef003ab30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T16:21:41.735 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.906+0000 7eff077fe640 1 -- 172.21.15.16:0/986404480 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7efef0038670 msgr2=0x7efef003ab30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1147765163 2024-09-13T16:21:41.735 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:37.906+0000 7eff077fe640 1 --2- 172.21.15.16:0/986404480 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7efef0038670 0x7efef003ab30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T16:21:41.735 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:38.707+0000 7eff077fe640 1 -- 172.21.15.16:0/986404480 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7efef0038670 msgr2=0x7efef003ab30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1147765163 2024-09-13T16:21:41.735 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:38.707+0000 7eff077fe640 1 --2- 172.21.15.16:0/986404480 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7efef0038670 0x7efef003ab30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T16:21:41.735 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:40.128+0000 7eff057fa640 1 -- 172.21.15.16:0/986404480 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mgrmap(e 6) v1 ==== 45308+0+0 (secure 0 0 0) 0x7efef806a750 con 0x7eff08104430 2024-09-13T16:21:41.735 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:40.128+0000 7eff057fa640 1 -- 172.21.15.16:0/986404480 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7efef0038670 msgr2=0x7efef003ab30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T16:21:41.735 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:40.128+0000 7eff057fa640 1 --2- 172.21.15.16:0/986404480 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7efef0038670 0x7efef003ab30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:41.735 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:41.129+0000 7eff057fa640 1 -- 172.21.15.16:0/986404480 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 7) v1 ==== 45435+0+0 (secure 0 0 0) 0x7efef8042db0 con 0x7eff08104430 2024-09-13T16:21:41.735 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:41.129+0000 7eff057fa640 1 --2- 172.21.15.16:0/986404480 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7efef003bf50 0x7efef003e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:41.735 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:41.129+0000 7eff057fa640 1 -- 172.21.15.16:0/986404480 --> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7efef003b240 con 0x7efef003bf50 2024-09-13T16:21:41.735 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:41.130+0000 7eff077fe640 1 --2- 172.21.15.16:0/986404480 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7efef003bf50 0x7efef003e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:41.736 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:41.130+0000 7eff077fe640 1 --2- 172.21.15.16:0/986404480 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7efef003bf50 0x7efef003e340 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7efefc003980 tx=0x7efefc0092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:41.736 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:41.131+0000 7eff057fa640 1 -- 172.21.15.16:0/986404480 <== mgr.14118 v2:172.21.15.16:6800/1756940569 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7efef003b240 con 0x7efef003bf50 2024-09-13T16:21:41.736 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:41.136+0000 7eff0e71b640 1 -- 172.21.15.16:0/986404480 --> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7eff08108860 con 0x7efef003bf50 2024-09-13T16:21:41.736 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:41.137+0000 7eff057fa640 1 -- 172.21.15.16:0/986404480 <== mgr.14118 v2:172.21.15.16:6800/1756940569 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7eff08108860 con 0x7efef003bf50 2024-09-13T16:21:41.736 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:41.137+0000 7eff0e71b640 1 -- 172.21.15.16:0/986404480 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7efef003bf50 msgr2=0x7efef003e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:41.736 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:41.137+0000 7eff0e71b640 1 --2- 172.21.15.16:0/986404480 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7efef003bf50 0x7efef003e340 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7efefc003980 tx=0x7efefc0092b0 comp rx=0 tx=0).stop 2024-09-13T16:21:41.736 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:41.137+0000 7eff0e71b640 1 -- 172.21.15.16:0/986404480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff08104430 msgr2=0x7eff08079a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:41.736 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:41.137+0000 7eff0e71b640 1 --2- 172.21.15.16:0/986404480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff08104430 0x7eff08079a20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efef8030560 tx=0x7efef8030d90 comp rx=0 tx=0).stop 2024-09-13T16:21:41.736 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:41.137+0000 7eff0e71b640 1 -- 172.21.15.16:0/986404480 shutdown_connections 2024-09-13T16:21:41.736 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:41.137+0000 7eff0e71b640 1 --2- 172.21.15.16:0/986404480 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7efef003bf50 0x7efef003e340 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:41.736 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:41.137+0000 7eff0e71b640 1 --2- 172.21.15.16:0/986404480 >> [v2:172.21.15.16:6800/1147765163,v1:172.21.15.16:6801/1147765163] conn(0x7efef0038670 0x7efef003ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:41.736 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:41.137+0000 7eff0e71b640 1 --2- 172.21.15.16:0/986404480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff08104430 0x7eff08079a20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:41.736 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:41.137+0000 7eff0e71b640 1 -- 172.21.15.16:0/986404480 >> 172.21.15.16:0/986404480 conn(0x7eff080ffbe0 msgr2=0x7eff081006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:41.737 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:41.137+0000 7eff0e71b640 1 -- 172.21.15.16:0/986404480 shutdown_connections 2024-09-13T16:21:41.737 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:41.137+0000 7eff0e71b640 1 -- 172.21.15.16:0/986404480 wait complete. 2024-09-13T16:21:41.737 INFO:teuthology.orchestra.run.smithi016.stdout:mgr epoch 5 is available 2024-09-13T16:21:41.737 INFO:teuthology.orchestra.run.smithi016.stdout:Setting orchestrator backend to cephadm... 2024-09-13T16:21:42.042 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:41 smithi016 ceph-mon[31187]: Found migration_current of "None". Setting to last migration. 2024-09-13T16:21:42.042 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:41 smithi016 ceph-mon[31187]: from='mgr.14118 172.21.15.16:0/600389341' entity='mgr.x' 2024-09-13T16:21:42.042 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:41 smithi016 ceph-mon[31187]: from='mgr.14118 172.21.15.16:0/600389341' entity='mgr.x' 2024-09-13T16:21:42.042 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:41 smithi016 ceph-mon[31187]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-13T16:21:42.042 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:41 smithi016 ceph-mon[31187]: mgrmap e7: x(active, since 1.00528s) 2024-09-13T16:21:42.319 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:42 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:42.041+0000 7f89bbabc640 -1 log_channel(cephadm) log [ERR] : [13/Sep/2024:16:21:42] ENGINE Error in HTTPServer.serve 2024-09-13T16:21:42.319 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:42 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: Traceback (most recent call last): 2024-09-13T16:21:42.319 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:42 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-13T16:21:42.319 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:42 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: self._connections.run(self.expiration_interval) 2024-09-13T16:21:42.319 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:42 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-13T16:21:42.319 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:42 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: self._run(expiration_interval) 2024-09-13T16:21:42.319 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:42 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-13T16:21:42.319 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:42 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: new_conn = self._from_server_socket(self.server.socket) 2024-09-13T16:21:42.319 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:42 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-13T16:21:42.319 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:42 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-13T16:21:42.320 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:42 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-13T16:21:42.320 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:42 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: s = self.context.wrap_socket( 2024-09-13T16:21:42.320 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:42 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-13T16:21:42.320 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:42 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: return self.sslsocket_class._create( 2024-09-13T16:21:42.320 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:42 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-13T16:21:42.320 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:42 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: self.do_handshake() 2024-09-13T16:21:42.320 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:42 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-13T16:21:42.320 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:42 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: self._sslobj.do_handshake() 2024-09-13T16:21:42.320 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:42 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-13T16:21:42.320 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:42 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:42.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:42 smithi016 ceph-mon[31187]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-13T16:21:42.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:42 smithi016 ceph-mon[31187]: [13/Sep/2024:16:21:41] ENGINE Bus STARTING 2024-09-13T16:21:42.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:42 smithi016 ceph-mon[31187]: [13/Sep/2024:16:21:41] ENGINE Serving on http://172.21.15.16:8765 2024-09-13T16:21:42.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:42 smithi016 ceph-mon[31187]: [13/Sep/2024:16:21:42] ENGINE Serving on https://172.21.15.16:7150 2024-09-13T16:21:42.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:42 smithi016 ceph-mon[31187]: [13/Sep/2024:16:21:42] ENGINE Bus STARTED 2024-09-13T16:21:42.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:42 smithi016 ceph-mon[31187]: from='mgr.14118 172.21.15.16:0/600389341' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:21:42.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:42 smithi016 ceph-mon[31187]: [13/Sep/2024:16:21:42] ENGINE Error in HTTPServer.serve 2024-09-13T16:21:42.913 INFO:journalctl@ceph.mon.a.smithi016.stdout: Traceback (most recent call last): 2024-09-13T16:21:42.913 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-13T16:21:42.913 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._connections.run(self.expiration_interval) 2024-09-13T16:21:42.913 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-13T16:21:42.913 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._run(expiration_interval) 2024-09-13T16:21:42.913 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-13T16:21:42.913 INFO:journalctl@ceph.mon.a.smithi016.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-13T16:21:42.913 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-13T16:21:42.913 INFO:journalctl@ceph.mon.a.smithi016.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-13T16:21:42.914 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-13T16:21:42.914 INFO:journalctl@ceph.mon.a.smithi016.stdout: s = self.context.wrap_socket( 2024-09-13T16:21:42.914 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-13T16:21:42.914 INFO:journalctl@ceph.mon.a.smithi016.stdout: return self.sslsocket_class._create( 2024-09-13T16:21:42.914 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-13T16:21:42.914 INFO:journalctl@ceph.mon.a.smithi016.stdout: self.do_handshake() 2024-09-13T16:21:42.914 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-13T16:21:42.914 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._sslobj.do_handshake() 2024-09-13T16:21:42.914 INFO:journalctl@ceph.mon.a.smithi016.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-13T16:21:42.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:42 smithi016 ceph-mon[31187]: from='mgr.14118 172.21.15.16:0/600389341' entity='mgr.x' 2024-09-13T16:21:42.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:42 smithi016 ceph-mon[31187]: from='mgr.14118 172.21.15.16:0/600389341' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:21:42.920 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.121+0000 7fc82c4f9640 1 Processor -- start 2024-09-13T16:21:42.920 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.121+0000 7fc82c4f9640 1 -- start start 2024-09-13T16:21:42.920 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.121+0000 7fc82c4f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc824105fd0 0x7fc8241083c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:42.920 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.121+0000 7fc82c4f9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc824079e00 con 0x7fc824105fd0 2024-09-13T16:21:42.920 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.122+0000 7fc82a295640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc824105fd0 0x7fc8241083c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:42.921 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.122+0000 7fc82a295640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc824105fd0 0x7fc8241083c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59638/0 (socket says 172.21.15.16:59638) 2024-09-13T16:21:42.921 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.122+0000 7fc82a295640 1 -- 172.21.15.16:0/3697398089 learned_addr learned my addr 172.21.15.16:0/3697398089 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:42.921 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.122+0000 7fc82a295640 1 -- 172.21.15.16:0/3697398089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc824108900 con 0x7fc824105fd0 2024-09-13T16:21:42.921 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.123+0000 7fc82a295640 1 --2- 172.21.15.16:0/3697398089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc824105fd0 0x7fc8241083c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc81c009820 tx=0x7fc81c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=950257b445911c11 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:42.921 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.123+0000 7fc829293640 1 -- 172.21.15.16:0/3697398089 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc81c02ae70 con 0x7fc824105fd0 2024-09-13T16:21:42.921 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.123+0000 7fc829293640 1 -- 172.21.15.16:0/3697398089 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fc81c0328b0 con 0x7fc824105fd0 2024-09-13T16:21:42.921 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.123+0000 7fc829293640 1 -- 172.21.15.16:0/3697398089 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc81c030850 con 0x7fc824105fd0 2024-09-13T16:21:42.921 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.123+0000 7fc82c4f9640 1 -- 172.21.15.16:0/3697398089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc824105fd0 msgr2=0x7fc8241083c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:42.921 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.123+0000 7fc82c4f9640 1 --2- 172.21.15.16:0/3697398089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc824105fd0 0x7fc8241083c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc81c009820 tx=0x7fc81c02a270 comp rx=0 tx=0).stop 2024-09-13T16:21:42.921 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.123+0000 7fc82c4f9640 1 -- 172.21.15.16:0/3697398089 shutdown_connections 2024-09-13T16:21:42.921 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.123+0000 7fc82c4f9640 1 --2- 172.21.15.16:0/3697398089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc824105fd0 0x7fc8241083c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:42.921 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.123+0000 7fc82c4f9640 1 -- 172.21.15.16:0/3697398089 >> 172.21.15.16:0/3697398089 conn(0x7fc8240ffbb0 msgr2=0x7fc824101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:42.921 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.124+0000 7fc82c4f9640 1 -- 172.21.15.16:0/3697398089 shutdown_connections 2024-09-13T16:21:42.922 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.124+0000 7fc82c4f9640 1 -- 172.21.15.16:0/3697398089 wait complete. 2024-09-13T16:21:42.922 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.124+0000 7fc82c4f9640 1 Processor -- start 2024-09-13T16:21:42.922 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.124+0000 7fc82c4f9640 1 -- start start 2024-09-13T16:21:42.922 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.124+0000 7fc82c4f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc824105fd0 0x7fc824199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:42.922 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.124+0000 7fc82c4f9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8241996d0 con 0x7fc824105fd0 2024-09-13T16:21:42.922 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.124+0000 7fc82a295640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc824105fd0 0x7fc824199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:42.922 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.124+0000 7fc82a295640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc824105fd0 0x7fc824199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59644/0 (socket says 172.21.15.16:59644) 2024-09-13T16:21:42.922 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.124+0000 7fc82a295640 1 -- 172.21.15.16:0/70669156 learned_addr learned my addr 172.21.15.16:0/70669156 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:42.922 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.125+0000 7fc82a295640 1 -- 172.21.15.16:0/70669156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81c0094d0 con 0x7fc824105fd0 2024-09-13T16:21:42.922 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.125+0000 7fc82a295640 1 --2- 172.21.15.16:0/70669156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc824105fd0 0x7fc824199190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc81c030560 tx=0x7fc81c030d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:42.922 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.125+0000 7fc8237fe640 1 -- 172.21.15.16:0/70669156 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc81c032d60 con 0x7fc824105fd0 2024-09-13T16:21:42.922 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.125+0000 7fc82c4f9640 1 -- 172.21.15.16:0/70669156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8241998d0 con 0x7fc824105fd0 2024-09-13T16:21:42.922 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.125+0000 7fc8237fe640 1 -- 172.21.15.16:0/70669156 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fc81c03b8a0 con 0x7fc824105fd0 2024-09-13T16:21:42.922 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.125+0000 7fc8237fe640 1 -- 172.21.15.16:0/70669156 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc81c03a740 con 0x7fc824105fd0 2024-09-13T16:21:42.923 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.125+0000 7fc82c4f9640 1 -- 172.21.15.16:0/70669156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc824199d70 con 0x7fc824105fd0 2024-09-13T16:21:42.923 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.126+0000 7fc8237fe640 1 -- 172.21.15.16:0/70669156 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 45435+0+0 (secure 0 0 0) 0x7fc81c039040 con 0x7fc824105fd0 2024-09-13T16:21:42.923 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.126+0000 7fc82c4f9640 1 -- 172.21.15.16:0/70669156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc824109660 con 0x7fc824105fd0 2024-09-13T16:21:42.923 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.126+0000 7fc8237fe640 1 --2- 172.21.15.16:0/70669156 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7fc80c038500 0x7fc80c03a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:42.923 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.126+0000 7fc8237fe640 1 -- 172.21.15.16:0/70669156 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc81c06b790 con 0x7fc824105fd0 2024-09-13T16:21:42.923 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.126+0000 7fc829a94640 1 --2- 172.21.15.16:0/70669156 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7fc80c038500 0x7fc80c03a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:42.923 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.127+0000 7fc829a94640 1 --2- 172.21.15.16:0/70669156 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7fc80c038500 0x7fc80c03a9c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc8180098c0 tx=0x7fc818006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:42.923 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.131+0000 7fc8237fe640 1 -- 172.21.15.16:0/70669156 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc81c031bd0 con 0x7fc824105fd0 2024-09-13T16:21:42.923 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.317+0000 7fc82c4f9640 1 -- 172.21.15.16:0/70669156 --> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fc824100e90 con 0x7fc80c038500 2024-09-13T16:21:42.923 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.325+0000 7fc8237fe640 1 -- 172.21.15.16:0/70669156 <== mgr.14118 v2:172.21.15.16:6800/1756940569 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc824100e90 con 0x7fc80c038500 2024-09-13T16:21:42.923 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.327+0000 7fc82c4f9640 1 -- 172.21.15.16:0/70669156 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7fc80c038500 msgr2=0x7fc80c03a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:42.923 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.327+0000 7fc82c4f9640 1 --2- 172.21.15.16:0/70669156 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7fc80c038500 0x7fc80c03a9c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc8180098c0 tx=0x7fc818006db0 comp rx=0 tx=0).stop 2024-09-13T16:21:42.923 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.327+0000 7fc82c4f9640 1 -- 172.21.15.16:0/70669156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc824105fd0 msgr2=0x7fc824199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:42.923 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.327+0000 7fc82c4f9640 1 --2- 172.21.15.16:0/70669156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc824105fd0 0x7fc824199190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc81c030560 tx=0x7fc81c030d90 comp rx=0 tx=0).stop 2024-09-13T16:21:42.923 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.327+0000 7fc82c4f9640 1 -- 172.21.15.16:0/70669156 shutdown_connections 2024-09-13T16:21:42.924 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.327+0000 7fc82c4f9640 1 --2- 172.21.15.16:0/70669156 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7fc80c038500 0x7fc80c03a9c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:42.924 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.327+0000 7fc82c4f9640 1 --2- 172.21.15.16:0/70669156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc824105fd0 0x7fc824199190 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:42.924 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.327+0000 7fc82c4f9640 1 -- 172.21.15.16:0/70669156 >> 172.21.15.16:0/70669156 conn(0x7fc8240ffbb0 msgr2=0x7fc824100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:42.924 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.327+0000 7fc82c4f9640 1 -- 172.21.15.16:0/70669156 shutdown_connections 2024-09-13T16:21:42.924 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:42.327+0000 7fc82c4f9640 1 -- 172.21.15.16:0/70669156 wait complete. 2024-09-13T16:21:44.068 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:43 smithi016 ceph-mon[31187]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:21:44.068 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:43 smithi016 ceph-mon[31187]: mgrmap e8: x(active, since 2s) 2024-09-13T16:21:44.076 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-13T16:21:44.076 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.324+0000 7f7b40a6a640 1 Processor -- start 2024-09-13T16:21:44.076 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.325+0000 7f7b40a6a640 1 -- start start 2024-09-13T16:21:44.076 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.325+0000 7f7b40a6a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b3c106000 0x7f7b3c1083f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:44.076 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.325+0000 7f7b40a6a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c079de0 con 0x7f7b3c106000 2024-09-13T16:21:44.076 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.325+0000 7f7b3a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b3c106000 0x7f7b3c1083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:44.076 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.325+0000 7f7b3a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b3c106000 0x7f7b3c1083f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59652/0 (socket says 172.21.15.16:59652) 2024-09-13T16:21:44.076 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.325+0000 7f7b3a59c640 1 -- 172.21.15.16:0/1950946778 learned_addr learned my addr 172.21.15.16:0/1950946778 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:44.076 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.325+0000 7f7b3a59c640 1 -- 172.21.15.16:0/1950946778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b3c108930 con 0x7f7b3c106000 2024-09-13T16:21:44.076 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.326+0000 7f7b3a59c640 1 --2- 172.21.15.16:0/1950946778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b3c106000 0x7f7b3c1083f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c009820 tx=0x7f7b2c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dccaa2f7841391a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:44.077 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.326+0000 7f7b3959a640 1 -- 172.21.15.16:0/1950946778 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b2c02ae70 con 0x7f7b3c106000 2024-09-13T16:21:44.077 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.327+0000 7f7b3959a640 1 -- 172.21.15.16:0/1950946778 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f7b2c0328b0 con 0x7f7b3c106000 2024-09-13T16:21:44.077 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.327+0000 7f7b3959a640 1 -- 172.21.15.16:0/1950946778 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b2c030850 con 0x7f7b3c106000 2024-09-13T16:21:44.077 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.327+0000 7f7b40a6a640 1 -- 172.21.15.16:0/1950946778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b3c106000 msgr2=0x7f7b3c1083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:44.077 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.327+0000 7f7b40a6a640 1 --2- 172.21.15.16:0/1950946778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b3c106000 0x7f7b3c1083f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c009820 tx=0x7f7b2c02a270 comp rx=0 tx=0).stop 2024-09-13T16:21:44.077 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.327+0000 7f7b40a6a640 1 -- 172.21.15.16:0/1950946778 shutdown_connections 2024-09-13T16:21:44.077 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.327+0000 7f7b40a6a640 1 --2- 172.21.15.16:0/1950946778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b3c106000 0x7f7b3c1083f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:44.077 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.327+0000 7f7b40a6a640 1 -- 172.21.15.16:0/1950946778 >> 172.21.15.16:0/1950946778 conn(0x7f7b3c0ffbe0 msgr2=0x7f7b3c102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:44.077 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.327+0000 7f7b40a6a640 1 -- 172.21.15.16:0/1950946778 shutdown_connections 2024-09-13T16:21:44.077 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.327+0000 7f7b40a6a640 1 -- 172.21.15.16:0/1950946778 wait complete. 2024-09-13T16:21:44.077 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.328+0000 7f7b40a6a640 1 Processor -- start 2024-09-13T16:21:44.077 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.328+0000 7f7b40a6a640 1 -- start start 2024-09-13T16:21:44.077 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.328+0000 7f7b40a6a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b3c106000 0x7f7b3c105880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:44.078 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.328+0000 7f7b40a6a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c105dc0 con 0x7f7b3c106000 2024-09-13T16:21:44.078 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.329+0000 7f7b3a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b3c106000 0x7f7b3c105880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:44.078 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.329+0000 7f7b3a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b3c106000 0x7f7b3c105880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59666/0 (socket says 172.21.15.16:59666) 2024-09-13T16:21:44.078 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.329+0000 7f7b3a59c640 1 -- 172.21.15.16:0/2619927355 learned_addr learned my addr 172.21.15.16:0/2619927355 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:44.078 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.329+0000 7f7b3a59c640 1 -- 172.21.15.16:0/2619927355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b2c0094d0 con 0x7f7b3c106000 2024-09-13T16:21:44.078 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.329+0000 7f7b3a59c640 1 --2- 172.21.15.16:0/2619927355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b3c106000 0x7f7b3c105880 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c030560 tx=0x7f7b2c030d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:44.078 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.329+0000 7f7b2b7fe640 1 -- 172.21.15.16:0/2619927355 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b2c032d60 con 0x7f7b3c106000 2024-09-13T16:21:44.078 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.329+0000 7f7b2b7fe640 1 -- 172.21.15.16:0/2619927355 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f7b2c03b8a0 con 0x7f7b3c106000 2024-09-13T16:21:44.078 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.329+0000 7f7b40a6a640 1 -- 172.21.15.16:0/2619927355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b3c103fb0 con 0x7f7b3c106000 2024-09-13T16:21:44.078 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.329+0000 7f7b2b7fe640 1 -- 172.21.15.16:0/2619927355 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b2c03a740 con 0x7f7b3c106000 2024-09-13T16:21:44.078 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.329+0000 7f7b40a6a640 1 -- 172.21.15.16:0/2619927355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b3c104450 con 0x7f7b3c106000 2024-09-13T16:21:44.078 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.330+0000 7f7b2b7fe640 1 -- 172.21.15.16:0/2619927355 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f7b2c039040 con 0x7f7b3c106000 2024-09-13T16:21:44.079 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.330+0000 7f7b2b7fe640 1 --2- 172.21.15.16:0/2619927355 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f7b24038620 0x7f7b2403aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:44.079 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.330+0000 7f7b2b7fe640 1 -- 172.21.15.16:0/2619927355 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7b2c06bcc0 con 0x7f7b3c106000 2024-09-13T16:21:44.079 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.330+0000 7f7b39d9b640 1 --2- 172.21.15.16:0/2619927355 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f7b24038620 0x7f7b2403aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:44.079 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.330+0000 7f7b40a6a640 1 -- 172.21.15.16:0/2619927355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b3c106460 con 0x7f7b3c106000 2024-09-13T16:21:44.079 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.331+0000 7f7b39d9b640 1 --2- 172.21.15.16:0/2619927355 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f7b24038620 0x7f7b2403aae0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7b3000ad30 tx=0x7f7b300093f0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:44.079 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.336+0000 7f7b2b7fe640 1 -- 172.21.15.16:0/2619927355 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7b2c037070 con 0x7f7b3c106000 2024-09-13T16:21:44.079 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.510+0000 7f7b40a6a640 1 -- 172.21.15.16:0/2619927355 --> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b3c104980 con 0x7f7b24038620 2024-09-13T16:21:44.079 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.511+0000 7f7b2b7fe640 1 -- 172.21.15.16:0/2619927355 <== mgr.14118 v2:172.21.15.16:6800/1756940569 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f7b3c104980 con 0x7f7b24038620 2024-09-13T16:21:44.079 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.513+0000 7f7b40a6a640 1 -- 172.21.15.16:0/2619927355 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f7b24038620 msgr2=0x7f7b2403aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:44.079 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.513+0000 7f7b40a6a640 1 --2- 172.21.15.16:0/2619927355 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f7b24038620 0x7f7b2403aae0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7b3000ad30 tx=0x7f7b300093f0 comp rx=0 tx=0).stop 2024-09-13T16:21:44.079 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.513+0000 7f7b40a6a640 1 -- 172.21.15.16:0/2619927355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b3c106000 msgr2=0x7f7b3c105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:44.079 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.513+0000 7f7b40a6a640 1 --2- 172.21.15.16:0/2619927355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b3c106000 0x7f7b3c105880 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c030560 tx=0x7f7b2c030d90 comp rx=0 tx=0).stop 2024-09-13T16:21:44.080 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.513+0000 7f7b40a6a640 1 -- 172.21.15.16:0/2619927355 shutdown_connections 2024-09-13T16:21:44.080 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.513+0000 7f7b40a6a640 1 --2- 172.21.15.16:0/2619927355 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f7b24038620 0x7f7b2403aae0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:44.080 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.513+0000 7f7b40a6a640 1 --2- 172.21.15.16:0/2619927355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b3c106000 0x7f7b3c105880 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:44.080 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.513+0000 7f7b40a6a640 1 -- 172.21.15.16:0/2619927355 >> 172.21.15.16:0/2619927355 conn(0x7f7b3c0ffbe0 msgr2=0x7f7b3c100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:44.080 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.514+0000 7f7b40a6a640 1 -- 172.21.15.16:0/2619927355 shutdown_connections 2024-09-13T16:21:44.080 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:43.514+0000 7f7b40a6a640 1 -- 172.21.15.16:0/2619927355 wait complete. 2024-09-13T16:21:44.080 INFO:teuthology.orchestra.run.smithi016.stdout:Generating ssh key... 2024-09-13T16:21:45.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:44 smithi016 ceph-mon[31187]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:21:45.474 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:45 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: Generating public/private rsa key pair. 2024-09-13T16:21:45.474 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:45 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: Your identification has been saved in /tmp/tmp4drmqkq3/key 2024-09-13T16:21:45.474 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:45 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: Your public key has been saved in /tmp/tmp4drmqkq3/key.pub 2024-09-13T16:21:45.475 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:45 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: The key fingerprint is: 2024-09-13T16:21:45.475 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:45 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: SHA256:ALiNsj+kv6gb6x9bhUPu5FTedB9bbfZmQHuN33akM+M ceph-0d241f6a-71ec-11ef-bceb-c7b262605968 2024-09-13T16:21:45.475 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:45 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: The key's randomart image is: 2024-09-13T16:21:45.475 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:45 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: +---[RSA 3072]----+ 2024-09-13T16:21:45.475 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:45 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: | .. . | 2024-09-13T16:21:45.475 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:45 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: | . . . oo| 2024-09-13T16:21:45.475 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:45 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: | + ... . . .+.B| 2024-09-13T16:21:45.475 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:45 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: |. o + +.o . . +Oo| 2024-09-13T16:21:45.475 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:45 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: | o * oS. o= O| 2024-09-13T16:21:45.475 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:45 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: |. . = o . B.| 2024-09-13T16:21:45.475 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:45 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: |.+ . + E | 2024-09-13T16:21:45.475 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:45 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: |.+o + | 2024-09-13T16:21:45.475 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:45 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: |B++= | 2024-09-13T16:21:45.475 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:45 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: +----[SHA256]-----+ 2024-09-13T16:21:45.749 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.521+0000 7f4280b7b640 1 Processor -- start 2024-09-13T16:21:45.749 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.521+0000 7f4280b7b640 1 -- start start 2024-09-13T16:21:45.749 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.521+0000 7f4280b7b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f427c105fe0 0x7f427c1083d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:45.749 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.521+0000 7f4280b7b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427c079e00 con 0x7f427c105fe0 2024-09-13T16:21:45.749 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.522+0000 7f427a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f427c105fe0 0x7f427c1083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:45.750 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.522+0000 7f427a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f427c105fe0 0x7f427c1083d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47842/0 (socket says 172.21.15.16:47842) 2024-09-13T16:21:45.750 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.522+0000 7f427a59c640 1 -- 172.21.15.16:0/3628559693 learned_addr learned my addr 172.21.15.16:0/3628559693 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:45.750 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.522+0000 7f427a59c640 1 -- 172.21.15.16:0/3628559693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f427c108910 con 0x7f427c105fe0 2024-09-13T16:21:45.750 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.522+0000 7f427a59c640 1 --2- 172.21.15.16:0/3628559693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f427c105fe0 0x7f427c1083d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4274009820 tx=0x7f427402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e0e641d0d920ef server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:45.750 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.523+0000 7f427959a640 1 -- 172.21.15.16:0/3628559693 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f427402ae70 con 0x7f427c105fe0 2024-09-13T16:21:45.750 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.523+0000 7f427959a640 1 -- 172.21.15.16:0/3628559693 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f42740328b0 con 0x7f427c105fe0 2024-09-13T16:21:45.750 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.523+0000 7f427959a640 1 -- 172.21.15.16:0/3628559693 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4274030850 con 0x7f427c105fe0 2024-09-13T16:21:45.750 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.523+0000 7f4280b7b640 1 -- 172.21.15.16:0/3628559693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f427c105fe0 msgr2=0x7f427c1083d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:45.750 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.523+0000 7f4280b7b640 1 --2- 172.21.15.16:0/3628559693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f427c105fe0 0x7f427c1083d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4274009820 tx=0x7f427402a270 comp rx=0 tx=0).stop 2024-09-13T16:21:45.750 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.523+0000 7f4280b7b640 1 -- 172.21.15.16:0/3628559693 shutdown_connections 2024-09-13T16:21:45.750 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.523+0000 7f4280b7b640 1 --2- 172.21.15.16:0/3628559693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f427c105fe0 0x7f427c1083d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:45.750 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.523+0000 7f4280b7b640 1 -- 172.21.15.16:0/3628559693 >> 172.21.15.16:0/3628559693 conn(0x7f427c0ffbe0 msgr2=0x7f427c102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:45.750 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.523+0000 7f4280b7b640 1 -- 172.21.15.16:0/3628559693 shutdown_connections 2024-09-13T16:21:45.751 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.523+0000 7f4280b7b640 1 -- 172.21.15.16:0/3628559693 wait complete. 2024-09-13T16:21:45.751 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.524+0000 7f4280b7b640 1 Processor -- start 2024-09-13T16:21:45.751 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.524+0000 7f4280b7b640 1 -- start start 2024-09-13T16:21:45.751 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.524+0000 7f4280b7b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f427c105fe0 0x7f427c199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:45.751 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.524+0000 7f4280b7b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427c1996b0 con 0x7f427c105fe0 2024-09-13T16:21:45.751 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.525+0000 7f427a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f427c105fe0 0x7f427c199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:45.751 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.525+0000 7f427a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f427c105fe0 0x7f427c199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47850/0 (socket says 172.21.15.16:47850) 2024-09-13T16:21:45.751 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.525+0000 7f427a59c640 1 -- 172.21.15.16:0/1430996965 learned_addr learned my addr 172.21.15.16:0/1430996965 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:45.751 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.525+0000 7f427a59c640 1 -- 172.21.15.16:0/1430996965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42740094d0 con 0x7f427c105fe0 2024-09-13T16:21:45.751 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.525+0000 7f427a59c640 1 --2- 172.21.15.16:0/1430996965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f427c105fe0 0x7f427c199170 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4274000c00 tx=0x7f4274030db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:45.751 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.525+0000 7f42737fe640 1 -- 172.21.15.16:0/1430996965 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4274032d60 con 0x7f427c105fe0 2024-09-13T16:21:45.751 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.525+0000 7f42737fe640 1 -- 172.21.15.16:0/1430996965 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f427403b8a0 con 0x7f427c105fe0 2024-09-13T16:21:45.751 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.525+0000 7f42737fe640 1 -- 172.21.15.16:0/1430996965 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f427403a640 con 0x7f427c105fe0 2024-09-13T16:21:45.751 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.525+0000 7f4280b7b640 1 -- 172.21.15.16:0/1430996965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f427c1998b0 con 0x7f427c105fe0 2024-09-13T16:21:45.752 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.525+0000 7f4280b7b640 1 -- 172.21.15.16:0/1430996965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f427c199d50 con 0x7f427c105fe0 2024-09-13T16:21:45.752 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.526+0000 7f42737fe640 1 -- 172.21.15.16:0/1430996965 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f4274039040 con 0x7f427c105fe0 2024-09-13T16:21:45.752 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.526+0000 7f4280b7b640 1 -- 172.21.15.16:0/1430996965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f427c109670 con 0x7f427c105fe0 2024-09-13T16:21:45.752 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.527+0000 7f42737fe640 1 --2- 172.21.15.16:0/1430996965 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f4268038670 0x7f426803ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:45.752 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.527+0000 7f42737fe640 1 -- 172.21.15.16:0/1430996965 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f427406b8c0 con 0x7f427c105fe0 2024-09-13T16:21:45.752 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.527+0000 7f4279d9b640 1 --2- 172.21.15.16:0/1430996965 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f4268038670 0x7f426803ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:45.752 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.527+0000 7f4279d9b640 1 --2- 172.21.15.16:0/1430996965 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f4268038670 0x7f426803ab30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f42640098c0 tx=0x7f4264006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:45.752 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.531+0000 7f42737fe640 1 -- 172.21.15.16:0/1430996965 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f4274031720 con 0x7f427c105fe0 2024-09-13T16:21:45.752 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:44.707+0000 7f4280b7b640 1 -- 172.21.15.16:0/1430996965 --> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f427c100e80 con 0x7f4268038670 2024-09-13T16:21:45.752 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:45.177+0000 7f42737fe640 1 -- 172.21.15.16:0/1430996965 <== mgr.14118 v2:172.21.15.16:6800/1756940569 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f427c100e80 con 0x7f4268038670 2024-09-13T16:21:45.752 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:45.179+0000 7f4280b7b640 1 -- 172.21.15.16:0/1430996965 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f4268038670 msgr2=0x7f426803ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:45.752 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:45.179+0000 7f4280b7b640 1 --2- 172.21.15.16:0/1430996965 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f4268038670 0x7f426803ab30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f42640098c0 tx=0x7f4264006db0 comp rx=0 tx=0).stop 2024-09-13T16:21:45.752 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:45.179+0000 7f4280b7b640 1 -- 172.21.15.16:0/1430996965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f427c105fe0 msgr2=0x7f427c199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:45.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:45.179+0000 7f4280b7b640 1 --2- 172.21.15.16:0/1430996965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f427c105fe0 0x7f427c199170 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4274000c00 tx=0x7f4274030db0 comp rx=0 tx=0).stop 2024-09-13T16:21:45.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:45.179+0000 7f4280b7b640 1 -- 172.21.15.16:0/1430996965 shutdown_connections 2024-09-13T16:21:45.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:45.179+0000 7f4280b7b640 1 --2- 172.21.15.16:0/1430996965 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f4268038670 0x7f426803ab30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:45.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:45.179+0000 7f4280b7b640 1 --2- 172.21.15.16:0/1430996965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f427c105fe0 0x7f427c199170 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:45.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:45.179+0000 7f4280b7b640 1 -- 172.21.15.16:0/1430996965 >> 172.21.15.16:0/1430996965 conn(0x7f427c0ffbe0 msgr2=0x7f427c100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:45.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:45.180+0000 7f4280b7b640 1 -- 172.21.15.16:0/1430996965 shutdown_connections 2024-09-13T16:21:45.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:45.180+0000 7f4280b7b640 1 -- 172.21.15.16:0/1430996965 wait complete. 2024-09-13T16:21:46.085 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:45 smithi016 ceph-mon[31187]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:21:46.086 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:45 smithi016 ceph-mon[31187]: Generating ssh key... 2024-09-13T16:21:46.086 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:45 smithi016 ceph-mon[31187]: from='mgr.14118 172.21.15.16:0/600389341' entity='mgr.x' 2024-09-13T16:21:46.086 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:45 smithi016 ceph-mon[31187]: from='mgr.14118 172.21.15.16:0/600389341' entity='mgr.x' 2024-09-13T16:21:46.930 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPbEGzctL4hKBPFguGj1c5vzIqAe1BPs1xQRbvm0ipZBRQMNGrNmdk9AtAqUe0gljBiudEei1iw6TWxgGM6bwh17ttPE1kgi9slR6ooRxrSPkGFvUe6XiqEZa8mt9cW2AzaU/XN4/ZiHxZtxd8eVB3h9dacWUuai+EHVlOoTeMsGk4ZlfYoXL7KaAx9A2PN8oaowCByu6MUB7hbOdckOozT06Bb89uzriqlSh2vlA8UBE06ZMOVJjy8ZjSG7ZHKd5ViQPTQn5vRhdnmx64KhA3F+Q4j1gTQ7V2HRFdgd6lnH1t5i2UcPQLmqkmUa1FAG1vfWxO4m/4JEfQ/yXJ9NZsZQkyp6r1VwksSGX9Fxw0rijUXAQ5W5lsvpNgsH+9Hv3PbydEakv3GH1MsdzQ2BDT7Tkgi2aG1u5rnUQgkJgk/yzOJuRd4qsdN7rBxP8ud+nLJ5mhE2ZY6CkfL2l/CmO41Wn4gQ0WOMdl7ow8RHU04RF9t6RaWFFeTFpMe4AbbBs= ceph-0d241f6a-71ec-11ef-bceb-c7b262605968 2024-09-13T16:21:46.930 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.168+0000 7efd9d7f0640 1 Processor -- start 2024-09-13T16:21:46.930 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.168+0000 7efd9d7f0640 1 -- start start 2024-09-13T16:21:46.930 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.168+0000 7efd9d7f0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd98105fa0 0x7efd98108390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:46.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.168+0000 7efd9d7f0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd98079e40 con 0x7efd98105fa0 2024-09-13T16:21:46.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.169+0000 7efd96ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd98105fa0 0x7efd98108390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:46.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.169+0000 7efd96ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd98105fa0 0x7efd98108390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47858/0 (socket says 172.21.15.16:47858) 2024-09-13T16:21:46.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.169+0000 7efd96ffd640 1 -- 172.21.15.16:0/3444851220 learned_addr learned my addr 172.21.15.16:0/3444851220 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:46.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.169+0000 7efd96ffd640 1 -- 172.21.15.16:0/3444851220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd981088d0 con 0x7efd98105fa0 2024-09-13T16:21:46.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.170+0000 7efd96ffd640 1 --2- 172.21.15.16:0/3444851220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd98105fa0 0x7efd98108390 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efd88009e80 tx=0x7efd8802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ebf6d9c189aec3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:46.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.170+0000 7efd95ffb640 1 -- 172.21.15.16:0/3444851220 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd88031400 con 0x7efd98105fa0 2024-09-13T16:21:46.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.170+0000 7efd95ffb640 1 -- 172.21.15.16:0/3444851220 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7efd8802d050 con 0x7efd98105fa0 2024-09-13T16:21:46.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.170+0000 7efd95ffb640 1 -- 172.21.15.16:0/3444851220 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd88007880 con 0x7efd98105fa0 2024-09-13T16:21:46.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.170+0000 7efd9d7f0640 1 -- 172.21.15.16:0/3444851220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd98105fa0 msgr2=0x7efd98108390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:46.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.170+0000 7efd9d7f0640 1 --2- 172.21.15.16:0/3444851220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd98105fa0 0x7efd98108390 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efd88009e80 tx=0x7efd8802a7a0 comp rx=0 tx=0).stop 2024-09-13T16:21:46.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.170+0000 7efd9d7f0640 1 -- 172.21.15.16:0/3444851220 shutdown_connections 2024-09-13T16:21:46.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.170+0000 7efd9d7f0640 1 --2- 172.21.15.16:0/3444851220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd98105fa0 0x7efd98108390 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:46.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.171+0000 7efd9d7f0640 1 -- 172.21.15.16:0/3444851220 >> 172.21.15.16:0/3444851220 conn(0x7efd980ffbe0 msgr2=0x7efd98102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:46.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.171+0000 7efd9d7f0640 1 -- 172.21.15.16:0/3444851220 shutdown_connections 2024-09-13T16:21:46.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.171+0000 7efd9d7f0640 1 -- 172.21.15.16:0/3444851220 wait complete. 2024-09-13T16:21:46.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.171+0000 7efd9d7f0640 1 Processor -- start 2024-09-13T16:21:46.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.171+0000 7efd9d7f0640 1 -- start start 2024-09-13T16:21:46.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.171+0000 7efd9d7f0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd98105fa0 0x7efd98199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:46.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.172+0000 7efd9d7f0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd981996c0 con 0x7efd98105fa0 2024-09-13T16:21:46.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.172+0000 7efd96ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd98105fa0 0x7efd98199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:46.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.172+0000 7efd96ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd98105fa0 0x7efd98199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47860/0 (socket says 172.21.15.16:47860) 2024-09-13T16:21:46.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.172+0000 7efd96ffd640 1 -- 172.21.15.16:0/4106779313 learned_addr learned my addr 172.21.15.16:0/4106779313 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:46.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.172+0000 7efd96ffd640 1 -- 172.21.15.16:0/4106779313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd88009b30 con 0x7efd98105fa0 2024-09-13T16:21:46.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.172+0000 7efd96ffd640 1 --2- 172.21.15.16:0/4106779313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd98105fa0 0x7efd98199180 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efd88031630 tx=0x7efd88031660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:46.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.172+0000 7efd7ffff640 1 -- 172.21.15.16:0/4106779313 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd88031b60 con 0x7efd98105fa0 2024-09-13T16:21:46.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.172+0000 7efd7ffff640 1 -- 172.21.15.16:0/4106779313 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7efd88031cc0 con 0x7efd98105fa0 2024-09-13T16:21:46.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.172+0000 7efd9d7f0640 1 -- 172.21.15.16:0/4106779313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd981998c0 con 0x7efd98105fa0 2024-09-13T16:21:46.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.172+0000 7efd7ffff640 1 -- 172.21.15.16:0/4106779313 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd8803b550 con 0x7efd98105fa0 2024-09-13T16:21:46.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.172+0000 7efd9d7f0640 1 -- 172.21.15.16:0/4106779313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd98199dc0 con 0x7efd98105fa0 2024-09-13T16:21:46.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.173+0000 7efd7ffff640 1 -- 172.21.15.16:0/4106779313 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7efd8803d450 con 0x7efd98105fa0 2024-09-13T16:21:46.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.173+0000 7efd7ffff640 1 --2- 172.21.15.16:0/4106779313 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7efd80038670 0x7efd8003ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:46.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.173+0000 7efd9d7f0640 1 -- 172.21.15.16:0/4106779313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd98109690 con 0x7efd98105fa0 2024-09-13T16:21:46.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.173+0000 7efd7ffff640 1 -- 172.21.15.16:0/4106779313 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7efd8806c350 con 0x7efd98105fa0 2024-09-13T16:21:46.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.173+0000 7efd967fc640 1 --2- 172.21.15.16:0/4106779313 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7efd80038670 0x7efd8003ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:46.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.174+0000 7efd967fc640 1 --2- 172.21.15.16:0/4106779313 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7efd80038670 0x7efd8003ab30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0098c0 tx=0x7efd8c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:46.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.178+0000 7efd7ffff640 1 -- 172.21.15.16:0/4106779313 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7efd8804d4e0 con 0x7efd98105fa0 2024-09-13T16:21:46.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.350+0000 7efd9d7f0640 1 -- 172.21.15.16:0/4106779313 --> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7efd98100e80 con 0x7efd80038670 2024-09-13T16:21:46.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.350+0000 7efd7ffff640 1 -- 172.21.15.16:0/4106779313 <== mgr.14118 v2:172.21.15.16:6800/1756940569 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7efd98100e80 con 0x7efd80038670 2024-09-13T16:21:46.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.352+0000 7efd9d7f0640 1 -- 172.21.15.16:0/4106779313 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7efd80038670 msgr2=0x7efd8003ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:46.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.352+0000 7efd9d7f0640 1 --2- 172.21.15.16:0/4106779313 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7efd80038670 0x7efd8003ab30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0098c0 tx=0x7efd8c006db0 comp rx=0 tx=0).stop 2024-09-13T16:21:46.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.352+0000 7efd9d7f0640 1 -- 172.21.15.16:0/4106779313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd98105fa0 msgr2=0x7efd98199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:46.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.352+0000 7efd9d7f0640 1 --2- 172.21.15.16:0/4106779313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd98105fa0 0x7efd98199180 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efd88031630 tx=0x7efd88031660 comp rx=0 tx=0).stop 2024-09-13T16:21:46.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.352+0000 7efd9d7f0640 1 -- 172.21.15.16:0/4106779313 shutdown_connections 2024-09-13T16:21:46.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.352+0000 7efd9d7f0640 1 --2- 172.21.15.16:0/4106779313 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7efd80038670 0x7efd8003ab30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:46.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.352+0000 7efd9d7f0640 1 --2- 172.21.15.16:0/4106779313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd98105fa0 0x7efd98199180 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:46.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.352+0000 7efd9d7f0640 1 -- 172.21.15.16:0/4106779313 >> 172.21.15.16:0/4106779313 conn(0x7efd980ffbe0 msgr2=0x7efd98100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:46.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.353+0000 7efd9d7f0640 1 -- 172.21.15.16:0/4106779313 shutdown_connections 2024-09-13T16:21:46.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:46.353+0000 7efd9d7f0640 1 -- 172.21.15.16:0/4106779313 wait complete. 2024-09-13T16:21:46.935 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-13T16:21:46.935 INFO:teuthology.orchestra.run.smithi016.stdout:Adding key to root@localhost authorized_keys... 2024-09-13T16:21:46.935 INFO:teuthology.orchestra.run.smithi016.stdout:Adding host smithi016... 2024-09-13T16:21:47.546 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:47 smithi016 ceph-mon[31187]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:21:48.599 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:48 smithi016 ceph-mon[31187]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi016", "addr": "172.21.15.16", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:21:49.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:49 smithi016 ceph-mon[31187]: Deploying cephadm binary to smithi016 2024-09-13T16:21:51.541 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Added host 'smithi016' with addr '172.21.15.16' 2024-09-13T16:21:51.541 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.358+0000 7f94abb07640 1 Processor -- start 2024-09-13T16:21:51.541 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.359+0000 7f94abb07640 1 -- start start 2024-09-13T16:21:51.541 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.359+0000 7f94abb07640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94a4104420 0x7f94a4104820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:51.541 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.359+0000 7f94abb07640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a4104df0 con 0x7f94a4104420 2024-09-13T16:21:51.542 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.359+0000 7f94a98a3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94a4104420 0x7f94a4104820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:51.542 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.359+0000 7f94a98a3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94a4104420 0x7f94a4104820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47866/0 (socket says 172.21.15.16:47866) 2024-09-13T16:21:51.542 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.359+0000 7f94a98a3640 1 -- 172.21.15.16:0/357999096 learned_addr learned my addr 172.21.15.16:0/357999096 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:51.542 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.359+0000 7f94a98a3640 1 -- 172.21.15.16:0/357999096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a4105620 con 0x7f94a4104420 2024-09-13T16:21:51.542 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.360+0000 7f94a98a3640 1 --2- 172.21.15.16:0/357999096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94a4104420 0x7f94a4104820 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009820 tx=0x7f94a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd5ccc20f94b5817 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:51.542 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.360+0000 7f94a88a1640 1 -- 172.21.15.16:0/357999096 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94a002ae70 con 0x7f94a4104420 2024-09-13T16:21:51.542 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.360+0000 7f94a88a1640 1 -- 172.21.15.16:0/357999096 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f94a00328b0 con 0x7f94a4104420 2024-09-13T16:21:51.542 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.360+0000 7f94a88a1640 1 -- 172.21.15.16:0/357999096 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94a0030850 con 0x7f94a4104420 2024-09-13T16:21:51.542 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.361+0000 7f94abb07640 1 -- 172.21.15.16:0/357999096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94a4104420 msgr2=0x7f94a4104820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:51.542 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.361+0000 7f94abb07640 1 --2- 172.21.15.16:0/357999096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94a4104420 0x7f94a4104820 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009820 tx=0x7f94a002a270 comp rx=0 tx=0).stop 2024-09-13T16:21:51.542 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.361+0000 7f94abb07640 1 -- 172.21.15.16:0/357999096 shutdown_connections 2024-09-13T16:21:51.542 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.361+0000 7f94abb07640 1 --2- 172.21.15.16:0/357999096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94a4104420 0x7f94a4104820 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:51.542 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.361+0000 7f94abb07640 1 -- 172.21.15.16:0/357999096 >> 172.21.15.16:0/357999096 conn(0x7f94a40ffbb0 msgr2=0x7f94a4101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:51.543 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.361+0000 7f94abb07640 1 -- 172.21.15.16:0/357999096 shutdown_connections 2024-09-13T16:21:51.543 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.361+0000 7f94abb07640 1 -- 172.21.15.16:0/357999096 wait complete. 2024-09-13T16:21:51.543 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.362+0000 7f94abb07640 1 Processor -- start 2024-09-13T16:21:51.543 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.362+0000 7f94abb07640 1 -- start start 2024-09-13T16:21:51.543 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.362+0000 7f94abb07640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94a4104420 0x7f94a41993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:51.543 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.362+0000 7f94abb07640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a41998e0 con 0x7f94a4104420 2024-09-13T16:21:51.543 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.362+0000 7f94a98a3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94a4104420 0x7f94a41993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:51.543 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.362+0000 7f94a98a3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94a4104420 0x7f94a41993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47872/0 (socket says 172.21.15.16:47872) 2024-09-13T16:21:51.543 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.362+0000 7f94a98a3640 1 -- 172.21.15.16:0/3549900916 learned_addr learned my addr 172.21.15.16:0/3549900916 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:51.543 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.362+0000 7f94a98a3640 1 -- 172.21.15.16:0/3549900916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a00094d0 con 0x7f94a4104420 2024-09-13T16:21:51.543 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.363+0000 7f94a98a3640 1 --2- 172.21.15.16:0/3549900916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94a4104420 0x7f94a41993a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f94a0030560 tx=0x7f94a0030d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:51.543 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.363+0000 7f949affd640 1 -- 172.21.15.16:0/3549900916 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94a0032d60 con 0x7f94a4104420 2024-09-13T16:21:51.543 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.363+0000 7f949affd640 1 -- 172.21.15.16:0/3549900916 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f94a003b8a0 con 0x7f94a4104420 2024-09-13T16:21:51.543 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.363+0000 7f94abb07640 1 -- 172.21.15.16:0/3549900916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a41064a0 con 0x7f94a4104420 2024-09-13T16:21:51.544 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.363+0000 7f949affd640 1 -- 172.21.15.16:0/3549900916 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94a003a680 con 0x7f94a4104420 2024-09-13T16:21:51.544 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.363+0000 7f94abb07640 1 -- 172.21.15.16:0/3549900916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a4199df0 con 0x7f94a4104420 2024-09-13T16:21:51.544 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.364+0000 7f949affd640 1 -- 172.21.15.16:0/3549900916 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f94a0039040 con 0x7f94a4104420 2024-09-13T16:21:51.544 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.364+0000 7f949affd640 1 --2- 172.21.15.16:0/3549900916 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f9490038670 0x7f949003ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:51.544 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.364+0000 7f949affd640 1 -- 172.21.15.16:0/3549900916 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f94a006b9a0 con 0x7f94a4104420 2024-09-13T16:21:51.544 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.364+0000 7f94abb07640 1 -- 172.21.15.16:0/3549900916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a41048a0 con 0x7f94a4104420 2024-09-13T16:21:51.544 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.364+0000 7f94a90a2640 1 --2- 172.21.15.16:0/3549900916 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f9490038670 0x7f949003ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:51.544 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.365+0000 7f94a90a2640 1 --2- 172.21.15.16:0/3549900916 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f9490038670 0x7f949003ab30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f94940098c0 tx=0x7f9494006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:51.544 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.369+0000 7f949affd640 1 -- 172.21.15.16:0/3549900916 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f94a0037070 con 0x7f94a4104420 2024-09-13T16:21:51.544 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:47.545+0000 7f94abb07640 1 -- 172.21.15.16:0/3549900916 --> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi016", "addr": "172.21.15.16", "target": ["mon-mgr", ""]}) v1 -- 0x7f94a40608f0 con 0x7f9490038670 2024-09-13T16:21:51.544 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:50.913+0000 7f949affd640 1 -- 172.21.15.16:0/3549900916 <== mgr.14118 v2:172.21.15.16:6800/1756940569 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f94a40608f0 con 0x7f9490038670 2024-09-13T16:21:51.544 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:50.916+0000 7f94abb07640 1 -- 172.21.15.16:0/3549900916 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f9490038670 msgr2=0x7f949003ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:51.544 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:50.916+0000 7f94abb07640 1 --2- 172.21.15.16:0/3549900916 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f9490038670 0x7f949003ab30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f94940098c0 tx=0x7f9494006db0 comp rx=0 tx=0).stop 2024-09-13T16:21:51.545 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:50.916+0000 7f94abb07640 1 -- 172.21.15.16:0/3549900916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94a4104420 msgr2=0x7f94a41993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:51.545 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:50.916+0000 7f94abb07640 1 --2- 172.21.15.16:0/3549900916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94a4104420 0x7f94a41993a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f94a0030560 tx=0x7f94a0030d90 comp rx=0 tx=0).stop 2024-09-13T16:21:51.545 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:50.916+0000 7f94abb07640 1 -- 172.21.15.16:0/3549900916 shutdown_connections 2024-09-13T16:21:51.545 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:50.916+0000 7f94abb07640 1 --2- 172.21.15.16:0/3549900916 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f9490038670 0x7f949003ab30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:51.545 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:50.916+0000 7f94abb07640 1 --2- 172.21.15.16:0/3549900916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94a4104420 0x7f94a41993a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:51.545 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:50.916+0000 7f94abb07640 1 -- 172.21.15.16:0/3549900916 >> 172.21.15.16:0/3549900916 conn(0x7f94a40ffbb0 msgr2=0x7f94a4100aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:51.545 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:50.916+0000 7f94abb07640 1 -- 172.21.15.16:0/3549900916 shutdown_connections 2024-09-13T16:21:51.545 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:50.916+0000 7f94abb07640 1 -- 172.21.15.16:0/3549900916 wait complete. 2024-09-13T16:21:51.545 INFO:teuthology.orchestra.run.smithi016.stdout:Deploying unmanaged mon service... 2024-09-13T16:21:52.159 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:51 smithi016 ceph-mon[31187]: from='mgr.14118 172.21.15.16:0/600389341' entity='mgr.x' 2024-09-13T16:21:52.159 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:51 smithi016 ceph-mon[31187]: Added host smithi016 2024-09-13T16:21:52.159 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:51 smithi016 ceph-mon[31187]: from='mgr.14118 172.21.15.16:0/600389341' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:21:53.718 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-13T16:21:53.719 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.676+0000 7f2760d60640 1 Processor -- start 2024-09-13T16:21:53.719 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.676+0000 7f2760d60640 1 -- start start 2024-09-13T16:21:53.719 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.677+0000 7f2760d60640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f275c105520 0x7f275c105920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:53.719 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.677+0000 7f2760d60640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275c105ef0 con 0x7f275c105520 2024-09-13T16:21:53.719 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.677+0000 7f275a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f275c105520 0x7f275c105920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:53.719 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.677+0000 7f275a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f275c105520 0x7f275c105920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47884/0 (socket says 172.21.15.16:47884) 2024-09-13T16:21:53.719 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.677+0000 7f275a59c640 1 -- 172.21.15.16:0/167394382 learned_addr learned my addr 172.21.15.16:0/167394382 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:53.719 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.677+0000 7f275a59c640 1 -- 172.21.15.16:0/167394382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f275c106720 con 0x7f275c105520 2024-09-13T16:21:53.720 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.678+0000 7f275a59c640 1 --2- 172.21.15.16:0/167394382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f275c105520 0x7f275c105920 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f274c009820 tx=0x7f274c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b18383d59c869c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:53.720 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.678+0000 7f275959a640 1 -- 172.21.15.16:0/167394382 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f274c02ae70 con 0x7f275c105520 2024-09-13T16:21:53.720 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.678+0000 7f275959a640 1 -- 172.21.15.16:0/167394382 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f274c0328b0 con 0x7f275c105520 2024-09-13T16:21:53.720 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.678+0000 7f275959a640 1 -- 172.21.15.16:0/167394382 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f274c030850 con 0x7f275c105520 2024-09-13T16:21:53.720 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.679+0000 7f2760d60640 1 -- 172.21.15.16:0/167394382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f275c105520 msgr2=0x7f275c105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:53.720 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.679+0000 7f2760d60640 1 --2- 172.21.15.16:0/167394382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f275c105520 0x7f275c105920 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f274c009820 tx=0x7f274c02a270 comp rx=0 tx=0).stop 2024-09-13T16:21:53.720 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.679+0000 7f2760d60640 1 -- 172.21.15.16:0/167394382 shutdown_connections 2024-09-13T16:21:53.720 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.679+0000 7f2760d60640 1 --2- 172.21.15.16:0/167394382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f275c105520 0x7f275c105920 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:53.720 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.679+0000 7f2760d60640 1 -- 172.21.15.16:0/167394382 >> 172.21.15.16:0/167394382 conn(0x7f275c100cb0 msgr2=0x7f275c1030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:53.720 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.679+0000 7f2760d60640 1 -- 172.21.15.16:0/167394382 shutdown_connections 2024-09-13T16:21:53.721 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.679+0000 7f2760d60640 1 -- 172.21.15.16:0/167394382 wait complete. 2024-09-13T16:21:53.721 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.680+0000 7f2760d60640 1 Processor -- start 2024-09-13T16:21:53.721 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.680+0000 7f2760d60640 1 -- start start 2024-09-13T16:21:53.721 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.680+0000 7f2760d60640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f275c105520 0x7f275c199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:53.721 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.680+0000 7f2760d60640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275c199880 con 0x7f275c105520 2024-09-13T16:21:53.721 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.680+0000 7f275a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f275c105520 0x7f275c199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:53.721 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.680+0000 7f275a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f275c105520 0x7f275c199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47886/0 (socket says 172.21.15.16:47886) 2024-09-13T16:21:53.721 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.680+0000 7f275a59c640 1 -- 172.21.15.16:0/1407699616 learned_addr learned my addr 172.21.15.16:0/1407699616 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:53.721 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.680+0000 7f275a59c640 1 -- 172.21.15.16:0/1407699616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f274c0094d0 con 0x7f275c105520 2024-09-13T16:21:53.722 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.681+0000 7f275a59c640 1 --2- 172.21.15.16:0/1407699616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f275c105520 0x7f275c199340 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f274c030560 tx=0x7f274c030d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:53.722 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.681+0000 7f274b7fe640 1 -- 172.21.15.16:0/1407699616 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f274c032d60 con 0x7f275c105520 2024-09-13T16:21:53.722 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.681+0000 7f2760d60640 1 -- 172.21.15.16:0/1407699616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f275c1075a0 con 0x7f275c105520 2024-09-13T16:21:53.722 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.681+0000 7f274b7fe640 1 -- 172.21.15.16:0/1407699616 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f274c03b8a0 con 0x7f275c105520 2024-09-13T16:21:53.722 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.681+0000 7f274b7fe640 1 -- 172.21.15.16:0/1407699616 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f274c03a680 con 0x7f275c105520 2024-09-13T16:21:53.722 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.681+0000 7f2760d60640 1 -- 172.21.15.16:0/1407699616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f275c199d90 con 0x7f275c105520 2024-09-13T16:21:53.722 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.682+0000 7f274b7fe640 1 -- 172.21.15.16:0/1407699616 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f274c039040 con 0x7f275c105520 2024-09-13T16:21:53.722 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.682+0000 7f2760d60640 1 -- 172.21.15.16:0/1407699616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f275c1059a0 con 0x7f275c105520 2024-09-13T16:21:53.722 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.682+0000 7f274b7fe640 1 --2- 172.21.15.16:0/1407699616 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f2744038620 0x7f274403aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:53.722 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.682+0000 7f274b7fe640 1 -- 172.21.15.16:0/1407699616 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f274c06bae0 con 0x7f275c105520 2024-09-13T16:21:53.722 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.682+0000 7f2759d9b640 1 --2- 172.21.15.16:0/1407699616 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f2744038620 0x7f274403aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:53.722 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.683+0000 7f2759d9b640 1 --2- 172.21.15.16:0/1407699616 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f2744038620 0x7f274403aae0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f27500098c0 tx=0x7f2750006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:53.722 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.687+0000 7f274b7fe640 1 -- 172.21.15.16:0/1407699616 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f274c031b90 con 0x7f275c105520 2024-09-13T16:21:53.723 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.873+0000 7f2760d60640 1 -- 172.21.15.16:0/1407699616 --> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f275c109920 con 0x7f2744038620 2024-09-13T16:21:53.723 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.877+0000 7f274b7fe640 1 -- 172.21.15.16:0/1407699616 <== mgr.14118 v2:172.21.15.16:6800/1756940569 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f275c109920 con 0x7f2744038620 2024-09-13T16:21:53.723 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.879+0000 7f2760d60640 1 -- 172.21.15.16:0/1407699616 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f2744038620 msgr2=0x7f274403aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:53.723 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.879+0000 7f2760d60640 1 --2- 172.21.15.16:0/1407699616 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f2744038620 0x7f274403aae0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f27500098c0 tx=0x7f2750006db0 comp rx=0 tx=0).stop 2024-09-13T16:21:53.723 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.879+0000 7f2760d60640 1 -- 172.21.15.16:0/1407699616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f275c105520 msgr2=0x7f275c199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:53.723 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.879+0000 7f2760d60640 1 --2- 172.21.15.16:0/1407699616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f275c105520 0x7f275c199340 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f274c030560 tx=0x7f274c030d90 comp rx=0 tx=0).stop 2024-09-13T16:21:53.723 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.879+0000 7f2760d60640 1 -- 172.21.15.16:0/1407699616 shutdown_connections 2024-09-13T16:21:53.723 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.879+0000 7f2760d60640 1 --2- 172.21.15.16:0/1407699616 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f2744038620 0x7f274403aae0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:53.723 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.879+0000 7f2760d60640 1 --2- 172.21.15.16:0/1407699616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f275c105520 0x7f275c199340 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:53.723 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.879+0000 7f2760d60640 1 -- 172.21.15.16:0/1407699616 >> 172.21.15.16:0/1407699616 conn(0x7f275c100cb0 msgr2=0x7f275c1017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:53.723 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.880+0000 7f2760d60640 1 -- 172.21.15.16:0/1407699616 shutdown_connections 2024-09-13T16:21:53.723 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:52.880+0000 7f2760d60640 1 -- 172.21.15.16:0/1407699616 wait complete. 2024-09-13T16:21:53.723 INFO:teuthology.orchestra.run.smithi016.stdout:Deploying unmanaged mgr service... 2024-09-13T16:21:54.038 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:53 smithi016 ceph-mon[31187]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:21:54.038 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:53 smithi016 ceph-mon[31187]: Saving service mon spec with placement count:5 2024-09-13T16:21:54.038 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:53 smithi016 ceph-mon[31187]: from='mgr.14118 172.21.15.16:0/600389341' entity='mgr.x' 2024-09-13T16:21:54.038 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:53 smithi016 ceph-mon[31187]: from='mgr.14118 172.21.15.16:0/600389341' entity='mgr.x' 2024-09-13T16:21:54.038 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:53 smithi016 ceph-mon[31187]: from='mgr.14118 172.21.15.16:0/600389341' entity='mgr.x' 2024-09-13T16:21:55.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-13T16:21:55.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.118+0000 7f2a7c19a640 1 Processor -- start 2024-09-13T16:21:55.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.118+0000 7f2a7c19a640 1 -- start start 2024-09-13T16:21:55.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.118+0000 7f2a7c19a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a7407be70 0x7f2a7407a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:55.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.118+0000 7f2a7c19a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7407a8a0 con 0x7f2a7407be70 2024-09-13T16:21:55.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.119+0000 7f2a79f36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a7407be70 0x7f2a7407a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:55.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.119+0000 7f2a79f36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a7407be70 0x7f2a7407a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49828/0 (socket says 172.21.15.16:49828) 2024-09-13T16:21:55.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.119+0000 7f2a79f36640 1 -- 172.21.15.16:0/299560618 learned_addr learned my addr 172.21.15.16:0/299560618 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:55.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.119+0000 7f2a79f36640 1 -- 172.21.15.16:0/299560618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a7407a9e0 con 0x7f2a7407be70 2024-09-13T16:21:55.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.120+0000 7f2a79f36640 1 --2- 172.21.15.16:0/299560618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a7407be70 0x7f2a7407a2d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2a70009820 tx=0x7f2a7002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c9e1d250ed691b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:55.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.120+0000 7f2a78f34640 1 -- 172.21.15.16:0/299560618 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a7002ae70 con 0x7f2a7407be70 2024-09-13T16:21:55.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.120+0000 7f2a78f34640 1 -- 172.21.15.16:0/299560618 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2a700328d0 con 0x7f2a7407be70 2024-09-13T16:21:55.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.120+0000 7f2a78f34640 1 -- 172.21.15.16:0/299560618 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a700307a0 con 0x7f2a7407be70 2024-09-13T16:21:55.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.120+0000 7f2a7c19a640 1 -- 172.21.15.16:0/299560618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a7407be70 msgr2=0x7f2a7407a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:55.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.120+0000 7f2a7c19a640 1 --2- 172.21.15.16:0/299560618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a7407be70 0x7f2a7407a2d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2a70009820 tx=0x7f2a7002a270 comp rx=0 tx=0).stop 2024-09-13T16:21:55.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.120+0000 7f2a7c19a640 1 -- 172.21.15.16:0/299560618 shutdown_connections 2024-09-13T16:21:55.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.120+0000 7f2a7c19a640 1 --2- 172.21.15.16:0/299560618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a7407be70 0x7f2a7407a2d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:55.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.120+0000 7f2a7c19a640 1 -- 172.21.15.16:0/299560618 >> 172.21.15.16:0/299560618 conn(0x7f2a74100cb0 msgr2=0x7f2a741030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:55.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.121+0000 7f2a7c19a640 1 -- 172.21.15.16:0/299560618 shutdown_connections 2024-09-13T16:21:55.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.121+0000 7f2a7c19a640 1 -- 172.21.15.16:0/299560618 wait complete. 2024-09-13T16:21:55.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.121+0000 7f2a7c19a640 1 Processor -- start 2024-09-13T16:21:55.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.121+0000 7f2a7c19a640 1 -- start start 2024-09-13T16:21:55.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.122+0000 7f2a7c19a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a7407be70 0x7f2a7419b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:55.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.122+0000 7f2a7c19a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7419b910 con 0x7f2a7407be70 2024-09-13T16:21:55.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.122+0000 7f2a79f36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a7407be70 0x7f2a7419b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:55.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.122+0000 7f2a79f36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a7407be70 0x7f2a7419b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49830/0 (socket says 172.21.15.16:49830) 2024-09-13T16:21:55.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.122+0000 7f2a79f36640 1 -- 172.21.15.16:0/1487645049 learned_addr learned my addr 172.21.15.16:0/1487645049 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:55.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.122+0000 7f2a79f36640 1 -- 172.21.15.16:0/1487645049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a700094d0 con 0x7f2a7407be70 2024-09-13T16:21:55.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.122+0000 7f2a79f36640 1 --2- 172.21.15.16:0/1487645049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a7407be70 0x7f2a7419b3d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2a700304f0 tx=0x7f2a70030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:55.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.122+0000 7f2a6affd640 1 -- 172.21.15.16:0/1487645049 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a70032c50 con 0x7f2a7407be70 2024-09-13T16:21:55.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.122+0000 7f2a6affd640 1 -- 172.21.15.16:0/1487645049 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2a70032db0 con 0x7f2a7407be70 2024-09-13T16:21:55.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.122+0000 7f2a7c19a640 1 -- 172.21.15.16:0/1487645049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a7419bb10 con 0x7f2a7407be70 2024-09-13T16:21:55.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.122+0000 7f2a6affd640 1 -- 172.21.15.16:0/1487645049 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a7003a5b0 con 0x7f2a7407be70 2024-09-13T16:21:55.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.122+0000 7f2a7c19a640 1 -- 172.21.15.16:0/1487645049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a7419bfb0 con 0x7f2a7407be70 2024-09-13T16:21:55.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.123+0000 7f2a6affd640 1 -- 172.21.15.16:0/1487645049 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f2a70039040 con 0x7f2a7407be70 2024-09-13T16:21:55.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.123+0000 7f2a7c19a640 1 -- 172.21.15.16:0/1487645049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a7410b910 con 0x7f2a7407be70 2024-09-13T16:21:55.372 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.124+0000 7f2a6affd640 1 --2- 172.21.15.16:0/1487645049 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f2a60038670 0x7f2a6003ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:55.372 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.124+0000 7f2a6affd640 1 -- 172.21.15.16:0/1487645049 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2a7006b960 con 0x7f2a7407be70 2024-09-13T16:21:55.372 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.124+0000 7f2a79735640 1 --2- 172.21.15.16:0/1487645049 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f2a60038670 0x7f2a6003ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:55.372 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.124+0000 7f2a79735640 1 --2- 172.21.15.16:0/1487645049 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f2a60038670 0x7f2a6003ab30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2a640098c0 tx=0x7f2a64006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:55.372 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.128+0000 7f2a6affd640 1 -- 172.21.15.16:0/1487645049 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f2a70031720 con 0x7f2a7407be70 2024-09-13T16:21:55.372 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.332+0000 7f2a7c19a640 1 -- 172.21.15.16:0/1487645049 --> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f2a7419e8f0 con 0x7f2a60038670 2024-09-13T16:21:55.372 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.337+0000 7f2a6affd640 1 -- 172.21.15.16:0/1487645049 <== mgr.14118 v2:172.21.15.16:6800/1756940569 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f2a7419e8f0 con 0x7f2a60038670 2024-09-13T16:21:55.372 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.339+0000 7f2a7c19a640 1 -- 172.21.15.16:0/1487645049 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f2a60038670 msgr2=0x7f2a6003ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:55.372 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.339+0000 7f2a7c19a640 1 --2- 172.21.15.16:0/1487645049 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f2a60038670 0x7f2a6003ab30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2a640098c0 tx=0x7f2a64006db0 comp rx=0 tx=0).stop 2024-09-13T16:21:55.372 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.339+0000 7f2a7c19a640 1 -- 172.21.15.16:0/1487645049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a7407be70 msgr2=0x7f2a7419b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:55.372 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.339+0000 7f2a7c19a640 1 --2- 172.21.15.16:0/1487645049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a7407be70 0x7f2a7419b3d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2a700304f0 tx=0x7f2a70030cc0 comp rx=0 tx=0).stop 2024-09-13T16:21:55.372 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.339+0000 7f2a7c19a640 1 -- 172.21.15.16:0/1487645049 shutdown_connections 2024-09-13T16:21:55.373 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.339+0000 7f2a7c19a640 1 --2- 172.21.15.16:0/1487645049 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f2a60038670 0x7f2a6003ab30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:55.373 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.339+0000 7f2a7c19a640 1 --2- 172.21.15.16:0/1487645049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a7407be70 0x7f2a7419b3d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:55.373 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.339+0000 7f2a7c19a640 1 -- 172.21.15.16:0/1487645049 >> 172.21.15.16:0/1487645049 conn(0x7f2a74100cb0 msgr2=0x7f2a74101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:55.373 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.339+0000 7f2a7c19a640 1 -- 172.21.15.16:0/1487645049 shutdown_connections 2024-09-13T16:21:55.373 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:54.339+0000 7f2a7c19a640 1 -- 172.21.15.16:0/1487645049 wait complete. 2024-09-13T16:21:55.374 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:55 smithi016 ceph-mon[31187]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:21:55.374 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:55 smithi016 ceph-mon[31187]: Saving service mgr spec with placement count:2 2024-09-13T16:21:55.374 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:55 smithi016 ceph-mon[31187]: from='mgr.14118 172.21.15.16:0/600389341' entity='mgr.x' 2024-09-13T16:21:55.374 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:55 smithi016 ceph-mon[31187]: from='mgr.14118 172.21.15.16:0/600389341' entity='mgr.x' 2024-09-13T16:21:56.600 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.821+0000 7fbf1478b640 1 Processor -- start 2024-09-13T16:21:56.601 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.821+0000 7fbf1478b640 1 -- start start 2024-09-13T16:21:56.601 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.821+0000 7fbf1478b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf0c105540 0x7fbf0c105940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:56.601 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.821+0000 7fbf1478b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c105f10 con 0x7fbf0c105540 2024-09-13T16:21:56.601 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.821+0000 7fbf12527640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf0c105540 0x7fbf0c105940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:56.601 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.821+0000 7fbf12527640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf0c105540 0x7fbf0c105940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49842/0 (socket says 172.21.15.16:49842) 2024-09-13T16:21:56.602 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.821+0000 7fbf12527640 1 -- 172.21.15.16:0/1885413702 learned_addr learned my addr 172.21.15.16:0/1885413702 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:56.602 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.822+0000 7fbf12527640 1 -- 172.21.15.16:0/1885413702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf0c106740 con 0x7fbf0c105540 2024-09-13T16:21:56.602 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.822+0000 7fbf12527640 1 --2- 172.21.15.16:0/1885413702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf0c105540 0x7fbf0c105940 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbf08009820 tx=0x7fbf0802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d6f98987d7e276 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:56.602 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.822+0000 7fbf11525640 1 -- 172.21.15.16:0/1885413702 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf0802ae70 con 0x7fbf0c105540 2024-09-13T16:21:56.602 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.823+0000 7fbf11525640 1 -- 172.21.15.16:0/1885413702 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf080328d0 con 0x7fbf0c105540 2024-09-13T16:21:56.602 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.823+0000 7fbf11525640 1 -- 172.21.15.16:0/1885413702 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf080307a0 con 0x7fbf0c105540 2024-09-13T16:21:56.602 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.823+0000 7fbf1478b640 1 -- 172.21.15.16:0/1885413702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf0c105540 msgr2=0x7fbf0c105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:56.602 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.823+0000 7fbf1478b640 1 --2- 172.21.15.16:0/1885413702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf0c105540 0x7fbf0c105940 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbf08009820 tx=0x7fbf0802a270 comp rx=0 tx=0).stop 2024-09-13T16:21:56.602 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.823+0000 7fbf1478b640 1 -- 172.21.15.16:0/1885413702 shutdown_connections 2024-09-13T16:21:56.603 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.823+0000 7fbf1478b640 1 --2- 172.21.15.16:0/1885413702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf0c105540 0x7fbf0c105940 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:56.603 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.823+0000 7fbf1478b640 1 -- 172.21.15.16:0/1885413702 >> 172.21.15.16:0/1885413702 conn(0x7fbf0c100cf0 msgr2=0x7fbf0c103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:56.603 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.823+0000 7fbf1478b640 1 -- 172.21.15.16:0/1885413702 shutdown_connections 2024-09-13T16:21:56.603 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.823+0000 7fbf1478b640 1 -- 172.21.15.16:0/1885413702 wait complete. 2024-09-13T16:21:56.603 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.824+0000 7fbf1478b640 1 Processor -- start 2024-09-13T16:21:56.603 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.824+0000 7fbf1478b640 1 -- start start 2024-09-13T16:21:56.603 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.824+0000 7fbf1478b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf0c105540 0x7fbf0c1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:56.603 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.824+0000 7fbf1478b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c199710 con 0x7fbf0c105540 2024-09-13T16:21:56.603 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.824+0000 7fbf12527640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf0c105540 0x7fbf0c1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:56.603 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.824+0000 7fbf12527640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf0c105540 0x7fbf0c1991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49850/0 (socket says 172.21.15.16:49850) 2024-09-13T16:21:56.603 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.824+0000 7fbf12527640 1 -- 172.21.15.16:0/3181288963 learned_addr learned my addr 172.21.15.16:0/3181288963 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:56.604 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.824+0000 7fbf12527640 1 -- 172.21.15.16:0/3181288963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf080094d0 con 0x7fbf0c105540 2024-09-13T16:21:56.604 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.824+0000 7fbf12527640 1 --2- 172.21.15.16:0/3181288963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf0c105540 0x7fbf0c1991d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbf080304f0 tx=0x7fbf08030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:56.604 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.825+0000 7fbf037fe640 1 -- 172.21.15.16:0/3181288963 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf08032c50 con 0x7fbf0c105540 2024-09-13T16:21:56.604 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.825+0000 7fbf037fe640 1 -- 172.21.15.16:0/3181288963 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf08032db0 con 0x7fbf0c105540 2024-09-13T16:21:56.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.825+0000 7fbf1478b640 1 -- 172.21.15.16:0/3181288963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf0c199910 con 0x7fbf0c105540 2024-09-13T16:21:56.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.825+0000 7fbf037fe640 1 -- 172.21.15.16:0/3181288963 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf0803a5b0 con 0x7fbf0c105540 2024-09-13T16:21:56.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.825+0000 7fbf1478b640 1 -- 172.21.15.16:0/3181288963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf0c199db0 con 0x7fbf0c105540 2024-09-13T16:21:56.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.825+0000 7fbf037fe640 1 -- 172.21.15.16:0/3181288963 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fbf08039040 con 0x7fbf0c105540 2024-09-13T16:21:56.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.826+0000 7fbf037fe640 1 --2- 172.21.15.16:0/3181288963 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7fbef8038670 0x7fbef803ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:56.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.826+0000 7fbf1478b640 1 -- 172.21.15.16:0/3181288963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf0c109970 con 0x7fbf0c105540 2024-09-13T16:21:56.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.826+0000 7fbf037fe640 1 -- 172.21.15.16:0/3181288963 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbf0806b8d0 con 0x7fbf0c105540 2024-09-13T16:21:56.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.826+0000 7fbf11d26640 1 --2- 172.21.15.16:0/3181288963 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7fbef8038670 0x7fbef803ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:56.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.826+0000 7fbf11d26640 1 --2- 172.21.15.16:0/3181288963 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7fbef8038670 0x7fbef803ab30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbefc0098c0 tx=0x7fbefc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:56.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:55.830+0000 7fbf037fe640 1 -- 172.21.15.16:0/3181288963 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fbf08037070 con 0x7fbf0c105540 2024-09-13T16:21:56.605 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.006+0000 7fbf1478b640 1 -- 172.21.15.16:0/3181288963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fbf0c19c690 con 0x7fbf0c105540 2024-09-13T16:21:56.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.010+0000 7fbf037fe640 1 -- 172.21.15.16:0/3181288963 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fbf0803f030 con 0x7fbf0c105540 2024-09-13T16:21:56.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.012+0000 7fbf1478b640 1 -- 172.21.15.16:0/3181288963 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7fbef8038670 msgr2=0x7fbef803ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:56.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.012+0000 7fbf1478b640 1 --2- 172.21.15.16:0/3181288963 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7fbef8038670 0x7fbef803ab30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbefc0098c0 tx=0x7fbefc006db0 comp rx=0 tx=0).stop 2024-09-13T16:21:56.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.012+0000 7fbf1478b640 1 -- 172.21.15.16:0/3181288963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf0c105540 msgr2=0x7fbf0c1991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:56.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.012+0000 7fbf1478b640 1 --2- 172.21.15.16:0/3181288963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf0c105540 0x7fbf0c1991d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbf080304f0 tx=0x7fbf08030cc0 comp rx=0 tx=0).stop 2024-09-13T16:21:56.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.012+0000 7fbf1478b640 1 -- 172.21.15.16:0/3181288963 shutdown_connections 2024-09-13T16:21:56.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.012+0000 7fbf1478b640 1 --2- 172.21.15.16:0/3181288963 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7fbef8038670 0x7fbef803ab30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:56.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.012+0000 7fbf1478b640 1 --2- 172.21.15.16:0/3181288963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf0c105540 0x7fbf0c1991d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:56.606 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.012+0000 7fbf1478b640 1 -- 172.21.15.16:0/3181288963 >> 172.21.15.16:0/3181288963 conn(0x7fbf0c100cf0 msgr2=0x7fbf0c1017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:56.607 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.012+0000 7fbf1478b640 1 -- 172.21.15.16:0/3181288963 shutdown_connections 2024-09-13T16:21:56.607 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.012+0000 7fbf1478b640 1 -- 172.21.15.16:0/3181288963 wait complete. 2024-09-13T16:21:57.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:56 smithi016 ceph-mon[31187]: from='mgr.14118 172.21.15.16:0/600389341' entity='mgr.x' 2024-09-13T16:21:57.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:56 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3181288963' entity='client.admin' 2024-09-13T16:21:58.336 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:58 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1503469933' entity='client.admin' 2024-09-13T16:21:58.338 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.977+0000 7f53ce126640 1 Processor -- start 2024-09-13T16:21:58.344 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.977+0000 7f53ce126640 1 -- start start 2024-09-13T16:21:58.344 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.977+0000 7f53ce126640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53c8105f90 0x7f53c8108380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:58.344 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.977+0000 7f53ce126640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c8079e40 con 0x7f53c8105f90 2024-09-13T16:21:58.344 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.978+0000 7f53c77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53c8105f90 0x7f53c8108380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:58.344 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.978+0000 7f53c77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53c8105f90 0x7f53c8108380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49858/0 (socket says 172.21.15.16:49858) 2024-09-13T16:21:58.344 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.978+0000 7f53c77fe640 1 -- 172.21.15.16:0/1450863496 learned_addr learned my addr 172.21.15.16:0/1450863496 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:58.344 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.978+0000 7f53c77fe640 1 -- 172.21.15.16:0/1450863496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c81088c0 con 0x7f53c8105f90 2024-09-13T16:21:58.344 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.979+0000 7f53c77fe640 1 --2- 172.21.15.16:0/1450863496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53c8105f90 0x7f53c8108380 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f53b8009820 tx=0x7f53b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7eefd66f6c690d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:58.344 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.979+0000 7f53c67fc640 1 -- 172.21.15.16:0/1450863496 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f53b802ae70 con 0x7f53c8105f90 2024-09-13T16:21:58.345 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.979+0000 7f53c67fc640 1 -- 172.21.15.16:0/1450863496 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f53b80328d0 con 0x7f53c8105f90 2024-09-13T16:21:58.345 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.979+0000 7f53c67fc640 1 -- 172.21.15.16:0/1450863496 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f53b80307a0 con 0x7f53c8105f90 2024-09-13T16:21:58.345 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.979+0000 7f53ce126640 1 -- 172.21.15.16:0/1450863496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53c8105f90 msgr2=0x7f53c8108380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:58.345 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.979+0000 7f53ce126640 1 --2- 172.21.15.16:0/1450863496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53c8105f90 0x7f53c8108380 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f53b8009820 tx=0x7f53b802a270 comp rx=0 tx=0).stop 2024-09-13T16:21:58.345 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.979+0000 7f53ce126640 1 -- 172.21.15.16:0/1450863496 shutdown_connections 2024-09-13T16:21:58.345 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.979+0000 7f53ce126640 1 --2- 172.21.15.16:0/1450863496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53c8105f90 0x7f53c8108380 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:58.345 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.979+0000 7f53ce126640 1 -- 172.21.15.16:0/1450863496 >> 172.21.15.16:0/1450863496 conn(0x7f53c80ffbb0 msgr2=0x7f53c8101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:58.345 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.980+0000 7f53ce126640 1 -- 172.21.15.16:0/1450863496 shutdown_connections 2024-09-13T16:21:58.345 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.980+0000 7f53ce126640 1 -- 172.21.15.16:0/1450863496 wait complete. 2024-09-13T16:21:58.345 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.980+0000 7f53ce126640 1 Processor -- start 2024-09-13T16:21:58.346 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.980+0000 7f53ce126640 1 -- start start 2024-09-13T16:21:58.346 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.980+0000 7f53ce126640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53c8105f90 0x7f53c819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:58.346 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.980+0000 7f53ce126640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c819b8e0 con 0x7f53c8105f90 2024-09-13T16:21:58.346 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.981+0000 7f53c77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53c8105f90 0x7f53c819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:58.346 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.981+0000 7f53c77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53c8105f90 0x7f53c819b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49874/0 (socket says 172.21.15.16:49874) 2024-09-13T16:21:58.346 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.981+0000 7f53c77fe640 1 -- 172.21.15.16:0/1503469933 learned_addr learned my addr 172.21.15.16:0/1503469933 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:21:58.346 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.981+0000 7f53c77fe640 1 -- 172.21.15.16:0/1503469933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b80094d0 con 0x7f53c8105f90 2024-09-13T16:21:58.346 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.981+0000 7f53c77fe640 1 --2- 172.21.15.16:0/1503469933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53c8105f90 0x7f53c819b3a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f53b80097f0 tx=0x7f53b8032db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:58.346 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.981+0000 7f53c4ff9640 1 -- 172.21.15.16:0/1503469933 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f53b8030da0 con 0x7f53c8105f90 2024-09-13T16:21:58.346 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.981+0000 7f53c4ff9640 1 -- 172.21.15.16:0/1503469933 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f53b803b8c0 con 0x7f53c8105f90 2024-09-13T16:21:58.347 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.981+0000 7f53c4ff9640 1 -- 172.21.15.16:0/1503469933 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f53b803a570 con 0x7f53c8105f90 2024-09-13T16:21:58.347 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.981+0000 7f53ce126640 1 -- 172.21.15.16:0/1503469933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c819bae0 con 0x7f53c8105f90 2024-09-13T16:21:58.347 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.981+0000 7f53ce126640 1 -- 172.21.15.16:0/1503469933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c819bf80 con 0x7f53c8105f90 2024-09-13T16:21:58.347 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.982+0000 7f53ce126640 1 -- 172.21.15.16:0/1503469933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c81096c0 con 0x7f53c8105f90 2024-09-13T16:21:58.347 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.983+0000 7f53c4ff9640 1 -- 172.21.15.16:0/1503469933 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f53b8039040 con 0x7f53c8105f90 2024-09-13T16:21:58.347 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.983+0000 7f53c4ff9640 1 --2- 172.21.15.16:0/1503469933 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f53b0038620 0x7f53b003aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:21:58.347 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.983+0000 7f53c4ff9640 1 -- 172.21.15.16:0/1503469933 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f53b806b960 con 0x7f53c8105f90 2024-09-13T16:21:58.347 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.983+0000 7f53c6ffd640 1 --2- 172.21.15.16:0/1503469933 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f53b0038620 0x7f53b003aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:21:58.347 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.983+0000 7f53c6ffd640 1 --2- 172.21.15.16:0/1503469933 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f53b0038620 0x7f53b003aae0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f53bc0098c0 tx=0x7f53bc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:21:58.347 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:56.987+0000 7f53c4ff9640 1 -- 172.21.15.16:0/1503469933 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f53b803bdb0 con 0x7f53c8105f90 2024-09-13T16:21:58.347 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:57.161+0000 7f53ce126640 1 -- 172.21.15.16:0/1503469933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f53c8109900 con 0x7f53c8105f90 2024-09-13T16:21:58.347 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:57.164+0000 7f53c4ff9640 1 -- 172.21.15.16:0/1503469933 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f53b803f070 con 0x7f53c8105f90 2024-09-13T16:21:58.347 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:57.166+0000 7f53ce126640 1 -- 172.21.15.16:0/1503469933 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f53b0038620 msgr2=0x7f53b003aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:58.348 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:57.166+0000 7f53ce126640 1 --2- 172.21.15.16:0/1503469933 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f53b0038620 0x7f53b003aae0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f53bc0098c0 tx=0x7f53bc006db0 comp rx=0 tx=0).stop 2024-09-13T16:21:58.348 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:57.166+0000 7f53ce126640 1 -- 172.21.15.16:0/1503469933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53c8105f90 msgr2=0x7f53c819b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:21:58.348 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:57.166+0000 7f53ce126640 1 --2- 172.21.15.16:0/1503469933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53c8105f90 0x7f53c819b3a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f53b80097f0 tx=0x7f53b8032db0 comp rx=0 tx=0).stop 2024-09-13T16:21:58.348 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:57.166+0000 7f53ce126640 1 -- 172.21.15.16:0/1503469933 shutdown_connections 2024-09-13T16:21:58.348 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:57.166+0000 7f53ce126640 1 --2- 172.21.15.16:0/1503469933 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f53b0038620 0x7f53b003aae0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:58.348 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:57.166+0000 7f53ce126640 1 --2- 172.21.15.16:0/1503469933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53c8105f90 0x7f53c819b3a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:21:58.348 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:57.166+0000 7f53ce126640 1 -- 172.21.15.16:0/1503469933 >> 172.21.15.16:0/1503469933 conn(0x7f53c80ffbb0 msgr2=0x7f53c81006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:21:58.348 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:57.167+0000 7f53ce126640 1 -- 172.21.15.16:0/1503469933 shutdown_connections 2024-09-13T16:21:58.348 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:57.167+0000 7f53ce126640 1 -- 172.21.15.16:0/1503469933 wait complete. 2024-09-13T16:21:58.348 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling the dashboard module... 2024-09-13T16:21:59.813 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:21:59 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/884196114' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-13T16:21:59.813 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:59 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: ignoring --setuser ceph since I am not root 2024-09-13T16:21:59.813 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:59 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: ignoring --setgroup ceph since I am not root 2024-09-13T16:21:59.814 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:59 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:59.629+0000 7fa3303b2040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T16:21:59.814 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:59 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:59.688+0000 7fa3303b2040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T16:22:00.068 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:59 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:59.846+0000 7fa3303b2040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T16:22:00.068 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:21:59 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:21:59.915+0000 7fa3303b2040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T16:22:00.069 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:00 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:22:00.067+0000 7fa3303b2040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T16:22:00.105 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.199+0000 7f7091845640 1 Processor -- start 2024-09-13T16:22:00.105 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.199+0000 7f7091845640 1 -- start start 2024-09-13T16:22:00.105 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.199+0000 7f7091845640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f708c07be30 0x7f708c07a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:00.105 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.199+0000 7f7091845640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708c07a860 con 0x7f708c07be30 2024-09-13T16:22:00.105 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.199+0000 7f708affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f708c07be30 0x7f708c07a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:00.105 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.199+0000 7f708affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f708c07be30 0x7f708c07a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49882/0 (socket says 172.21.15.16:49882) 2024-09-13T16:22:00.105 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.199+0000 7f708affd640 1 -- 172.21.15.16:0/2392963375 learned_addr learned my addr 172.21.15.16:0/2392963375 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:22:00.105 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.199+0000 7f708affd640 1 -- 172.21.15.16:0/2392963375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708c07a9a0 con 0x7f708c07be30 2024-09-13T16:22:00.105 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.200+0000 7f708affd640 1 --2- 172.21.15.16:0/2392963375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f708c07be30 0x7f708c07a290 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f707c009820 tx=0x7f707c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6e047062154919 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:00.106 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.200+0000 7f708a7fc640 1 -- 172.21.15.16:0/2392963375 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f707c02ae70 con 0x7f708c07be30 2024-09-13T16:22:00.106 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.200+0000 7f708a7fc640 1 -- 172.21.15.16:0/2392963375 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f707c0328d0 con 0x7f708c07be30 2024-09-13T16:22:00.106 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.200+0000 7f708a7fc640 1 -- 172.21.15.16:0/2392963375 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f707c0307a0 con 0x7f708c07be30 2024-09-13T16:22:00.106 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.201+0000 7f7091845640 1 -- 172.21.15.16:0/2392963375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f708c07be30 msgr2=0x7f708c07a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:00.106 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.201+0000 7f7091845640 1 --2- 172.21.15.16:0/2392963375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f708c07be30 0x7f708c07a290 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f707c009820 tx=0x7f707c02a270 comp rx=0 tx=0).stop 2024-09-13T16:22:00.106 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.201+0000 7f7091845640 1 -- 172.21.15.16:0/2392963375 shutdown_connections 2024-09-13T16:22:00.106 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.201+0000 7f7091845640 1 --2- 172.21.15.16:0/2392963375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f708c07be30 0x7f708c07a290 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:00.106 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.201+0000 7f7091845640 1 -- 172.21.15.16:0/2392963375 >> 172.21.15.16:0/2392963375 conn(0x7f708c100cb0 msgr2=0x7f708c1030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:00.106 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.201+0000 7f7091845640 1 -- 172.21.15.16:0/2392963375 shutdown_connections 2024-09-13T16:22:00.106 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.201+0000 7f7091845640 1 -- 172.21.15.16:0/2392963375 wait complete. 2024-09-13T16:22:00.106 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.202+0000 7f7091845640 1 Processor -- start 2024-09-13T16:22:00.106 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.202+0000 7f7091845640 1 -- start start 2024-09-13T16:22:00.106 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.202+0000 7f7091845640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f708c07be30 0x7f708c19b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:00.106 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.202+0000 7f7091845640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708c19b930 con 0x7f708c07be30 2024-09-13T16:22:00.107 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.202+0000 7f708affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f708c07be30 0x7f708c19b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:00.107 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.202+0000 7f708affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f708c07be30 0x7f708c19b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49898/0 (socket says 172.21.15.16:49898) 2024-09-13T16:22:00.107 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.202+0000 7f708affd640 1 -- 172.21.15.16:0/884196114 learned_addr learned my addr 172.21.15.16:0/884196114 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:22:00.107 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.203+0000 7f708affd640 1 -- 172.21.15.16:0/884196114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707c0094d0 con 0x7f708c07be30 2024-09-13T16:22:00.107 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.203+0000 7f708affd640 1 --2- 172.21.15.16:0/884196114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f708c07be30 0x7f708c19b3f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f707c0097f0 tx=0x7f707c032d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:00.107 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.203+0000 7f7090843640 1 -- 172.21.15.16:0/884196114 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f707c030da0 con 0x7f708c07be30 2024-09-13T16:22:00.107 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.203+0000 7f7090843640 1 -- 172.21.15.16:0/884196114 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f707c03c890 con 0x7f708c07be30 2024-09-13T16:22:00.107 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.203+0000 7f7091845640 1 -- 172.21.15.16:0/884196114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f708c19bb30 con 0x7f708c07be30 2024-09-13T16:22:00.107 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.203+0000 7f7091845640 1 -- 172.21.15.16:0/884196114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f708c19bfd0 con 0x7f708c07be30 2024-09-13T16:22:00.107 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.203+0000 7f7090843640 1 -- 172.21.15.16:0/884196114 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f707c03a5d0 con 0x7f708c07be30 2024-09-13T16:22:00.107 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.204+0000 7f7090843640 1 -- 172.21.15.16:0/884196114 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f707c03c400 con 0x7f708c07be30 2024-09-13T16:22:00.107 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.204+0000 7f7090843640 1 --2- 172.21.15.16:0/884196114 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f7078038670 0x7f707803ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:00.107 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.204+0000 7f7091845640 1 -- 172.21.15.16:0/884196114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f708c10b950 con 0x7f708c07be30 2024-09-13T16:22:00.108 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.205+0000 7f7090843640 1 -- 172.21.15.16:0/884196114 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f707c06bb20 con 0x7f708c07be30 2024-09-13T16:22:00.108 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.205+0000 7f7083fff640 1 --2- 172.21.15.16:0/884196114 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f7078038670 0x7f707803ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:00.108 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.205+0000 7f7083fff640 1 --2- 172.21.15.16:0/884196114 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f7078038670 0x7f707803ab30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f70740098c0 tx=0x7f7074006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:00.108 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.209+0000 7f7090843640 1 -- 172.21.15.16:0/884196114 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f707c031d30 con 0x7f708c07be30 2024-09-13T16:22:00.108 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.426+0000 7f7091845640 1 -- 172.21.15.16:0/884196114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f708c10bb90 con 0x7f708c07be30 2024-09-13T16:22:00.108 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.479+0000 7f7090843640 1 -- 172.21.15.16:0/884196114 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f707c043d70 con 0x7f708c07be30 2024-09-13T16:22:00.108 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.480+0000 7f7090843640 1 -- 172.21.15.16:0/884196114 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f707c03cce0 con 0x7f708c07be30 2024-09-13T16:22:00.108 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.483+0000 7f7091845640 1 -- 172.21.15.16:0/884196114 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f7078038670 msgr2=0x7f707803ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:00.108 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.483+0000 7f7091845640 1 --2- 172.21.15.16:0/884196114 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f7078038670 0x7f707803ab30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f70740098c0 tx=0x7f7074006db0 comp rx=0 tx=0).stop 2024-09-13T16:22:00.108 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.483+0000 7f7091845640 1 -- 172.21.15.16:0/884196114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f708c07be30 msgr2=0x7f708c19b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:00.108 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.483+0000 7f7091845640 1 --2- 172.21.15.16:0/884196114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f708c07be30 0x7f708c19b3f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f707c0097f0 tx=0x7f707c032d90 comp rx=0 tx=0).stop 2024-09-13T16:22:00.108 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.483+0000 7f7091845640 1 -- 172.21.15.16:0/884196114 shutdown_connections 2024-09-13T16:22:00.109 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.483+0000 7f7091845640 1 --2- 172.21.15.16:0/884196114 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f7078038670 0x7f707803ab30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:00.109 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.483+0000 7f7091845640 1 --2- 172.21.15.16:0/884196114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f708c07be30 0x7f708c19b3f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:00.109 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.483+0000 7f7091845640 1 -- 172.21.15.16:0/884196114 >> 172.21.15.16:0/884196114 conn(0x7f708c100cb0 msgr2=0x7f708c101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:00.109 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.484+0000 7f7091845640 1 -- 172.21.15.16:0/884196114 shutdown_connections 2024-09-13T16:22:00.109 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:21:59.484+0000 7f7091845640 1 -- 172.21.15.16:0/884196114 wait complete. 2024-09-13T16:22:00.354 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:00 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:22:00.216+0000 7fa3303b2040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T16:22:00.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:00 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/884196114' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-13T16:22:00.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:00 smithi016 ceph-mon[31187]: mgrmap e9: x(active, since 19s) 2024-09-13T16:22:00.614 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:00 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:22:00.549+0000 7fa3303b2040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T16:22:00.614 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:00 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:22:00.613+0000 7fa3303b2040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T16:22:01.077 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:00 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:22:00.807+0000 7fa3303b2040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T16:22:01.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-13T16:22:01.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-13T16:22:01.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-13T16:22:01.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-13T16:22:01.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-13T16:22:01.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-13T16:22:01.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.537+0000 7f96d25b8640 1 Processor -- start 2024-09-13T16:22:01.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.537+0000 7f96d25b8640 1 -- start start 2024-09-13T16:22:01.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.537+0000 7f96d25b8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96cc105fe0 0x7f96cc1083d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:01.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.537+0000 7f96d25b8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc079e00 con 0x7f96cc105fe0 2024-09-13T16:22:01.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.538+0000 7f96cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96cc105fe0 0x7f96cc1083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:01.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.538+0000 7f96cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96cc105fe0 0x7f96cc1083d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49914/0 (socket says 172.21.15.16:49914) 2024-09-13T16:22:01.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.538+0000 7f96cbfff640 1 -- 172.21.15.16:0/3441415876 learned_addr learned my addr 172.21.15.16:0/3441415876 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:22:01.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.538+0000 7f96cbfff640 1 -- 172.21.15.16:0/3441415876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96cc108910 con 0x7f96cc105fe0 2024-09-13T16:22:01.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.539+0000 7f96cbfff640 1 --2- 172.21.15.16:0/3441415876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96cc105fe0 0x7f96cc1083d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f96bc009820 tx=0x7f96bc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0a15ee0425eede server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:01.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.539+0000 7f96cb7fe640 1 -- 172.21.15.16:0/3441415876 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96bc02ae70 con 0x7f96cc105fe0 2024-09-13T16:22:01.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.539+0000 7f96cb7fe640 1 -- 172.21.15.16:0/3441415876 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f96bc0328d0 con 0x7f96cc105fe0 2024-09-13T16:22:01.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.539+0000 7f96cb7fe640 1 -- 172.21.15.16:0/3441415876 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96bc0307a0 con 0x7f96cc105fe0 2024-09-13T16:22:01.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.539+0000 7f96d25b8640 1 -- 172.21.15.16:0/3441415876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96cc105fe0 msgr2=0x7f96cc1083d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:01.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.539+0000 7f96d25b8640 1 --2- 172.21.15.16:0/3441415876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96cc105fe0 0x7f96cc1083d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f96bc009820 tx=0x7f96bc02a270 comp rx=0 tx=0).stop 2024-09-13T16:22:01.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.539+0000 7f96d25b8640 1 -- 172.21.15.16:0/3441415876 shutdown_connections 2024-09-13T16:22:01.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.539+0000 7f96d25b8640 1 --2- 172.21.15.16:0/3441415876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96cc105fe0 0x7f96cc1083d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:01.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.539+0000 7f96d25b8640 1 -- 172.21.15.16:0/3441415876 >> 172.21.15.16:0/3441415876 conn(0x7f96cc0ffbe0 msgr2=0x7f96cc102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:01.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.540+0000 7f96d25b8640 1 -- 172.21.15.16:0/3441415876 shutdown_connections 2024-09-13T16:22:01.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.540+0000 7f96d25b8640 1 -- 172.21.15.16:0/3441415876 wait complete. 2024-09-13T16:22:01.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.540+0000 7f96d25b8640 1 Processor -- start 2024-09-13T16:22:01.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.540+0000 7f96d25b8640 1 -- start start 2024-09-13T16:22:01.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.540+0000 7f96d25b8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96cc105fe0 0x7f96cc19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:01.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.540+0000 7f96d25b8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc19b900 con 0x7f96cc105fe0 2024-09-13T16:22:01.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.540+0000 7f96cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96cc105fe0 0x7f96cc19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:01.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.540+0000 7f96cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96cc105fe0 0x7f96cc19b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49920/0 (socket says 172.21.15.16:49920) 2024-09-13T16:22:01.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.541+0000 7f96cbfff640 1 -- 172.21.15.16:0/3227335636 learned_addr learned my addr 172.21.15.16:0/3227335636 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:22:01.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.541+0000 7f96cbfff640 1 -- 172.21.15.16:0/3227335636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96bc0094d0 con 0x7f96cc105fe0 2024-09-13T16:22:01.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.541+0000 7f96cbfff640 1 --2- 172.21.15.16:0/3227335636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96cc105fe0 0x7f96cc19b3c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f96bc0097f0 tx=0x7f96bc032db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:01.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.541+0000 7f96c9ffb640 1 -- 172.21.15.16:0/3227335636 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96bc030da0 con 0x7f96cc105fe0 2024-09-13T16:22:01.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.541+0000 7f96c9ffb640 1 -- 172.21.15.16:0/3227335636 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f96bc03b8c0 con 0x7f96cc105fe0 2024-09-13T16:22:01.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.541+0000 7f96d25b8640 1 -- 172.21.15.16:0/3227335636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96cc19bb00 con 0x7f96cc105fe0 2024-09-13T16:22:01.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.541+0000 7f96c9ffb640 1 -- 172.21.15.16:0/3227335636 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96bc03a570 con 0x7f96cc105fe0 2024-09-13T16:22:01.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.541+0000 7f96d25b8640 1 -- 172.21.15.16:0/3227335636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96cc19bfa0 con 0x7f96cc105fe0 2024-09-13T16:22:01.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.542+0000 7f96d25b8640 1 -- 172.21.15.16:0/3227335636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96cc1096b0 con 0x7f96cc105fe0 2024-09-13T16:22:01.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.542+0000 7f96c9ffb640 1 -- 172.21.15.16:0/3227335636 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f96bc039040 con 0x7f96cc105fe0 2024-09-13T16:22:01.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.543+0000 7f96c9ffb640 1 --2- 172.21.15.16:0/3227335636 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f96b8038670 0x7f96b803ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:01.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.543+0000 7f96c9ffb640 1 -- 172.21.15.16:0/3227335636 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f96bc06b930 con 0x7f96cc105fe0 2024-09-13T16:22:01.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.543+0000 7f96c3fff640 1 -- 172.21.15.16:0/3227335636 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f96b8038670 msgr2=0x7f96b803ab30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1756940569 2024-09-13T16:22:01.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.543+0000 7f96c3fff640 1 --2- 172.21.15.16:0/3227335636 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f96b8038670 0x7f96b803ab30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T16:22:01.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.547+0000 7f96c9ffb640 1 -- 172.21.15.16:0/3227335636 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f96bc03ba30 con 0x7f96cc105fe0 2024-09-13T16:22:01.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.744+0000 7f96c3fff640 1 -- 172.21.15.16:0/3227335636 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f96b8038670 msgr2=0x7f96b803ab30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1756940569 2024-09-13T16:22:01.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.744+0000 7f96c3fff640 1 --2- 172.21.15.16:0/3227335636 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f96b8038670 0x7f96b803ab30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T16:22:01.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.770+0000 7f96d25b8640 1 -- 172.21.15.16:0/3227335636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f96cc19e8e0 con 0x7f96cc105fe0 2024-09-13T16:22:01.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.770+0000 7f96c9ffb640 1 -- 172.21.15.16:0/3227335636 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f96bc03f070 con 0x7f96cc105fe0 2024-09-13T16:22:01.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.773+0000 7f96d25b8640 1 -- 172.21.15.16:0/3227335636 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f96b8038670 msgr2=0x7f96b803ab30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T16:22:01.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.773+0000 7f96d25b8640 1 --2- 172.21.15.16:0/3227335636 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f96b8038670 0x7f96b803ab30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:01.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.773+0000 7f96d25b8640 1 -- 172.21.15.16:0/3227335636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96cc105fe0 msgr2=0x7f96cc19b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:01.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.773+0000 7f96d25b8640 1 --2- 172.21.15.16:0/3227335636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96cc105fe0 0x7f96cc19b3c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f96bc0097f0 tx=0x7f96bc032db0 comp rx=0 tx=0).stop 2024-09-13T16:22:01.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.773+0000 7f96d25b8640 1 -- 172.21.15.16:0/3227335636 shutdown_connections 2024-09-13T16:22:01.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.773+0000 7f96d25b8640 1 --2- 172.21.15.16:0/3227335636 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7f96b8038670 0x7f96b803ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:01.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.773+0000 7f96d25b8640 1 --2- 172.21.15.16:0/3227335636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96cc105fe0 0x7f96cc19b3c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:01.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.773+0000 7f96d25b8640 1 -- 172.21.15.16:0/3227335636 >> 172.21.15.16:0/3227335636 conn(0x7f96cc0ffbe0 msgr2=0x7f96cc1006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:01.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.773+0000 7f96d25b8640 1 -- 172.21.15.16:0/3227335636 shutdown_connections 2024-09-13T16:22:01.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:00.773+0000 7f96d25b8640 1 -- 172.21.15.16:0/3227335636 wait complete. 2024-09-13T16:22:01.383 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for the mgr to restart... 2024-09-13T16:22:01.383 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr epoch 9... 2024-09-13T16:22:01.722 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:01 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3227335636' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-13T16:22:02.010 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:01 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:22:01.731+0000 7fa3303b2040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T16:22:02.010 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:01 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:22:01.802+0000 7fa3303b2040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T16:22:02.669 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:02 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:22:02.547+0000 7fa3303b2040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-13T16:22:02.669 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:02 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:22:02.667+0000 7fa3303b2040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T16:22:03.032 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:02 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:22:02.814+0000 7fa3303b2040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T16:22:03.033 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:02 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:22:02.877+0000 7fa3303b2040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T16:22:03.033 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:03 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:22:03.030+0000 7fa3303b2040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T16:22:03.328 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:03 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:22:03.101+0000 7fa3303b2040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T16:22:03.328 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:03 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:22:03.163+0000 7fa3303b2040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T16:22:03.328 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:03 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:22:03.326+0000 7fa3303b2040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T16:22:03.604 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:03 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:22:03.393+0000 7fa3303b2040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T16:22:03.604 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:03 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:22:03.462+0000 7fa3303b2040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T16:22:03.604 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:03 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:22:03.548+0000 7fa3303b2040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T16:22:04.104 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:03 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:22:03.609+0000 7fa3303b2040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T16:22:04.104 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:03 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:22:03.671+0000 7fa3303b2040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T16:22:04.735 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:04 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:22:04.482+0000 7fa3303b2040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T16:22:04.735 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:04 smithi016 ceph-mon[31187]: Active manager daemon x restarted 2024-09-13T16:22:04.736 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:04 smithi016 ceph-mon[31187]: Activating manager daemon x 2024-09-13T16:22:04.736 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:04 smithi016 ceph-mon[31187]: osdmap e3: 0 total, 0 up, 0 in 2024-09-13T16:22:04.736 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:04 smithi016 ceph-mon[31187]: mgrmap e10: x(active, starting, since 0.0033059s) 2024-09-13T16:22:04.736 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T16:22:04.736 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-13T16:22:04.736 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-13T16:22:04.736 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-13T16:22:04.736 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-13T16:22:04.736 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:04 smithi016 ceph-mon[31187]: Manager daemon x is now available 2024-09-13T16:22:04.736 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:22:04.736 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-13T16:22:04.736 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-13T16:22:06.364 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-13T16:22:06.364 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-13T16:22:06.364 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-13T16:22:06.364 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-13T16:22:06.365 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.801+0000 7febbf715640 1 Processor -- start 2024-09-13T16:22:06.365 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.801+0000 7febbf715640 1 -- start start 2024-09-13T16:22:06.365 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.801+0000 7febbf715640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febb8105fc0 0x7febb81083b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:06.365 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.801+0000 7febbf715640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb8079e00 con 0x7febb8105fc0 2024-09-13T16:22:06.365 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.802+0000 7febbd4b1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febb8105fc0 0x7febb81083b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:06.365 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.802+0000 7febbd4b1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febb8105fc0 0x7febb81083b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49934/0 (socket says 172.21.15.16:49934) 2024-09-13T16:22:06.365 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.802+0000 7febbd4b1640 1 -- 172.21.15.16:0/1138753677 learned_addr learned my addr 172.21.15.16:0/1138753677 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:22:06.365 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.802+0000 7febbd4b1640 1 -- 172.21.15.16:0/1138753677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb81088f0 con 0x7febb8105fc0 2024-09-13T16:22:06.365 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.803+0000 7febbd4b1640 1 --2- 172.21.15.16:0/1138753677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febb8105fc0 0x7febb81083b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7febb4009820 tx=0x7febb402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d17c16d40d907ca6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:06.365 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.803+0000 7febaffff640 1 -- 172.21.15.16:0/1138753677 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7febb402ae70 con 0x7febb8105fc0 2024-09-13T16:22:06.365 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.803+0000 7febaffff640 1 -- 172.21.15.16:0/1138753677 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7febb40328d0 con 0x7febb8105fc0 2024-09-13T16:22:06.365 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.803+0000 7febaffff640 1 -- 172.21.15.16:0/1138753677 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7febb40307a0 con 0x7febb8105fc0 2024-09-13T16:22:06.365 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.803+0000 7febbf715640 1 -- 172.21.15.16:0/1138753677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febb8105fc0 msgr2=0x7febb81083b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:06.366 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.803+0000 7febbf715640 1 --2- 172.21.15.16:0/1138753677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febb8105fc0 0x7febb81083b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7febb4009820 tx=0x7febb402a270 comp rx=0 tx=0).stop 2024-09-13T16:22:06.366 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.804+0000 7febbf715640 1 -- 172.21.15.16:0/1138753677 shutdown_connections 2024-09-13T16:22:06.366 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.804+0000 7febbf715640 1 --2- 172.21.15.16:0/1138753677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febb8105fc0 0x7febb81083b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:06.366 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.804+0000 7febbf715640 1 -- 172.21.15.16:0/1138753677 >> 172.21.15.16:0/1138753677 conn(0x7febb80ffb80 msgr2=0x7febb8101fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:06.366 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.804+0000 7febbf715640 1 -- 172.21.15.16:0/1138753677 shutdown_connections 2024-09-13T16:22:06.366 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.804+0000 7febbf715640 1 -- 172.21.15.16:0/1138753677 wait complete. 2024-09-13T16:22:06.366 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.804+0000 7febbf715640 1 Processor -- start 2024-09-13T16:22:06.367 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.804+0000 7febbf715640 1 -- start start 2024-09-13T16:22:06.367 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.805+0000 7febbf715640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febb8105fc0 0x7febb8105840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:06.367 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.805+0000 7febbf715640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb8105d80 con 0x7febb8105fc0 2024-09-13T16:22:06.367 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.805+0000 7febbd4b1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febb8105fc0 0x7febb8105840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:06.367 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.805+0000 7febbd4b1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febb8105fc0 0x7febb8105840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49942/0 (socket says 172.21.15.16:49942) 2024-09-13T16:22:06.367 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.805+0000 7febbd4b1640 1 -- 172.21.15.16:0/1281470293 learned_addr learned my addr 172.21.15.16:0/1281470293 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:22:06.367 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.805+0000 7febbd4b1640 1 -- 172.21.15.16:0/1281470293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb40094d0 con 0x7febb8105fc0 2024-09-13T16:22:06.367 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.805+0000 7febbd4b1640 1 --2- 172.21.15.16:0/1281470293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febb8105fc0 0x7febb8105840 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7febb40097f0 tx=0x7febb4032db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:06.367 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.805+0000 7febae7fc640 1 -- 172.21.15.16:0/1281470293 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7febb4030da0 con 0x7febb8105fc0 2024-09-13T16:22:06.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.805+0000 7febae7fc640 1 -- 172.21.15.16:0/1281470293 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7febb403b8c0 con 0x7febb8105fc0 2024-09-13T16:22:06.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.805+0000 7febbf715640 1 -- 172.21.15.16:0/1281470293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febb8103f70 con 0x7febb8105fc0 2024-09-13T16:22:06.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.805+0000 7febae7fc640 1 -- 172.21.15.16:0/1281470293 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7febb403a570 con 0x7febb8105fc0 2024-09-13T16:22:06.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.805+0000 7febbf715640 1 -- 172.21.15.16:0/1281470293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb8104410 con 0x7febb8105fc0 2024-09-13T16:22:06.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.806+0000 7febae7fc640 1 -- 172.21.15.16:0/1281470293 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7febb4039040 con 0x7febb8105fc0 2024-09-13T16:22:06.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.807+0000 7febae7fc640 1 --2- 172.21.15.16:0/1281470293 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7feba40386c0 0x7feba403ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:06.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.807+0000 7febae7fc640 1 -- 172.21.15.16:0/1281470293 --> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7feba403b290 con 0x7feba40386c0 2024-09-13T16:22:06.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.807+0000 7febae7fc640 1 -- 172.21.15.16:0/1281470293 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7febb406b8e0 con 0x7febb8105fc0 2024-09-13T16:22:06.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.807+0000 7febbccb0640 1 -- 172.21.15.16:0/1281470293 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7feba40386c0 msgr2=0x7feba403ab80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1756940569 2024-09-13T16:22:06.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:01.807+0000 7febbccb0640 1 --2- 172.21.15.16:0/1281470293 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7feba40386c0 0x7feba403ab80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T16:22:06.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:02.008+0000 7febbccb0640 1 -- 172.21.15.16:0/1281470293 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7feba40386c0 msgr2=0x7feba403ab80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1756940569 2024-09-13T16:22:06.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:02.008+0000 7febbccb0640 1 --2- 172.21.15.16:0/1281470293 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7feba40386c0 0x7feba403ab80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T16:22:06.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:02.409+0000 7febbccb0640 1 -- 172.21.15.16:0/1281470293 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7feba40386c0 msgr2=0x7feba403ab80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1756940569 2024-09-13T16:22:06.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:02.409+0000 7febbccb0640 1 --2- 172.21.15.16:0/1281470293 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7feba40386c0 0x7feba403ab80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T16:22:06.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:03.210+0000 7febbccb0640 1 -- 172.21.15.16:0/1281470293 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7feba40386c0 msgr2=0x7feba403ab80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1756940569 2024-09-13T16:22:06.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:03.210+0000 7febbccb0640 1 --2- 172.21.15.16:0/1281470293 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7feba40386c0 0x7feba403ab80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T16:22:06.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:04.485+0000 7febae7fc640 1 -- 172.21.15.16:0/1281470293 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mgrmap(e 10) v1 ==== 45321+0+0 (secure 0 0 0) 0x7febb406a710 con 0x7febb8105fc0 2024-09-13T16:22:06.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:04.486+0000 7febae7fc640 1 -- 172.21.15.16:0/1281470293 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7feba40386c0 msgr2=0x7feba403ab80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T16:22:06.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:04.486+0000 7febae7fc640 1 --2- 172.21.15.16:0/1281470293 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7feba40386c0 0x7feba403ab80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:06.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:05.487+0000 7febae7fc640 1 -- 172.21.15.16:0/1281470293 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 11) v1 ==== 45448+0+0 (secure 0 0 0) 0x7febb4031d50 con 0x7febb8105fc0 2024-09-13T16:22:06.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:05.487+0000 7febae7fc640 1 --2- 172.21.15.16:0/1281470293 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7feba403bfb0 0x7feba403e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:06.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:05.487+0000 7febae7fc640 1 -- 172.21.15.16:0/1281470293 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7feba403b290 con 0x7feba403bfb0 2024-09-13T16:22:06.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:05.488+0000 7febbccb0640 1 --2- 172.21.15.16:0/1281470293 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7feba403bfb0 0x7feba403e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:06.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:05.488+0000 7febbccb0640 1 --2- 172.21.15.16:0/1281470293 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7feba403bfb0 0x7feba403e3a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7feba8003980 tx=0x7feba80092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:06.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:05.489+0000 7febae7fc640 1 -- 172.21.15.16:0/1281470293 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7feba403b290 con 0x7feba403bfb0 2024-09-13T16:22:06.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:05.495+0000 7febbf715640 1 -- 172.21.15.16:0/1281470293 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7febb8106420 con 0x7feba403bfb0 2024-09-13T16:22:06.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:05.495+0000 7febae7fc640 1 -- 172.21.15.16:0/1281470293 <== mgr.14150 v2:172.21.15.16:6800/3739009683 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7febb8106420 con 0x7feba403bfb0 2024-09-13T16:22:06.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:05.495+0000 7febbf715640 1 -- 172.21.15.16:0/1281470293 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7feba403bfb0 msgr2=0x7feba403e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:06.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:05.495+0000 7febbf715640 1 --2- 172.21.15.16:0/1281470293 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7feba403bfb0 0x7feba403e3a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7feba8003980 tx=0x7feba80092b0 comp rx=0 tx=0).stop 2024-09-13T16:22:06.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:05.495+0000 7febbf715640 1 -- 172.21.15.16:0/1281470293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febb8105fc0 msgr2=0x7febb8105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:06.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:05.495+0000 7febbf715640 1 --2- 172.21.15.16:0/1281470293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febb8105fc0 0x7febb8105840 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7febb40097f0 tx=0x7febb4032db0 comp rx=0 tx=0).stop 2024-09-13T16:22:06.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:05.495+0000 7febbf715640 1 -- 172.21.15.16:0/1281470293 shutdown_connections 2024-09-13T16:22:06.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:05.495+0000 7febbf715640 1 --2- 172.21.15.16:0/1281470293 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7feba403bfb0 0x7feba403e3a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:06.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:05.495+0000 7febbf715640 1 --2- 172.21.15.16:0/1281470293 >> [v2:172.21.15.16:6800/1756940569,v1:172.21.15.16:6801/1756940569] conn(0x7feba40386c0 0x7feba403ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:06.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:05.495+0000 7febbf715640 1 --2- 172.21.15.16:0/1281470293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febb8105fc0 0x7febb8105840 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:06.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:05.495+0000 7febbf715640 1 -- 172.21.15.16:0/1281470293 >> 172.21.15.16:0/1281470293 conn(0x7febb80ffb80 msgr2=0x7febb8100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:06.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:05.495+0000 7febbf715640 1 -- 172.21.15.16:0/1281470293 shutdown_connections 2024-09-13T16:22:06.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:05.495+0000 7febbf715640 1 -- 172.21.15.16:0/1281470293 wait complete. 2024-09-13T16:22:06.370 INFO:teuthology.orchestra.run.smithi016.stdout:mgr epoch 9 is available 2024-09-13T16:22:06.370 INFO:teuthology.orchestra.run.smithi016.stdout:Generating a dashboard self-signed certificate... 2024-09-13T16:22:06.637 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:06 smithi016 ceph-mon[31187]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-13T16:22:06.637 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:06 smithi016 ceph-mon[31187]: mgrmap e11: x(active, since 1.00554s) 2024-09-13T16:22:06.637 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:06 smithi016 ceph-mon[31187]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-13T16:22:06.637 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:06 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:06.967 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:06 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:22:06.764+0000 7fa2967fc640 -1 log_channel(cephadm) log [ERR] : [13/Sep/2024:16:22:06] ENGINE Error in HTTPServer.serve 2024-09-13T16:22:06.968 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:06 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: Traceback (most recent call last): 2024-09-13T16:22:06.968 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:06 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-13T16:22:06.968 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:06 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: self._connections.run(self.expiration_interval) 2024-09-13T16:22:06.968 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:06 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-13T16:22:06.968 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:06 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: self._run(expiration_interval) 2024-09-13T16:22:06.968 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:06 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-13T16:22:06.968 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:06 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: new_conn = self._from_server_socket(self.server.socket) 2024-09-13T16:22:06.968 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:06 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-13T16:22:06.968 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:06 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-13T16:22:06.968 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:06 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-13T16:22:06.968 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:06 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: s = self.context.wrap_socket( 2024-09-13T16:22:06.968 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:06 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-13T16:22:06.969 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:06 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: return self.sslsocket_class._create( 2024-09-13T16:22:06.969 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:06 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-13T16:22:06.969 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:06 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: self.do_handshake() 2024-09-13T16:22:06.969 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:06 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-13T16:22:06.969 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:06 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: self._sslobj.do_handshake() 2024-09-13T16:22:06.969 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:06 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-13T16:22:06.969 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:22:06 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:22:07.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:07 smithi016 ceph-mon[31187]: [13/Sep/2024:16:22:06] ENGINE Bus STARTING 2024-09-13T16:22:07.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:07 smithi016 ceph-mon[31187]: [13/Sep/2024:16:22:06] ENGINE Serving on http://172.21.15.16:8765 2024-09-13T16:22:07.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:07 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:07.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:07 smithi016 ceph-mon[31187]: [13/Sep/2024:16:22:06] ENGINE Serving on https://172.21.15.16:7150 2024-09-13T16:22:07.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:07 smithi016 ceph-mon[31187]: [13/Sep/2024:16:22:06] ENGINE Bus STARTED 2024-09-13T16:22:07.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:07 smithi016 ceph-mon[31187]: [13/Sep/2024:16:22:06] ENGINE Error in HTTPServer.serve 2024-09-13T16:22:07.855 INFO:journalctl@ceph.mon.a.smithi016.stdout: Traceback (most recent call last): 2024-09-13T16:22:07.855 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-13T16:22:07.855 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._connections.run(self.expiration_interval) 2024-09-13T16:22:07.855 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-13T16:22:07.855 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._run(expiration_interval) 2024-09-13T16:22:07.855 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-13T16:22:07.855 INFO:journalctl@ceph.mon.a.smithi016.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-13T16:22:07.856 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-13T16:22:07.856 INFO:journalctl@ceph.mon.a.smithi016.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-13T16:22:07.856 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-13T16:22:07.856 INFO:journalctl@ceph.mon.a.smithi016.stdout: s = self.context.wrap_socket( 2024-09-13T16:22:07.856 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-13T16:22:07.856 INFO:journalctl@ceph.mon.a.smithi016.stdout: return self.sslsocket_class._create( 2024-09-13T16:22:07.856 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-13T16:22:07.856 INFO:journalctl@ceph.mon.a.smithi016.stdout: self.do_handshake() 2024-09-13T16:22:07.856 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-13T16:22:07.856 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._sslobj.do_handshake() 2024-09-13T16:22:07.856 INFO:journalctl@ceph.mon.a.smithi016.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-13T16:22:07.856 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:07 smithi016 ceph-mon[31187]: mgrmap e12: x(active, since 2s) 2024-09-13T16:22:07.856 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:07 smithi016 ceph-mon[31187]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:22:07.856 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:07 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:07.857 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:07 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:08.241 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-13T16:22:08.241 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.856+0000 7fe2d1017640 1 Processor -- start 2024-09-13T16:22:08.241 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.856+0000 7fe2d1017640 1 -- start start 2024-09-13T16:22:08.241 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.856+0000 7fe2d1017640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2cc106020 0x7fe2cc108410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:08.241 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.856+0000 7fe2d1017640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2cc079dc0 con 0x7fe2cc106020 2024-09-13T16:22:08.241 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.856+0000 7fe2cad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2cc106020 0x7fe2cc108410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:08.241 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.856+0000 7fe2cad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2cc106020 0x7fe2cc108410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43442/0 (socket says 172.21.15.16:43442) 2024-09-13T16:22:08.242 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.856+0000 7fe2cad9d640 1 -- 172.21.15.16:0/2487501281 learned_addr learned my addr 172.21.15.16:0/2487501281 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:22:08.242 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.856+0000 7fe2cad9d640 1 -- 172.21.15.16:0/2487501281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2cc108950 con 0x7fe2cc106020 2024-09-13T16:22:08.242 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.857+0000 7fe2cad9d640 1 --2- 172.21.15.16:0/2487501281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2cc106020 0x7fe2cc108410 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc009820 tx=0x7fe2bc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a66671d105dd914a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:08.242 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.857+0000 7fe2ca59c640 1 -- 172.21.15.16:0/2487501281 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2bc02ae70 con 0x7fe2cc106020 2024-09-13T16:22:08.242 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.857+0000 7fe2ca59c640 1 -- 172.21.15.16:0/2487501281 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe2bc0328d0 con 0x7fe2cc106020 2024-09-13T16:22:08.242 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.857+0000 7fe2ca59c640 1 -- 172.21.15.16:0/2487501281 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2bc0307a0 con 0x7fe2cc106020 2024-09-13T16:22:08.242 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.858+0000 7fe2d1017640 1 -- 172.21.15.16:0/2487501281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2cc106020 msgr2=0x7fe2cc108410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:08.242 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.858+0000 7fe2d1017640 1 --2- 172.21.15.16:0/2487501281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2cc106020 0x7fe2cc108410 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc009820 tx=0x7fe2bc02a270 comp rx=0 tx=0).stop 2024-09-13T16:22:08.242 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.858+0000 7fe2d1017640 1 -- 172.21.15.16:0/2487501281 shutdown_connections 2024-09-13T16:22:08.242 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.858+0000 7fe2d1017640 1 --2- 172.21.15.16:0/2487501281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2cc106020 0x7fe2cc108410 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:08.242 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.858+0000 7fe2d1017640 1 -- 172.21.15.16:0/2487501281 >> 172.21.15.16:0/2487501281 conn(0x7fe2cc0ffbe0 msgr2=0x7fe2cc102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:08.242 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.858+0000 7fe2d1017640 1 -- 172.21.15.16:0/2487501281 shutdown_connections 2024-09-13T16:22:08.243 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.858+0000 7fe2d1017640 1 -- 172.21.15.16:0/2487501281 wait complete. 2024-09-13T16:22:08.243 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.859+0000 7fe2d1017640 1 Processor -- start 2024-09-13T16:22:08.243 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.859+0000 7fe2d1017640 1 -- start start 2024-09-13T16:22:08.243 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.859+0000 7fe2d1017640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2cc106020 0x7fe2cc199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:08.243 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.859+0000 7fe2d1017640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2cc199740 con 0x7fe2cc106020 2024-09-13T16:22:08.243 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.859+0000 7fe2cad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2cc106020 0x7fe2cc199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:08.243 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.859+0000 7fe2cad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2cc106020 0x7fe2cc199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43450/0 (socket says 172.21.15.16:43450) 2024-09-13T16:22:08.243 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.859+0000 7fe2cad9d640 1 -- 172.21.15.16:0/3438820608 learned_addr learned my addr 172.21.15.16:0/3438820608 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:22:08.243 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.859+0000 7fe2cad9d640 1 -- 172.21.15.16:0/3438820608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2bc0094d0 con 0x7fe2cc106020 2024-09-13T16:22:08.243 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.859+0000 7fe2cad9d640 1 --2- 172.21.15.16:0/3438820608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2cc106020 0x7fe2cc199200 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc0304f0 tx=0x7fe2bc030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:08.243 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.860+0000 7fe2c8d99640 1 -- 172.21.15.16:0/3438820608 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2bc032c50 con 0x7fe2cc106020 2024-09-13T16:22:08.243 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.860+0000 7fe2c8d99640 1 -- 172.21.15.16:0/3438820608 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe2bc032db0 con 0x7fe2cc106020 2024-09-13T16:22:08.244 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.860+0000 7fe2d1017640 1 -- 172.21.15.16:0/3438820608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2cc199940 con 0x7fe2cc106020 2024-09-13T16:22:08.244 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.860+0000 7fe2c8d99640 1 -- 172.21.15.16:0/3438820608 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2bc03a640 con 0x7fe2cc106020 2024-09-13T16:22:08.244 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.860+0000 7fe2d1017640 1 -- 172.21.15.16:0/3438820608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2cc199de0 con 0x7fe2cc106020 2024-09-13T16:22:08.244 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.860+0000 7fe2c8d99640 1 -- 172.21.15.16:0/3438820608 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 11) v1 ==== 45448+0+0 (secure 0 0 0) 0x7fe2bc039040 con 0x7fe2cc106020 2024-09-13T16:22:08.244 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.861+0000 7fe2c8d99640 1 --2- 172.21.15.16:0/3438820608 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe2b8038550 0x7fe2b803aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:08.244 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.861+0000 7fe2c8d99640 1 -- 172.21.15.16:0/3438820608 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe2bc06b830 con 0x7fe2cc106020 2024-09-13T16:22:08.244 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.861+0000 7fe2d1017640 1 -- 172.21.15.16:0/3438820608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2cc1096f0 con 0x7fe2cc106020 2024-09-13T16:22:08.244 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.861+0000 7fe2c3fff640 1 --2- 172.21.15.16:0/3438820608 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe2b8038550 0x7fe2b803aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:08.244 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.861+0000 7fe2c3fff640 1 --2- 172.21.15.16:0/3438820608 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe2b8038550 0x7fe2b803aa10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40098c0 tx=0x7fe2b4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:08.244 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.866+0000 7fe2c8d99640 1 -- 172.21.15.16:0/3438820608 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fe2bc037070 con 0x7fe2cc106020 2024-09-13T16:22:08.244 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:06.964+0000 7fe2c8d99640 1 -- 172.21.15.16:0/3438820608 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7fe2bc044400 con 0x7fe2cc106020 2024-09-13T16:22:08.244 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:07.060+0000 7fe2d1017640 1 -- 172.21.15.16:0/3438820608 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2cc100e50 con 0x7fe2b8038550 2024-09-13T16:22:08.244 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:07.119+0000 7fe2c8d99640 1 -- 172.21.15.16:0/3438820608 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fe2cc100e50 con 0x7fe2b8038550 2024-09-13T16:22:08.244 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:07.121+0000 7fe2d1017640 1 -- 172.21.15.16:0/3438820608 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe2b8038550 msgr2=0x7fe2b803aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:08.245 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:07.121+0000 7fe2d1017640 1 --2- 172.21.15.16:0/3438820608 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe2b8038550 0x7fe2b803aa10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40098c0 tx=0x7fe2b4006db0 comp rx=0 tx=0).stop 2024-09-13T16:22:08.245 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:07.121+0000 7fe2d1017640 1 -- 172.21.15.16:0/3438820608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2cc106020 msgr2=0x7fe2cc199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:08.245 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:07.121+0000 7fe2d1017640 1 --2- 172.21.15.16:0/3438820608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2cc106020 0x7fe2cc199200 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc0304f0 tx=0x7fe2bc030cc0 comp rx=0 tx=0).stop 2024-09-13T16:22:08.245 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:07.121+0000 7fe2d1017640 1 -- 172.21.15.16:0/3438820608 shutdown_connections 2024-09-13T16:22:08.245 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:07.121+0000 7fe2d1017640 1 --2- 172.21.15.16:0/3438820608 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe2b8038550 0x7fe2b803aa10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:08.245 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:07.121+0000 7fe2d1017640 1 --2- 172.21.15.16:0/3438820608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2cc106020 0x7fe2cc199200 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:08.245 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:07.122+0000 7fe2d1017640 1 -- 172.21.15.16:0/3438820608 >> 172.21.15.16:0/3438820608 conn(0x7fe2cc0ffbe0 msgr2=0x7fe2cc100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:08.245 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:07.122+0000 7fe2d1017640 1 -- 172.21.15.16:0/3438820608 shutdown_connections 2024-09-13T16:22:08.245 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:07.122+0000 7fe2d1017640 1 -- 172.21.15.16:0/3438820608 wait complete. 2024-09-13T16:22:08.245 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial admin user... 2024-09-13T16:22:09.634 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$qVvBX91XlwDQ8NarT8i04uUjSw3viqhjiWyXne.xhUN6.qgrobGma", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726244529, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-13T16:22:09.634 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.630+0000 7f282b76b640 1 Processor -- start 2024-09-13T16:22:09.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.630+0000 7f282b76b640 1 -- start start 2024-09-13T16:22:09.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.630+0000 7f282b76b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2824105540 0x7f2824105940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:09.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.630+0000 7f282b76b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2824105f10 con 0x7f2824105540 2024-09-13T16:22:09.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.631+0000 7f2829507640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2824105540 0x7f2824105940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:09.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.631+0000 7f2829507640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2824105540 0x7f2824105940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43466/0 (socket says 172.21.15.16:43466) 2024-09-13T16:22:09.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.631+0000 7f2829507640 1 -- 172.21.15.16:0/2860930114 learned_addr learned my addr 172.21.15.16:0/2860930114 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:22:09.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.631+0000 7f2829507640 1 -- 172.21.15.16:0/2860930114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2824106740 con 0x7f2824105540 2024-09-13T16:22:09.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.632+0000 7f2829507640 1 --2- 172.21.15.16:0/2860930114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2824105540 0x7f2824105940 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2820009820 tx=0x7f282002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a9ba91cb8c1b24 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:09.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.632+0000 7f281bfff640 1 -- 172.21.15.16:0/2860930114 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f282002ae70 con 0x7f2824105540 2024-09-13T16:22:09.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.632+0000 7f281bfff640 1 -- 172.21.15.16:0/2860930114 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28200328d0 con 0x7f2824105540 2024-09-13T16:22:09.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.632+0000 7f281bfff640 1 -- 172.21.15.16:0/2860930114 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f28200307a0 con 0x7f2824105540 2024-09-13T16:22:09.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.632+0000 7f282b76b640 1 -- 172.21.15.16:0/2860930114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2824105540 msgr2=0x7f2824105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:09.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.632+0000 7f282b76b640 1 --2- 172.21.15.16:0/2860930114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2824105540 0x7f2824105940 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2820009820 tx=0x7f282002a270 comp rx=0 tx=0).stop 2024-09-13T16:22:09.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.633+0000 7f282b76b640 1 -- 172.21.15.16:0/2860930114 shutdown_connections 2024-09-13T16:22:09.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.633+0000 7f282b76b640 1 --2- 172.21.15.16:0/2860930114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2824105540 0x7f2824105940 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:09.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.633+0000 7f282b76b640 1 -- 172.21.15.16:0/2860930114 >> 172.21.15.16:0/2860930114 conn(0x7f2824100cf0 msgr2=0x7f2824103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:09.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.633+0000 7f282b76b640 1 -- 172.21.15.16:0/2860930114 shutdown_connections 2024-09-13T16:22:09.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.633+0000 7f282b76b640 1 -- 172.21.15.16:0/2860930114 wait complete. 2024-09-13T16:22:09.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.633+0000 7f282b76b640 1 Processor -- start 2024-09-13T16:22:09.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.633+0000 7f282b76b640 1 -- start start 2024-09-13T16:22:09.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.634+0000 7f282b76b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2824105540 0x7f282407bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:09.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.634+0000 7f282b76b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282407c040 con 0x7f2824105540 2024-09-13T16:22:09.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.634+0000 7f2829507640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2824105540 0x7f282407bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:09.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.634+0000 7f2829507640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2824105540 0x7f282407bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43470/0 (socket says 172.21.15.16:43470) 2024-09-13T16:22:09.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.634+0000 7f2829507640 1 -- 172.21.15.16:0/1431857584 learned_addr learned my addr 172.21.15.16:0/1431857584 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:22:09.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.634+0000 7f2829507640 1 -- 172.21.15.16:0/1431857584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28200094d0 con 0x7f2824105540 2024-09-13T16:22:09.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.634+0000 7f2829507640 1 --2- 172.21.15.16:0/1431857584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2824105540 0x7f282407bb00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2820030380 tx=0x7f2820032d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:09.637 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.634+0000 7f281a7fc640 1 -- 172.21.15.16:0/1431857584 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2820030c70 con 0x7f2824105540 2024-09-13T16:22:09.637 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.634+0000 7f281a7fc640 1 -- 172.21.15.16:0/1431857584 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2820030dd0 con 0x7f2824105540 2024-09-13T16:22:09.637 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.634+0000 7f282b76b640 1 -- 172.21.15.16:0/1431857584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f282407a230 con 0x7f2824105540 2024-09-13T16:22:09.637 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.634+0000 7f281a7fc640 1 -- 172.21.15.16:0/1431857584 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f282003a5b0 con 0x7f2824105540 2024-09-13T16:22:09.637 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.634+0000 7f282b76b640 1 -- 172.21.15.16:0/1431857584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f282407a6d0 con 0x7f2824105540 2024-09-13T16:22:09.637 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.635+0000 7f281a7fc640 1 -- 172.21.15.16:0/1431857584 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f2820039040 con 0x7f2824105540 2024-09-13T16:22:09.637 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.635+0000 7f282b76b640 1 -- 172.21.15.16:0/1431857584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28241059c0 con 0x7f2824105540 2024-09-13T16:22:09.637 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.636+0000 7f281a7fc640 1 --2- 172.21.15.16:0/1431857584 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f2810038670 0x7f281003ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:09.637 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.636+0000 7f281a7fc640 1 -- 172.21.15.16:0/1431857584 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f282006ba70 con 0x7f2824105540 2024-09-13T16:22:09.637 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.636+0000 7f2828d06640 1 --2- 172.21.15.16:0/1431857584 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f2810038670 0x7f281003ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:09.637 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.636+0000 7f2828d06640 1 --2- 172.21.15.16:0/1431857584 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f2810038670 0x7f281003ab30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f28140098c0 tx=0x7f2814006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:09.637 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.641+0000 7f281a7fc640 1 -- 172.21.15.16:0/1431857584 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f282003b430 con 0x7f2824105540 2024-09-13T16:22:09.638 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:08.826+0000 7f282b76b640 1 -- 172.21.15.16:0/1431857584 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f282407aa80 con 0x7f2810038670 2024-09-13T16:22:09.638 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:09.074+0000 7f281a7fc640 1 -- 172.21.15.16:0/1431857584 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f282407aa80 con 0x7f2810038670 2024-09-13T16:22:09.638 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:09.076+0000 7f282b76b640 1 -- 172.21.15.16:0/1431857584 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f2810038670 msgr2=0x7f281003ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:09.638 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:09.076+0000 7f282b76b640 1 --2- 172.21.15.16:0/1431857584 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f2810038670 0x7f281003ab30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f28140098c0 tx=0x7f2814006db0 comp rx=0 tx=0).stop 2024-09-13T16:22:09.638 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:09.076+0000 7f282b76b640 1 -- 172.21.15.16:0/1431857584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2824105540 msgr2=0x7f282407bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:09.638 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:09.076+0000 7f282b76b640 1 --2- 172.21.15.16:0/1431857584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2824105540 0x7f282407bb00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2820030380 tx=0x7f2820032d00 comp rx=0 tx=0).stop 2024-09-13T16:22:09.638 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:09.076+0000 7f282b76b640 1 -- 172.21.15.16:0/1431857584 shutdown_connections 2024-09-13T16:22:09.638 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:09.076+0000 7f282b76b640 1 --2- 172.21.15.16:0/1431857584 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f2810038670 0x7f281003ab30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:09.638 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:09.076+0000 7f282b76b640 1 --2- 172.21.15.16:0/1431857584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2824105540 0x7f282407bb00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:09.638 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:09.076+0000 7f282b76b640 1 -- 172.21.15.16:0/1431857584 >> 172.21.15.16:0/1431857584 conn(0x7f2824100cf0 msgr2=0x7f28241017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:09.638 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:09.077+0000 7f282b76b640 1 -- 172.21.15.16:0/1431857584 shutdown_connections 2024-09-13T16:22:09.638 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:09.077+0000 7f282b76b640 1 -- 172.21.15.16:0/1431857584 wait complete. 2024-09-13T16:22:09.638 INFO:teuthology.orchestra.run.smithi016.stdout:Fetching dashboard port number... 2024-09-13T16:22:10.316 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:10 smithi016 ceph-mon[31187]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:22:10.316 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:10 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:10.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 8443 2024-09-13T16:22:10.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.132+0000 7f1db90b2640 1 Processor -- start 2024-09-13T16:22:10.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.132+0000 7f1db90b2640 1 -- start start 2024-09-13T16:22:10.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.132+0000 7f1db90b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db4104420 0x7f1db4104820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:10.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.132+0000 7f1db90b2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db4104df0 con 0x7f1db4104420 2024-09-13T16:22:10.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.133+0000 7f1db2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db4104420 0x7f1db4104820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:10.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.133+0000 7f1db2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db4104420 0x7f1db4104820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43482/0 (socket says 172.21.15.16:43482) 2024-09-13T16:22:10.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.133+0000 7f1db2d9d640 1 -- 172.21.15.16:0/3024926850 learned_addr learned my addr 172.21.15.16:0/3024926850 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:22:10.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.133+0000 7f1db2d9d640 1 -- 172.21.15.16:0/3024926850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db4105620 con 0x7f1db4104420 2024-09-13T16:22:10.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.134+0000 7f1db2d9d640 1 --2- 172.21.15.16:0/3024926850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db4104420 0x7f1db4104820 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1da4009e80 tx=0x7f1da402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63780d59af8a56a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:10.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.134+0000 7f1db1d9b640 1 -- 172.21.15.16:0/3024926850 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1da4031400 con 0x7f1db4104420 2024-09-13T16:22:10.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.134+0000 7f1db1d9b640 1 -- 172.21.15.16:0/3024926850 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1da402d050 con 0x7f1db4104420 2024-09-13T16:22:10.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.134+0000 7f1db1d9b640 1 -- 172.21.15.16:0/3024926850 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1da4007830 con 0x7f1db4104420 2024-09-13T16:22:10.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.134+0000 7f1db90b2640 1 -- 172.21.15.16:0/3024926850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db4104420 msgr2=0x7f1db4104820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:10.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.134+0000 7f1db90b2640 1 --2- 172.21.15.16:0/3024926850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db4104420 0x7f1db4104820 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1da4009e80 tx=0x7f1da402a7a0 comp rx=0 tx=0).stop 2024-09-13T16:22:10.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.134+0000 7f1db90b2640 1 -- 172.21.15.16:0/3024926850 shutdown_connections 2024-09-13T16:22:10.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.134+0000 7f1db90b2640 1 --2- 172.21.15.16:0/3024926850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db4104420 0x7f1db4104820 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:10.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.134+0000 7f1db90b2640 1 -- 172.21.15.16:0/3024926850 >> 172.21.15.16:0/3024926850 conn(0x7f1db40ffbb0 msgr2=0x7f1db4101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:10.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.134+0000 7f1db90b2640 1 -- 172.21.15.16:0/3024926850 shutdown_connections 2024-09-13T16:22:10.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.134+0000 7f1db90b2640 1 -- 172.21.15.16:0/3024926850 wait complete. 2024-09-13T16:22:10.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.135+0000 7f1db90b2640 1 Processor -- start 2024-09-13T16:22:10.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.135+0000 7f1db90b2640 1 -- start start 2024-09-13T16:22:10.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.136+0000 7f1db90b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db4104420 0x7f1db40799e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:10.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.136+0000 7f1db90b2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db4079f20 con 0x7f1db4104420 2024-09-13T16:22:10.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.136+0000 7f1db2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db4104420 0x7f1db40799e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:10.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.136+0000 7f1db2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db4104420 0x7f1db40799e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43494/0 (socket says 172.21.15.16:43494) 2024-09-13T16:22:10.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.136+0000 7f1db2d9d640 1 -- 172.21.15.16:0/551659852 learned_addr learned my addr 172.21.15.16:0/551659852 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:22:10.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.136+0000 7f1db2d9d640 1 -- 172.21.15.16:0/551659852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da4009b30 con 0x7f1db4104420 2024-09-13T16:22:10.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.136+0000 7f1db2d9d640 1 --2- 172.21.15.16:0/551659852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db4104420 0x7f1db40799e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1da402acc0 tx=0x7f1da4031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:10.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.136+0000 7f1d9bfff640 1 -- 172.21.15.16:0/551659852 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1da4031b30 con 0x7f1db4104420 2024-09-13T16:22:10.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.136+0000 7f1d9bfff640 1 -- 172.21.15.16:0/551659852 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1da4031c90 con 0x7f1db4104420 2024-09-13T16:22:10.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.136+0000 7f1db90b2640 1 -- 172.21.15.16:0/551659852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1db419bf20 con 0x7f1db4104420 2024-09-13T16:22:10.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.136+0000 7f1d9bfff640 1 -- 172.21.15.16:0/551659852 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1da403a500 con 0x7f1db4104420 2024-09-13T16:22:10.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.136+0000 7f1db90b2640 1 -- 172.21.15.16:0/551659852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db419c3c0 con 0x7f1db4104420 2024-09-13T16:22:10.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.137+0000 7f1d9bfff640 1 -- 172.21.15.16:0/551659852 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f1da4038050 con 0x7f1db4104420 2024-09-13T16:22:10.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.137+0000 7f1db90b2640 1 -- 172.21.15.16:0/551659852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db4108850 con 0x7f1db4104420 2024-09-13T16:22:10.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.137+0000 7f1d9bfff640 1 --2- 172.21.15.16:0/551659852 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f1d9c038670 0x7f1d9c03ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:10.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.137+0000 7f1d9bfff640 1 -- 172.21.15.16:0/551659852 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1da406b620 con 0x7f1db4104420 2024-09-13T16:22:10.950 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.138+0000 7f1db259c640 1 --2- 172.21.15.16:0/551659852 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f1d9c038670 0x7f1d9c03ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:10.950 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.138+0000 7f1db259c640 1 --2- 172.21.15.16:0/551659852 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f1d9c038670 0x7f1d9c03ab30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1da80098c0 tx=0x7f1da8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:10.950 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.142+0000 7f1d9bfff640 1 -- 172.21.15.16:0/551659852 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f1da4044360 con 0x7f1db4104420 2024-09-13T16:22:10.950 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.314+0000 7f1db90b2640 1 -- 172.21.15.16:0/551659852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f1db4108a60 con 0x7f1db4104420 2024-09-13T16:22:10.950 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.314+0000 7f1d9bfff640 1 -- 172.21.15.16:0/551659852 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f1da4036020 con 0x7f1db4104420 2024-09-13T16:22:10.950 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.317+0000 7f1db90b2640 1 -- 172.21.15.16:0/551659852 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f1d9c038670 msgr2=0x7f1d9c03ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:10.950 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.317+0000 7f1db90b2640 1 --2- 172.21.15.16:0/551659852 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f1d9c038670 0x7f1d9c03ab30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1da80098c0 tx=0x7f1da8006db0 comp rx=0 tx=0).stop 2024-09-13T16:22:10.950 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.317+0000 7f1db90b2640 1 -- 172.21.15.16:0/551659852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db4104420 msgr2=0x7f1db40799e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:10.950 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.317+0000 7f1db90b2640 1 --2- 172.21.15.16:0/551659852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db4104420 0x7f1db40799e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1da402acc0 tx=0x7f1da4031630 comp rx=0 tx=0).stop 2024-09-13T16:22:10.950 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.317+0000 7f1db90b2640 1 -- 172.21.15.16:0/551659852 shutdown_connections 2024-09-13T16:22:10.950 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.317+0000 7f1db90b2640 1 --2- 172.21.15.16:0/551659852 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f1d9c038670 0x7f1d9c03ab30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:10.950 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.317+0000 7f1db90b2640 1 --2- 172.21.15.16:0/551659852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db4104420 0x7f1db40799e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:10.950 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.317+0000 7f1db90b2640 1 -- 172.21.15.16:0/551659852 >> 172.21.15.16:0/551659852 conn(0x7f1db40ffbb0 msgr2=0x7f1db41006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:10.950 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.317+0000 7f1db90b2640 1 -- 172.21.15.16:0/551659852 shutdown_connections 2024-09-13T16:22:10.951 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:10.317+0000 7f1db90b2640 1 -- 172.21.15.16:0/551659852 wait complete. 2024-09-13T16:22:10.957 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-13T16:22:10.957 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-09-13T16:22:10.965 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-13T16:22:10.965 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-13T16:22:10.965 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-09-13T16:22:10.965 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-13T16:22:10.967 INFO:teuthology.orchestra.run.smithi016.stdout:Ceph Dashboard is now available at: 2024-09-13T16:22:10.967 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:22:10.968 INFO:teuthology.orchestra.run.smithi016.stdout: URL: https://smithi016.front.sepia.ceph.com:8443/ 2024-09-13T16:22:10.968 INFO:teuthology.orchestra.run.smithi016.stdout: User: admin 2024-09-13T16:22:10.968 INFO:teuthology.orchestra.run.smithi016.stdout: Password: ga991id4tm 2024-09-13T16:22:10.968 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:22:10.968 INFO:teuthology.orchestra.run.smithi016.stdout:Saving cluster configuration to /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/config directory 2024-09-13T16:22:10.968 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling autotune for osd_memory_target 2024-09-13T16:22:11.197 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:11 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/551659852' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-13T16:22:12.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:12 smithi016 ceph-mon[31187]: mgrmap e13: x(active, since 6s) 2024-09-13T16:22:12.118 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.367+0000 7fc8b9282640 1 Processor -- start 2024-09-13T16:22:12.119 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.367+0000 7fc8b9282640 1 -- start start 2024-09-13T16:22:12.119 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.367+0000 7fc8b9282640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b4105a90 0x7fc8b4105e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:12.119 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.367+0000 7fc8b9282640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b4106460 con 0x7fc8b4105a90 2024-09-13T16:22:12.119 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.367+0000 7fc8b2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b4105a90 0x7fc8b4105e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:12.120 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.367+0000 7fc8b2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b4105a90 0x7fc8b4105e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43496/0 (socket says 172.21.15.16:43496) 2024-09-13T16:22:12.120 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.367+0000 7fc8b2ffd640 1 -- 172.21.15.16:0/2824962488 learned_addr learned my addr 172.21.15.16:0/2824962488 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:22:12.120 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.368+0000 7fc8b2ffd640 1 -- 172.21.15.16:0/2824962488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b41065a0 con 0x7fc8b4105a90 2024-09-13T16:22:12.120 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.368+0000 7fc8b2ffd640 1 --2- 172.21.15.16:0/2824962488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b4105a90 0x7fc8b4105e90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4009820 tx=0x7fc8a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d801aed34d0c812 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:12.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.368+0000 7fc8b27fc640 1 -- 172.21.15.16:0/2824962488 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8a402ae70 con 0x7fc8b4105a90 2024-09-13T16:22:12.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.369+0000 7fc8b27fc640 1 -- 172.21.15.16:0/2824962488 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc8a40328d0 con 0x7fc8b4105a90 2024-09-13T16:22:12.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.369+0000 7fc8b27fc640 1 -- 172.21.15.16:0/2824962488 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8a40307a0 con 0x7fc8b4105a90 2024-09-13T16:22:12.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.369+0000 7fc8b9282640 1 -- 172.21.15.16:0/2824962488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b4105a90 msgr2=0x7fc8b4105e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:12.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.369+0000 7fc8b9282640 1 --2- 172.21.15.16:0/2824962488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b4105a90 0x7fc8b4105e90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4009820 tx=0x7fc8a402a270 comp rx=0 tx=0).stop 2024-09-13T16:22:12.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.369+0000 7fc8b9282640 1 -- 172.21.15.16:0/2824962488 shutdown_connections 2024-09-13T16:22:12.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.369+0000 7fc8b9282640 1 --2- 172.21.15.16:0/2824962488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b4105a90 0x7fc8b4105e90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:12.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.369+0000 7fc8b9282640 1 -- 172.21.15.16:0/2824962488 >> 172.21.15.16:0/2824962488 conn(0x7fc8b40ff320 msgr2=0x7fc8b4101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:12.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.369+0000 7fc8b9282640 1 -- 172.21.15.16:0/2824962488 shutdown_connections 2024-09-13T16:22:12.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.369+0000 7fc8b9282640 1 -- 172.21.15.16:0/2824962488 wait complete. 2024-09-13T16:22:12.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.370+0000 7fc8b9282640 1 Processor -- start 2024-09-13T16:22:12.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.370+0000 7fc8b9282640 1 -- start start 2024-09-13T16:22:12.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.370+0000 7fc8b9282640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b4105a90 0x7fc8b419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:12.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.370+0000 7fc8b9282640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b419b8b0 con 0x7fc8b4105a90 2024-09-13T16:22:12.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.370+0000 7fc8b2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b4105a90 0x7fc8b419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:12.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.370+0000 7fc8b2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b4105a90 0x7fc8b419b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43502/0 (socket says 172.21.15.16:43502) 2024-09-13T16:22:12.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.370+0000 7fc8b2ffd640 1 -- 172.21.15.16:0/3532364568 learned_addr learned my addr 172.21.15.16:0/3532364568 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:22:12.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.370+0000 7fc8b2ffd640 1 -- 172.21.15.16:0/3532364568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a40094d0 con 0x7fc8b4105a90 2024-09-13T16:22:12.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.371+0000 7fc8b2ffd640 1 --2- 172.21.15.16:0/3532364568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b4105a90 0x7fc8b419b370 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc8a40097f0 tx=0x7fc8a4032db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:12.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.371+0000 7fc8b0ff9640 1 -- 172.21.15.16:0/3532364568 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8a4030da0 con 0x7fc8b4105a90 2024-09-13T16:22:12.128 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.371+0000 7fc8b0ff9640 1 -- 172.21.15.16:0/3532364568 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc8a403b8c0 con 0x7fc8b4105a90 2024-09-13T16:22:12.128 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.371+0000 7fc8b9282640 1 -- 172.21.15.16:0/3532364568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b419bab0 con 0x7fc8b4105a90 2024-09-13T16:22:12.128 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.371+0000 7fc8b0ff9640 1 -- 172.21.15.16:0/3532364568 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8a403a570 con 0x7fc8b4105a90 2024-09-13T16:22:12.128 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.371+0000 7fc8b9282640 1 -- 172.21.15.16:0/3532364568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b419bf50 con 0x7fc8b4105a90 2024-09-13T16:22:12.128 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.372+0000 7fc8b0ff9640 1 -- 172.21.15.16:0/3532364568 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fc8a4039040 con 0x7fc8b4105a90 2024-09-13T16:22:12.128 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.372+0000 7fc8b9282640 1 -- 172.21.15.16:0/3532364568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b4104250 con 0x7fc8b4105a90 2024-09-13T16:22:12.128 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.372+0000 7fc8b0ff9640 1 --2- 172.21.15.16:0/3532364568 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc8a0038760 0x7fc8a003ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:12.128 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.372+0000 7fc8b0ff9640 1 -- 172.21.15.16:0/3532364568 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc8a406bab0 con 0x7fc8b4105a90 2024-09-13T16:22:12.128 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.372+0000 7fc8abfff640 1 --2- 172.21.15.16:0/3532364568 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc8a0038760 0x7fc8a003ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:12.129 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.373+0000 7fc8abfff640 1 --2- 172.21.15.16:0/3532364568 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc8a0038760 0x7fc8a003ac20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc89c0098c0 tx=0x7fc89c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:12.129 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.377+0000 7fc8b0ff9640 1 -- 172.21.15.16:0/3532364568 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc8a404b6c0 con 0x7fc8b4105a90 2024-09-13T16:22:12.129 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.548+0000 7fc8b9282640 1 -- 172.21.15.16:0/3532364568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fc8b4104490 con 0x7fc8b4105a90 2024-09-13T16:22:12.129 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.549+0000 7fc8b0ff9640 1 -- 172.21.15.16:0/3532364568 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fc8a403f030 con 0x7fc8b4105a90 2024-09-13T16:22:12.129 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.551+0000 7fc8b9282640 1 -- 172.21.15.16:0/3532364568 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc8a0038760 msgr2=0x7fc8a003ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:12.129 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.551+0000 7fc8b9282640 1 --2- 172.21.15.16:0/3532364568 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc8a0038760 0x7fc8a003ac20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc89c0098c0 tx=0x7fc89c006db0 comp rx=0 tx=0).stop 2024-09-13T16:22:12.129 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.551+0000 7fc8b9282640 1 -- 172.21.15.16:0/3532364568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b4105a90 msgr2=0x7fc8b419b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:12.129 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.551+0000 7fc8b9282640 1 --2- 172.21.15.16:0/3532364568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b4105a90 0x7fc8b419b370 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc8a40097f0 tx=0x7fc8a4032db0 comp rx=0 tx=0).stop 2024-09-13T16:22:12.129 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.551+0000 7fc8b9282640 1 -- 172.21.15.16:0/3532364568 shutdown_connections 2024-09-13T16:22:12.129 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.551+0000 7fc8b9282640 1 --2- 172.21.15.16:0/3532364568 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc8a0038760 0x7fc8a003ac20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:12.129 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.551+0000 7fc8b9282640 1 --2- 172.21.15.16:0/3532364568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b4105a90 0x7fc8b419b370 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:12.129 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.551+0000 7fc8b9282640 1 -- 172.21.15.16:0/3532364568 >> 172.21.15.16:0/3532364568 conn(0x7fc8b40ff320 msgr2=0x7fc8b4191c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:12.129 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.551+0000 7fc8b9282640 1 -- 172.21.15.16:0/3532364568 shutdown_connections 2024-09-13T16:22:12.129 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:11.551+0000 7fc8b9282640 1 -- 172.21.15.16:0/3532364568 wait complete. 2024-09-13T16:22:13.366 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.522+0000 7fbae8fbc640 1 Processor -- start 2024-09-13T16:22:13.367 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.522+0000 7fbae8fbc640 1 -- start start 2024-09-13T16:22:13.367 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.522+0000 7fbae8fbc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae4105ff0 0x7fbae41083e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:13.367 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.522+0000 7fbae8fbc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae4079de0 con 0x7fbae4105ff0 2024-09-13T16:22:13.367 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.522+0000 7fbae259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae4105ff0 0x7fbae41083e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:13.367 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.522+0000 7fbae259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae4105ff0 0x7fbae41083e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43504/0 (socket says 172.21.15.16:43504) 2024-09-13T16:22:13.367 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.522+0000 7fbae259c640 1 -- 172.21.15.16:0/1532359984 learned_addr learned my addr 172.21.15.16:0/1532359984 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:22:13.367 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.522+0000 7fbae259c640 1 -- 172.21.15.16:0/1532359984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae4108920 con 0x7fbae4105ff0 2024-09-13T16:22:13.367 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.523+0000 7fbae259c640 1 --2- 172.21.15.16:0/1532359984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae4105ff0 0x7fbae41083e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbad4009820 tx=0x7fbad402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80a483cd5f2908b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:13.367 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.523+0000 7fbae1d9b640 1 -- 172.21.15.16:0/1532359984 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbad402ae70 con 0x7fbae4105ff0 2024-09-13T16:22:13.367 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.523+0000 7fbae1d9b640 1 -- 172.21.15.16:0/1532359984 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbad40328d0 con 0x7fbae4105ff0 2024-09-13T16:22:13.367 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.523+0000 7fbae1d9b640 1 -- 172.21.15.16:0/1532359984 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbad40307a0 con 0x7fbae4105ff0 2024-09-13T16:22:13.367 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.524+0000 7fbae8fbc640 1 -- 172.21.15.16:0/1532359984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae4105ff0 msgr2=0x7fbae41083e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:13.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.524+0000 7fbae8fbc640 1 --2- 172.21.15.16:0/1532359984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae4105ff0 0x7fbae41083e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbad4009820 tx=0x7fbad402a270 comp rx=0 tx=0).stop 2024-09-13T16:22:13.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.524+0000 7fbae8fbc640 1 -- 172.21.15.16:0/1532359984 shutdown_connections 2024-09-13T16:22:13.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.524+0000 7fbae8fbc640 1 --2- 172.21.15.16:0/1532359984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae4105ff0 0x7fbae41083e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:13.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.524+0000 7fbae8fbc640 1 -- 172.21.15.16:0/1532359984 >> 172.21.15.16:0/1532359984 conn(0x7fbae40ffbb0 msgr2=0x7fbae4101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:13.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.524+0000 7fbae8fbc640 1 -- 172.21.15.16:0/1532359984 shutdown_connections 2024-09-13T16:22:13.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.524+0000 7fbae8fbc640 1 -- 172.21.15.16:0/1532359984 wait complete. 2024-09-13T16:22:13.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.525+0000 7fbae8fbc640 1 Processor -- start 2024-09-13T16:22:13.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.525+0000 7fbae8fbc640 1 -- start start 2024-09-13T16:22:13.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.525+0000 7fbae8fbc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae4105ff0 0x7fbae41991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:13.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.525+0000 7fbae8fbc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae41996e0 con 0x7fbae4105ff0 2024-09-13T16:22:13.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.525+0000 7fbae259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae4105ff0 0x7fbae41991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:13.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.525+0000 7fbae259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae4105ff0 0x7fbae41991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43508/0 (socket says 172.21.15.16:43508) 2024-09-13T16:22:13.368 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.525+0000 7fbae259c640 1 -- 172.21.15.16:0/3974179317 learned_addr learned my addr 172.21.15.16:0/3974179317 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:22:13.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.525+0000 7fbae259c640 1 -- 172.21.15.16:0/3974179317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad40094d0 con 0x7fbae4105ff0 2024-09-13T16:22:13.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.525+0000 7fbae259c640 1 --2- 172.21.15.16:0/3974179317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae4105ff0 0x7fbae41991a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbad40304f0 tx=0x7fbad4030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:13.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.525+0000 7fbadb7fe640 1 -- 172.21.15.16:0/3974179317 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbad4032c50 con 0x7fbae4105ff0 2024-09-13T16:22:13.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.525+0000 7fbadb7fe640 1 -- 172.21.15.16:0/3974179317 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbad4032db0 con 0x7fbae4105ff0 2024-09-13T16:22:13.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.525+0000 7fbae8fbc640 1 -- 172.21.15.16:0/3974179317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae41998e0 con 0x7fbae4105ff0 2024-09-13T16:22:13.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.525+0000 7fbadb7fe640 1 -- 172.21.15.16:0/3974179317 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbad403a640 con 0x7fbae4105ff0 2024-09-13T16:22:13.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.526+0000 7fbae8fbc640 1 -- 172.21.15.16:0/3974179317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae4199d80 con 0x7fbae4105ff0 2024-09-13T16:22:13.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.526+0000 7fbadb7fe640 1 -- 172.21.15.16:0/3974179317 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fbad4039040 con 0x7fbae4105ff0 2024-09-13T16:22:13.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.526+0000 7fbae8fbc640 1 -- 172.21.15.16:0/3974179317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbae41096c0 con 0x7fbae4105ff0 2024-09-13T16:22:13.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.527+0000 7fbadb7fe640 1 --2- 172.21.15.16:0/3974179317 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fbad0038760 0x7fbad003ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:13.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.527+0000 7fbadb7fe640 1 -- 172.21.15.16:0/3974179317 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbad406bb40 con 0x7fbae4105ff0 2024-09-13T16:22:13.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.527+0000 7fbadbfff640 1 --2- 172.21.15.16:0/3974179317 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fbad0038760 0x7fbad003ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:13.369 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.527+0000 7fbadbfff640 1 --2- 172.21.15.16:0/3974179317 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fbad0038760 0x7fbad003ac20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0098c0 tx=0x7fbacc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:13.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.531+0000 7fbadb7fe640 1 -- 172.21.15.16:0/3974179317 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fbad406de60 con 0x7fbae4105ff0 2024-09-13T16:22:13.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.790+0000 7fbae8fbc640 1 -- 172.21.15.16:0/3974179317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fbae4109900 con 0x7fbae4105ff0 2024-09-13T16:22:13.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.793+0000 7fbadb7fe640 1 -- 172.21.15.16:0/3974179317 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7fbad403f070 con 0x7fbae4105ff0 2024-09-13T16:22:13.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-13T16:22:13.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.795+0000 7fbae8fbc640 1 -- 172.21.15.16:0/3974179317 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fbad0038760 msgr2=0x7fbad003ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:13.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.795+0000 7fbae8fbc640 1 --2- 172.21.15.16:0/3974179317 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fbad0038760 0x7fbad003ac20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0098c0 tx=0x7fbacc006db0 comp rx=0 tx=0).stop 2024-09-13T16:22:13.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.795+0000 7fbae8fbc640 1 -- 172.21.15.16:0/3974179317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae4105ff0 msgr2=0x7fbae41991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:13.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.795+0000 7fbae8fbc640 1 --2- 172.21.15.16:0/3974179317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae4105ff0 0x7fbae41991a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbad40304f0 tx=0x7fbad4030cc0 comp rx=0 tx=0).stop 2024-09-13T16:22:13.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.795+0000 7fbae8fbc640 1 -- 172.21.15.16:0/3974179317 shutdown_connections 2024-09-13T16:22:13.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.795+0000 7fbae8fbc640 1 --2- 172.21.15.16:0/3974179317 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fbad0038760 0x7fbad003ac20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:13.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.795+0000 7fbae8fbc640 1 --2- 172.21.15.16:0/3974179317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae4105ff0 0x7fbae41991a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:13.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.795+0000 7fbae8fbc640 1 -- 172.21.15.16:0/3974179317 >> 172.21.15.16:0/3974179317 conn(0x7fbae40ffbb0 msgr2=0x7fbae4100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:13.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.795+0000 7fbae8fbc640 1 -- 172.21.15.16:0/3974179317 shutdown_connections 2024-09-13T16:22:13.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-13T16:22:12.795+0000 7fbae8fbc640 1 -- 172.21.15.16:0/3974179317 wait complete. 2024-09-13T16:22:13.371 INFO:teuthology.orchestra.run.smithi016.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-13T16:22:13.371 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:22:13.371 INFO:teuthology.orchestra.run.smithi016.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-13T16:22:13.371 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:22:13.371 INFO:teuthology.orchestra.run.smithi016.stdout:Or, if you are only running a single cluster on this host: 2024-09-13T16:22:13.371 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:22:13.371 INFO:teuthology.orchestra.run.smithi016.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-13T16:22:13.371 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:22:13.371 INFO:teuthology.orchestra.run.smithi016.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-13T16:22:13.371 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:22:13.371 INFO:teuthology.orchestra.run.smithi016.stdout: ceph telemetry on 2024-09-13T16:22:13.371 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:22:13.371 INFO:teuthology.orchestra.run.smithi016.stdout:For more information see: 2024-09-13T16:22:13.371 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:22:13.371 INFO:teuthology.orchestra.run.smithi016.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-13T16:22:13.372 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:22:13.372 INFO:teuthology.orchestra.run.smithi016.stdout:Bootstrap complete. 2024-09-13T16:22:13.399 INFO:tasks.cephadm:Fetching config... 2024-09-13T16:22:13.400 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-13T16:22:13.400 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-13T16:22:13.417 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-13T16:22:13.417 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-13T16:22:13.418 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-13T16:22:13.474 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-13T16:22:13.474 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-13T16:22:13.474 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-13T16:22:13.539 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-13T16:22:13.539 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-13T16:22:13.540 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-13T16:22:13.596 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-13T16:22:13.596 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPbEGzctL4hKBPFguGj1c5vzIqAe1BPs1xQRbvm0ipZBRQMNGrNmdk9AtAqUe0gljBiudEei1iw6TWxgGM6bwh17ttPE1kgi9slR6ooRxrSPkGFvUe6XiqEZa8mt9cW2AzaU/XN4/ZiHxZtxd8eVB3h9dacWUuai+EHVlOoTeMsGk4ZlfYoXL7KaAx9A2PN8oaowCByu6MUB7hbOdckOozT06Bb89uzriqlSh2vlA8UBE06ZMOVJjy8ZjSG7ZHKd5ViQPTQn5vRhdnmx64KhA3F+Q4j1gTQ7V2HRFdgd6lnH1t5i2UcPQLmqkmUa1FAG1vfWxO4m/4JEfQ/yXJ9NZsZQkyp6r1VwksSGX9Fxw0rijUXAQ5W5lsvpNgsH+9Hv3PbydEakv3GH1MsdzQ2BDT7Tkgi2aG1u5rnUQgkJgk/yzOJuRd4qsdN7rBxP8ud+nLJ5mhE2ZY6CkfL2l/CmO41Wn4gQ0WOMdl7ow8RHU04RF9t6RaWFFeTFpMe4AbbBs= ceph-0d241f6a-71ec-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-13T16:22:13.683 INFO:teuthology.orchestra.run.smithi016.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPbEGzctL4hKBPFguGj1c5vzIqAe1BPs1xQRbvm0ipZBRQMNGrNmdk9AtAqUe0gljBiudEei1iw6TWxgGM6bwh17ttPE1kgi9slR6ooRxrSPkGFvUe6XiqEZa8mt9cW2AzaU/XN4/ZiHxZtxd8eVB3h9dacWUuai+EHVlOoTeMsGk4ZlfYoXL7KaAx9A2PN8oaowCByu6MUB7hbOdckOozT06Bb89uzriqlSh2vlA8UBE06ZMOVJjy8ZjSG7ZHKd5ViQPTQn5vRhdnmx64KhA3F+Q4j1gTQ7V2HRFdgd6lnH1t5i2UcPQLmqkmUa1FAG1vfWxO4m/4JEfQ/yXJ9NZsZQkyp6r1VwksSGX9Fxw0rijUXAQ5W5lsvpNgsH+9Hv3PbydEakv3GH1MsdzQ2BDT7Tkgi2aG1u5rnUQgkJgk/yzOJuRd4qsdN7rBxP8ud+nLJ5mhE2ZY6CkfL2l/CmO41Wn4gQ0WOMdl7ow8RHU04RF9t6RaWFFeTFpMe4AbbBs= ceph-0d241f6a-71ec-11ef-bceb-c7b262605968 2024-09-13T16:22:13.695 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPbEGzctL4hKBPFguGj1c5vzIqAe1BPs1xQRbvm0ipZBRQMNGrNmdk9AtAqUe0gljBiudEei1iw6TWxgGM6bwh17ttPE1kgi9slR6ooRxrSPkGFvUe6XiqEZa8mt9cW2AzaU/XN4/ZiHxZtxd8eVB3h9dacWUuai+EHVlOoTeMsGk4ZlfYoXL7KaAx9A2PN8oaowCByu6MUB7hbOdckOozT06Bb89uzriqlSh2vlA8UBE06ZMOVJjy8ZjSG7ZHKd5ViQPTQn5vRhdnmx64KhA3F+Q4j1gTQ7V2HRFdgd6lnH1t5i2UcPQLmqkmUa1FAG1vfWxO4m/4JEfQ/yXJ9NZsZQkyp6r1VwksSGX9Fxw0rijUXAQ5W5lsvpNgsH+9Hv3PbydEakv3GH1MsdzQ2BDT7Tkgi2aG1u5rnUQgkJgk/yzOJuRd4qsdN7rBxP8ud+nLJ5mhE2ZY6CkfL2l/CmO41Wn4gQ0WOMdl7ow8RHU04RF9t6RaWFFeTFpMe4AbbBs= ceph-0d241f6a-71ec-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-13T16:22:13.801 INFO:teuthology.orchestra.run.smithi107.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPbEGzctL4hKBPFguGj1c5vzIqAe1BPs1xQRbvm0ipZBRQMNGrNmdk9AtAqUe0gljBiudEei1iw6TWxgGM6bwh17ttPE1kgi9slR6ooRxrSPkGFvUe6XiqEZa8mt9cW2AzaU/XN4/ZiHxZtxd8eVB3h9dacWUuai+EHVlOoTeMsGk4ZlfYoXL7KaAx9A2PN8oaowCByu6MUB7hbOdckOozT06Bb89uzriqlSh2vlA8UBE06ZMOVJjy8ZjSG7ZHKd5ViQPTQn5vRhdnmx64KhA3F+Q4j1gTQ7V2HRFdgd6lnH1t5i2UcPQLmqkmUa1FAG1vfWxO4m/4JEfQ/yXJ9NZsZQkyp6r1VwksSGX9Fxw0rijUXAQ5W5lsvpNgsH+9Hv3PbydEakv3GH1MsdzQ2BDT7Tkgi2aG1u5rnUQgkJgk/yzOJuRd4qsdN7rBxP8ud+nLJ5mhE2ZY6CkfL2l/CmO41Wn4gQ0WOMdl7ow8RHU04RF9t6RaWFFeTFpMe4AbbBs= ceph-0d241f6a-71ec-11ef-bceb-c7b262605968 2024-09-13T16:22:13.815 DEBUG:teuthology.orchestra.run.smithi111:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPbEGzctL4hKBPFguGj1c5vzIqAe1BPs1xQRbvm0ipZBRQMNGrNmdk9AtAqUe0gljBiudEei1iw6TWxgGM6bwh17ttPE1kgi9slR6ooRxrSPkGFvUe6XiqEZa8mt9cW2AzaU/XN4/ZiHxZtxd8eVB3h9dacWUuai+EHVlOoTeMsGk4ZlfYoXL7KaAx9A2PN8oaowCByu6MUB7hbOdckOozT06Bb89uzriqlSh2vlA8UBE06ZMOVJjy8ZjSG7ZHKd5ViQPTQn5vRhdnmx64KhA3F+Q4j1gTQ7V2HRFdgd6lnH1t5i2UcPQLmqkmUa1FAG1vfWxO4m/4JEfQ/yXJ9NZsZQkyp6r1VwksSGX9Fxw0rijUXAQ5W5lsvpNgsH+9Hv3PbydEakv3GH1MsdzQ2BDT7Tkgi2aG1u5rnUQgkJgk/yzOJuRd4qsdN7rBxP8ud+nLJ5mhE2ZY6CkfL2l/CmO41Wn4gQ0WOMdl7ow8RHU04RF9t6RaWFFeTFpMe4AbbBs= ceph-0d241f6a-71ec-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-13T16:22:13.862 INFO:teuthology.orchestra.run.smithi111.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPbEGzctL4hKBPFguGj1c5vzIqAe1BPs1xQRbvm0ipZBRQMNGrNmdk9AtAqUe0gljBiudEei1iw6TWxgGM6bwh17ttPE1kgi9slR6ooRxrSPkGFvUe6XiqEZa8mt9cW2AzaU/XN4/ZiHxZtxd8eVB3h9dacWUuai+EHVlOoTeMsGk4ZlfYoXL7KaAx9A2PN8oaowCByu6MUB7hbOdckOozT06Bb89uzriqlSh2vlA8UBE06ZMOVJjy8ZjSG7ZHKd5ViQPTQn5vRhdnmx64KhA3F+Q4j1gTQ7V2HRFdgd6lnH1t5i2UcPQLmqkmUa1FAG1vfWxO4m/4JEfQ/yXJ9NZsZQkyp6r1VwksSGX9Fxw0rijUXAQ5W5lsvpNgsH+9Hv3PbydEakv3GH1MsdzQ2BDT7Tkgi2aG1u5rnUQgkJgk/yzOJuRd4qsdN7rBxP8ud+nLJ5mhE2ZY6CkfL2l/CmO41Wn4gQ0WOMdl7ow8RHU04RF9t6RaWFFeTFpMe4AbbBs= ceph-0d241f6a-71ec-11ef-bceb-c7b262605968 2024-09-13T16:22:13.875 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-13T16:22:13.897 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:13 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3974179317' entity='client.admin' 2024-09-13T16:22:14.078 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:22:16.471 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:16 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:16.471 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:16 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:16.471 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:16 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi016", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:22:16.471 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:16 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:16.471 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:16 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:22:16.471 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:16 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:16.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.553+0000 7f777f4d5640 1 -- 172.21.15.16:0/1697905096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7778100260 msgr2=0x7f7778102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:16.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.553+0000 7f777f4d5640 1 --2- 172.21.15.16:0/1697905096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7778100260 0x7f7778102650 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7768009880 tx=0x7f776802a5d0 comp rx=0 tx=0).stop 2024-09-13T16:22:16.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.553+0000 7f777f4d5640 1 -- 172.21.15.16:0/1697905096 shutdown_connections 2024-09-13T16:22:16.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.553+0000 7f777f4d5640 1 --2- 172.21.15.16:0/1697905096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7778100260 0x7f7778102650 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:16.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.553+0000 7f777f4d5640 1 -- 172.21.15.16:0/1697905096 >> 172.21.15.16:0/1697905096 conn(0x7f77780f9e80 msgr2=0x7f77780fc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:16.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.554+0000 7f777f4d5640 1 -- 172.21.15.16:0/1697905096 shutdown_connections 2024-09-13T16:22:16.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.554+0000 7f777f4d5640 1 -- 172.21.15.16:0/1697905096 wait complete. 2024-09-13T16:22:16.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.554+0000 7f777f4d5640 1 Processor -- start 2024-09-13T16:22:16.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.554+0000 7f777f4d5640 1 -- start start 2024-09-13T16:22:16.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.554+0000 7f777f4d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7778100260 0x7f7778194c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:16.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.554+0000 7f777f4d5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7778195190 con 0x7f7778100260 2024-09-13T16:22:16.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.555+0000 7f777d271640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7778100260 0x7f7778194c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:16.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.555+0000 7f777d271640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7778100260 0x7f7778194c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38764/0 (socket says 172.21.15.16:38764) 2024-09-13T16:22:16.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.555+0000 7f777d271640 1 -- 172.21.15.16:0/3309689595 learned_addr learned my addr 172.21.15.16:0/3309689595 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:22:16.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.555+0000 7f777d271640 1 -- 172.21.15.16:0/3309689595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7768009560 con 0x7f7778100260 2024-09-13T16:22:16.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.555+0000 7f777d271640 1 --2- 172.21.15.16:0/3309689595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7778100260 0x7f7778194c50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7768006010 tx=0x7f7768004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:16.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.555+0000 7f776e7fc640 1 -- 172.21.15.16:0/3309689595 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77680055a0 con 0x7f7778100260 2024-09-13T16:22:16.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.555+0000 7f776e7fc640 1 -- 172.21.15.16:0/3309689595 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7768003a80 con 0x7f7778100260 2024-09-13T16:22:16.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.555+0000 7f777f4d5640 1 -- 172.21.15.16:0/3309689595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7778195390 con 0x7f7778100260 2024-09-13T16:22:16.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.555+0000 7f776e7fc640 1 -- 172.21.15.16:0/3309689595 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f776803c910 con 0x7f7778100260 2024-09-13T16:22:16.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.555+0000 7f777f4d5640 1 -- 172.21.15.16:0/3309689595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7778195830 con 0x7f7778100260 2024-09-13T16:22:16.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.556+0000 7f776e7fc640 1 -- 172.21.15.16:0/3309689595 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f776803ca70 con 0x7f7778100260 2024-09-13T16:22:16.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.556+0000 7f777f4d5640 1 -- 172.21.15.16:0/3309689595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77781006c0 con 0x7f7778100260 2024-09-13T16:22:16.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.556+0000 7f776e7fc640 1 --2- 172.21.15.16:0/3309689595 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7764038760 0x7f776403ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:16.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.556+0000 7f776e7fc640 1 -- 172.21.15.16:0/3309689595 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f776806c3b0 con 0x7f7778100260 2024-09-13T16:22:16.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.557+0000 7f777ca70640 1 --2- 172.21.15.16:0/3309689595 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7764038760 0x7f776403ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:16.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.557+0000 7f777ca70640 1 --2- 172.21.15.16:0/3309689595 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7764038760 0x7f776403ac20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f77740098c0 tx=0x7f7774006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:16.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.561+0000 7f776e7fc640 1 -- 172.21.15.16:0/3309689595 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f776804d350 con 0x7f7778100260 2024-09-13T16:22:16.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.734+0000 7f777f4d5640 1 -- 172.21.15.16:0/3309689595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f777818e380 con 0x7f7778100260 2024-09-13T16:22:16.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.737+0000 7f776e7fc640 1 -- 172.21.15.16:0/3309689595 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f777818e380 con 0x7f7778100260 2024-09-13T16:22:16.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.740+0000 7f777f4d5640 1 -- 172.21.15.16:0/3309689595 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7764038760 msgr2=0x7f776403ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:16.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.740+0000 7f777f4d5640 1 --2- 172.21.15.16:0/3309689595 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7764038760 0x7f776403ac20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f77740098c0 tx=0x7f7774006db0 comp rx=0 tx=0).stop 2024-09-13T16:22:16.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.740+0000 7f777f4d5640 1 -- 172.21.15.16:0/3309689595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7778100260 msgr2=0x7f7778194c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:16.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.740+0000 7f777f4d5640 1 --2- 172.21.15.16:0/3309689595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7778100260 0x7f7778194c50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7768006010 tx=0x7f7768004600 comp rx=0 tx=0).stop 2024-09-13T16:22:16.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.740+0000 7f777f4d5640 1 -- 172.21.15.16:0/3309689595 shutdown_connections 2024-09-13T16:22:16.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.740+0000 7f777f4d5640 1 --2- 172.21.15.16:0/3309689595 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7764038760 0x7f776403ac20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:16.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.740+0000 7f777f4d5640 1 --2- 172.21.15.16:0/3309689595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7778100260 0x7f7778194c50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:16.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.740+0000 7f777f4d5640 1 -- 172.21.15.16:0/3309689595 >> 172.21.15.16:0/3309689595 conn(0x7f77780f9e80 msgr2=0x7f77780fab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:16.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.741+0000 7f777f4d5640 1 -- 172.21.15.16:0/3309689595 shutdown_connections 2024-09-13T16:22:16.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:16.741+0000 7f777f4d5640 1 -- 172.21.15.16:0/3309689595 wait complete. 2024-09-13T16:22:17.370 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-13T16:22:17.371 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-13T16:22:17.988 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:17 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3309689595' entity='client.admin' 2024-09-13T16:22:17.989 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:17 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:22:17.989 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:17 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:18.119 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:22:19.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.622+0000 7f2215594640 1 -- 172.21.15.16:0/2222967318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22100fe6f0 msgr2=0x7f22100feaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:19.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.622+0000 7f2215594640 1 --2- 172.21.15.16:0/2222967318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22100fe6f0 0x7f22100feaf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2200009880 tx=0x7f220002a5d0 comp rx=0 tx=0).stop 2024-09-13T16:22:19.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.622+0000 7f2215594640 1 -- 172.21.15.16:0/2222967318 shutdown_connections 2024-09-13T16:22:19.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.622+0000 7f2215594640 1 --2- 172.21.15.16:0/2222967318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22100fe6f0 0x7f22100feaf0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:19.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.622+0000 7f2215594640 1 -- 172.21.15.16:0/2222967318 >> 172.21.15.16:0/2222967318 conn(0x7f22100f9e80 msgr2=0x7f22100fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:19.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.622+0000 7f2215594640 1 -- 172.21.15.16:0/2222967318 shutdown_connections 2024-09-13T16:22:19.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.622+0000 7f2215594640 1 -- 172.21.15.16:0/2222967318 wait complete. 2024-09-13T16:22:19.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.623+0000 7f2215594640 1 Processor -- start 2024-09-13T16:22:19.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.623+0000 7f2215594640 1 -- start start 2024-09-13T16:22:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.623+0000 7f2215594640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22100fe6f0 0x7f2210194bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.623+0000 7f2215594640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2210195100 con 0x7f22100fe6f0 2024-09-13T16:22:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.624+0000 7f220effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22100fe6f0 0x7f2210194bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.624+0000 7f220effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22100fe6f0 0x7f2210194bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38786/0 (socket says 172.21.15.16:38786) 2024-09-13T16:22:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.624+0000 7f220effd640 1 -- 172.21.15.16:0/4274277718 learned_addr learned my addr 172.21.15.16:0/4274277718 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:22:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.624+0000 7f220effd640 1 -- 172.21.15.16:0/4274277718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2200009560 con 0x7f22100fe6f0 2024-09-13T16:22:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.624+0000 7f220effd640 1 --2- 172.21.15.16:0/4274277718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22100fe6f0 0x7f2210194bc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2200005230 tx=0x7f2200005260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.624+0000 7f21f7fff640 1 -- 172.21.15.16:0/4274277718 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2200004b70 con 0x7f22100fe6f0 2024-09-13T16:22:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.624+0000 7f21f7fff640 1 -- 172.21.15.16:0/4274277718 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2200003a80 con 0x7f22100fe6f0 2024-09-13T16:22:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.624+0000 7f2215594640 1 -- 172.21.15.16:0/4274277718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2210195300 con 0x7f22100fe6f0 2024-09-13T16:22:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.624+0000 7f21f7fff640 1 -- 172.21.15.16:0/4274277718 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f220003c970 con 0x7f22100fe6f0 2024-09-13T16:22:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.624+0000 7f2215594640 1 -- 172.21.15.16:0/4274277718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22101957a0 con 0x7f22100fe6f0 2024-09-13T16:22:19.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.625+0000 7f21f7fff640 1 -- 172.21.15.16:0/4274277718 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f220003cc30 con 0x7f22100fe6f0 2024-09-13T16:22:19.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.625+0000 7f2215594640 1 -- 172.21.15.16:0/4274277718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22100feb70 con 0x7f22100fe6f0 2024-09-13T16:22:19.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.626+0000 7f21f7fff640 1 --2- 172.21.15.16:0/4274277718 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f21f8038710 0x7f21f803abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:19.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.626+0000 7f21f7fff640 1 -- 172.21.15.16:0/4274277718 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f220006c5b0 con 0x7f22100fe6f0 2024-09-13T16:22:19.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.626+0000 7f220e7fc640 1 --2- 172.21.15.16:0/4274277718 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f21f8038710 0x7f21f803abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:19.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.626+0000 7f220e7fc640 1 --2- 172.21.15.16:0/4274277718 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f21f8038710 0x7f21f803abd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f22040098c0 tx=0x7f2204006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:19.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.631+0000 7f21f7fff640 1 -- 172.21.15.16:0/4274277718 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f22000443b0 con 0x7f22100fe6f0 2024-09-13T16:22:19.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.804+0000 7f2215594640 1 -- 172.21.15.16:0/4274277718 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f2210060980 con 0x7f21f8038710 2024-09-13T16:22:19.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.807+0000 7f21f7fff640 1 -- 172.21.15.16:0/4274277718 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2210060980 con 0x7f21f8038710 2024-09-13T16:22:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.810+0000 7f2215594640 1 -- 172.21.15.16:0/4274277718 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f21f8038710 msgr2=0x7f21f803abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.810+0000 7f2215594640 1 --2- 172.21.15.16:0/4274277718 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f21f8038710 0x7f21f803abd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f22040098c0 tx=0x7f2204006db0 comp rx=0 tx=0).stop 2024-09-13T16:22:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.810+0000 7f2215594640 1 -- 172.21.15.16:0/4274277718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22100fe6f0 msgr2=0x7f2210194bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.810+0000 7f2215594640 1 --2- 172.21.15.16:0/4274277718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22100fe6f0 0x7f2210194bc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2200005230 tx=0x7f2200005260 comp rx=0 tx=0).stop 2024-09-13T16:22:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.810+0000 7f2215594640 1 -- 172.21.15.16:0/4274277718 shutdown_connections 2024-09-13T16:22:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.810+0000 7f2215594640 1 --2- 172.21.15.16:0/4274277718 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f21f8038710 0x7f21f803abd0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.810+0000 7f2215594640 1 --2- 172.21.15.16:0/4274277718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22100fe6f0 0x7f2210194bc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.810+0000 7f2215594640 1 -- 172.21.15.16:0/4274277718 >> 172.21.15.16:0/4274277718 conn(0x7f22100f9e80 msgr2=0x7f22100686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:19.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.810+0000 7f2215594640 1 -- 172.21.15.16:0/4274277718 shutdown_connections 2024-09-13T16:22:19.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:19.810+0000 7f2215594640 1 -- 172.21.15.16:0/4274277718 wait complete. 2024-09-13T16:22:20.627 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi107 2024-09-13T16:22:20.627 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-13T16:22:20.627 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/etc/ceph/ceph.conf 2024-09-13T16:22:20.648 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-13T16:22:20.648 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-13T16:22:20.706 INFO:tasks.cephadm:Adding host smithi107 to orchestrator... 2024-09-13T16:22:20.706 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph orch host add smithi107 2024-09-13T16:22:20.906 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:22:21.027 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:20 smithi016 ceph-mon[31187]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:22:21.028 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:20 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:21.028 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:20 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:22:21.028 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:20 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:22:21.028 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:20 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:22:21.028 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:20 smithi016 ceph-mon[31187]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-13T16:22:21.028 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:20 smithi016 ceph-mon[31187]: Updating smithi016:/var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T16:22:21.028 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:20 smithi016 ceph-mon[31187]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-09-13T16:22:21.028 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:20 smithi016 ceph-mon[31187]: Updating smithi016:/var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T16:22:21.028 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:20 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:21.028 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:20 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:21.028 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:20 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:22.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:22.396+0000 7f42dc5aa640 1 -- 172.21.15.16:0/3507934519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42d41057a0 msgr2=0x7f42d4105b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:22.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:22.396+0000 7f42dc5aa640 1 --2- 172.21.15.16:0/3507934519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42d41057a0 0x7f42d4105b80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009880 tx=0x7f42d002a5d0 comp rx=0 tx=0).stop 2024-09-13T16:22:22.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:22.396+0000 7f42dc5aa640 1 -- 172.21.15.16:0/3507934519 shutdown_connections 2024-09-13T16:22:22.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:22.396+0000 7f42dc5aa640 1 --2- 172.21.15.16:0/3507934519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42d41057a0 0x7f42d4105b80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:22.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:22.396+0000 7f42dc5aa640 1 -- 172.21.15.16:0/3507934519 >> 172.21.15.16:0/3507934519 conn(0x7f42d4076500 msgr2=0x7f42d4076910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:22.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:22.396+0000 7f42dc5aa640 1 -- 172.21.15.16:0/3507934519 shutdown_connections 2024-09-13T16:22:22.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:22.396+0000 7f42dc5aa640 1 -- 172.21.15.16:0/3507934519 wait complete. 2024-09-13T16:22:22.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:22.397+0000 7f42dc5aa640 1 Processor -- start 2024-09-13T16:22:22.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:22.397+0000 7f42dc5aa640 1 -- start start 2024-09-13T16:22:22.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:22.397+0000 7f42dc5aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42d41057a0 0x7f42d4197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:22.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:22.397+0000 7f42dc5aa640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d41976b0 con 0x7f42d41057a0 2024-09-13T16:22:22.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:22.397+0000 7f42da346640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42d41057a0 0x7f42d4197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:22.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:22.397+0000 7f42da346640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42d41057a0 0x7f42d4197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38802/0 (socket says 172.21.15.16:38802) 2024-09-13T16:22:22.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:22.397+0000 7f42da346640 1 -- 172.21.15.16:0/2579681792 learned_addr learned my addr 172.21.15.16:0/2579681792 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:22:22.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:22.397+0000 7f42da346640 1 -- 172.21.15.16:0/2579681792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d0009560 con 0x7f42d41057a0 2024-09-13T16:22:22.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:22.398+0000 7f42da346640 1 --2- 172.21.15.16:0/2579681792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42d41057a0 0x7f42d4197170 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f42d0005ab0 tx=0x7f42d0005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:22.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:22.398+0000 7f42cb7fe640 1 -- 172.21.15.16:0/2579681792 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f42d0005440 con 0x7f42d41057a0 2024-09-13T16:22:22.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:22.398+0000 7f42dc5aa640 1 -- 172.21.15.16:0/2579681792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d4197910 con 0x7f42d41057a0 2024-09-13T16:22:22.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:22.398+0000 7f42cb7fe640 1 -- 172.21.15.16:0/2579681792 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f42d00055a0 con 0x7f42d41057a0 2024-09-13T16:22:22.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:22.398+0000 7f42cb7fe640 1 -- 172.21.15.16:0/2579681792 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f42d003c740 con 0x7f42d41057a0 2024-09-13T16:22:22.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:22.398+0000 7f42dc5aa640 1 -- 172.21.15.16:0/2579681792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d4197d90 con 0x7f42d41057a0 2024-09-13T16:22:22.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:22.399+0000 7f42cb7fe640 1 -- 172.21.15.16:0/2579681792 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f42d00035f0 con 0x7f42d41057a0 2024-09-13T16:22:22.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:22.399+0000 7f42dc5aa640 1 -- 172.21.15.16:0/2579681792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d4100e70 con 0x7f42d41057a0 2024-09-13T16:22:22.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:22.399+0000 7f42cb7fe640 1 --2- 172.21.15.16:0/2579681792 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f42c0038710 0x7f42c003abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:22.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:22.399+0000 7f42cb7fe640 1 -- 172.21.15.16:0/2579681792 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f42d006c330 con 0x7f42d41057a0 2024-09-13T16:22:22.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:22.399+0000 7f42d9b45640 1 --2- 172.21.15.16:0/2579681792 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f42c0038710 0x7f42c003abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:22.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:22.400+0000 7f42d9b45640 1 --2- 172.21.15.16:0/2579681792 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f42c0038710 0x7f42c003abd0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f42c40098c0 tx=0x7f42c4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:22.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:22.404+0000 7f42cb7fe640 1 -- 172.21.15.16:0/2579681792 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f42d003a360 con 0x7f42d41057a0 2024-09-13T16:22:22.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:22.579+0000 7f42dc5aa640 1 -- 172.21.15.16:0/2579681792 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi107", "target": ["mon-mgr", ""]}) v1 -- 0x7f42d41099a0 con 0x7f42c0038710 2024-09-13T16:22:23.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:23 smithi016 ceph-mon[31187]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi107", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:22:25.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:25 smithi016 ceph-mon[31187]: Deploying cephadm binary to smithi107 2024-09-13T16:22:25.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:25 smithi016 ceph-mon[31187]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:22:25.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:25.868+0000 7f42cb7fe640 1 -- 172.21.15.16:0/2579681792 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f42d41099a0 con 0x7f42c0038710 2024-09-13T16:22:25.870 INFO:teuthology.orchestra.run.smithi016.stdout:Added host 'smithi107' with addr '172.21.15.107' 2024-09-13T16:22:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:25.871+0000 7f42dc5aa640 1 -- 172.21.15.16:0/2579681792 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f42c0038710 msgr2=0x7f42c003abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:25.871+0000 7f42dc5aa640 1 --2- 172.21.15.16:0/2579681792 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f42c0038710 0x7f42c003abd0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f42c40098c0 tx=0x7f42c4006db0 comp rx=0 tx=0).stop 2024-09-13T16:22:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:25.871+0000 7f42dc5aa640 1 -- 172.21.15.16:0/2579681792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42d41057a0 msgr2=0x7f42d4197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:25.871+0000 7f42dc5aa640 1 --2- 172.21.15.16:0/2579681792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42d41057a0 0x7f42d4197170 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f42d0005ab0 tx=0x7f42d0005e70 comp rx=0 tx=0).stop 2024-09-13T16:22:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:25.871+0000 7f42dc5aa640 1 -- 172.21.15.16:0/2579681792 shutdown_connections 2024-09-13T16:22:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:25.871+0000 7f42dc5aa640 1 --2- 172.21.15.16:0/2579681792 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f42c0038710 0x7f42c003abd0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:25.871+0000 7f42dc5aa640 1 --2- 172.21.15.16:0/2579681792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42d41057a0 0x7f42d4197170 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:25.871+0000 7f42dc5aa640 1 -- 172.21.15.16:0/2579681792 >> 172.21.15.16:0/2579681792 conn(0x7f42d4076500 msgr2=0x7f42d40fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:25.871+0000 7f42dc5aa640 1 -- 172.21.15.16:0/2579681792 shutdown_connections 2024-09-13T16:22:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:25.871+0000 7f42dc5aa640 1 -- 172.21.15.16:0/2579681792 wait complete. 2024-09-13T16:22:26.505 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-13T16:22:26.705 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:22:27.094 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:26 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:27.094 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:26 smithi016 ceph-mon[31187]: Added host smithi107 2024-09-13T16:22:27.094 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:26 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:22:27.095 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:26 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:27.095 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:26 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:27.095 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:26 smithi016 ceph-mon[31187]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:22:28.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.222+0000 7fa9ca1e5640 1 -- 172.21.15.16:0/2996269170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9c4104650 msgr2=0x7fa9c4104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:28.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.222+0000 7fa9ca1e5640 1 --2- 172.21.15.16:0/2996269170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9c4104650 0x7fa9c4104a30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4009880 tx=0x7fa9b402a5d0 comp rx=0 tx=0).stop 2024-09-13T16:22:28.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.222+0000 7fa9ca1e5640 1 -- 172.21.15.16:0/2996269170 shutdown_connections 2024-09-13T16:22:28.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.222+0000 7fa9ca1e5640 1 --2- 172.21.15.16:0/2996269170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9c4104650 0x7fa9c4104a30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:28.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.222+0000 7fa9ca1e5640 1 -- 172.21.15.16:0/2996269170 >> 172.21.15.16:0/2996269170 conn(0x7fa9c4075490 msgr2=0x7fa9c40fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:28.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.222+0000 7fa9ca1e5640 1 -- 172.21.15.16:0/2996269170 shutdown_connections 2024-09-13T16:22:28.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.222+0000 7fa9ca1e5640 1 -- 172.21.15.16:0/2996269170 wait complete. 2024-09-13T16:22:28.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.223+0000 7fa9ca1e5640 1 Processor -- start 2024-09-13T16:22:28.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.223+0000 7fa9ca1e5640 1 -- start start 2024-09-13T16:22:28.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.223+0000 7fa9ca1e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9c4104650 0x7fa9c4195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:28.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.223+0000 7fa9ca1e5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c4068620 con 0x7fa9c4104650 2024-09-13T16:22:28.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.223+0000 7fa9c37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9c4104650 0x7fa9c4195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:28.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.223+0000 7fa9c37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9c4104650 0x7fa9c4195e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50076/0 (socket says 172.21.15.16:50076) 2024-09-13T16:22:28.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.223+0000 7fa9c37fe640 1 -- 172.21.15.16:0/674479138 learned_addr learned my addr 172.21.15.16:0/674479138 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:22:28.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.223+0000 7fa9c37fe640 1 -- 172.21.15.16:0/674479138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b4009560 con 0x7fa9c4104650 2024-09-13T16:22:28.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.224+0000 7fa9c37fe640 1 --2- 172.21.15.16:0/674479138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9c4104650 0x7fa9c4195e20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4005ab0 tx=0x7fa9b4005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:28.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.224+0000 7fa9c17fa640 1 -- 172.21.15.16:0/674479138 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9b4005440 con 0x7fa9c4104650 2024-09-13T16:22:28.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.224+0000 7fa9ca1e5640 1 -- 172.21.15.16:0/674479138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9c41963c0 con 0x7fa9c4104650 2024-09-13T16:22:28.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.224+0000 7fa9c17fa640 1 -- 172.21.15.16:0/674479138 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa9b40055a0 con 0x7fa9c4104650 2024-09-13T16:22:28.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.224+0000 7fa9c17fa640 1 -- 172.21.15.16:0/674479138 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9b403c740 con 0x7fa9c4104650 2024-09-13T16:22:28.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.224+0000 7fa9ca1e5640 1 -- 172.21.15.16:0/674479138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9c4196840 con 0x7fa9c4104650 2024-09-13T16:22:28.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.225+0000 7fa9c17fa640 1 -- 172.21.15.16:0/674479138 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fa9b40035f0 con 0x7fa9c4104650 2024-09-13T16:22:28.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.225+0000 7fa9ca1e5640 1 -- 172.21.15.16:0/674479138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9c40ffd90 con 0x7fa9c4104650 2024-09-13T16:22:28.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.225+0000 7fa9c17fa640 1 --2- 172.21.15.16:0/674479138 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa9b0038710 0x7fa9b003abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:28.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.225+0000 7fa9c17fa640 1 -- 172.21.15.16:0/674479138 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa9b406c450 con 0x7fa9c4104650 2024-09-13T16:22:28.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.225+0000 7fa9baffd640 1 --2- 172.21.15.16:0/674479138 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa9b0038710 0x7fa9b003abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:28.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.226+0000 7fa9baffd640 1 --2- 172.21.15.16:0/674479138 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa9b0038710 0x7fa9b003abd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0098c0 tx=0x7fa9ac006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:28.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.230+0000 7fa9c17fa640 1 -- 172.21.15.16:0/674479138 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa9b404d350 con 0x7fa9c4104650 2024-09-13T16:22:28.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:28 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:28.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.422+0000 7fa9ca1e5640 1 -- 172.21.15.16:0/674479138 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa9c4196af0 con 0x7fa9b0038710 2024-09-13T16:22:28.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.423+0000 7fa9c17fa640 1 -- 172.21.15.16:0/674479138 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7fa9c4196af0 con 0x7fa9b0038710 2024-09-13T16:22:28.424 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:22:28.424 INFO:teuthology.orchestra.run.smithi016.stdout:[{"addr": "172.21.15.16", "hostname": "smithi016", "labels": [], "status": ""}, {"addr": "172.21.15.107", "hostname": "smithi107", "labels": [], "status": ""}] 2024-09-13T16:22:28.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.425+0000 7fa9ca1e5640 1 -- 172.21.15.16:0/674479138 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa9b0038710 msgr2=0x7fa9b003abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:28.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.425+0000 7fa9ca1e5640 1 --2- 172.21.15.16:0/674479138 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa9b0038710 0x7fa9b003abd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0098c0 tx=0x7fa9ac006db0 comp rx=0 tx=0).stop 2024-09-13T16:22:28.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.425+0000 7fa9ca1e5640 1 -- 172.21.15.16:0/674479138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9c4104650 msgr2=0x7fa9c4195e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:28.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.425+0000 7fa9ca1e5640 1 --2- 172.21.15.16:0/674479138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9c4104650 0x7fa9c4195e20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4005ab0 tx=0x7fa9b4005e70 comp rx=0 tx=0).stop 2024-09-13T16:22:28.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.425+0000 7fa9ca1e5640 1 -- 172.21.15.16:0/674479138 shutdown_connections 2024-09-13T16:22:28.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.425+0000 7fa9ca1e5640 1 --2- 172.21.15.16:0/674479138 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa9b0038710 0x7fa9b003abd0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:28.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.425+0000 7fa9ca1e5640 1 --2- 172.21.15.16:0/674479138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9c4104650 0x7fa9c4195e20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:28.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.425+0000 7fa9ca1e5640 1 -- 172.21.15.16:0/674479138 >> 172.21.15.16:0/674479138 conn(0x7fa9c4075490 msgr2=0x7fa9c40fb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:28.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.425+0000 7fa9ca1e5640 1 -- 172.21.15.16:0/674479138 shutdown_connections 2024-09-13T16:22:28.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:28.425+0000 7fa9ca1e5640 1 -- 172.21.15.16:0/674479138 wait complete. 2024-09-13T16:22:29.000 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi111 2024-09-13T16:22:29.000 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T16:22:29.000 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/etc/ceph/ceph.conf 2024-09-13T16:22:29.018 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T16:22:29.019 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-13T16:22:29.078 INFO:tasks.cephadm:Adding host smithi111 to orchestrator... 2024-09-13T16:22:29.078 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph orch host add smithi111 2024-09-13T16:22:29.281 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:22:29.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:29 smithi016 ceph-mon[31187]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T16:22:29.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:29 smithi016 ceph-mon[31187]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:22:30.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:30.743+0000 7f0e8a7bb640 1 -- 172.21.15.16:0/1098752397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e840fe520 msgr2=0x7f0e840fe900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:30.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:30.743+0000 7f0e8a7bb640 1 --2- 172.21.15.16:0/1098752397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e840fe520 0x7f0e840fe900 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0e74009880 tx=0x7f0e7402a5d0 comp rx=0 tx=0).stop 2024-09-13T16:22:30.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:30.743+0000 7f0e8a7bb640 1 -- 172.21.15.16:0/1098752397 shutdown_connections 2024-09-13T16:22:30.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:30.743+0000 7f0e8a7bb640 1 --2- 172.21.15.16:0/1098752397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e840fe520 0x7f0e840fe900 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:30.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:30.743+0000 7f0e8a7bb640 1 -- 172.21.15.16:0/1098752397 >> 172.21.15.16:0/1098752397 conn(0x7f0e840fa3b0 msgr2=0x7f0e840fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:30.743+0000 7f0e8a7bb640 1 -- 172.21.15.16:0/1098752397 shutdown_connections 2024-09-13T16:22:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:30.743+0000 7f0e8a7bb640 1 -- 172.21.15.16:0/1098752397 wait complete. 2024-09-13T16:22:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:30.744+0000 7f0e8a7bb640 1 Processor -- start 2024-09-13T16:22:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:30.744+0000 7f0e8a7bb640 1 -- start start 2024-09-13T16:22:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:30.744+0000 7f0e8a7bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e840fe520 0x7f0e841972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:30.744+0000 7f0e8a7bb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e84197830 con 0x7f0e840fe520 2024-09-13T16:22:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:30.744+0000 7f0e83fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e840fe520 0x7f0e841972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:30.744+0000 7f0e83fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e840fe520 0x7f0e841972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50084/0 (socket says 172.21.15.16:50084) 2024-09-13T16:22:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:30.744+0000 7f0e83fff640 1 -- 172.21.15.16:0/268179634 learned_addr learned my addr 172.21.15.16:0/268179634 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:22:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:30.745+0000 7f0e83fff640 1 -- 172.21.15.16:0/268179634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e74009560 con 0x7f0e840fe520 2024-09-13T16:22:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:30.745+0000 7f0e83fff640 1 --2- 172.21.15.16:0/268179634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e840fe520 0x7f0e841972f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0e74005ab0 tx=0x7f0e74005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:30.746+0000 7f0e817fa640 1 -- 172.21.15.16:0/268179634 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e74005440 con 0x7f0e840fe520 2024-09-13T16:22:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:30.746+0000 7f0e8a7bb640 1 -- 172.21.15.16:0/268179634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e84197a30 con 0x7f0e840fe520 2024-09-13T16:22:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:30.747+0000 7f0e8a7bb640 1 -- 172.21.15.16:0/268179634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e84197e50 con 0x7f0e840fe520 2024-09-13T16:22:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:30.747+0000 7f0e817fa640 1 -- 172.21.15.16:0/268179634 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0e740055a0 con 0x7f0e840fe520 2024-09-13T16:22:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:30.747+0000 7f0e817fa640 1 -- 172.21.15.16:0/268179634 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e7403c760 con 0x7f0e840fe520 2024-09-13T16:22:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:30.747+0000 7f0e817fa640 1 -- 172.21.15.16:0/268179634 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f0e7403c8e0 con 0x7f0e840fe520 2024-09-13T16:22:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:30.747+0000 7f0e817fa640 1 --2- 172.21.15.16:0/268179634 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f0e6c038710 0x7f0e6c03abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:30.748+0000 7f0e817fa640 1 -- 172.21.15.16:0/268179634 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0e7406d1b0 con 0x7f0e840fe520 2024-09-13T16:22:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:30.748+0000 7f0e837fe640 1 --2- 172.21.15.16:0/268179634 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f0e6c038710 0x7f0e6c03abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:30.748+0000 7f0e8a7bb640 1 -- 172.21.15.16:0/268179634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e84068740 con 0x7f0e840fe520 2024-09-13T16:22:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:30.748+0000 7f0e837fe640 1 --2- 172.21.15.16:0/268179634 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f0e6c038710 0x7f0e6c03abd0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0e780098c0 tx=0x7f0e78006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:30.753+0000 7f0e817fa640 1 -- 172.21.15.16:0/268179634 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0e74035080 con 0x7f0e840fe520 2024-09-13T16:22:30.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:30.926+0000 7f0e8a7bb640 1 -- 172.21.15.16:0/268179634 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi111", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e841986c0 con 0x7f0e6c038710 2024-09-13T16:22:30.928 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:30 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:30.928 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:30 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:30.928 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:30 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:30.928 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:30 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:30.928 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:30 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi107", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:22:30.928 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:30 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:22:30.928 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:30 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:22:30.928 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:30 smithi016 ceph-mon[31187]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-13T16:22:30.928 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:30 smithi016 ceph-mon[31187]: Updating smithi107:/var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T16:22:30.928 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:30 smithi016 ceph-mon[31187]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-09-13T16:22:30.929 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:30 smithi016 ceph-mon[31187]: Updating smithi107:/var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T16:22:30.929 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:30 smithi016 ceph-mon[31187]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:22:30.929 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:30 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:30.929 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:30 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:30.929 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:30 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:32.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:31 smithi016 ceph-mon[31187]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi111", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:22:33.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:32 smithi016 ceph-mon[31187]: Deploying cephadm binary to smithi111 2024-09-13T16:22:33.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:32 smithi016 ceph-mon[31187]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:22:34.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:34.223+0000 7f0e817fa640 1 -- 172.21.15.16:0/268179634 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f0e841986c0 con 0x7f0e6c038710 2024-09-13T16:22:34.224 INFO:teuthology.orchestra.run.smithi016.stdout:Added host 'smithi111' with addr '172.21.15.111' 2024-09-13T16:22:34.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:34.225+0000 7f0e8a7bb640 1 -- 172.21.15.16:0/268179634 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f0e6c038710 msgr2=0x7f0e6c03abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:34.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:34.225+0000 7f0e8a7bb640 1 --2- 172.21.15.16:0/268179634 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f0e6c038710 0x7f0e6c03abd0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0e780098c0 tx=0x7f0e78006db0 comp rx=0 tx=0).stop 2024-09-13T16:22:34.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:34.225+0000 7f0e8a7bb640 1 -- 172.21.15.16:0/268179634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e840fe520 msgr2=0x7f0e841972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:34.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:34.225+0000 7f0e8a7bb640 1 --2- 172.21.15.16:0/268179634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e840fe520 0x7f0e841972f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0e74005ab0 tx=0x7f0e74005e70 comp rx=0 tx=0).stop 2024-09-13T16:22:34.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:34.225+0000 7f0e8a7bb640 1 -- 172.21.15.16:0/268179634 shutdown_connections 2024-09-13T16:22:34.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:34.225+0000 7f0e8a7bb640 1 --2- 172.21.15.16:0/268179634 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f0e6c038710 0x7f0e6c03abd0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:34.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:34.225+0000 7f0e8a7bb640 1 --2- 172.21.15.16:0/268179634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e840fe520 0x7f0e841972f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:34.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:34.225+0000 7f0e8a7bb640 1 -- 172.21.15.16:0/268179634 >> 172.21.15.16:0/268179634 conn(0x7f0e840fa3b0 msgr2=0x7f0e840fb550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:34.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:34.225+0000 7f0e8a7bb640 1 -- 172.21.15.16:0/268179634 shutdown_connections 2024-09-13T16:22:34.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:34.225+0000 7f0e8a7bb640 1 -- 172.21.15.16:0/268179634 wait complete. 2024-09-13T16:22:34.820 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-13T16:22:35.021 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:22:35.450 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:35 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:35.450 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:35 smithi016 ceph-mon[31187]: Added host smithi111 2024-09-13T16:22:35.450 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:35 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:22:35.451 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:35 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:35.451 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:35 smithi016 ceph-mon[31187]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:22:35.451 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:35 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:36.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.547+0000 7fa6cd4a9640 1 -- 172.21.15.16:0/763126523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6c81057a0 msgr2=0x7fa6c8105b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:36.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.547+0000 7fa6cd4a9640 1 --2- 172.21.15.16:0/763126523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6c81057a0 0x7fa6c8105b80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009880 tx=0x7fa6b802a5d0 comp rx=0 tx=0).stop 2024-09-13T16:22:36.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.547+0000 7fa6cd4a9640 1 -- 172.21.15.16:0/763126523 shutdown_connections 2024-09-13T16:22:36.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.547+0000 7fa6cd4a9640 1 --2- 172.21.15.16:0/763126523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6c81057a0 0x7fa6c8105b80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:36.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.547+0000 7fa6cd4a9640 1 -- 172.21.15.16:0/763126523 >> 172.21.15.16:0/763126523 conn(0x7fa6c8076500 msgr2=0x7fa6c8076910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:36.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.547+0000 7fa6cd4a9640 1 -- 172.21.15.16:0/763126523 shutdown_connections 2024-09-13T16:22:36.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.547+0000 7fa6cd4a9640 1 -- 172.21.15.16:0/763126523 wait complete. 2024-09-13T16:22:36.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.548+0000 7fa6cd4a9640 1 Processor -- start 2024-09-13T16:22:36.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.548+0000 7fa6cd4a9640 1 -- start start 2024-09-13T16:22:36.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.549+0000 7fa6cd4a9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6c81057a0 0x7fa6c8194f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:36.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.549+0000 7fa6cd4a9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c8195470 con 0x7fa6c81057a0 2024-09-13T16:22:36.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.549+0000 7fa6c6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6c81057a0 0x7fa6c8194f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:36.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.549+0000 7fa6c6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6c81057a0 0x7fa6c8194f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38530/0 (socket says 172.21.15.16:38530) 2024-09-13T16:22:36.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.549+0000 7fa6c6ffd640 1 -- 172.21.15.16:0/1092991503 learned_addr learned my addr 172.21.15.16:0/1092991503 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:22:36.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.549+0000 7fa6c6ffd640 1 -- 172.21.15.16:0/1092991503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b8009560 con 0x7fa6c81057a0 2024-09-13T16:22:36.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.549+0000 7fa6c6ffd640 1 --2- 172.21.15.16:0/1092991503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6c81057a0 0x7fa6c8194f30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8005ab0 tx=0x7fa6b8005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:36.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.549+0000 7fa6c4ff9640 1 -- 172.21.15.16:0/1092991503 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6b8005440 con 0x7fa6c81057a0 2024-09-13T16:22:36.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.549+0000 7fa6c4ff9640 1 -- 172.21.15.16:0/1092991503 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa6b80055a0 con 0x7fa6c81057a0 2024-09-13T16:22:36.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.549+0000 7fa6cd4a9640 1 -- 172.21.15.16:0/1092991503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c81956d0 con 0x7fa6c81057a0 2024-09-13T16:22:36.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.549+0000 7fa6c4ff9640 1 -- 172.21.15.16:0/1092991503 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6b803c740 con 0x7fa6c81057a0 2024-09-13T16:22:36.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.549+0000 7fa6cd4a9640 1 -- 172.21.15.16:0/1092991503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c8195b50 con 0x7fa6c81057a0 2024-09-13T16:22:36.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.550+0000 7fa6c4ff9640 1 -- 172.21.15.16:0/1092991503 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fa6b80035f0 con 0x7fa6c81057a0 2024-09-13T16:22:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.550+0000 7fa6cd4a9640 1 -- 172.21.15.16:0/1092991503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c8100e70 con 0x7fa6c81057a0 2024-09-13T16:22:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.550+0000 7fa6c4ff9640 1 --2- 172.21.15.16:0/1092991503 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa6b4038710 0x7fa6b403abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.551+0000 7fa6c4ff9640 1 -- 172.21.15.16:0/1092991503 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa6b806c410 con 0x7fa6c81057a0 2024-09-13T16:22:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.551+0000 7fa6bffff640 1 --2- 172.21.15.16:0/1092991503 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa6b4038710 0x7fa6b403abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:36.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.551+0000 7fa6bffff640 1 --2- 172.21.15.16:0/1092991503 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa6b4038710 0x7fa6b403abd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa6b00098c0 tx=0x7fa6b0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:36.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.555+0000 7fa6c4ff9640 1 -- 172.21.15.16:0/1092991503 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa6b804d350 con 0x7fa6c81057a0 2024-09-13T16:22:36.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:36 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:36.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.732+0000 7fa6cd4a9640 1 -- 172.21.15.16:0/1092991503 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa6c81099a0 con 0x7fa6b4038710 2024-09-13T16:22:36.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.733+0000 7fa6c4ff9640 1 -- 172.21.15.16:0/1092991503 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7fa6c81099a0 con 0x7fa6b4038710 2024-09-13T16:22:36.734 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:22:36.734 INFO:teuthology.orchestra.run.smithi016.stdout:[{"addr": "172.21.15.16", "hostname": "smithi016", "labels": [], "status": ""}, {"addr": "172.21.15.107", "hostname": "smithi107", "labels": [], "status": ""}, {"addr": "172.21.15.111", "hostname": "smithi111", "labels": [], "status": ""}] 2024-09-13T16:22:36.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.735+0000 7fa6cd4a9640 1 -- 172.21.15.16:0/1092991503 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa6b4038710 msgr2=0x7fa6b403abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:36.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.735+0000 7fa6cd4a9640 1 --2- 172.21.15.16:0/1092991503 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa6b4038710 0x7fa6b403abd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa6b00098c0 tx=0x7fa6b0006db0 comp rx=0 tx=0).stop 2024-09-13T16:22:36.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.735+0000 7fa6cd4a9640 1 -- 172.21.15.16:0/1092991503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6c81057a0 msgr2=0x7fa6c8194f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:36.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.735+0000 7fa6cd4a9640 1 --2- 172.21.15.16:0/1092991503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6c81057a0 0x7fa6c8194f30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8005ab0 tx=0x7fa6b8005e70 comp rx=0 tx=0).stop 2024-09-13T16:22:36.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.735+0000 7fa6cd4a9640 1 -- 172.21.15.16:0/1092991503 shutdown_connections 2024-09-13T16:22:36.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.735+0000 7fa6cd4a9640 1 --2- 172.21.15.16:0/1092991503 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa6b4038710 0x7fa6b403abd0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:36.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.735+0000 7fa6cd4a9640 1 --2- 172.21.15.16:0/1092991503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6c81057a0 0x7fa6c8194f30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:36.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.735+0000 7fa6cd4a9640 1 -- 172.21.15.16:0/1092991503 >> 172.21.15.16:0/1092991503 conn(0x7fa6c8076500 msgr2=0x7fa6c80fc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:36.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.735+0000 7fa6cd4a9640 1 -- 172.21.15.16:0/1092991503 shutdown_connections 2024-09-13T16:22:36.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:36.735+0000 7fa6cd4a9640 1 -- 172.21.15.16:0/1092991503 wait complete. 2024-09-13T16:22:37.393 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-13T16:22:37.393 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-13T16:22:37.594 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:22:37.715 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:37 smithi016 ceph-mon[31187]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:22:38.708 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:38 smithi016 ceph-mon[31187]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T16:22:38.708 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:38 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:38.709 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:38 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:38.709 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:38 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:38.709 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:38 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:38.709 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:38 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi111", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:22:38.709 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:38 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:22:38.709 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:38 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:22:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:39.221+0000 7fb860567640 1 -- 172.21.15.16:0/3272044947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8581057a0 msgr2=0x7fb858105b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:39.221+0000 7fb860567640 1 --2- 172.21.15.16:0/3272044947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8581057a0 0x7fb858105b80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb854009880 tx=0x7fb85402a5d0 comp rx=0 tx=0).stop 2024-09-13T16:22:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:39.221+0000 7fb860567640 1 -- 172.21.15.16:0/3272044947 shutdown_connections 2024-09-13T16:22:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:39.221+0000 7fb860567640 1 --2- 172.21.15.16:0/3272044947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8581057a0 0x7fb858105b80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:39.221+0000 7fb860567640 1 -- 172.21.15.16:0/3272044947 >> 172.21.15.16:0/3272044947 conn(0x7fb858076500 msgr2=0x7fb858076910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:39.221+0000 7fb860567640 1 -- 172.21.15.16:0/3272044947 shutdown_connections 2024-09-13T16:22:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:39.221+0000 7fb860567640 1 -- 172.21.15.16:0/3272044947 wait complete. 2024-09-13T16:22:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:39.222+0000 7fb860567640 1 Processor -- start 2024-09-13T16:22:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:39.222+0000 7fb860567640 1 -- start start 2024-09-13T16:22:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:39.222+0000 7fb860567640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8581057a0 0x7fb858194ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:39.222+0000 7fb860567640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb858195420 con 0x7fb8581057a0 2024-09-13T16:22:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:39.222+0000 7fb85e303640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8581057a0 0x7fb858194ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:39.222+0000 7fb85e303640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8581057a0 0x7fb858194ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38542/0 (socket says 172.21.15.16:38542) 2024-09-13T16:22:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:39.222+0000 7fb85e303640 1 -- 172.21.15.16:0/2320325142 learned_addr learned my addr 172.21.15.16:0/2320325142 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:22:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:39.222+0000 7fb85e303640 1 -- 172.21.15.16:0/2320325142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb854009560 con 0x7fb8581057a0 2024-09-13T16:22:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:39.222+0000 7fb85e303640 1 --2- 172.21.15.16:0/2320325142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8581057a0 0x7fb858194ee0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb854005ab0 tx=0x7fb854005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:39.223+0000 7fb84f7fe640 1 -- 172.21.15.16:0/2320325142 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb854005440 con 0x7fb8581057a0 2024-09-13T16:22:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:39.223+0000 7fb84f7fe640 1 -- 172.21.15.16:0/2320325142 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb8540055a0 con 0x7fb8581057a0 2024-09-13T16:22:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:39.223+0000 7fb860567640 1 -- 172.21.15.16:0/2320325142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb858195680 con 0x7fb8581057a0 2024-09-13T16:22:39.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:39.223+0000 7fb84f7fe640 1 -- 172.21.15.16:0/2320325142 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb85403c740 con 0x7fb8581057a0 2024-09-13T16:22:39.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:39.223+0000 7fb860567640 1 -- 172.21.15.16:0/2320325142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb858195b00 con 0x7fb8581057a0 2024-09-13T16:22:39.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:39.224+0000 7fb84f7fe640 1 -- 172.21.15.16:0/2320325142 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fb8540035f0 con 0x7fb8581057a0 2024-09-13T16:22:39.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:39.224+0000 7fb860567640 1 -- 172.21.15.16:0/2320325142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb858100e70 con 0x7fb8581057a0 2024-09-13T16:22:39.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:39.224+0000 7fb84f7fe640 1 --2- 172.21.15.16:0/2320325142 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb844038710 0x7fb84403abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:39.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:39.224+0000 7fb84f7fe640 1 -- 172.21.15.16:0/2320325142 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb85406c350 con 0x7fb8581057a0 2024-09-13T16:22:39.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:39.224+0000 7fb85db02640 1 --2- 172.21.15.16:0/2320325142 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb844038710 0x7fb84403abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:39.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:39.224+0000 7fb85db02640 1 --2- 172.21.15.16:0/2320325142 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb844038710 0x7fb84403abd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb8480098c0 tx=0x7fb848006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:39.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:39.229+0000 7fb84f7fe640 1 -- 172.21.15.16:0/2320325142 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb85404d350 con 0x7fb8581057a0 2024-09-13T16:22:39.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:39.409+0000 7fb860567640 1 -- 172.21.15.16:0/2320325142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fb858109990 con 0x7fb8581057a0 2024-09-13T16:22:39.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:39 smithi016 ceph-mon[31187]: Updating smithi111:/etc/ceph/ceph.conf 2024-09-13T16:22:39.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:39 smithi016 ceph-mon[31187]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:22:39.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:39 smithi016 ceph-mon[31187]: Updating smithi111:/var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T16:22:39.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:39 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:39.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:39 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:39.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:39 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:39.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:39 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2320325142' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-13T16:22:40.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:40.268+0000 7fb84f7fe640 1 -- 172.21.15.16:0/2320325142 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fb854035020 con 0x7fb8581057a0 2024-09-13T16:22:40.269 INFO:teuthology.orchestra.run.smithi016.stderr:adjusted tunables profile to default 2024-09-13T16:22:40.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:40.271+0000 7fb860567640 1 -- 172.21.15.16:0/2320325142 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb844038710 msgr2=0x7fb84403abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:40.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:40.271+0000 7fb860567640 1 --2- 172.21.15.16:0/2320325142 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb844038710 0x7fb84403abd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb8480098c0 tx=0x7fb848006db0 comp rx=0 tx=0).stop 2024-09-13T16:22:40.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:40.271+0000 7fb860567640 1 -- 172.21.15.16:0/2320325142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8581057a0 msgr2=0x7fb858194ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:40.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:40.271+0000 7fb860567640 1 --2- 172.21.15.16:0/2320325142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8581057a0 0x7fb858194ee0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb854005ab0 tx=0x7fb854005e70 comp rx=0 tx=0).stop 2024-09-13T16:22:40.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:40.271+0000 7fb860567640 1 -- 172.21.15.16:0/2320325142 shutdown_connections 2024-09-13T16:22:40.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:40.271+0000 7fb860567640 1 --2- 172.21.15.16:0/2320325142 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb844038710 0x7fb84403abd0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:40.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:40.271+0000 7fb860567640 1 --2- 172.21.15.16:0/2320325142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8581057a0 0x7fb858194ee0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:40.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:40.271+0000 7fb860567640 1 -- 172.21.15.16:0/2320325142 >> 172.21.15.16:0/2320325142 conn(0x7fb858076500 msgr2=0x7fb8580fc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:40.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:40.271+0000 7fb860567640 1 -- 172.21.15.16:0/2320325142 shutdown_connections 2024-09-13T16:22:40.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:22:40.271+0000 7fb860567640 1 -- 172.21.15.16:0/2320325142 wait complete. 2024-09-13T16:22:40.565 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:40 smithi016 ceph-mon[31187]: Updating smithi111:/etc/ceph/ceph.client.admin.keyring 2024-09-13T16:22:40.565 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:40 smithi016 ceph-mon[31187]: Updating smithi111:/var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T16:22:40.565 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:40 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2320325142' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-13T16:22:40.565 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:40 smithi016 ceph-mon[31187]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T16:22:40.860 INFO:tasks.cephadm:Adding mon.a on smithi016 2024-09-13T16:22:40.860 INFO:tasks.cephadm:Adding mon.b on smithi107 2024-09-13T16:22:40.860 INFO:tasks.cephadm:Adding mon.c on smithi111 2024-09-13T16:22:40.860 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi016:172.21.15.16=a;smithi107:172.21.15.107=b;smithi111:172.21.15.111=c' 2024-09-13T16:22:41.159 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T16:22:41.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:41 smithi016 ceph-mon[31187]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:22:42.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.667+0000 7f49c47fc640 1 -- 172.21.15.111:0/3477950096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49bc1057a0 msgr2=0x7f49bc105b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:42.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.667+0000 7f49c47fc640 1 --2- 172.21.15.111:0/3477950096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49bc1057a0 0x7f49bc105b80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f49b8009880 tx=0x7f49b802a5d0 comp rx=0 tx=0).stop 2024-09-13T16:22:42.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.667+0000 7f49c47fc640 1 -- 172.21.15.111:0/3477950096 shutdown_connections 2024-09-13T16:22:42.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.667+0000 7f49c47fc640 1 --2- 172.21.15.111:0/3477950096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49bc1057a0 0x7f49bc105b80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:42.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.667+0000 7f49c47fc640 1 -- 172.21.15.111:0/3477950096 >> 172.21.15.111:0/3477950096 conn(0x7f49bc076500 msgr2=0x7f49bc076910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:42.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.667+0000 7f49c47fc640 1 -- 172.21.15.111:0/3477950096 shutdown_connections 2024-09-13T16:22:42.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.667+0000 7f49c47fc640 1 -- 172.21.15.111:0/3477950096 wait complete. 2024-09-13T16:22:42.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.668+0000 7f49c47fc640 1 Processor -- start 2024-09-13T16:22:42.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.668+0000 7f49c47fc640 1 -- start start 2024-09-13T16:22:42.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.668+0000 7f49c47fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49bc1057a0 0x7f49bc1970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:42.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.668+0000 7f49c47fc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc197630 con 0x7f49bc1057a0 2024-09-13T16:22:42.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.668+0000 7f49c2598640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49bc1057a0 0x7f49bc1970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:42.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.668+0000 7f49c2598640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49bc1057a0 0x7f49bc1970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.111:58392/0 (socket says 172.21.15.111:58392) 2024-09-13T16:22:42.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.668+0000 7f49c2598640 1 -- 172.21.15.111:0/3386773892 learned_addr learned my addr 172.21.15.111:0/3386773892 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T16:22:42.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.669+0000 7f49c2598640 1 -- 172.21.15.111:0/3386773892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b8009560 con 0x7f49bc1057a0 2024-09-13T16:22:42.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.669+0000 7f49c2598640 1 --2- 172.21.15.111:0/3386773892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49bc1057a0 0x7f49bc1970f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f49b8005ab0 tx=0x7f49b8005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:42.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.669+0000 7f49b37fe640 1 -- 172.21.15.111:0/3386773892 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49b8005440 con 0x7f49bc1057a0 2024-09-13T16:22:42.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.669+0000 7f49c47fc640 1 -- 172.21.15.111:0/3386773892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49bc197890 con 0x7f49bc1057a0 2024-09-13T16:22:42.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.669+0000 7f49b37fe640 1 -- 172.21.15.111:0/3386773892 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f49b80055a0 con 0x7f49bc1057a0 2024-09-13T16:22:42.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.669+0000 7f49b37fe640 1 -- 172.21.15.111:0/3386773892 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49b803c740 con 0x7f49bc1057a0 2024-09-13T16:22:42.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.669+0000 7f49c47fc640 1 -- 172.21.15.111:0/3386773892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49bc197d10 con 0x7f49bc1057a0 2024-09-13T16:22:42.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.670+0000 7f49b37fe640 1 -- 172.21.15.111:0/3386773892 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f49b803ca00 con 0x7f49bc1057a0 2024-09-13T16:22:42.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.670+0000 7f49b37fe640 1 --2- 172.21.15.111:0/3386773892 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f49a8038710 0x7f49a803abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:42.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.670+0000 7f49b37fe640 1 -- 172.21.15.111:0/3386773892 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f49b806c3c0 con 0x7f49bc1057a0 2024-09-13T16:22:42.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.670+0000 7f49c47fc640 1 -- 172.21.15.111:0/3386773892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49bc100e70 con 0x7f49bc1057a0 2024-09-13T16:22:42.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.671+0000 7f49c1d97640 1 --2- 172.21.15.111:0/3386773892 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f49a8038710 0x7f49a803abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:42.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.671+0000 7f49c1d97640 1 --2- 172.21.15.111:0/3386773892 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f49a8038710 0x7f49a803abd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f49ac0098c0 tx=0x7f49ac006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:42.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.676+0000 7f49b37fe640 1 -- 172.21.15.111:0/3386773892 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f49b8040080 con 0x7f49bc1057a0 2024-09-13T16:22:42.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.857+0000 7f49c47fc640 1 -- 172.21.15.111:0/3386773892 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi016:172.21.15.16=a;smithi107:172.21.15.107=b;smithi111:172.21.15.111=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f49bc1099f0 con 0x7f49a8038710 2024-09-13T16:22:42.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.863+0000 7f49b37fe640 1 -- 172.21.15.111:0/3386773892 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f49bc1099f0 con 0x7f49a8038710 2024-09-13T16:22:42.864 INFO:teuthology.orchestra.run.smithi111.stdout:Scheduled mon update... 2024-09-13T16:22:42.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.865+0000 7f49c47fc640 1 -- 172.21.15.111:0/3386773892 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f49a8038710 msgr2=0x7f49a803abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:42.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.865+0000 7f49c47fc640 1 --2- 172.21.15.111:0/3386773892 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f49a8038710 0x7f49a803abd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f49ac0098c0 tx=0x7f49ac006db0 comp rx=0 tx=0).stop 2024-09-13T16:22:42.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.865+0000 7f49c47fc640 1 -- 172.21.15.111:0/3386773892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49bc1057a0 msgr2=0x7f49bc1970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:42.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.865+0000 7f49c47fc640 1 --2- 172.21.15.111:0/3386773892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49bc1057a0 0x7f49bc1970f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f49b8005ab0 tx=0x7f49b8005e70 comp rx=0 tx=0).stop 2024-09-13T16:22:42.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.865+0000 7f49c47fc640 1 -- 172.21.15.111:0/3386773892 shutdown_connections 2024-09-13T16:22:42.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.865+0000 7f49c47fc640 1 --2- 172.21.15.111:0/3386773892 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f49a8038710 0x7f49a803abd0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:42.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.865+0000 7f49c47fc640 1 --2- 172.21.15.111:0/3386773892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49bc1057a0 0x7f49bc1970f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:42.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.865+0000 7f49c47fc640 1 -- 172.21.15.111:0/3386773892 >> 172.21.15.111:0/3386773892 conn(0x7f49bc076500 msgr2=0x7f49bc0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:42.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.865+0000 7f49c47fc640 1 -- 172.21.15.111:0/3386773892 shutdown_connections 2024-09-13T16:22:42.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:42.865+0000 7f49c47fc640 1 -- 172.21.15.111:0/3386773892 wait complete. 2024-09-13T16:22:43.520 DEBUG:teuthology.orchestra.run.smithi107:mon.b> sudo journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mon.b.service 2024-09-13T16:22:43.523 DEBUG:teuthology.orchestra.run.smithi111:mon.c> sudo journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mon.c.service 2024-09-13T16:22:43.525 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-13T16:22:43.526 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-13T16:22:43.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:43 smithi016 ceph-mon[31187]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:22:43.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:43 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:43.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:43 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:22:43.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:43 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:22:43.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:43 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:22:43.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:43 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:43.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:43 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-13T16:22:43.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:43 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:22:44.249 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T16:22:44.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:44 smithi016 ceph-mon[31187]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi016:172.21.15.16=a;smithi107:172.21.15.107=b;smithi111:172.21.15.111=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:22:44.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:44 smithi016 ceph-mon[31187]: Saving service mon spec with placement smithi016:172.21.15.16=a;smithi107:172.21.15.107=b;smithi111:172.21.15.111=c;count:3 2024-09-13T16:22:44.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:44 smithi016 ceph-mon[31187]: Deploying daemon mon.c on smithi111 2024-09-13T16:22:45.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:45 smithi016 ceph-mon[31187]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:22:47.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.412+0000 7f98f7858640 1 -- 172.21.15.111:0/939371013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f00749e0 msgr2=0x7f98f0074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:47.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.412+0000 7f98f7858640 1 --2- 172.21.15.111:0/939371013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f00749e0 0x7f98f0074dc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f98ec009880 tx=0x7f98ec02a5d0 comp rx=0 tx=0).stop 2024-09-13T16:22:47.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.412+0000 7f98f7858640 1 -- 172.21.15.111:0/939371013 shutdown_connections 2024-09-13T16:22:47.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.412+0000 7f98f7858640 1 --2- 172.21.15.111:0/939371013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f00749e0 0x7f98f0074dc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:47.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.412+0000 7f98f7858640 1 -- 172.21.15.111:0/939371013 >> 172.21.15.111:0/939371013 conn(0x7f98f00fd690 msgr2=0x7f98f00ffab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:47.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.412+0000 7f98f7858640 1 -- 172.21.15.111:0/939371013 shutdown_connections 2024-09-13T16:22:47.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.412+0000 7f98f7858640 1 -- 172.21.15.111:0/939371013 wait complete. 2024-09-13T16:22:47.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.412+0000 7f98f7858640 1 Processor -- start 2024-09-13T16:22:47.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.412+0000 7f98f7858640 1 -- start start 2024-09-13T16:22:47.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.413+0000 7f98f7858640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f00749e0 0x7f98f019b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:47.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.413+0000 7f98f7858640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f019bad0 con 0x7f98f00749e0 2024-09-13T16:22:47.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.413+0000 7f98f55f4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f00749e0 0x7f98f019b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:47.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.413+0000 7f98f55f4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f00749e0 0x7f98f019b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.111:60594/0 (socket says 172.21.15.111:60594) 2024-09-13T16:22:47.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.413+0000 7f98f55f4640 1 -- 172.21.15.111:0/2492182994 learned_addr learned my addr 172.21.15.111:0/2492182994 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T16:22:47.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.413+0000 7f98f55f4640 1 -- 172.21.15.111:0/2492182994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ec009560 con 0x7f98f00749e0 2024-09-13T16:22:47.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.413+0000 7f98f55f4640 1 --2- 172.21.15.111:0/2492182994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f00749e0 0x7f98f019b590 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f98ec005ab0 tx=0x7f98ec005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:47.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.414+0000 7f98e67fc640 1 -- 172.21.15.111:0/2492182994 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98ec005440 con 0x7f98f00749e0 2024-09-13T16:22:47.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.414+0000 7f98f7858640 1 -- 172.21.15.111:0/2492182994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f019bcd0 con 0x7f98f00749e0 2024-09-13T16:22:47.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.414+0000 7f98f7858640 1 -- 172.21.15.111:0/2492182994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f019c0f0 con 0x7f98f00749e0 2024-09-13T16:22:47.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.414+0000 7f98e67fc640 1 -- 172.21.15.111:0/2492182994 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f98ec0055a0 con 0x7f98f00749e0 2024-09-13T16:22:47.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.414+0000 7f98e67fc640 1 -- 172.21.15.111:0/2492182994 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98ec03c760 con 0x7f98f00749e0 2024-09-13T16:22:47.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.415+0000 7f98f7858640 1 -- 172.21.15.111:0/2492182994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98d4005250 con 0x7f98f00749e0 2024-09-13T16:22:47.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.415+0000 7f98e67fc640 1 -- 172.21.15.111:0/2492182994 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f98ec0035f0 con 0x7f98f00749e0 2024-09-13T16:22:47.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.415+0000 7f98e67fc640 1 --2- 172.21.15.111:0/2492182994 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f98dc0386c0 0x7f98dc03ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:47.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.415+0000 7f98e67fc640 1 -- 172.21.15.111:0/2492182994 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f98ec06c370 con 0x7f98f00749e0 2024-09-13T16:22:47.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.415+0000 7f98f4df3640 1 --2- 172.21.15.111:0/2492182994 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f98dc0386c0 0x7f98dc03ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:47.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.416+0000 7f98f4df3640 1 --2- 172.21.15.111:0/2492182994 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f98dc0386c0 0x7f98dc03ab80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f98e00098c0 tx=0x7f98e0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:47.421 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.420+0000 7f98e67fc640 1 -- 172.21.15.111:0/2492182994 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f98ec03a540 con 0x7f98f00749e0 2024-09-13T16:22:47.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.652+0000 7f98f7858640 1 -- 172.21.15.111:0/2492182994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f98d40050c0 con 0x7f98f00749e0 2024-09-13T16:22:47.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.652+0000 7f98e67fc640 1 -- 172.21.15.111:0/2492182994 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7f98ec035090 con 0x7f98f00749e0 2024-09-13T16:22:47.654 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T16:22:47.654 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":1,"fsid":"0d241f6a-71ec-11ef-bceb-c7b262605968","modified":"2024-09-13T16:21:13.217034Z","created":"2024-09-13T16:21:13.217034Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-13T16:22:47.654 INFO:teuthology.orchestra.run.smithi111.stderr:dumped monmap epoch 1 2024-09-13T16:22:47.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.654+0000 7f98f7858640 1 -- 172.21.15.111:0/2492182994 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f98dc0386c0 msgr2=0x7f98dc03ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:47.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.654+0000 7f98f7858640 1 --2- 172.21.15.111:0/2492182994 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f98dc0386c0 0x7f98dc03ab80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f98e00098c0 tx=0x7f98e0006db0 comp rx=0 tx=0).stop 2024-09-13T16:22:47.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.654+0000 7f98f7858640 1 -- 172.21.15.111:0/2492182994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f00749e0 msgr2=0x7f98f019b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:47.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.654+0000 7f98f7858640 1 --2- 172.21.15.111:0/2492182994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f00749e0 0x7f98f019b590 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f98ec005ab0 tx=0x7f98ec005e70 comp rx=0 tx=0).stop 2024-09-13T16:22:47.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.655+0000 7f98f7858640 1 -- 172.21.15.111:0/2492182994 shutdown_connections 2024-09-13T16:22:47.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.655+0000 7f98f7858640 1 --2- 172.21.15.111:0/2492182994 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f98dc0386c0 0x7f98dc03ab80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:47.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.655+0000 7f98f7858640 1 --2- 172.21.15.111:0/2492182994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f00749e0 0x7f98f019b590 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:47.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.655+0000 7f98f7858640 1 -- 172.21.15.111:0/2492182994 >> 172.21.15.111:0/2492182994 conn(0x7f98f00fd690 msgr2=0x7f98f010c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:47.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.655+0000 7f98f7858640 1 -- 172.21.15.111:0/2492182994 shutdown_connections 2024-09-13T16:22:47.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:47.655+0000 7f98f7858640 1 -- 172.21.15.111:0/2492182994 wait complete. 2024-09-13T16:22:47.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:47 smithi016 ceph-mon[31187]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:22:49.446 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-13T16:22:49.446 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-13T16:22:49.656 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T16:22:53.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:22:53 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:22:53.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:53 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T16:22:53.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:53 smithi016 ceph-mon[31187]: mon.a calling monitor election 2024-09-13T16:22:53.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:53 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T16:22:53.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:53 smithi016 ceph-mon[31187]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:22:53.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:53 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T16:22:53.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:53 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T16:22:53.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:53 smithi016 ceph-mon[31187]: mon.c calling monitor election 2024-09-13T16:22:53.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:53 smithi016 ceph-mon[31187]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:22:53.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:53 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T16:22:53.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:53 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T16:22:53.606 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:53 smithi016 ceph-mon[31187]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:22:53.606 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:53 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T16:22:53.606 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:53 smithi016 ceph-mon[31187]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-13T16:22:53.606 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:53 smithi016 ceph-mon[31187]: monmap e2: 2 mons at {a=[v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0],c=[v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-13T16:22:53.606 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:53 smithi016 ceph-mon[31187]: fsmap 2024-09-13T16:22:53.606 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:53 smithi016 ceph-mon[31187]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T16:22:53.606 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:53 smithi016 ceph-mon[31187]: mgrmap e13: x(active, since 48s) 2024-09-13T16:22:53.606 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:53 smithi016 ceph-mon[31187]: overall HEALTH_OK 2024-09-13T16:22:53.606 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:53 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:53.606 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:53 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:53.606 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:53 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:22:53.606 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:53 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-13T16:22:53.607 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:53 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:22:54.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.162+0000 7f8c87d65640 1 -- 172.21.15.111:0/3238111707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c68003530 msgr2=0x7f8c680059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:54.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.162+0000 7f8c87d65640 1 --2- 172.21.15.111:0/3238111707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c68003530 0x7f8c680059c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8c80100e70 tx=0x7f8c7002bc80 comp rx=0 tx=0).stop 2024-09-13T16:22:54.163 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.162+0000 7f8c87d65640 1 -- 172.21.15.111:0/3238111707 shutdown_connections 2024-09-13T16:22:54.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.162+0000 7f8c87d65640 1 --2- 172.21.15.111:0/3238111707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c68003530 0x7f8c680059c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:54.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.162+0000 7f8c87d65640 1 -- 172.21.15.111:0/3238111707 >> 172.21.15.111:0/3238111707 conn(0x7f8c80074620 msgr2=0x7f8c80074a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:54.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.162+0000 7f8c87d65640 1 -- 172.21.15.111:0/3238111707 shutdown_connections 2024-09-13T16:22:54.164 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.162+0000 7f8c87d65640 1 -- 172.21.15.111:0/3238111707 wait complete. 2024-09-13T16:22:54.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.163+0000 7f8c87d65640 1 Processor -- start 2024-09-13T16:22:54.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.163+0000 7f8c87d65640 1 -- start start 2024-09-13T16:22:54.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.163+0000 7f8c87d65640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c68003530 0x7f8c800feb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:54.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.163+0000 7f8c87d65640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c800ff0c0 0x7f8c800ff520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:54.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.163+0000 7f8c87d65640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c80100a90 con 0x7f8c68003530 2024-09-13T16:22:54.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.163+0000 7f8c87d65640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c801a1350 con 0x7f8c800ff0c0 2024-09-13T16:22:54.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.164+0000 7f8c85300640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c800ff0c0 0x7f8c800ff520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:54.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.164+0000 7f8c85300640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c800ff0c0 0x7f8c800ff520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56178/0 (socket says 172.21.15.111:56178) 2024-09-13T16:22:54.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.164+0000 7f8c85300640 1 -- 172.21.15.111:0/2394854031 learned_addr learned my addr 172.21.15.111:0/2394854031 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T16:22:54.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.164+0000 7f8c85b01640 1 --2- 172.21.15.111:0/2394854031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c68003530 0x7f8c800feb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:54.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.164+0000 7f8c85300640 1 -- 172.21.15.111:0/2394854031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c800ff0c0 msgr2=0x7f8c800ff520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-13T16:22:54.166 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.164+0000 7f8c85300640 1 -- 172.21.15.111:0/2394854031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c800ff0c0 msgr2=0x7f8c800ff520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T16:22:54.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.164+0000 7f8c85300640 1 --2- 172.21.15.111:0/2394854031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c800ff0c0 0x7f8c800ff520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T16:22:54.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.164+0000 7f8c85300640 1 --2- 172.21.15.111:0/2394854031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c800ff0c0 0x7f8c800ff520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T16:22:54.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.164+0000 7f8c85b01640 1 -- 172.21.15.111:0/2394854031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c800ff0c0 msgr2=0x7f8c800ff520 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T16:22:54.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.164+0000 7f8c85b01640 1 --2- 172.21.15.111:0/2394854031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c800ff0c0 0x7f8c800ff520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:54.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.164+0000 7f8c85b01640 1 -- 172.21.15.111:0/2394854031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c70009a20 con 0x7f8c68003530 2024-09-13T16:22:54.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.164+0000 7f8c85b01640 1 --2- 172.21.15.111:0/2394854031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c68003530 0x7f8c800feb80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8c700092e0 tx=0x7f8c7002d6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:54.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.164+0000 7f8c7effd640 1 -- 172.21.15.111:0/2394854031 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f8c700049e0 con 0x7f8c68003530 2024-09-13T16:22:54.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.165+0000 7f8c7effd640 1 -- 172.21.15.111:0/2394854031 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c7002e470 con 0x7f8c68003530 2024-09-13T16:22:54.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.165+0000 7f8c7effd640 1 -- 172.21.15.111:0/2394854031 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f8c70005220 con 0x7f8c68003530 2024-09-13T16:22:54.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.165+0000 7f8c87d65640 1 -- 172.21.15.111:0/2394854031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c801a1490 con 0x7f8c68003530 2024-09-13T16:22:54.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.165+0000 7f8c87d65640 1 -- 172.21.15.111:0/2394854031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c801a19b0 con 0x7f8c68003530 2024-09-13T16:22:54.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.166+0000 7f8c7effd640 1 -- 172.21.15.111:0/2394854031 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f8c7000a040 con 0x7f8c68003530 2024-09-13T16:22:54.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.166+0000 7f8c7effd640 1 --2- 172.21.15.111:0/2394854031 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8c6c038550 0x7f8c6c03aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:22:54.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.166+0000 7f8c7effd640 1 -- 172.21.15.111:0/2394854031 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8c7006c9f0 con 0x7f8c68003530 2024-09-13T16:22:54.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.166+0000 7f8c87d65640 1 -- 172.21.15.111:0/2394854031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c801010b0 con 0x7f8c68003530 2024-09-13T16:22:54.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.166+0000 7f8c85300640 1 --2- 172.21.15.111:0/2394854031 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8c6c038550 0x7f8c6c03aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:22:54.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.167+0000 7f8c85300640 1 --2- 172.21.15.111:0/2394854031 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8c6c038550 0x7f8c6c03aa10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8c74009100 tx=0x7f8c74008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:22:54.173 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.172+0000 7f8c7effd640 1 -- 172.21.15.111:0/2394854031 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f8c70004c80 con 0x7f8c68003530 2024-09-13T16:22:54.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.404+0000 7f8c87d65640 1 -- 172.21.15.111:0/2394854031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8c80104b90 con 0x7f8c68003530 2024-09-13T16:22:54.405 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:22:54 smithi111 ceph-mon[34600]: Deploying daemon mon.b on smithi107 2024-09-13T16:22:54.405 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:22:54 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T16:22:54.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.405+0000 7f8c7effd640 1 -- 172.21.15.111:0/2394854031 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7f8c70036c90 con 0x7f8c68003530 2024-09-13T16:22:54.406 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T16:22:54.406 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":2,"fsid":"0d241f6a-71ec-11ef-bceb-c7b262605968","modified":"2024-09-13T16:22:48.044749Z","created":"2024-09-13T16:21:13.217034Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:3300","nonce":0},{"type":"v1","addr":"172.21.15.111:6789","nonce":0}]},"addr":"172.21.15.111:6789/0","public_addr":"172.21.15.111:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-13T16:22:54.406 INFO:teuthology.orchestra.run.smithi111.stderr:dumped monmap epoch 2 2024-09-13T16:22:54.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.407+0000 7f8c87d65640 1 -- 172.21.15.111:0/2394854031 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8c6c038550 msgr2=0x7f8c6c03aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:54.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.407+0000 7f8c87d65640 1 --2- 172.21.15.111:0/2394854031 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8c6c038550 0x7f8c6c03aa10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8c74009100 tx=0x7f8c74008040 comp rx=0 tx=0).stop 2024-09-13T16:22:54.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.407+0000 7f8c87d65640 1 -- 172.21.15.111:0/2394854031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c68003530 msgr2=0x7f8c800feb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:22:54.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.407+0000 7f8c87d65640 1 --2- 172.21.15.111:0/2394854031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c68003530 0x7f8c800feb80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8c700092e0 tx=0x7f8c7002d6c0 comp rx=0 tx=0).stop 2024-09-13T16:22:54.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.407+0000 7f8c87d65640 1 -- 172.21.15.111:0/2394854031 shutdown_connections 2024-09-13T16:22:54.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.407+0000 7f8c87d65640 1 --2- 172.21.15.111:0/2394854031 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8c6c038550 0x7f8c6c03aa10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:54.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.407+0000 7f8c87d65640 1 --2- 172.21.15.111:0/2394854031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c800ff0c0 0x7f8c800ff520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:54.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.407+0000 7f8c87d65640 1 --2- 172.21.15.111:0/2394854031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c68003530 0x7f8c800feb80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:22:54.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.407+0000 7f8c87d65640 1 -- 172.21.15.111:0/2394854031 >> 172.21.15.111:0/2394854031 conn(0x7f8c80074620 msgr2=0x7f8c8018dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:22:54.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.407+0000 7f8c87d65640 1 -- 172.21.15.111:0/2394854031 shutdown_connections 2024-09-13T16:22:54.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:22:54.407+0000 7f8c87d65640 1 -- 172.21.15.111:0/2394854031 wait complete. 2024-09-13T16:22:54.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:54 smithi016 ceph-mon[31187]: Deploying daemon mon.b on smithi107 2024-09-13T16:22:54.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:54 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T16:22:55.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:22:55 smithi111 ceph-mon[34600]: from='client.? 172.21.15.111:0/2394854031' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-13T16:22:55.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:22:55 smithi111 ceph-mon[34600]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:22:55.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:55 smithi016 ceph-mon[31187]: from='client.? 172.21.15.111:0/2394854031' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-13T16:22:55.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:55 smithi016 ceph-mon[31187]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:22:56.010 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-13T16:22:56.010 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-13T16:22:56.233 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T16:22:56.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:22:56 smithi016 ceph-mon[31187]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:22:56.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:22:56 smithi111 ceph-mon[34600]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:22:57.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:22:56 smithi107 systemd[1]: Started Ceph mon.b for 0d241f6a-71ec-11ef-bceb-c7b262605968. 2024-09-13T16:23:01.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.986+0000 7f2edcd9a640 1 -- 172.21.15.111:0/2146800272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec0003530 msgr2=0x7f2ec00059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:23:01.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.986+0000 7f2edcd9a640 1 --2- 172.21.15.111:0/2146800272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec0003530 0x7f2ec00059c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2ed80ffc10 tx=0x7f2ec802c490 comp rx=0 tx=0).stop 2024-09-13T16:23:01.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.986+0000 7f2edcd9a640 1 -- 172.21.15.111:0/2146800272 shutdown_connections 2024-09-13T16:23:01.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.986+0000 7f2edcd9a640 1 --2- 172.21.15.111:0/2146800272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec0003530 0x7f2ec00059c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:01.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.986+0000 7f2edcd9a640 1 -- 172.21.15.111:0/2146800272 >> 172.21.15.111:0/2146800272 conn(0x7f2ed80fa440 msgr2=0x7f2ed80fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:23:01.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.986+0000 7f2edcd9a640 1 -- 172.21.15.111:0/2146800272 shutdown_connections 2024-09-13T16:23:01.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.986+0000 7f2edcd9a640 1 -- 172.21.15.111:0/2146800272 wait complete. 2024-09-13T16:23:01.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.987+0000 7f2edcd9a640 1 Processor -- start 2024-09-13T16:23:01.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.987+0000 7f2edcd9a640 1 -- start start 2024-09-13T16:23:01.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.988+0000 7f2edcd9a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec0003530 0x7f2ed8195250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:23:01.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.988+0000 7f2edcd9a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ed8195790 0x7f2ed8199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:23:01.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.988+0000 7f2edcd9a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ed819a080 0x7f2ed819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:23:01.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.988+0000 7f2edcd9a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed8195e60 con 0x7f2ec0003530 2024-09-13T16:23:01.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.988+0000 7f2edcd9a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed81a1df0 con 0x7f2ed819a080 2024-09-13T16:23:01.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.988+0000 7f2edcd9a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed81a1f30 con 0x7f2ed8195790 2024-09-13T16:23:01.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.988+0000 7f2ed5d9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ed8195790 0x7f2ed8199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:23:01.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.988+0000 7f2ed5d9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ed8195790 0x7f2ed8199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44500/0 (socket says 172.21.15.111:44500) 2024-09-13T16:23:01.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.988+0000 7f2ed5d9b640 1 -- 172.21.15.111:0/3654446637 learned_addr learned my addr 172.21.15.111:0/3654446637 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T16:23:01.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.988+0000 7f2ed659c640 1 --2- 172.21.15.111:0/3654446637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec0003530 0x7f2ed8195250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:23:01.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.988+0000 7f2ed6d9d640 1 --2- 172.21.15.111:0/3654446637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ed819a080 0x7f2ed819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:23:01.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.988+0000 7f2ed5d9b640 1 -- 172.21.15.111:0/3654446637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ed8195790 msgr2=0x7f2ed8199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-13T16:23:01.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.988+0000 7f2ed5d9b640 1 -- 172.21.15.111:0/3654446637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ed8195790 msgr2=0x7f2ed8199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T16:23:01.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.988+0000 7f2ed5d9b640 1 --2- 172.21.15.111:0/3654446637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ed8195790 0x7f2ed8199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T16:23:01.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.988+0000 7f2ed5d9b640 1 --2- 172.21.15.111:0/3654446637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ed8195790 0x7f2ed8199b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T16:23:01.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.988+0000 7f2ed6d9d640 1 -- 172.21.15.111:0/3654446637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ed819a080 msgr2=0x7f2ed819a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-13T16:23:01.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.988+0000 7f2ed6d9d640 1 -- 172.21.15.111:0/3654446637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ed819a080 msgr2=0x7f2ed819a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T16:23:01.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.988+0000 7f2ed6d9d640 1 --2- 172.21.15.111:0/3654446637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ed819a080 0x7f2ed819a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T16:23:01.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.988+0000 7f2ed6d9d640 1 --2- 172.21.15.111:0/3654446637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ed819a080 0x7f2ed819a530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T16:23:01.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.988+0000 7f2ed659c640 1 -- 172.21.15.111:0/3654446637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ed8195790 msgr2=0x7f2ed8199b40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T16:23:01.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.988+0000 7f2ed659c640 1 --2- 172.21.15.111:0/3654446637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ed8195790 0x7f2ed8199b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:01.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.988+0000 7f2ed659c640 1 -- 172.21.15.111:0/3654446637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ed819a080 msgr2=0x7f2ed819a530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T16:23:01.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.988+0000 7f2ed659c640 1 --2- 172.21.15.111:0/3654446637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ed819a080 0x7f2ed819a530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:01.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.988+0000 7f2ed659c640 1 -- 172.21.15.111:0/3654446637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec8012880 con 0x7f2ec0003530 2024-09-13T16:23:01.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.989+0000 7f2ed659c640 1 --2- 172.21.15.111:0/3654446637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec0003530 0x7f2ed8195250 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8005f30 tx=0x7f2ec8034410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:23:01.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.989+0000 7f2ec77fe640 1 -- 172.21.15.111:0/3654446637 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8034a10 con 0x7f2ec0003530 2024-09-13T16:23:01.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.989+0000 7f2ec77fe640 1 -- 172.21.15.111:0/3654446637 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2ec803c410 con 0x7f2ec0003530 2024-09-13T16:23:01.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.989+0000 7f2edcd9a640 1 -- 172.21.15.111:0/3654446637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ed81a2180 con 0x7f2ec0003530 2024-09-13T16:23:01.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.989+0000 7f2ec77fe640 1 -- 172.21.15.111:0/3654446637 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec803b5f0 con 0x7f2ec0003530 2024-09-13T16:23:01.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.989+0000 7f2edcd9a640 1 -- 172.21.15.111:0/3654446637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ed81a26a0 con 0x7f2ec0003530 2024-09-13T16:23:01.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.990+0000 7f2ec77fe640 1 -- 172.21.15.111:0/3654446637 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f2ec803a040 con 0x7f2ec0003530 2024-09-13T16:23:01.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.990+0000 7f2edcd9a640 1 -- 172.21.15.111:0/3654446637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ed80ffec0 con 0x7f2ec0003530 2024-09-13T16:23:01.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.990+0000 7f2ec77fe640 1 --2- 172.21.15.111:0/3654446637 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f2eb8038b90 0x7f2eb803b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:23:01.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.990+0000 7f2ec77fe640 1 -- 172.21.15.111:0/3654446637 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2ec8000e40 con 0x7f2ec0003530 2024-09-13T16:23:01.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.991+0000 7f2ed5d9b640 1 --2- 172.21.15.111:0/3654446637 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f2eb8038b90 0x7f2eb803b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:23:01.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.991+0000 7f2ed5d9b640 1 --2- 172.21.15.111:0/3654446637 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f2eb8038b90 0x7f2eb803b050 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009100 tx=0x7f2ecc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:23:02.000 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:01.999+0000 7f2ec77fe640 1 -- 172.21.15.111:0/3654446637 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f2ec804cd30 con 0x7f2ec0003530 2024-09-13T16:23:02.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:01 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T16:23:02.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:01 smithi111 ceph-mon[34600]: mon.a calling monitor election 2024-09-13T16:23:02.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:01 smithi111 ceph-mon[34600]: mon.c calling monitor election 2024-09-13T16:23:02.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:01 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T16:23:02.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:01 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T16:23:02.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:01 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T16:23:02.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:01 smithi111 ceph-mon[34600]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:02.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:01 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T16:23:02.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:01 smithi111 ceph-mon[34600]: mon.b calling monitor election 2024-09-13T16:23:02.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:01 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T16:23:02.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:01 smithi111 ceph-mon[34600]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:02.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:01 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T16:23:02.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:01 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T16:23:02.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:01 smithi111 ceph-mon[34600]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-13T16:23:02.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:01 smithi111 ceph-mon[34600]: monmap e3: 3 mons at {a=[v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0],b=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],c=[v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-13T16:23:02.189 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:01 smithi111 ceph-mon[34600]: fsmap 2024-09-13T16:23:02.189 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:01 smithi111 ceph-mon[34600]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T16:23:02.189 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:01 smithi111 ceph-mon[34600]: mgrmap e13: x(active, since 57s) 2024-09-13T16:23:02.189 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:01 smithi111 ceph-mon[34600]: overall HEALTH_OK 2024-09-13T16:23:02.189 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:01 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:02.189 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:01 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:02.189 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:01 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:02.189 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:01 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:02.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:02.246+0000 7f2edcd9a640 1 -- 172.21.15.111:0/3654446637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2ed81063b0 con 0x7f2ec0003530 2024-09-13T16:23:02.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:02.246+0000 7f2ec77fe640 1 -- 172.21.15.111:0/3654446637 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1276 (secure 0 0 0) 0x7f2ec8040070 con 0x7f2ec0003530 2024-09-13T16:23:02.248 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-13T16:23:02.248 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":3,"fsid":"0d241f6a-71ec-11ef-bceb-c7b262605968","modified":"2024-09-13T16:22:56.796110Z","created":"2024-09-13T16:21:13.217034Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:3300","nonce":0},{"type":"v1","addr":"172.21.15.111:6789","nonce":0}]},"addr":"172.21.15.111:6789/0","public_addr":"172.21.15.111:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-13T16:23:02.248 INFO:teuthology.orchestra.run.smithi111.stderr:dumped monmap epoch 3 2024-09-13T16:23:02.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:02.248+0000 7f2edcd9a640 1 -- 172.21.15.111:0/3654446637 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f2eb8038b90 msgr2=0x7f2eb803b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:23:02.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:02.248+0000 7f2edcd9a640 1 --2- 172.21.15.111:0/3654446637 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f2eb8038b90 0x7f2eb803b050 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009100 tx=0x7f2ecc008040 comp rx=0 tx=0).stop 2024-09-13T16:23:02.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:02.248+0000 7f2edcd9a640 1 -- 172.21.15.111:0/3654446637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec0003530 msgr2=0x7f2ed8195250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:23:02.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:02.248+0000 7f2edcd9a640 1 --2- 172.21.15.111:0/3654446637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec0003530 0x7f2ed8195250 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8005f30 tx=0x7f2ec8034410 comp rx=0 tx=0).stop 2024-09-13T16:23:02.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:02.248+0000 7f2edcd9a640 1 -- 172.21.15.111:0/3654446637 shutdown_connections 2024-09-13T16:23:02.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:02.248+0000 7f2edcd9a640 1 --2- 172.21.15.111:0/3654446637 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f2eb8038b90 0x7f2eb803b050 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:02.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:02.248+0000 7f2edcd9a640 1 --2- 172.21.15.111:0/3654446637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ed819a080 0x7f2ed819a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:02.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:02.248+0000 7f2edcd9a640 1 --2- 172.21.15.111:0/3654446637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ed8195790 0x7f2ed8199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:02.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:02.248+0000 7f2edcd9a640 1 --2- 172.21.15.111:0/3654446637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec0003530 0x7f2ed8195250 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:02.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:02.248+0000 7f2edcd9a640 1 -- 172.21.15.111:0/3654446637 >> 172.21.15.111:0/3654446637 conn(0x7f2ed80fa440 msgr2=0x7f2ed80683c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:23:02.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:02.248+0000 7f2edcd9a640 1 -- 172.21.15.111:0/3654446637 shutdown_connections 2024-09-13T16:23:02.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:02.248+0000 7f2edcd9a640 1 -- 172.21.15.111:0/3654446637 wait complete. 2024-09-13T16:23:02.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:01 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T16:23:02.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:01 smithi107 ceph-mon[34139]: mon.a calling monitor election 2024-09-13T16:23:02.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:01 smithi107 ceph-mon[34139]: mon.c calling monitor election 2024-09-13T16:23:02.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:01 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T16:23:02.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:01 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T16:23:02.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:01 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T16:23:02.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:01 smithi107 ceph-mon[34139]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:02.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:01 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T16:23:02.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:01 smithi107 ceph-mon[34139]: mon.b calling monitor election 2024-09-13T16:23:02.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:01 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T16:23:02.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:01 smithi107 ceph-mon[34139]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:02.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:01 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T16:23:02.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:01 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T16:23:02.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:01 smithi107 ceph-mon[34139]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-13T16:23:02.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:01 smithi107 ceph-mon[34139]: monmap e3: 3 mons at {a=[v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0],b=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],c=[v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-13T16:23:02.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:01 smithi107 ceph-mon[34139]: fsmap 2024-09-13T16:23:02.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:01 smithi107 ceph-mon[34139]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T16:23:02.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:01 smithi107 ceph-mon[34139]: mgrmap e13: x(active, since 57s) 2024-09-13T16:23:02.351 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:01 smithi107 ceph-mon[34139]: overall HEALTH_OK 2024-09-13T16:23:02.351 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:01 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:02.351 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:01 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:02.351 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:01 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:02.351 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:01 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:02.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:01 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T16:23:02.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:01 smithi016 ceph-mon[31187]: mon.a calling monitor election 2024-09-13T16:23:02.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:01 smithi016 ceph-mon[31187]: mon.c calling monitor election 2024-09-13T16:23:02.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:01 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T16:23:02.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:01 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T16:23:02.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:01 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T16:23:02.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:01 smithi016 ceph-mon[31187]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:02.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:01 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T16:23:02.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:01 smithi016 ceph-mon[31187]: mon.b calling monitor election 2024-09-13T16:23:02.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:01 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T16:23:02.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:01 smithi016 ceph-mon[31187]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:02.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:01 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T16:23:02.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:01 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T16:23:02.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:01 smithi016 ceph-mon[31187]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-13T16:23:02.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:01 smithi016 ceph-mon[31187]: monmap e3: 3 mons at {a=[v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0],b=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],c=[v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-13T16:23:02.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:01 smithi016 ceph-mon[31187]: fsmap 2024-09-13T16:23:02.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:01 smithi016 ceph-mon[31187]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T16:23:02.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:01 smithi016 ceph-mon[31187]: mgrmap e13: x(active, since 57s) 2024-09-13T16:23:02.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:01 smithi016 ceph-mon[31187]: overall HEALTH_OK 2024-09-13T16:23:02.356 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:01 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:02.356 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:01 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:02.356 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:01 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:02.356 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:01 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:02.895 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-13T16:23:02.895 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-13T16:23:03.111 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:23:03.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:02 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:23:03.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:02 smithi111 ceph-mon[34600]: from='client.? 172.21.15.111:0/3654446637' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-13T16:23:03.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:02 smithi111 ceph-mon[34600]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:03.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:02 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T16:23:03.233 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:02 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:23:03.233 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:02 smithi016 ceph-mon[31187]: from='client.? 172.21.15.111:0/3654446637' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-13T16:23:03.233 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:02 smithi016 ceph-mon[31187]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:03.233 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:02 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T16:23:03.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:02 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:23:03.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:02 smithi107 ceph-mon[34139]: from='client.? 172.21.15.111:0/3654446637' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-13T16:23:03.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:02 smithi107 ceph-mon[34139]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:03.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:02 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T16:23:04.284 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:03 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:04.285 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:03 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:04.285 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:03 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:04.285 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:03 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:04.285 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:03 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:04.285 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:03 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:23:04.285 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:03 smithi016 ceph-mon[31187]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-13T16:23:04.285 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:03 smithi016 ceph-mon[31187]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-13T16:23:04.285 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:03 smithi016 ceph-mon[31187]: Updating smithi111:/etc/ceph/ceph.conf 2024-09-13T16:23:04.285 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:03 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:04.285 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:03 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:04.285 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:03 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:04.285 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:03 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:04.286 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:03 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:04.286 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:03 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:04.286 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:03 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:04.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:03 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:04.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:03 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:04.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:03 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:04.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:03 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:04.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:03 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:04.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:03 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:23:04.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:03 smithi107 ceph-mon[34139]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-13T16:23:04.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:03 smithi107 ceph-mon[34139]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-13T16:23:04.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:03 smithi107 ceph-mon[34139]: Updating smithi111:/etc/ceph/ceph.conf 2024-09-13T16:23:04.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:03 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:04.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:03 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:04.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:03 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:04.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:03 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:04.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:03 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:04.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:03 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:04.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:03 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:04.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:03 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:04.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:03 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:04.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:03 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:04.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:03 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:04.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:03 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:04.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:03 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:23:04.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:03 smithi111 ceph-mon[34600]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-13T16:23:04.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:03 smithi111 ceph-mon[34600]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-13T16:23:04.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:03 smithi111 ceph-mon[34600]: Updating smithi111:/etc/ceph/ceph.conf 2024-09-13T16:23:04.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:03 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:04.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:03 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:04.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:03 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:04.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:03 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:04.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:03 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:04.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:03 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:04.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:03 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:05.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:05 smithi107 ceph-mon[34139]: Updating smithi111:/var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T16:23:05.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:05 smithi107 ceph-mon[34139]: Updating smithi107:/var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T16:23:05.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:05 smithi107 ceph-mon[34139]: Updating smithi016:/var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T16:23:05.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:05 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:05.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:05 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:05.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:05 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:05.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:05 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:05.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:05 smithi107 ceph-mon[34139]: Reconfiguring mon.a (unknown last config time)... 2024-09-13T16:23:05.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:05 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-13T16:23:05.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:05 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-13T16:23:05.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:05 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:05.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:05 smithi107 ceph-mon[34139]: Reconfiguring daemon mon.a on smithi016 2024-09-13T16:23:05.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:05 smithi107 ceph-mon[34139]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:05.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:05 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:05.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:05 smithi016 ceph-mon[31187]: Updating smithi111:/var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T16:23:05.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:05 smithi016 ceph-mon[31187]: Updating smithi107:/var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T16:23:05.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:05 smithi016 ceph-mon[31187]: Updating smithi016:/var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T16:23:05.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:05 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:05.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:05 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:05.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:05 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:05.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:05 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:05.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:05 smithi016 ceph-mon[31187]: Reconfiguring mon.a (unknown last config time)... 2024-09-13T16:23:05.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:05 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-13T16:23:05.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:05 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-13T16:23:05.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:05 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:05.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:05 smithi016 ceph-mon[31187]: Reconfiguring daemon mon.a on smithi016 2024-09-13T16:23:05.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:05 smithi016 ceph-mon[31187]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:05.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:05 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:05.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:05 smithi111 ceph-mon[34600]: Updating smithi111:/var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T16:23:05.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:05 smithi111 ceph-mon[34600]: Updating smithi107:/var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T16:23:05.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:05 smithi111 ceph-mon[34600]: Updating smithi016:/var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T16:23:05.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:05 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:05.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:05 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:05.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:05 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:05.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:05 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:05.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:05 smithi111 ceph-mon[34600]: Reconfiguring mon.a (unknown last config time)... 2024-09-13T16:23:05.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:05 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-13T16:23:05.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:05 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-13T16:23:05.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:05 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:05.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:05 smithi111 ceph-mon[34600]: Reconfiguring daemon mon.a on smithi016 2024-09-13T16:23:05.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:05 smithi111 ceph-mon[34600]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:05.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:05 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:05.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.460+0000 7fd3028ce640 1 -- 172.21.15.16:0/2033056174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2fc074390 msgr2=0x7fd2fc0726e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:23:05.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.460+0000 7fd3028ce640 1 --2- 172.21.15.16:0/2033056174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2fc074390 0x7fd2fc0726e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec009880 tx=0x7fd2ec02a5d0 comp rx=0 tx=0).stop 2024-09-13T16:23:05.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.460+0000 7fd3028ce640 1 -- 172.21.15.16:0/2033056174 shutdown_connections 2024-09-13T16:23:05.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.460+0000 7fd3028ce640 1 --2- 172.21.15.16:0/2033056174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2fc074390 0x7fd2fc0726e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:05.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.460+0000 7fd3028ce640 1 -- 172.21.15.16:0/2033056174 >> 172.21.15.16:0/2033056174 conn(0x7fd2fc076500 msgr2=0x7fd2fc076910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:23:05.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.460+0000 7fd3028ce640 1 -- 172.21.15.16:0/2033056174 shutdown_connections 2024-09-13T16:23:05.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.460+0000 7fd3028ce640 1 -- 172.21.15.16:0/2033056174 wait complete. 2024-09-13T16:23:05.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.461+0000 7fd3028ce640 1 Processor -- start 2024-09-13T16:23:05.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.461+0000 7fd3028ce640 1 -- start start 2024-09-13T16:23:05.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.461+0000 7fd3028ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2fc074390 0x7fd2fc19b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:23:05.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.461+0000 7fd3028ce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2fc19bc30 0x7fd2fc19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:23:05.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.461+0000 7fd3028ce640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2fc19f4c0 0x7fd2fc19f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:23:05.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.461+0000 7fd3028ce640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc19c2b0 con 0x7fd2fc074390 2024-09-13T16:23:05.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.461+0000 7fd3028ce640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc1a8310 con 0x7fd2fc19f4c0 2024-09-13T16:23:05.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.461+0000 7fd3028ce640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc1a8450 con 0x7fd2fc19bc30 2024-09-13T16:23:05.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.461+0000 7fd2fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2fc074390 0x7fd2fc19b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:23:05.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.461+0000 7fd2fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2fc074390 0x7fd2fc19b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49770/0 (socket says 172.21.15.16:49770) 2024-09-13T16:23:05.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.461+0000 7fd2fbfff640 1 -- 172.21.15.16:0/4047278453 learned_addr learned my addr 172.21.15.16:0/4047278453 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:23:05.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.462+0000 7fd2f3fff640 1 --2- 172.21.15.16:0/4047278453 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2fc19bc30 0x7fd2fc19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:23:05.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.462+0000 7fd300e6b640 1 --2- 172.21.15.16:0/4047278453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2fc19f4c0 0x7fd2fc19f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:23:05.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.462+0000 7fd2fbfff640 1 -- 172.21.15.16:0/4047278453 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2fc19bc30 msgr2=0x7fd2fc19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:23:05.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.462+0000 7fd2fbfff640 1 --2- 172.21.15.16:0/4047278453 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2fc19bc30 0x7fd2fc19ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:05.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.462+0000 7fd2fbfff640 1 -- 172.21.15.16:0/4047278453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2fc19f4c0 msgr2=0x7fd2fc19f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:23:05.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.462+0000 7fd2fbfff640 1 --2- 172.21.15.16:0/4047278453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2fc19f4c0 0x7fd2fc19f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:05.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.462+0000 7fd2fbfff640 1 -- 172.21.15.16:0/4047278453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ec009560 con 0x7fd2fc074390 2024-09-13T16:23:05.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.462+0000 7fd300e6b640 1 --2- 172.21.15.16:0/4047278453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2fc19f4c0 0x7fd2fc19f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:23:05.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.462+0000 7fd2fbfff640 1 --2- 172.21.15.16:0/4047278453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2fc074390 0x7fd2fc19b6f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec032c20 tx=0x7fd2ec032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:23:05.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.462+0000 7fd2f9ffb640 1 -- 172.21.15.16:0/4047278453 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec005210 con 0x7fd2fc074390 2024-09-13T16:23:05.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.462+0000 7fd2f9ffb640 1 -- 172.21.15.16:0/4047278453 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd2ec005370 con 0x7fd2fc074390 2024-09-13T16:23:05.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.462+0000 7fd3028ce640 1 -- 172.21.15.16:0/4047278453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2fc1a8650 con 0x7fd2fc074390 2024-09-13T16:23:05.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.462+0000 7fd2f9ffb640 1 -- 172.21.15.16:0/4047278453 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec003970 con 0x7fd2fc074390 2024-09-13T16:23:05.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.462+0000 7fd3028ce640 1 -- 172.21.15.16:0/4047278453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2fc1a8aa0 con 0x7fd2fc074390 2024-09-13T16:23:05.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.463+0000 7fd2f9ffb640 1 -- 172.21.15.16:0/4047278453 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fd2ec003ad0 con 0x7fd2fc074390 2024-09-13T16:23:05.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.463+0000 7fd2f9ffb640 1 --2- 172.21.15.16:0/4047278453 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd2e8038b90 0x7fd2e803b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:23:05.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.463+0000 7fd2f9ffb640 1 -- 172.21.15.16:0/4047278453 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd2ec06c8b0 con 0x7fd2fc074390 2024-09-13T16:23:05.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.463+0000 7fd3028ce640 1 -- 172.21.15.16:0/4047278453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2fc073c40 con 0x7fd2fc074390 2024-09-13T16:23:05.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.463+0000 7fd2f3fff640 1 --2- 172.21.15.16:0/4047278453 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd2e8038b90 0x7fd2e803b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:23:05.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.464+0000 7fd2f3fff640 1 --2- 172.21.15.16:0/4047278453 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd2e8038b90 0x7fd2e803b050 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4006fd0 tx=0x7fd2e4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:23:05.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.468+0000 7fd2f9ffb640 1 -- 172.21.15.16:0/4047278453 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd2ec040020 con 0x7fd2fc074390 2024-09-13T16:23:05.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.640+0000 7fd3028ce640 1 -- 172.21.15.16:0/4047278453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fd2fc110060 con 0x7fd2fc074390 2024-09-13T16:23:05.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.641+0000 7fd2f9ffb640 1 -- 172.21.15.16:0/4047278453 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7fd2ec035070 con 0x7fd2fc074390 2024-09-13T16:23:05.642 INFO:teuthology.orchestra.run.smithi016.stdout:# minimal ceph.conf for 0d241f6a-71ec-11ef-bceb-c7b262605968 2024-09-13T16:23:05.642 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2024-09-13T16:23:05.642 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = 0d241f6a-71ec-11ef-bceb-c7b262605968 2024-09-13T16:23:05.642 INFO:teuthology.orchestra.run.smithi016.stdout: mon_host = [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] 2024-09-13T16:23:05.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.643+0000 7fd3028ce640 1 -- 172.21.15.16:0/4047278453 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd2e8038b90 msgr2=0x7fd2e803b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:23:05.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.643+0000 7fd3028ce640 1 --2- 172.21.15.16:0/4047278453 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd2e8038b90 0x7fd2e803b050 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4006fd0 tx=0x7fd2e4008040 comp rx=0 tx=0).stop 2024-09-13T16:23:05.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.643+0000 7fd3028ce640 1 -- 172.21.15.16:0/4047278453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2fc074390 msgr2=0x7fd2fc19b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:23:05.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.643+0000 7fd3028ce640 1 --2- 172.21.15.16:0/4047278453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2fc074390 0x7fd2fc19b6f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec032c20 tx=0x7fd2ec032c50 comp rx=0 tx=0).stop 2024-09-13T16:23:05.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.643+0000 7fd3028ce640 1 -- 172.21.15.16:0/4047278453 shutdown_connections 2024-09-13T16:23:05.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.643+0000 7fd3028ce640 1 --2- 172.21.15.16:0/4047278453 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd2e8038b90 0x7fd2e803b050 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:05.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.643+0000 7fd3028ce640 1 --2- 172.21.15.16:0/4047278453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2fc19f4c0 0x7fd2fc19f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:05.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.643+0000 7fd3028ce640 1 --2- 172.21.15.16:0/4047278453 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2fc19bc30 0x7fd2fc19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:05.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.643+0000 7fd3028ce640 1 --2- 172.21.15.16:0/4047278453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2fc074390 0x7fd2fc19b6f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:05.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.643+0000 7fd3028ce640 1 -- 172.21.15.16:0/4047278453 >> 172.21.15.16:0/4047278453 conn(0x7fd2fc076500 msgr2=0x7fd2fc0fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:23:05.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.643+0000 7fd3028ce640 1 -- 172.21.15.16:0/4047278453 shutdown_connections 2024-09-13T16:23:05.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:05.643+0000 7fd3028ce640 1 -- 172.21.15.16:0/4047278453 wait complete. 2024-09-13T16:23:06.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:06 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/4047278453' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:06.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:06 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:06.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:06 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:06.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:06 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-13T16:23:06.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:06 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-13T16:23:06.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:06 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:06.351 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:06 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/4047278453' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:06.351 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:06 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:06.352 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:06 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:06.352 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:06 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-13T16:23:06.352 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:06 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-13T16:23:06.352 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:06 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:06.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:06 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/4047278453' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:06.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:06 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:06.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:06 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:06.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:06 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-13T16:23:06.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:06 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-13T16:23:06.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:06 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:06.643 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-13T16:23:06.643 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-13T16:23:06.643 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.conf 2024-09-13T16:23:06.675 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-13T16:23:06.675 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-13T16:23:06.741 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-13T16:23:06.742 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.conf 2024-09-13T16:23:06.772 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-13T16:23:06.772 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-13T16:23:06.837 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T16:23:06.837 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/ceph/ceph.conf 2024-09-13T16:23:06.870 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T16:23:06.870 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-13T16:23:06.939 INFO:tasks.cephadm:Adding mgr.x on smithi016 2024-09-13T16:23:06.939 INFO:tasks.cephadm:Adding mgr.y on smithi107 2024-09-13T16:23:06.939 INFO:tasks.cephadm:Adding mgr.z on smithi111 2024-09-13T16:23:06.939 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi016=x;smithi107=y;smithi111=z' 2024-09-13T16:23:07.176 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T16:23:07.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:07 smithi111 ceph-mon[34600]: Reconfiguring mon.b (monmap changed)... 2024-09-13T16:23:07.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:07 smithi111 ceph-mon[34600]: Reconfiguring daemon mon.b on smithi107 2024-09-13T16:23:07.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:07 smithi111 ceph-mon[34600]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:07.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:07 smithi107 ceph-mon[34139]: Reconfiguring mon.b (monmap changed)... 2024-09-13T16:23:07.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:07 smithi107 ceph-mon[34139]: Reconfiguring daemon mon.b on smithi107 2024-09-13T16:23:07.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:07 smithi107 ceph-mon[34139]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:07.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:07 smithi016 ceph-mon[31187]: Reconfiguring mon.b (monmap changed)... 2024-09-13T16:23:07.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:07 smithi016 ceph-mon[31187]: Reconfiguring daemon mon.b on smithi107 2024-09-13T16:23:07.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:07 smithi016 ceph-mon[31187]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:08.261 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:08 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:08.261 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:08 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:08.261 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:08 smithi111 ceph-mon[34600]: Reconfiguring mon.c (monmap changed)... 2024-09-13T16:23:08.261 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:08 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-13T16:23:08.262 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:08 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-13T16:23:08.262 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:08 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:08.262 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:08 smithi111 ceph-mon[34600]: Reconfiguring daemon mon.c on smithi111 2024-09-13T16:23:08.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:08 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:08.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:08 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:08.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:08 smithi016 ceph-mon[31187]: Reconfiguring mon.c (monmap changed)... 2024-09-13T16:23:08.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:08 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-13T16:23:08.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:08 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-13T16:23:08.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:08 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:08.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:08 smithi016 ceph-mon[31187]: Reconfiguring daemon mon.c on smithi111 2024-09-13T16:23:08.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:08 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:08.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:08 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:08.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:08 smithi107 ceph-mon[34139]: Reconfiguring mon.c (monmap changed)... 2024-09-13T16:23:08.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:08 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-13T16:23:08.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:08 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-13T16:23:08.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:08 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:08.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:08 smithi107 ceph-mon[34139]: Reconfiguring daemon mon.c on smithi111 2024-09-13T16:23:09.184 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:09 smithi111 ceph-mon[34600]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:09.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:09 smithi107 ceph-mon[34139]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:09.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:09 smithi016 ceph-mon[31187]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:09.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.632+0000 7fac53c6d640 1 -- 172.21.15.111:0/2091484687 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac4c1008a0 msgr2=0x7fac4c107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:23:09.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.632+0000 7fac53c6d640 1 --2- 172.21.15.111:0/2091484687 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac4c1008a0 0x7fac4c107130 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fac44009ca0 tx=0x7fac4402a8c0 comp rx=0 tx=0).stop 2024-09-13T16:23:09.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.632+0000 7fac53c6d640 1 -- 172.21.15.111:0/2091484687 shutdown_connections 2024-09-13T16:23:09.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.632+0000 7fac53c6d640 1 --2- 172.21.15.111:0/2091484687 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac4c1008a0 0x7fac4c107130 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:09.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.632+0000 7fac53c6d640 1 --2- 172.21.15.111:0/2091484687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac4c0ffee0 0x7fac4c100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:09.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.632+0000 7fac53c6d640 1 --2- 172.21.15.111:0/2091484687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac4c0fece0 0x7fac4c0ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:09.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.632+0000 7fac53c6d640 1 -- 172.21.15.111:0/2091484687 >> 172.21.15.111:0/2091484687 conn(0x7fac4c0fa490 msgr2=0x7fac4c0fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:23:09.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.632+0000 7fac53c6d640 1 -- 172.21.15.111:0/2091484687 shutdown_connections 2024-09-13T16:23:09.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.632+0000 7fac53c6d640 1 -- 172.21.15.111:0/2091484687 wait complete. 2024-09-13T16:23:09.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.633+0000 7fac53c6d640 1 Processor -- start 2024-09-13T16:23:09.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.633+0000 7fac53c6d640 1 -- start start 2024-09-13T16:23:09.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.633+0000 7fac53c6d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac4c0fece0 0x7fac4c102c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:23:09.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.633+0000 7fac53c6d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac4c0ffee0 0x7fac4c1012c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:23:09.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.633+0000 7fac53c6d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac4c1008a0 0x7fac4c101800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:23:09.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.633+0000 7fac53c6d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c101f00 con 0x7fac4c1008a0 2024-09-13T16:23:09.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.633+0000 7fac53c6d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c102070 con 0x7fac4c0ffee0 2024-09-13T16:23:09.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.633+0000 7fac53c6d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c19fd20 con 0x7fac4c0fece0 2024-09-13T16:23:09.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.633+0000 7fac51a09640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac4c0fece0 0x7fac4c102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:23:09.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.633+0000 7fac51a09640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac4c0fece0 0x7fac4c102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40654/0 (socket says 172.21.15.111:40654) 2024-09-13T16:23:09.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.633+0000 7fac51a09640 1 -- 172.21.15.111:0/114836200 learned_addr learned my addr 172.21.15.111:0/114836200 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T16:23:09.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.633+0000 7fac5220a640 1 --2- 172.21.15.111:0/114836200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac4c1008a0 0x7fac4c101800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:23:09.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.633+0000 7fac51208640 1 --2- 172.21.15.111:0/114836200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac4c0ffee0 0x7fac4c1012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:23:09.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.633+0000 7fac51a09640 1 -- 172.21.15.111:0/114836200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac4c0ffee0 msgr2=0x7fac4c1012c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:23:09.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.633+0000 7fac51a09640 1 --2- 172.21.15.111:0/114836200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac4c0ffee0 0x7fac4c1012c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:09.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.633+0000 7fac51a09640 1 -- 172.21.15.111:0/114836200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac4c1008a0 msgr2=0x7fac4c101800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:23:09.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.633+0000 7fac51a09640 1 --2- 172.21.15.111:0/114836200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac4c1008a0 0x7fac4c101800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:09.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.633+0000 7fac51a09640 1 -- 172.21.15.111:0/114836200 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac44009900 con 0x7fac4c0fece0 2024-09-13T16:23:09.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.633+0000 7fac51a09640 1 --2- 172.21.15.111:0/114836200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac4c0fece0 0x7fac4c102c20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fac4000d950 tx=0x7fac4000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:23:09.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.634+0000 7fac3effd640 1 -- 172.21.15.111:0/114836200 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac40009c30 con 0x7fac4c0fece0 2024-09-13T16:23:09.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.634+0000 7fac3effd640 1 -- 172.21.15.111:0/114836200 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fac40009d90 con 0x7fac4c0fece0 2024-09-13T16:23:09.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.634+0000 7fac53c6d640 1 -- 172.21.15.111:0/114836200 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac4c19ff80 con 0x7fac4c0fece0 2024-09-13T16:23:09.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.634+0000 7fac3effd640 1 -- 172.21.15.111:0/114836200 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4000f620 con 0x7fac4c0fece0 2024-09-13T16:23:09.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.634+0000 7fac53c6d640 1 -- 172.21.15.111:0/114836200 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac4c1a0390 con 0x7fac4c0fece0 2024-09-13T16:23:09.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.635+0000 7fac53c6d640 1 -- 172.21.15.111:0/114836200 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac4c10a4a0 con 0x7fac4c0fece0 2024-09-13T16:23:09.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.635+0000 7fac3effd640 1 -- 172.21.15.111:0/114836200 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fac4000f780 con 0x7fac4c0fece0 2024-09-13T16:23:09.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.635+0000 7fac3effd640 1 --2- 172.21.15.111:0/114836200 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fac38038930 0x7fac3803adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:23:09.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.635+0000 7fac3effd640 1 -- 172.21.15.111:0/114836200 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fac4004b1f0 con 0x7fac4c0fece0 2024-09-13T16:23:09.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.635+0000 7fac51208640 1 --2- 172.21.15.111:0/114836200 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fac38038930 0x7fac3803adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:23:09.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.636+0000 7fac51208640 1 --2- 172.21.15.111:0/114836200 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fac38038930 0x7fac3803adf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fac48005fd0 tx=0x7fac48005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:23:09.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.640+0000 7fac3effd640 1 -- 172.21.15.111:0/114836200 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fac400238d0 con 0x7fac4c0fece0 2024-09-13T16:23:09.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.816+0000 7fac53c6d640 1 -- 172.21.15.111:0/114836200 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi107=y;smithi111=z", "target": ["mon-mgr", ""]}) v1 -- 0x7fac4c068640 con 0x7fac38038930 2024-09-13T16:23:09.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.822+0000 7fac3effd640 1 -- 172.21.15.111:0/114836200 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fac4c068640 con 0x7fac38038930 2024-09-13T16:23:09.824 INFO:teuthology.orchestra.run.smithi111.stdout:Scheduled mgr update... 2024-09-13T16:23:09.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.824+0000 7fac53c6d640 1 -- 172.21.15.111:0/114836200 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fac38038930 msgr2=0x7fac3803adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:23:09.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.824+0000 7fac53c6d640 1 --2- 172.21.15.111:0/114836200 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fac38038930 0x7fac3803adf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fac48005fd0 tx=0x7fac48005850 comp rx=0 tx=0).stop 2024-09-13T16:23:09.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.824+0000 7fac53c6d640 1 -- 172.21.15.111:0/114836200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac4c0fece0 msgr2=0x7fac4c102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:23:09.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.824+0000 7fac53c6d640 1 --2- 172.21.15.111:0/114836200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac4c0fece0 0x7fac4c102c20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fac4000d950 tx=0x7fac4000de10 comp rx=0 tx=0).stop 2024-09-13T16:23:09.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.824+0000 7fac53c6d640 1 -- 172.21.15.111:0/114836200 shutdown_connections 2024-09-13T16:23:09.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.824+0000 7fac53c6d640 1 --2- 172.21.15.111:0/114836200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac4c1008a0 0x7fac4c101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:09.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.824+0000 7fac53c6d640 1 --2- 172.21.15.111:0/114836200 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fac38038930 0x7fac3803adf0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:09.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.824+0000 7fac53c6d640 1 --2- 172.21.15.111:0/114836200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac4c0ffee0 0x7fac4c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:09.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.824+0000 7fac53c6d640 1 --2- 172.21.15.111:0/114836200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac4c0fece0 0x7fac4c102c20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:09.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.824+0000 7fac53c6d640 1 -- 172.21.15.111:0/114836200 >> 172.21.15.111:0/114836200 conn(0x7fac4c0fa490 msgr2=0x7fac4c067f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:23:09.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.824+0000 7fac53c6d640 1 -- 172.21.15.111:0/114836200 shutdown_connections 2024-09-13T16:23:09.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:23:09.824+0000 7fac53c6d640 1 -- 172.21.15.111:0/114836200 wait complete. 2024-09-13T16:23:10.437 DEBUG:teuthology.orchestra.run.smithi107:mgr.y> sudo journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mgr.y.service 2024-09-13T16:23:10.439 DEBUG:teuthology.orchestra.run.smithi111:mgr.z> sudo journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mgr.z.service 2024-09-13T16:23:10.442 INFO:tasks.cephadm:Deploying OSDs... 2024-09-13T16:23:10.443 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-13T16:23:10.443 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T16:23:10.461 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T16:23:10.462 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_1 2024-09-13T16:23:10.519 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T16:23:10.520 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T16:23:10.520 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-13T16:23:10.520 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T16:23:10.520 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T16:23:10.520 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-13 16:22:16.671772075 +0000 2024-09-13T16:23:10.520 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-13 16:18:11.834878818 +0000 2024-09-13T16:23:10.520 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-13 16:18:11.834878818 +0000 2024-09-13T16:23:10.520 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-13 16:18:11.834878818 +0000 2024-09-13T16:23:10.520 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T16:23:10.586 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:10 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:10.586 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:10 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:10.586 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:10 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:23:10.586 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:10 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:10.586 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:10 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:23:10.586 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:10 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:10.586 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:10 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:10.586 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:10 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:23:10.587 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:10 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:10.587 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:10 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:23:10.587 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:10 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:10.587 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:10 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-13T16:23:10.587 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:10 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-13T16:23:10.587 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:10 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-13T16:23:10.587 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:10 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:10.588 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-13T16:23:10.588 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-13T16:23:10.589 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000131192 s, 3.9 MB/s 2024-09-13T16:23:10.590 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T16:23:10.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:10 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:10.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:10 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:10.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:10 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:23:10.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:10 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:10.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:10 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:23:10.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:10 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:10.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:10 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:10.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:10 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:23:10.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:10 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:10.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:10 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:23:10.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:10 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:10.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:10 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-13T16:23:10.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:10 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-13T16:23:10.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:10 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-13T16:23:10.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:10 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:10.648 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_2 2024-09-13T16:23:10.650 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:10 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:10.650 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:10 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:10.650 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:10 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:23:10.650 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:10 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:10.650 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:10 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:23:10.651 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:10 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:10.651 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:10 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:10.651 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:10 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:23:10.651 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:10 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:10.651 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:10 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:23:10.651 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:10 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:10.651 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:10 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-13T16:23:10.651 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:10 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-13T16:23:10.651 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:10 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-13T16:23:10.651 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:10 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:10.706 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T16:23:10.706 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T16:23:10.706 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-13T16:23:10.706 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T16:23:10.706 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T16:23:10.706 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-13 16:22:16.704771576 +0000 2024-09-13T16:23:10.706 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-13 16:18:12.175873659 +0000 2024-09-13T16:23:10.706 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-13 16:18:12.175873659 +0000 2024-09-13T16:23:10.706 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-13 16:18:12.175873659 +0000 2024-09-13T16:23:10.707 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T16:23:10.772 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-13T16:23:10.772 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-13T16:23:10.772 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000115649 s, 4.4 MB/s 2024-09-13T16:23:10.774 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T16:23:10.831 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_3 2024-09-13T16:23:10.888 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T16:23:10.888 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T16:23:10.889 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-13T16:23:10.889 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T16:23:10.889 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T16:23:10.889 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-13 16:22:16.735771107 +0000 2024-09-13T16:23:10.889 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-13 16:18:12.497868787 +0000 2024-09-13T16:23:10.889 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-13 16:18:12.497868787 +0000 2024-09-13T16:23:10.889 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-13 16:18:12.497868787 +0000 2024-09-13T16:23:10.889 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T16:23:10.954 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-13T16:23:10.954 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-13T16:23:10.954 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000117054 s, 4.4 MB/s 2024-09-13T16:23:10.956 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T16:23:11.013 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_4 2024-09-13T16:23:11.070 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T16:23:11.070 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T16:23:11.071 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-13T16:23:11.071 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T16:23:11.071 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T16:23:11.071 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-13 16:22:16.775770503 +0000 2024-09-13T16:23:11.071 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-13 16:18:12.812864021 +0000 2024-09-13T16:23:11.071 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-13 16:18:12.812864021 +0000 2024-09-13T16:23:11.072 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-13 16:18:12.812864021 +0000 2024-09-13T16:23:11.072 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T16:23:11.135 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-13T16:23:11.135 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-13T16:23:11.135 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000112771 s, 4.5 MB/s 2024-09-13T16:23:11.137 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T16:23:11.193 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-13T16:23:11.194 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T16:23:11.213 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T16:23:11.213 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_1 2024-09-13T16:23:11.272 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T16:23:11.273 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T16:23:11.273 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-13T16:23:11.273 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T16:23:11.273 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T16:23:11.273 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-13 16:22:56.189426618 +0000 2024-09-13T16:23:11.274 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-13 16:18:06.878138919 +0000 2024-09-13T16:23:11.274 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-13 16:18:06.878138919 +0000 2024-09-13T16:23:11.274 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-09-13 16:18:06.878138919 +0000 2024-09-13T16:23:11.274 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T16:23:11.340 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-09-13T16:23:11.341 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-09-13T16:23:11.341 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000232886 s, 2.2 MB/s 2024-09-13T16:23:11.342 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T16:23:11.400 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_2 2024-09-13T16:23:11.457 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T16:23:11.458 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T16:23:11.458 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 688 Links: 1 2024-09-13T16:23:11.458 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T16:23:11.458 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T16:23:11.458 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-13 16:22:56.189426618 +0000 2024-09-13T16:23:11.458 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-13 16:18:07.213133368 +0000 2024-09-13T16:23:11.458 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-13 16:18:07.213133368 +0000 2024-09-13T16:23:11.458 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-09-13 16:18:07.213133368 +0000 2024-09-13T16:23:11.458 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T16:23:11.520 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:11 smithi107 ceph-mon[34139]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi107=y;smithi111=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:23:11.521 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:11 smithi107 ceph-mon[34139]: Saving service mgr spec with placement smithi016=x;smithi107=y;smithi111=z;count:3 2024-09-13T16:23:11.521 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:11 smithi107 ceph-mon[34139]: Deploying daemon mgr.z on smithi111 2024-09-13T16:23:11.521 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:11 smithi107 ceph-mon[34139]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:11.523 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-09-13T16:23:11.523 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-09-13T16:23:11.523 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000259873 s, 2.0 MB/s 2024-09-13T16:23:11.525 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T16:23:11.530 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:11 smithi111 ceph-mon[34600]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi107=y;smithi111=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:23:11.530 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:11 smithi111 ceph-mon[34600]: Saving service mgr spec with placement smithi016=x;smithi107=y;smithi111=z;count:3 2024-09-13T16:23:11.530 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:11 smithi111 ceph-mon[34600]: Deploying daemon mgr.z on smithi111 2024-09-13T16:23:11.530 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:11 smithi111 ceph-mon[34600]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:11.582 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_3 2024-09-13T16:23:11.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:11 smithi016 ceph-mon[31187]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi107=y;smithi111=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:23:11.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:11 smithi016 ceph-mon[31187]: Saving service mgr spec with placement smithi016=x;smithi107=y;smithi111=z;count:3 2024-09-13T16:23:11.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:11 smithi016 ceph-mon[31187]: Deploying daemon mgr.z on smithi111 2024-09-13T16:23:11.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:11 smithi016 ceph-mon[31187]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:11.638 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T16:23:11.638 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T16:23:11.639 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-13T16:23:11.639 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T16:23:11.639 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T16:23:11.639 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-13 16:22:56.189426618 +0000 2024-09-13T16:23:11.639 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-13 16:18:07.542127917 +0000 2024-09-13T16:23:11.639 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-13 16:18:07.542127917 +0000 2024-09-13T16:23:11.640 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-09-13 16:18:07.542127917 +0000 2024-09-13T16:23:11.640 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T16:23:11.703 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-09-13T16:23:11.703 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-09-13T16:23:11.703 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000222298 s, 2.3 MB/s 2024-09-13T16:23:11.705 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T16:23:11.762 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_4 2024-09-13T16:23:11.818 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T16:23:11.818 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T16:23:11.818 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 726 Links: 1 2024-09-13T16:23:11.819 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T16:23:11.819 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T16:23:11.819 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-13 16:22:56.190426602 +0000 2024-09-13T16:23:11.819 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-13 16:18:07.869122499 +0000 2024-09-13T16:23:11.819 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-13 16:18:07.869122499 +0000 2024-09-13T16:23:11.819 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-09-13 16:18:07.869122499 +0000 2024-09-13T16:23:11.820 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T16:23:11.882 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-09-13T16:23:11.882 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-09-13T16:23:11.882 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000216538 s, 2.4 MB/s 2024-09-13T16:23:11.883 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T16:23:11.939 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-13T16:23:11.940 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T16:23:11.956 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T16:23:11.956 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_1 2024-09-13T16:23:12.013 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T16:23:12.013 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T16:23:12.013 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-13T16:23:12.013 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T16:23:12.013 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T16:23:12.013 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-09-13 16:23:11.892944820 +0000 2024-09-13T16:23:12.013 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-09-13 16:18:06.882741177 +0000 2024-09-13T16:23:12.013 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-09-13 16:18:06.882741177 +0000 2024-09-13T16:23:12.013 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-09-13 16:18:06.882741177 +0000 2024-09-13T16:23:12.014 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T16:23:12.076 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:11 smithi111 systemd[1]: Starting Ceph mgr.z for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:23:12.079 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-09-13T16:23:12.079 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-09-13T16:23:12.079 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000187518 s, 2.7 MB/s 2024-09-13T16:23:12.080 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T16:23:12.136 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_2 2024-09-13T16:23:12.193 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T16:23:12.193 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T16:23:12.193 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-13T16:23:12.193 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T16:23:12.193 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T16:23:12.193 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-09-13 16:23:11.893944802 +0000 2024-09-13T16:23:12.193 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-09-13 16:18:07.206735693 +0000 2024-09-13T16:23:12.194 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-09-13 16:18:07.206735693 +0000 2024-09-13T16:23:12.194 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-09-13 16:18:07.206735693 +0000 2024-09-13T16:23:12.194 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T16:23:12.257 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-09-13T16:23:12.257 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-09-13T16:23:12.257 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000202074 s, 2.5 MB/s 2024-09-13T16:23:12.259 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T16:23:12.317 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_3 2024-09-13T16:23:12.333 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:12 smithi111 podman[36249]: 2024-09-13 16:23:12.067645901 +0000 UTC m=+0.016546528 image pull 152d8bc6b19e354515e8ba109d4619445de0e09b4d4a747c2f16eb5c0edb670c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a 2024-09-13T16:23:12.333 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:12 smithi111 podman[36249]: 2024-09-13 16:23:12.207828995 +0000 UTC m=+0.156729653 container create 4bfb8e2c8cc57d4b61926c77254bff27477b652038b19e847ff19fe2b0dd0547 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-13T16:23:12.347 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T16:23:12.347 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T16:23:12.347 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-13T16:23:12.347 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T16:23:12.347 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T16:23:12.347 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-09-13 16:23:11.893944802 +0000 2024-09-13T16:23:12.348 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-09-13 16:18:07.537730090 +0000 2024-09-13T16:23:12.348 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-09-13 16:18:07.537730090 +0000 2024-09-13T16:23:12.348 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-09-13 16:18:07.537730090 +0000 2024-09-13T16:23:12.348 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T16:23:12.411 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-09-13T16:23:12.411 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-09-13T16:23:12.411 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000224005 s, 2.3 MB/s 2024-09-13T16:23:12.412 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T16:23:12.470 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_4 2024-09-13T16:23:12.526 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T16:23:12.526 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T16:23:12.527 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-13T16:23:12.527 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T16:23:12.527 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T16:23:12.527 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-09-13 16:23:11.893944802 +0000 2024-09-13T16:23:12.527 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-09-13 16:18:07.861724607 +0000 2024-09-13T16:23:12.527 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-09-13 16:18:07.861724607 +0000 2024-09-13T16:23:12.527 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-09-13 16:18:07.861724607 +0000 2024-09-13T16:23:12.527 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T16:23:12.591 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:12 smithi111 podman[36249]: 2024-09-13 16:23:12.43545461 +0000 UTC m=+0.384355227 container init 4bfb8e2c8cc57d4b61926c77254bff27477b652038b19e847ff19fe2b0dd0547 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-13T16:23:12.591 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:12 smithi111 podman[36249]: 2024-09-13 16:23:12.438378493 +0000 UTC m=+0.387279107 container start 4bfb8e2c8cc57d4b61926c77254bff27477b652038b19e847ff19fe2b0dd0547 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T16:23:12.591 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:12 smithi111 ceph-mgr[36392]: -- 172.21.15.111:0/369120554 <== mon.1 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55713fe32d00 con 0x55713fe07400 2024-09-13T16:23:12.591 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:12 smithi111 bash[36249]: 4bfb8e2c8cc57d4b61926c77254bff27477b652038b19e847ff19fe2b0dd0547 2024-09-13T16:23:12.591 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:12 smithi111 systemd[1]: Started Ceph mgr.z for 0d241f6a-71ec-11ef-bceb-c7b262605968. 2024-09-13T16:23:12.591 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:12 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-z[36341]: 2024-09-13T16:23:12.561+0000 7fab184c2040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T16:23:12.593 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-09-13T16:23:12.593 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-09-13T16:23:12.593 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000186588 s, 2.7 MB/s 2024-09-13T16:23:12.595 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T16:23:12.651 INFO:tasks.cephadm:Deploying osd.0 on smithi016 with /dev/vg_nvme/lv_4... 2024-09-13T16:23:12.651 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-13T16:23:12.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:12 smithi016 ceph-mon[31187]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:12.857 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:23:12.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:12 smithi111 ceph-mon[34600]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:12.937 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:12 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-z[36341]: 2024-09-13T16:23:12.622+0000 7fab184c2040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T16:23:12.937 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:12 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-z[36341]: 2024-09-13T16:23:12.772+0000 7fab184c2040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T16:23:12.937 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:12 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-z[36341]: 2024-09-13T16:23:12.836+0000 7fab184c2040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T16:23:12.995 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:12 smithi107 ceph-mon[34139]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:13.420 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:12 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-z[36341]: 2024-09-13T16:23:12.980+0000 7fab184c2040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T16:23:13.420 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:13 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-z[36341]: 2024-09-13T16:23:13.116+0000 7fab184c2040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T16:23:13.420 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:13 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-z[36341]: 2024-09-13T16:23:13.417+0000 7fab184c2040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T16:23:13.672 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:13 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-z[36341]: 2024-09-13T16:23:13.480+0000 7fab184c2040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T16:23:13.672 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:13 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:13.672 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:13 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:13.672 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:13 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:13.672 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:13 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-13T16:23:13.672 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:13 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-13T16:23:13.672 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:13 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-13T16:23:13.673 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:13 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:13.673 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:13 smithi111 ceph-mon[34600]: Deploying daemon mgr.y on smithi107 2024-09-13T16:23:13.881 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:13 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:13.882 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:13 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:13.882 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:13 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:13.882 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:13 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-13T16:23:13.882 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:13 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-13T16:23:13.882 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:13 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-13T16:23:13.882 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:13 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:13.882 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:13 smithi107 ceph-mon[34139]: Deploying daemon mgr.y on smithi107 2024-09-13T16:23:13.899 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:13 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:13.899 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:13 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:13.899 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:13 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:13.899 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:13 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-13T16:23:13.900 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:13 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-13T16:23:13.900 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:13 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-13T16:23:13.900 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:13 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:13.900 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:13 smithi016 ceph-mon[31187]: Deploying daemon mgr.y on smithi107 2024-09-13T16:23:13.937 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:13 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-z[36341]: 2024-09-13T16:23:13.669+0000 7fab184c2040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T16:23:14.533 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:14 smithi107 systemd[1]: Starting Ceph mgr.y for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:23:14.559 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-13T16:23:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-13T16:23:14.652 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-09-13T16:23:14.652 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-09-13T16:23:14.652 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00964262 s, 1.1 GB/s 2024-09-13T16:23:14.652 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-09-13T16:23:14.794 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:14 smithi107 ceph-mon[34139]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:14.794 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:14 smithi107 podman[35533]: 2024-09-13 16:23:14.435014242 +0000 UTC m=+0.015562327 image pull 152d8bc6b19e354515e8ba109d4619445de0e09b4d4a747c2f16eb5c0edb670c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a 2024-09-13T16:23:14.794 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:14 smithi107 podman[35533]: 2024-09-13 16:23:14.532544126 +0000 UTC m=+0.113092200 container create 787cdddb9b5ceae45ab6557ef27b5c1cfca57404a4d229cd1a08afc638396fbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-13T16:23:14.794 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:14 smithi107 podman[35533]: 2024-09-13 16:23:14.735027679 +0000 UTC m=+0.315575753 container init 787cdddb9b5ceae45ab6557ef27b5c1cfca57404a4d229cd1a08afc638396fbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f) 2024-09-13T16:23:14.794 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:14 smithi107 podman[35533]: 2024-09-13 16:23:14.737869675 +0000 UTC m=+0.318417764 container start 787cdddb9b5ceae45ab6557ef27b5c1cfca57404a4d229cd1a08afc638396fbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-y, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-13T16:23:14.794 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:14 smithi107 ceph-mgr[35552]: -- 172.21.15.107:0/3496250568 <== mon.2 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56067c2f6d00 con 0x56067c2cb000 2024-09-13T16:23:14.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:14 smithi016 ceph-mon[31187]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:14.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:14 smithi111 ceph-mon[34600]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:14.937 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:14 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-z[36341]: 2024-09-13T16:23:14.574+0000 7fab184c2040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T16:23:14.937 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:14 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-z[36341]: 2024-09-13T16:23:14.641+0000 7fab184c2040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T16:23:15.085 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:14 smithi107 bash[35533]: 787cdddb9b5ceae45ab6557ef27b5c1cfca57404a4d229cd1a08afc638396fbe 2024-09-13T16:23:15.085 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:14 smithi107 systemd[1]: Started Ceph mgr.y for 0d241f6a-71ec-11ef-bceb-c7b262605968. 2024-09-13T16:23:15.085 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:14 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-y[35548]: 2024-09-13T16:23:14.868+0000 7fdde4aa8040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T16:23:15.085 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:14 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-y[35548]: 2024-09-13T16:23:14.926+0000 7fdde4aa8040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T16:23:15.086 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:15 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-y[35548]: 2024-09-13T16:23:15.083+0000 7fdde4aa8040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T16:23:15.251 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_4 2024-09-13T16:23:15.349 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:15 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-y[35548]: 2024-09-13T16:23:15.153+0000 7fdde4aa8040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T16:23:15.349 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:15 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-y[35548]: 2024-09-13T16:23:15.324+0000 7fdde4aa8040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T16:23:15.449 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:23:15.651 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:15 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-z[36341]: 2024-09-13T16:23:15.388+0000 7fab184c2040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-13T16:23:15.651 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:15 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-z[36341]: 2024-09-13T16:23:15.500+0000 7fab184c2040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T16:23:15.805 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:15 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-y[35548]: 2024-09-13T16:23:15.472+0000 7fdde4aa8040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T16:23:15.805 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:15 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-y[35548]: 2024-09-13T16:23:15.803+0000 7fdde4aa8040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T16:23:15.927 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:15 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:15.927 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:15 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:15.927 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:15 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:15.927 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:15 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:15.927 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:15 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:23:15.928 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:15 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-z[36341]: 2024-09-13T16:23:15.648+0000 7fab184c2040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T16:23:15.928 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:15 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-z[36341]: 2024-09-13T16:23:15.709+0000 7fab184c2040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T16:23:15.928 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:15 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-z[36341]: 2024-09-13T16:23:15.858+0000 7fab184c2040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T16:23:15.928 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:15 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-z[36341]: 2024-09-13T16:23:15.925+0000 7fab184c2040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T16:23:16.061 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:15 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-y[35548]: 2024-09-13T16:23:15.865+0000 7fdde4aa8040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T16:23:16.061 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:15 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:16.062 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:15 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:16.062 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:15 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:16.062 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:15 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:16.062 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:15 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:23:16.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:15 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:16.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:15 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:16.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:15 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:16.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:15 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:16.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:15 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:23:16.187 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:15 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-z[36341]: 2024-09-13T16:23:15.987+0000 7fab184c2040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T16:23:16.187 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:16 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-z[36341]: 2024-09-13T16:23:16.151+0000 7fab184c2040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T16:23:16.318 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:16 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-y[35548]: 2024-09-13T16:23:16.060+0000 7fdde4aa8040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T16:23:16.480 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:16 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-z[36341]: 2024-09-13T16:23:16.216+0000 7fab184c2040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T16:23:16.480 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:16 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-z[36341]: 2024-09-13T16:23:16.281+0000 7fab184c2040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T16:23:16.480 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:16 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-z[36341]: 2024-09-13T16:23:16.363+0000 7fab184c2040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T16:23:16.480 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:16 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-z[36341]: 2024-09-13T16:23:16.421+0000 7fab184c2040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T16:23:16.481 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:16 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-z[36341]: 2024-09-13T16:23:16.478+0000 7fab184c2040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T16:23:17.083 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:16 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:17.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:16 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:17.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:16 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:17.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:16 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:17.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:16 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:17.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:16 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:23:17.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:16 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:17.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:16 smithi016 ceph-mon[31187]: Reconfiguring mgr.x (unknown last config time)... 2024-09-13T16:23:17.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:16 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-13T16:23:17.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:16 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-13T16:23:17.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:16 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:17.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:16 smithi016 ceph-mon[31187]: Reconfiguring daemon mgr.x on smithi016 2024-09-13T16:23:17.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:16 smithi016 ceph-mon[31187]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:17.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.166+0000 7f7964ef6640 1 -- 172.21.15.16:0/2691269776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79600fed00 msgr2=0x7f79600ff100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:23:17.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.166+0000 7f7964ef6640 1 --2- 172.21.15.16:0/2691269776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79600fed00 0x7f79600ff100 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7950009900 tx=0x7f795002a410 comp rx=0 tx=0).stop 2024-09-13T16:23:17.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.167+0000 7f7964ef6640 1 -- 172.21.15.16:0/2691269776 shutdown_connections 2024-09-13T16:23:17.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.167+0000 7f7964ef6640 1 --2- 172.21.15.16:0/2691269776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79601008c0 0x7f7960107150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:17.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.167+0000 7f7964ef6640 1 --2- 172.21.15.16:0/2691269776 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79600fff00 0x7f7960100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:17.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.167+0000 7f7964ef6640 1 --2- 172.21.15.16:0/2691269776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79600fed00 0x7f79600ff100 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:17.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.167+0000 7f7964ef6640 1 -- 172.21.15.16:0/2691269776 >> 172.21.15.16:0/2691269776 conn(0x7f79600fa490 msgr2=0x7f79600fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:23:17.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.167+0000 7f7964ef6640 1 -- 172.21.15.16:0/2691269776 shutdown_connections 2024-09-13T16:23:17.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.167+0000 7f7964ef6640 1 -- 172.21.15.16:0/2691269776 wait complete. 2024-09-13T16:23:17.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.167+0000 7f7964ef6640 1 Processor -- start 2024-09-13T16:23:17.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.167+0000 7f7964ef6640 1 -- start start 2024-09-13T16:23:17.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.168+0000 7f7964ef6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79600fed00 0x7f7960109480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:23:17.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.168+0000 7f7964ef6640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79600fff00 0x7f7960107ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:23:17.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.168+0000 7f7964ef6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79601008c0 0x7f7960108010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:23:17.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.168+0000 7f7964ef6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79601087a0 con 0x7f79600fed00 2024-09-13T16:23:17.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.168+0000 7f7964ef6640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7960108910 con 0x7f79601008c0 2024-09-13T16:23:17.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.168+0000 7f7964ef6640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796019fd50 con 0x7f79600fff00 2024-09-13T16:23:17.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.168+0000 7f795e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79600fed00 0x7f7960109480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:23:17.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.168+0000 7f795e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79600fed00 0x7f7960109480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45888/0 (socket says 172.21.15.16:45888) 2024-09-13T16:23:17.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.168+0000 7f795e59c640 1 -- 172.21.15.16:0/1973532423 learned_addr learned my addr 172.21.15.16:0/1973532423 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:23:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.168+0000 7f795ed9d640 1 --2- 172.21.15.16:0/1973532423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79601008c0 0x7f7960108010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:23:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.168+0000 7f795dd9b640 1 --2- 172.21.15.16:0/1973532423 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79600fff00 0x7f7960107ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:23:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.168+0000 7f795e59c640 1 -- 172.21.15.16:0/1973532423 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79600fff00 msgr2=0x7f7960107ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:23:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.168+0000 7f795e59c640 1 --2- 172.21.15.16:0/1973532423 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79600fff00 0x7f7960107ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.168+0000 7f795e59c640 1 -- 172.21.15.16:0/1973532423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79601008c0 msgr2=0x7f7960108010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:23:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.168+0000 7f795e59c640 1 --2- 172.21.15.16:0/1973532423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79601008c0 0x7f7960108010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.168+0000 7f795e59c640 1 -- 172.21.15.16:0/1973532423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7950009560 con 0x7f79600fed00 2024-09-13T16:23:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.168+0000 7f795e59c640 1 --2- 172.21.15.16:0/1973532423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79600fed00 0x7f7960109480 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7950004920 tx=0x7f7950004950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:23:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.168+0000 7f794f7fe640 1 -- 172.21.15.16:0/1973532423 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79500044d0 con 0x7f79600fed00 2024-09-13T16:23:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.168+0000 7f794f7fe640 1 -- 172.21.15.16:0/1973532423 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7950003a80 con 0x7f79600fed00 2024-09-13T16:23:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.168+0000 7f7964ef6640 1 -- 172.21.15.16:0/1973532423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f796019ffa0 con 0x7f79600fed00 2024-09-13T16:23:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.168+0000 7f794f7fe640 1 -- 172.21.15.16:0/1973532423 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795003cb10 con 0x7f79600fed00 2024-09-13T16:23:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.168+0000 7f7964ef6640 1 -- 172.21.15.16:0/1973532423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79601a0360 con 0x7f79600fed00 2024-09-13T16:23:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.169+0000 7f794f7fe640 1 -- 172.21.15.16:0/1973532423 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f795003cc70 con 0x7f79600fed00 2024-09-13T16:23:17.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.169+0000 7f7964ef6640 1 -- 172.21.15.16:0/1973532423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79601061e0 con 0x7f79600fed00 2024-09-13T16:23:17.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.170+0000 7f794f7fe640 1 --2- 172.21.15.16:0/1973532423 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7948038930 0x7f794803adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:23:17.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.170+0000 7f794f7fe640 1 -- 172.21.15.16:0/1973532423 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f795006ca20 con 0x7f79600fed00 2024-09-13T16:23:17.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.170+0000 7f795dd9b640 1 --2- 172.21.15.16:0/1973532423 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7948038930 0x7f794803adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:23:17.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.170+0000 7f795dd9b640 1 --2- 172.21.15.16:0/1973532423 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7948038930 0x7f794803adf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7960109200 tx=0x7f795400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:23:17.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.174+0000 7f794f7fe640 1 -- 172.21.15.16:0/1973532423 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f795004d350 con 0x7f79600fed00 2024-09-13T16:23:17.250 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:16 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:17.250 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:16 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:17.250 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:16 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:17.250 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:16 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:17.250 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:16 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:17.250 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:16 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:23:17.250 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:16 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:17.250 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:16 smithi111 ceph-mon[34600]: Reconfiguring mgr.x (unknown last config time)... 2024-09-13T16:23:17.251 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:16 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-13T16:23:17.251 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:16 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-13T16:23:17.251 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:16 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:17.251 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:16 smithi111 ceph-mon[34600]: Reconfiguring daemon mgr.x on smithi016 2024-09-13T16:23:17.251 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:16 smithi111 ceph-mon[34600]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:17.349 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:16 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-y[35548]: 2024-09-13T16:23:16.923+0000 7fdde4aa8040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T16:23:17.349 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:16 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-y[35548]: 2024-09-13T16:23:16.987+0000 7fdde4aa8040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T16:23:17.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:16 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:17.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:16 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:17.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:16 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:17.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:16 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:17.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:16 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:17.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:16 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:23:17.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:16 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:17.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:16 smithi107 ceph-mon[34139]: Reconfiguring mgr.x (unknown last config time)... 2024-09-13T16:23:17.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:16 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-13T16:23:17.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:16 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-13T16:23:17.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:16 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:17.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:16 smithi107 ceph-mon[34139]: Reconfiguring daemon mgr.x on smithi016 2024-09-13T16:23:17.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:16 smithi107 ceph-mon[34139]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:17.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:17.358+0000 7f7964ef6640 1 -- 172.21.15.16:0/1973532423 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f796002c550 con 0x7f7948038930 2024-09-13T16:23:17.687 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:23:17 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-z[36341]: 2024-09-13T16:23:17.247+0000 7fab184c2040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T16:23:18.001 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:17 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-y[35548]: 2024-09-13T16:23:17.690+0000 7fdde4aa8040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-13T16:23:18.001 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:17 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-y[35548]: 2024-09-13T16:23:17.801+0000 7fdde4aa8040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T16:23:18.002 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:17 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-y[35548]: 2024-09-13T16:23:17.940+0000 7fdde4aa8040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T16:23:18.002 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:18 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-y[35548]: 2024-09-13T16:23:18.000+0000 7fdde4aa8040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T16:23:18.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:18.048+0000 7f794f7fe640 1 -- 172.21.15.16:0/1973532423 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 14) v1 ==== 90707+0+0 (secure 0 0 0) 0x7f7950003bf0 con 0x7f79600fed00 2024-09-13T16:23:18.274 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:18 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-y[35548]: 2024-09-13T16:23:18.149+0000 7fdde4aa8040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T16:23:18.275 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:18 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-y[35548]: 2024-09-13T16:23:18.215+0000 7fdde4aa8040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T16:23:18.275 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:18 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-y[35548]: 2024-09-13T16:23:18.273+0000 7fdde4aa8040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T16:23:18.275 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:18 smithi107 ceph-mon[34139]: Standby manager daemon z started 2024-09-13T16:23:18.275 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:18 smithi107 ceph-mon[34139]: from='mgr.? 172.21.15.111:0/2911885930' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-13T16:23:18.275 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:18 smithi107 ceph-mon[34139]: from='mgr.? 172.21.15.111:0/2911885930' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-13T16:23:18.275 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:18 smithi107 ceph-mon[34139]: from='mgr.? 172.21.15.111:0/2911885930' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-13T16:23:18.275 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:18 smithi107 ceph-mon[34139]: from='mgr.? 172.21.15.111:0/2911885930' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-13T16:23:18.275 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:18 smithi107 ceph-mon[34139]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:23:18.275 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:18 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:23:18.275 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:18 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:23:18.276 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:18 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:18.276 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:18 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:18.276 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:18 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:18.276 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:18 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:23:18.276 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:18 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:18.276 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:18 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:23:18.276 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:18 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:18.299 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:18 smithi016 ceph-mon[31187]: Standby manager daemon z started 2024-09-13T16:23:18.299 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:18 smithi016 ceph-mon[31187]: from='mgr.? 172.21.15.111:0/2911885930' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-13T16:23:18.299 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:18 smithi016 ceph-mon[31187]: from='mgr.? 172.21.15.111:0/2911885930' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-13T16:23:18.299 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:18 smithi016 ceph-mon[31187]: from='mgr.? 172.21.15.111:0/2911885930' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-13T16:23:18.299 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:18 smithi016 ceph-mon[31187]: from='mgr.? 172.21.15.111:0/2911885930' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-13T16:23:18.299 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:18 smithi016 ceph-mon[31187]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:23:18.299 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:18 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:23:18.299 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:18 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:23:18.299 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:18 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:18.300 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:18 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:18.300 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:18 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:18.300 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:18 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:23:18.300 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:18 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:18.300 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:18 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:23:18.300 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:18 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:18.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:18 smithi111 ceph-mon[34600]: Standby manager daemon z started 2024-09-13T16:23:18.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:18 smithi111 ceph-mon[34600]: from='mgr.? 172.21.15.111:0/2911885930' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-13T16:23:18.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:18 smithi111 ceph-mon[34600]: from='mgr.? 172.21.15.111:0/2911885930' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-13T16:23:18.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:18 smithi111 ceph-mon[34600]: from='mgr.? 172.21.15.111:0/2911885930' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-13T16:23:18.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:18 smithi111 ceph-mon[34600]: from='mgr.? 172.21.15.111:0/2911885930' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-13T16:23:18.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:18 smithi111 ceph-mon[34600]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:23:18.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:18 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:23:18.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:18 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:23:18.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:18 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:18.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:18 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:18.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:18 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:18.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:18 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:23:18.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:18 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:18.439 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:18 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:23:18.439 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:18 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:18.557 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:18 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-y[35548]: 2024-09-13T16:23:18.427+0000 7fdde4aa8040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T16:23:18.557 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:18 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-y[35548]: 2024-09-13T16:23:18.491+0000 7fdde4aa8040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T16:23:18.557 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:18 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-y[35548]: 2024-09-13T16:23:18.556+0000 7fdde4aa8040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T16:23:18.849 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:18 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-y[35548]: 2024-09-13T16:23:18.636+0000 7fdde4aa8040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T16:23:18.849 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:18 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-y[35548]: 2024-09-13T16:23:18.694+0000 7fdde4aa8040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T16:23:18.849 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:18 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-y[35548]: 2024-09-13T16:23:18.753+0000 7fdde4aa8040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T16:23:19.182 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:19 smithi016 ceph-mon[31187]: mgrmap e14: x(active, since 73s), standbys: z 2024-09-13T16:23:19.182 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:19 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-13T16:23:19.182 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:19 smithi016 ceph-mon[31187]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:19.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:19 smithi107 ceph-mon[34139]: mgrmap e14: x(active, since 73s), standbys: z 2024-09-13T16:23:19.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:19 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-13T16:23:19.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:19 smithi107 ceph-mon[34139]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:19.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:19 smithi111 ceph-mon[34600]: mgrmap e14: x(active, since 73s), standbys: z 2024-09-13T16:23:19.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:19 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-13T16:23:19.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:19 smithi111 ceph-mon[34600]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:19.849 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:23:19 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-y[35548]: 2024-09-13T16:23:19.516+0000 7fdde4aa8040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T16:23:20.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:20.060+0000 7f794f7fe640 1 -- 172.21.15.16:0/1973532423 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mgrmap(e 15) v1 ==== 135817+0+0 (secure 0 0 0) 0x7f795003bc50 con 0x7f79600fed00 2024-09-13T16:23:20.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:20 smithi107 ceph-mon[34139]: Standby manager daemon y started 2024-09-13T16:23:20.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:20 smithi107 ceph-mon[34139]: from='mgr.? 172.21.15.107:0/3476713050' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-13T16:23:20.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:20 smithi107 ceph-mon[34139]: from='mgr.? 172.21.15.107:0/3476713050' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-13T16:23:20.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:20 smithi107 ceph-mon[34139]: from='mgr.? 172.21.15.107:0/3476713050' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-13T16:23:20.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:20 smithi107 ceph-mon[34139]: from='mgr.? 172.21.15.107:0/3476713050' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-13T16:23:20.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:20 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:20.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:20 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3186530717' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e11ef8f-08d6-4e74-afd6-f49f4bd40032"}]: dispatch 2024-09-13T16:23:20.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:20 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3186530717' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e11ef8f-08d6-4e74-afd6-f49f4bd40032"}]': finished 2024-09-13T16:23:20.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:20 smithi107 ceph-mon[34139]: osdmap e5: 1 total, 0 up, 1 in 2024-09-13T16:23:20.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:20 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T16:23:20.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:20 smithi016 ceph-mon[31187]: Standby manager daemon y started 2024-09-13T16:23:20.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:20 smithi016 ceph-mon[31187]: from='mgr.? 172.21.15.107:0/3476713050' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-13T16:23:20.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:20 smithi016 ceph-mon[31187]: from='mgr.? 172.21.15.107:0/3476713050' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-13T16:23:20.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:20 smithi016 ceph-mon[31187]: from='mgr.? 172.21.15.107:0/3476713050' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-13T16:23:20.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:20 smithi016 ceph-mon[31187]: from='mgr.? 172.21.15.107:0/3476713050' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-13T16:23:20.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:20 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:20.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:20 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3186530717' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e11ef8f-08d6-4e74-afd6-f49f4bd40032"}]: dispatch 2024-09-13T16:23:20.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:20 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3186530717' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e11ef8f-08d6-4e74-afd6-f49f4bd40032"}]': finished 2024-09-13T16:23:20.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:20 smithi016 ceph-mon[31187]: osdmap e5: 1 total, 0 up, 1 in 2024-09-13T16:23:20.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:20 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T16:23:20.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:20 smithi111 ceph-mon[34600]: Standby manager daemon y started 2024-09-13T16:23:20.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:20 smithi111 ceph-mon[34600]: from='mgr.? 172.21.15.107:0/3476713050' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-13T16:23:20.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:20 smithi111 ceph-mon[34600]: from='mgr.? 172.21.15.107:0/3476713050' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-13T16:23:20.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:20 smithi111 ceph-mon[34600]: from='mgr.? 172.21.15.107:0/3476713050' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-13T16:23:20.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:20 smithi111 ceph-mon[34600]: from='mgr.? 172.21.15.107:0/3476713050' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-13T16:23:20.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:20 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:20.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:20 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3186530717' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e11ef8f-08d6-4e74-afd6-f49f4bd40032"}]: dispatch 2024-09-13T16:23:20.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:20 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3186530717' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e11ef8f-08d6-4e74-afd6-f49f4bd40032"}]': finished 2024-09-13T16:23:20.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:20 smithi111 ceph-mon[34600]: osdmap e5: 1 total, 0 up, 1 in 2024-09-13T16:23:20.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:20 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T16:23:21.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:21 smithi107 ceph-mon[34139]: mgrmap e15: x(active, since 75s), standbys: z, y 2024-09-13T16:23:21.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:21 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-13T16:23:21.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:21 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1107805541' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:23:21.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:21 smithi107 ceph-mon[34139]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:21.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:21 smithi016 ceph-mon[31187]: mgrmap e15: x(active, since 75s), standbys: z, y 2024-09-13T16:23:21.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:21 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-13T16:23:21.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:21 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1107805541' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:23:21.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:21 smithi016 ceph-mon[31187]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:21.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:21 smithi111 ceph-mon[34600]: mgrmap e15: x(active, since 75s), standbys: z, y 2024-09-13T16:23:21.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:21 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-13T16:23:21.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:21 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1107805541' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:23:21.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:21 smithi111 ceph-mon[34600]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:22.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:22 smithi107 ceph-mon[34139]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:22.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:22 smithi016 ceph-mon[31187]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:22.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:22 smithi111 ceph-mon[34600]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:24.796 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:24 smithi016 ceph-mon[31187]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:24.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:24 smithi107 ceph-mon[34139]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:24.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:24 smithi111 ceph-mon[34600]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:26.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:26 smithi107 ceph-mon[34139]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:26.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:26 smithi016 ceph-mon[31187]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:26.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:26 smithi111 ceph-mon[34600]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:27.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:27 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-13T16:23:27.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:27 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:27.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:27 smithi016 ceph-mon[31187]: Deploying daemon osd.0 on smithi016 2024-09-13T16:23:27.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:27 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-13T16:23:27.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:27 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:27.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:27 smithi111 ceph-mon[34600]: Deploying daemon osd.0 on smithi016 2024-09-13T16:23:28.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:27 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-13T16:23:28.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:27 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:28.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:27 smithi107 ceph-mon[34139]: Deploying daemon osd.0 on smithi016 2024-09-13T16:23:28.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:28 smithi016 ceph-mon[31187]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:28.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:28 smithi111 ceph-mon[34600]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:29.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:28 smithi107 ceph-mon[34139]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:30.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:30 smithi016 ceph-mon[31187]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:30.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:30 smithi111 ceph-mon[34600]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:31.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:30 smithi107 ceph-mon[34139]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:31.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:31 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:23:31.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:31 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:31.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:31 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:31.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:31 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:23:31.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:31 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:31.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:31 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:32.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:31 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:23:32.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:31 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:32.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:31 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:32.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:32 smithi016 ceph-mon[31187]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:32.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:32 smithi111 ceph-mon[34600]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:33.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:32 smithi107 ceph-mon[34139]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:34.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:34 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:34.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:34 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:34.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:34 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:34.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:34 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:23:34.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:34 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:34.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:34 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:34.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:34 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:34.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:34 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:34.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:34 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:23:34.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:34 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:34.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:34 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:34.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:34 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:34.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:34 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:34.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:34 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:23:34.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:34 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:34.733+0000 7f794f7fe640 1 -- 172.21.15.16:0/1973532423 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f796002c550 con 0x7f7948038930 2024-09-13T16:23:34.735 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 0 on host 'smithi016' 2024-09-13T16:23:34.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:34.736+0000 7f7964ef6640 1 -- 172.21.15.16:0/1973532423 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7948038930 msgr2=0x7f794803adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:23:34.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:34.736+0000 7f7964ef6640 1 --2- 172.21.15.16:0/1973532423 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7948038930 0x7f794803adf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7960109200 tx=0x7f795400b410 comp rx=0 tx=0).stop 2024-09-13T16:23:34.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:34.736+0000 7f7964ef6640 1 -- 172.21.15.16:0/1973532423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79600fed00 msgr2=0x7f7960109480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:23:34.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:34.736+0000 7f7964ef6640 1 --2- 172.21.15.16:0/1973532423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79600fed00 0x7f7960109480 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7950004920 tx=0x7f7950004950 comp rx=0 tx=0).stop 2024-09-13T16:23:34.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:34.736+0000 7f7964ef6640 1 -- 172.21.15.16:0/1973532423 shutdown_connections 2024-09-13T16:23:34.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:34.736+0000 7f7964ef6640 1 --2- 172.21.15.16:0/1973532423 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7948038930 0x7f794803adf0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:34.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:34.736+0000 7f7964ef6640 1 --2- 172.21.15.16:0/1973532423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79601008c0 0x7f7960108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:34.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:34.736+0000 7f7964ef6640 1 --2- 172.21.15.16:0/1973532423 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79600fff00 0x7f7960107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:34.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:34.736+0000 7f7964ef6640 1 --2- 172.21.15.16:0/1973532423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79600fed00 0x7f7960109480 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:34.736+0000 7f7964ef6640 1 -- 172.21.15.16:0/1973532423 >> 172.21.15.16:0/1973532423 conn(0x7f79600fa490 msgr2=0x7f79601033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:23:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:34.736+0000 7f7964ef6640 1 -- 172.21.15.16:0/1973532423 shutdown_connections 2024-09-13T16:23:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:23:34.736+0000 7f7964ef6640 1 -- 172.21.15.16:0/1973532423 wait complete. 2024-09-13T16:23:35.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:35 smithi107 ceph-mon[34139]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:35.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:35 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:23:35.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:35 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:35.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:35 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:35.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:35 smithi016 ceph-mon[31187]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:35.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:35 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:23:35.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:35 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:35.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:35 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:35.387 DEBUG:teuthology.orchestra.run.smithi016:osd.0> sudo journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.0.service 2024-09-13T16:23:35.390 INFO:tasks.cephadm:Deploying osd.1 on smithi107 with /dev/vg_nvme/lv_4... 2024-09-13T16:23:35.390 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-13T16:23:35.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:35 smithi111 ceph-mon[34600]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:35.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:35 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:23:35.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:35 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:35.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:35 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:35.594 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T16:23:36.341 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:36 smithi107 ceph-mon[34139]: from='osd.0 [v2:172.21.15.16:6802/2079394186,v1:172.21.15.16:6803/2079394186]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-13T16:23:36.356 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:36 smithi016 ceph-mon[31187]: from='osd.0 [v2:172.21.15.16:6802/2079394186,v1:172.21.15.16:6803/2079394186]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-13T16:23:36.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:36 smithi111 ceph-mon[34600]: from='osd.0 [v2:172.21.15.16:6802/2079394186,v1:172.21.15.16:6803/2079394186]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-13T16:23:37.188 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-13T16:23:37.267 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-13T16:23:37.267 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:37 smithi107 ceph-mon[34139]: from='osd.0 [v2:172.21.15.16:6802/2079394186,v1:172.21.15.16:6803/2079394186]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-13T16:23:37.267 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:37 smithi107 ceph-mon[34139]: osdmap e6: 1 total, 0 up, 1 in 2024-09-13T16:23:37.267 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:37 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T16:23:37.267 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:37 smithi107 ceph-mon[34139]: from='osd.0 [v2:172.21.15.16:6802/2079394186,v1:172.21.15.16:6803/2079394186]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-13T16:23:37.267 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:37 smithi107 ceph-mon[34139]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:37.282 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-09-13T16:23:37.282 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-09-13T16:23:37.282 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0135086 s, 776 MB/s 2024-09-13T16:23:37.282 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-09-13T16:23:37.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:37 smithi016 ceph-mon[31187]: from='osd.0 [v2:172.21.15.16:6802/2079394186,v1:172.21.15.16:6803/2079394186]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-13T16:23:37.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:37 smithi016 ceph-mon[31187]: osdmap e6: 1 total, 0 up, 1 in 2024-09-13T16:23:37.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:37 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T16:23:37.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:37 smithi016 ceph-mon[31187]: from='osd.0 [v2:172.21.15.16:6802/2079394186,v1:172.21.15.16:6803/2079394186]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-13T16:23:37.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:37 smithi016 ceph-mon[31187]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:37.355 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 13 16:23:37 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-0[45834]: 2024-09-13T16:23:37.066+0000 7fc70066f640 -1 osd.0 0 waiting for initial osdmap 2024-09-13T16:23:37.355 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 13 16:23:37 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-0[45834]: 2024-09-13T16:23:37.069+0000 7fc6fc498640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T16:23:37.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:37 smithi111 ceph-mon[34600]: from='osd.0 [v2:172.21.15.16:6802/2079394186,v1:172.21.15.16:6803/2079394186]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-13T16:23:37.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:37 smithi111 ceph-mon[34600]: osdmap e6: 1 total, 0 up, 1 in 2024-09-13T16:23:37.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:37 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T16:23:37.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:37 smithi111 ceph-mon[34600]: from='osd.0 [v2:172.21.15.16:6802/2079394186,v1:172.21.15.16:6803/2079394186]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-13T16:23:37.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:37 smithi111 ceph-mon[34600]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:37.920 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_4 2024-09-13T16:23:38.116 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T16:23:38.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:38 smithi107 ceph-mon[34139]: from='osd.0 [v2:172.21.15.16:6802/2079394186,v1:172.21.15.16:6803/2079394186]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-13T16:23:38.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:38 smithi107 ceph-mon[34139]: osdmap e7: 1 total, 0 up, 1 in 2024-09-13T16:23:38.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:38 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T16:23:38.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:38 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T16:23:38.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:38 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T16:23:38.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:38 smithi016 ceph-mon[31187]: from='osd.0 [v2:172.21.15.16:6802/2079394186,v1:172.21.15.16:6803/2079394186]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-13T16:23:38.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:38 smithi016 ceph-mon[31187]: osdmap e7: 1 total, 0 up, 1 in 2024-09-13T16:23:38.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:38 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T16:23:38.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:38 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T16:23:38.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:38 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T16:23:38.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:38 smithi111 ceph-mon[34600]: from='osd.0 [v2:172.21.15.16:6802/2079394186,v1:172.21.15.16:6803/2079394186]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-13T16:23:38.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:38 smithi111 ceph-mon[34600]: osdmap e7: 1 total, 0 up, 1 in 2024-09-13T16:23:38.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:38 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T16:23:38.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:38 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T16:23:38.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:38 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T16:23:39.313 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:39 smithi107 ceph-mon[34139]: purged_snaps scrub starts 2024-09-13T16:23:39.313 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:39 smithi107 ceph-mon[34139]: purged_snaps scrub ok 2024-09-13T16:23:39.313 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:39 smithi107 ceph-mon[34139]: osd.0 [v2:172.21.15.16:6802/2079394186,v1:172.21.15.16:6803/2079394186] boot 2024-09-13T16:23:39.314 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:39 smithi107 ceph-mon[34139]: osdmap e8: 1 total, 1 up, 1 in 2024-09-13T16:23:39.314 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:39 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T16:23:39.314 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:39 smithi107 ceph-mon[34139]: Detected new or changed devices on smithi016 2024-09-13T16:23:39.314 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:39 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:39.314 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:39 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:39.314 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:39 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:23:39.314 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:39 smithi107 ceph-mon[34139]: Adjusting osd_memory_target on smithi016 to 17092M 2024-09-13T16:23:39.314 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:39 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:39.314 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:39 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:39.314 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:39 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:23:39.314 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:39 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:39.314 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:39 smithi107 ceph-mon[34139]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:39.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:39 smithi016 ceph-mon[31187]: purged_snaps scrub starts 2024-09-13T16:23:39.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:39 smithi016 ceph-mon[31187]: purged_snaps scrub ok 2024-09-13T16:23:39.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:39 smithi016 ceph-mon[31187]: osd.0 [v2:172.21.15.16:6802/2079394186,v1:172.21.15.16:6803/2079394186] boot 2024-09-13T16:23:39.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:39 smithi016 ceph-mon[31187]: osdmap e8: 1 total, 1 up, 1 in 2024-09-13T16:23:39.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:39 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T16:23:39.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:39 smithi016 ceph-mon[31187]: Detected new or changed devices on smithi016 2024-09-13T16:23:39.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:39 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:39.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:39 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:39.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:39 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:23:39.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:39 smithi016 ceph-mon[31187]: Adjusting osd_memory_target on smithi016 to 17092M 2024-09-13T16:23:39.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:39 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:39.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:39 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:39.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:39 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:23:39.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:39 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:39.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:39 smithi016 ceph-mon[31187]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:39.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:39 smithi111 ceph-mon[34600]: purged_snaps scrub starts 2024-09-13T16:23:39.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:39 smithi111 ceph-mon[34600]: purged_snaps scrub ok 2024-09-13T16:23:39.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:39 smithi111 ceph-mon[34600]: osd.0 [v2:172.21.15.16:6802/2079394186,v1:172.21.15.16:6803/2079394186] boot 2024-09-13T16:23:39.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:39 smithi111 ceph-mon[34600]: osdmap e8: 1 total, 1 up, 1 in 2024-09-13T16:23:39.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:39 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T16:23:39.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:39 smithi111 ceph-mon[34600]: Detected new or changed devices on smithi016 2024-09-13T16:23:39.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:39 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:39.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:39 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:39.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:39 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:23:39.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:39 smithi111 ceph-mon[34600]: Adjusting osd_memory_target on smithi016 to 17092M 2024-09-13T16:23:39.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:39 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:39.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:39 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:39.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:39 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:23:39.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:39 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:39.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:39 smithi111 ceph-mon[34600]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T16:23:39.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.644+0000 7f6a4a7e6640 1 -- 172.21.15.107:0/963718241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a441008c0 msgr2=0x7f6a44107150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:23:39.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.644+0000 7f6a4a7e6640 1 --2- 172.21.15.107:0/963718241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a441008c0 0x7f6a44107150 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c009ca0 tx=0x7f6a3c02a8c0 comp rx=0 tx=0).stop 2024-09-13T16:23:39.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.644+0000 7f6a4a7e6640 1 -- 172.21.15.107:0/963718241 shutdown_connections 2024-09-13T16:23:39.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.644+0000 7f6a4a7e6640 1 --2- 172.21.15.107:0/963718241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a441008c0 0x7f6a44107150 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:39.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.644+0000 7f6a4a7e6640 1 --2- 172.21.15.107:0/963718241 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a440fff00 0x7f6a44100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:39.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.644+0000 7f6a4a7e6640 1 --2- 172.21.15.107:0/963718241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a440fed00 0x7f6a440ff100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:39.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.644+0000 7f6a4a7e6640 1 -- 172.21.15.107:0/963718241 >> 172.21.15.107:0/963718241 conn(0x7f6a440fa490 msgr2=0x7f6a440fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:23:39.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.644+0000 7f6a4a7e6640 1 -- 172.21.15.107:0/963718241 shutdown_connections 2024-09-13T16:23:39.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.644+0000 7f6a4a7e6640 1 -- 172.21.15.107:0/963718241 wait complete. 2024-09-13T16:23:39.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.645+0000 7f6a4a7e6640 1 Processor -- start 2024-09-13T16:23:39.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.645+0000 7f6a4a7e6640 1 -- start start 2024-09-13T16:23:39.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.645+0000 7f6a4a7e6640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a440fed00 0x7f6a44194a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:23:39.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.645+0000 7f6a4a7e6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a440fff00 0x7f6a44194f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:23:39.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.645+0000 7f6a4a7e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a441008c0 0x7f6a4419c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:23:39.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.645+0000 7f6a4a7e6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4419c540 con 0x7f6a441008c0 2024-09-13T16:23:39.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.645+0000 7f6a4a7e6640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4419c6b0 con 0x7f6a440fff00 2024-09-13T16:23:39.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.645+0000 7f6a4a7e6640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4419c870 con 0x7f6a440fed00 2024-09-13T16:23:39.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.645+0000 7f6a437fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a440fff00 0x7f6a44194f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:23:39.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.645+0000 7f6a437fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a440fff00 0x7f6a44194f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41822/0 (socket says 172.21.15.107:41822) 2024-09-13T16:23:39.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.645+0000 7f6a437fe640 1 -- 172.21.15.107:0/3569023017 learned_addr learned my addr 172.21.15.107:0/3569023017 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-13T16:23:39.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.646+0000 7f6a48d83640 1 --2- 172.21.15.107:0/3569023017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a441008c0 0x7f6a4419c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:23:39.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.646+0000 7f6a43fff640 1 --2- 172.21.15.107:0/3569023017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a440fed00 0x7f6a44194a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:23:39.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.646+0000 7f6a437fe640 1 -- 172.21.15.107:0/3569023017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a440fed00 msgr2=0x7f6a44194a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:23:39.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.646+0000 7f6a437fe640 1 --2- 172.21.15.107:0/3569023017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a440fed00 0x7f6a44194a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:39.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.646+0000 7f6a437fe640 1 -- 172.21.15.107:0/3569023017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a441008c0 msgr2=0x7f6a4419c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:23:39.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.646+0000 7f6a437fe640 1 --2- 172.21.15.107:0/3569023017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a441008c0 0x7f6a4419c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:39.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.646+0000 7f6a437fe640 1 -- 172.21.15.107:0/3569023017 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3c009900 con 0x7f6a440fff00 2024-09-13T16:23:39.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.646+0000 7f6a43fff640 1 --2- 172.21.15.107:0/3569023017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a440fed00 0x7f6a44194a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:23:39.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.646+0000 7f6a437fe640 1 --2- 172.21.15.107:0/3569023017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a440fff00 0x7f6a44194f80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800c920 tx=0x7f6a3800cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:23:39.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.646+0000 7f6a417fa640 1 -- 172.21.15.107:0/3569023017 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a38007c70 con 0x7f6a440fff00 2024-09-13T16:23:39.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.646+0000 7f6a4a7e6640 1 -- 172.21.15.107:0/3569023017 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a4419cb50 con 0x7f6a440fff00 2024-09-13T16:23:39.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.646+0000 7f6a417fa640 1 -- 172.21.15.107:0/3569023017 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a38007dd0 con 0x7f6a440fff00 2024-09-13T16:23:39.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.646+0000 7f6a417fa640 1 -- 172.21.15.107:0/3569023017 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a38018730 con 0x7f6a440fff00 2024-09-13T16:23:39.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.646+0000 7f6a4a7e6640 1 -- 172.21.15.107:0/3569023017 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a4419d050 con 0x7f6a440fff00 2024-09-13T16:23:39.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.647+0000 7f6a4a7e6640 1 -- 172.21.15.107:0/3569023017 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a44106050 con 0x7f6a440fff00 2024-09-13T16:23:39.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.648+0000 7f6a417fa640 1 -- 172.21.15.107:0/3569023017 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 15) v1 ==== 135817+0+0 (secure 0 0 0) 0x7f6a3800f450 con 0x7f6a440fff00 2024-09-13T16:23:39.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.649+0000 7f6a417fa640 1 --2- 172.21.15.107:0/3569023017 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f6a2c0a1020 0x7f6a2c0a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:23:39.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.649+0000 7f6a417fa640 1 -- 172.21.15.107:0/3569023017 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f6a380cb280 con 0x7f6a440fff00 2024-09-13T16:23:39.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.649+0000 7f6a43fff640 1 --2- 172.21.15.107:0/3569023017 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f6a2c0a1020 0x7f6a2c0a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:23:39.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.650+0000 7f6a43fff640 1 --2- 172.21.15.107:0/3569023017 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f6a2c0a1020 0x7f6a2c0a34e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6a440ffd60 tx=0x7f6a34005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:23:39.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.652+0000 7f6a417fa640 1 -- 172.21.15.107:0/3569023017 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f6a38096510 con 0x7f6a440fff00 2024-09-13T16:23:39.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:39.822+0000 7f6a4a7e6640 1 -- 172.21.15.107:0/3569023017 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a4402c4f0 con 0x7f6a2c0a1020 2024-09-13T16:23:40.477 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:40 smithi107 ceph-mon[34139]: osdmap e9: 1 total, 1 up, 1 in 2024-09-13T16:23:40.477 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:40 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:23:40.477 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:40 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:23:40.477 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:40 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:40.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:40 smithi016 ceph-mon[31187]: osdmap e9: 1 total, 1 up, 1 in 2024-09-13T16:23:40.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:40 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:23:40.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:40 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:23:40.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:40 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:40.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:40 smithi111 ceph-mon[34600]: osdmap e9: 1 total, 1 up, 1 in 2024-09-13T16:23:40.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:40 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:23:40.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:40 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:23:40.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:40 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:41.407 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:41 smithi107 ceph-mon[34139]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:23:41.407 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:41 smithi107 ceph-mon[34139]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:23:41.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:41 smithi016 ceph-mon[31187]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:23:41.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:41 smithi016 ceph-mon[31187]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:23:41.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:41 smithi111 ceph-mon[34600]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:23:41.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:41 smithi111 ceph-mon[34600]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:23:42.431 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:42 smithi107 ceph-mon[34139]: from='client.? 172.21.15.107:0/3204300328' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2cd3694-9c9e-4da7-976e-f80a8df120f5"}]: dispatch 2024-09-13T16:23:42.431 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:42 smithi107 ceph-mon[34139]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2cd3694-9c9e-4da7-976e-f80a8df120f5"}]: dispatch 2024-09-13T16:23:42.431 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:42 smithi107 ceph-mon[34139]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2cd3694-9c9e-4da7-976e-f80a8df120f5"}]': finished 2024-09-13T16:23:42.431 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:42 smithi107 ceph-mon[34139]: osdmap e10: 2 total, 1 up, 2 in 2024-09-13T16:23:42.431 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:42 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T16:23:42.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:42 smithi016 ceph-mon[31187]: from='client.? 172.21.15.107:0/3204300328' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2cd3694-9c9e-4da7-976e-f80a8df120f5"}]: dispatch 2024-09-13T16:23:42.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:42 smithi016 ceph-mon[31187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2cd3694-9c9e-4da7-976e-f80a8df120f5"}]: dispatch 2024-09-13T16:23:42.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:42 smithi016 ceph-mon[31187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2cd3694-9c9e-4da7-976e-f80a8df120f5"}]': finished 2024-09-13T16:23:42.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:42 smithi016 ceph-mon[31187]: osdmap e10: 2 total, 1 up, 2 in 2024-09-13T16:23:42.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:42 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T16:23:42.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:42 smithi111 ceph-mon[34600]: from='client.? 172.21.15.107:0/3204300328' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2cd3694-9c9e-4da7-976e-f80a8df120f5"}]: dispatch 2024-09-13T16:23:42.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:42 smithi111 ceph-mon[34600]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2cd3694-9c9e-4da7-976e-f80a8df120f5"}]: dispatch 2024-09-13T16:23:42.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:42 smithi111 ceph-mon[34600]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2cd3694-9c9e-4da7-976e-f80a8df120f5"}]': finished 2024-09-13T16:23:42.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:42 smithi111 ceph-mon[34600]: osdmap e10: 2 total, 1 up, 2 in 2024-09-13T16:23:42.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:42 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T16:23:43.589 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:43 smithi111 ceph-mon[34600]: from='client.? 172.21.15.107:0/3558492901' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:23:43.589 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:43 smithi111 ceph-mon[34600]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:23:43.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:43 smithi107 ceph-mon[34139]: from='client.? 172.21.15.107:0/3558492901' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:23:43.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:43 smithi107 ceph-mon[34139]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:23:43.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:43 smithi016 ceph-mon[31187]: from='client.? 172.21.15.107:0/3558492901' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:23:43.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:43 smithi016 ceph-mon[31187]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:23:44.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:44 smithi016 ceph-mon[31187]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:23:44.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:44 smithi111 ceph-mon[34600]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:23:45.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:44 smithi107 ceph-mon[34139]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:23:46.621 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:46 smithi107 ceph-mon[34139]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:23:46.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:46 smithi016 ceph-mon[31187]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:23:46.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:46 smithi111 ceph-mon[34600]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:23:48.833 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:48 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-13T16:23:48.833 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:48 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:48.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:48 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-13T16:23:48.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:48 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:48.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:48 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-13T16:23:48.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:48 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:49.788 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:49 smithi107 ceph-mon[34139]: Deploying daemon osd.1 on smithi107 2024-09-13T16:23:49.788 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:49 smithi107 ceph-mon[34139]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:23:49.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:49 smithi016 ceph-mon[31187]: Deploying daemon osd.1 on smithi107 2024-09-13T16:23:49.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:49 smithi016 ceph-mon[31187]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:23:49.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:49 smithi111 ceph-mon[34600]: Deploying daemon osd.1 on smithi107 2024-09-13T16:23:49.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:49 smithi111 ceph-mon[34600]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:23:51.765 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:51 smithi107 ceph-mon[34139]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:23:51.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:51 smithi016 ceph-mon[31187]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:23:51.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:51 smithi111 ceph-mon[34600]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:23:53.765 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:53 smithi107 ceph-mon[34139]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:23:53.765 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:53 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:23:53.765 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:53 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:53.765 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:53 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:53.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:53 smithi016 ceph-mon[31187]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:23:53.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:53 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:23:53.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:53 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:53.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:53 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:53.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:53 smithi111 ceph-mon[34600]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:23:53.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:53 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:23:53.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:53 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:53.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:53 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:55.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:55 smithi107 ceph-mon[34139]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:23:55.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:55 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:55.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:55 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:55.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:55 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:55.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:55 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:23:55.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:55 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:55.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:55 smithi016 ceph-mon[31187]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:23:55.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:55 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:55.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:55 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:55.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:55 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:55.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:55 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:23:55.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:55 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:55.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:55 smithi111 ceph-mon[34600]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:23:55.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:55 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:55.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:55 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:55.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:55 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:55.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:55 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:23:55.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:55 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:56.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:56.159+0000 7f6a417fa640 1 -- 172.21.15.107:0/3569023017 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6a4402c4f0 con 0x7f6a2c0a1020 2024-09-13T16:23:56.160 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 1 on host 'smithi107' 2024-09-13T16:23:56.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:56.161+0000 7f6a4a7e6640 1 -- 172.21.15.107:0/3569023017 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f6a2c0a1020 msgr2=0x7f6a2c0a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:23:56.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:56.161+0000 7f6a4a7e6640 1 --2- 172.21.15.107:0/3569023017 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f6a2c0a1020 0x7f6a2c0a34e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6a440ffd60 tx=0x7f6a34005db0 comp rx=0 tx=0).stop 2024-09-13T16:23:56.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:56.161+0000 7f6a4a7e6640 1 -- 172.21.15.107:0/3569023017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a440fff00 msgr2=0x7f6a44194f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:23:56.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:56.161+0000 7f6a4a7e6640 1 --2- 172.21.15.107:0/3569023017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a440fff00 0x7f6a44194f80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800c920 tx=0x7f6a3800cde0 comp rx=0 tx=0).stop 2024-09-13T16:23:56.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:56.161+0000 7f6a4a7e6640 1 -- 172.21.15.107:0/3569023017 shutdown_connections 2024-09-13T16:23:56.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:56.161+0000 7f6a4a7e6640 1 --2- 172.21.15.107:0/3569023017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a441008c0 0x7f6a4419c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:56.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:56.161+0000 7f6a4a7e6640 1 --2- 172.21.15.107:0/3569023017 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f6a2c0a1020 0x7f6a2c0a34e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:56.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:56.161+0000 7f6a4a7e6640 1 --2- 172.21.15.107:0/3569023017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a440fff00 0x7f6a44194f80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:56.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:56.161+0000 7f6a4a7e6640 1 --2- 172.21.15.107:0/3569023017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a440fed00 0x7f6a44194a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:23:56.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:56.161+0000 7f6a4a7e6640 1 -- 172.21.15.107:0/3569023017 >> 172.21.15.107:0/3569023017 conn(0x7f6a440fa490 msgr2=0x7f6a44103220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:23:56.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:56.161+0000 7f6a4a7e6640 1 -- 172.21.15.107:0/3569023017 shutdown_connections 2024-09-13T16:23:56.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:23:56.161+0000 7f6a4a7e6640 1 -- 172.21.15.107:0/3569023017 wait complete. 2024-09-13T16:23:56.833 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:56 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:23:56.834 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:56 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:56.834 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:56 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:56.852 DEBUG:teuthology.orchestra.run.smithi107:osd.1> sudo journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.1.service 2024-09-13T16:23:56.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:56 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:23:56.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:56 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:56.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:56 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:56.855 INFO:tasks.cephadm:Deploying osd.2 on smithi111 with /dev/vg_nvme/lv_4... 2024-09-13T16:23:56.855 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-13T16:23:56.882 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:56 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:23:56.883 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:56 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:56.883 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:56 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:57.057 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T16:23:57.097 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 13 16:23:57 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-1[38658]: 2024-09-13T16:23:57.001+0000 7fbdea085640 -1 osd.1 0 log_to_monitors true 2024-09-13T16:23:57.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:57 smithi107 ceph-mon[34139]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:23:57.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:57 smithi107 ceph-mon[34139]: from='osd.1 [v2:172.21.15.107:6800/4021360958,v1:172.21.15.107:6801/4021360958]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-13T16:23:57.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:57 smithi107 ceph-mon[34139]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-13T16:23:57.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:57 smithi016 ceph-mon[31187]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:23:57.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:57 smithi016 ceph-mon[31187]: from='osd.1 [v2:172.21.15.107:6800/4021360958,v1:172.21.15.107:6801/4021360958]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-13T16:23:57.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:57 smithi016 ceph-mon[31187]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-13T16:23:57.910 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:57 smithi111 ceph-mon[34600]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:23:57.910 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:57 smithi111 ceph-mon[34600]: from='osd.1 [v2:172.21.15.107:6800/4021360958,v1:172.21.15.107:6801/4021360958]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-13T16:23:57.910 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:57 smithi111 ceph-mon[34600]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-13T16:23:58.840 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-13T16:23:58.841 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:58 smithi111 ceph-mon[34600]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-13T16:23:58.841 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:58 smithi111 ceph-mon[34600]: osdmap e11: 2 total, 1 up, 2 in 2024-09-13T16:23:58.841 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:58 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T16:23:58.841 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:58 smithi111 ceph-mon[34600]: from='osd.1 [v2:172.21.15.107:6800/4021360958,v1:172.21.15.107:6801/4021360958]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-13T16:23:58.841 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:58 smithi111 ceph-mon[34600]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-13T16:23:58.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:58 smithi107 ceph-mon[34139]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-13T16:23:58.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:58 smithi107 ceph-mon[34139]: osdmap e11: 2 total, 1 up, 2 in 2024-09-13T16:23:58.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:58 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T16:23:58.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:58 smithi107 ceph-mon[34139]: from='osd.1 [v2:172.21.15.107:6800/4021360958,v1:172.21.15.107:6801/4021360958]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-13T16:23:58.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:58 smithi107 ceph-mon[34139]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-13T16:23:58.850 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 13 16:23:58 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-1[38658]: 2024-09-13T16:23:58.548+0000 7fbde6006640 -1 osd.1 0 waiting for initial osdmap 2024-09-13T16:23:58.850 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 13 16:23:58 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-1[38658]: 2024-09-13T16:23:58.551+0000 7fbde1e2f640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T16:23:58.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:58 smithi016 ceph-mon[31187]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-13T16:23:58.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:58 smithi016 ceph-mon[31187]: osdmap e11: 2 total, 1 up, 2 in 2024-09-13T16:23:58.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:58 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T16:23:58.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:58 smithi016 ceph-mon[31187]: from='osd.1 [v2:172.21.15.107:6800/4021360958,v1:172.21.15.107:6801/4021360958]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-13T16:23:58.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:58 smithi016 ceph-mon[31187]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-13T16:23:58.923 INFO:teuthology.orchestra.run.smithi111.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-13T16:23:58.936 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10+0 records in 2024-09-13T16:23:58.936 INFO:teuthology.orchestra.run.smithi111.stderr:10+0 records out 2024-09-13T16:23:58.936 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111313 s, 942 MB/s 2024-09-13T16:23:58.936 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping successful for: 2024-09-13T16:23:59.495 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi111:vg_nvme/lv_4 2024-09-13T16:23:59.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:59 smithi111 ceph-mon[34600]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:23:59.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:59 smithi111 ceph-mon[34600]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-13T16:23:59.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:59 smithi111 ceph-mon[34600]: osdmap e12: 2 total, 1 up, 2 in 2024-09-13T16:23:59.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:59 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T16:23:59.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:59 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T16:23:59.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:59 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:59.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:59 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:59.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:59 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:23:59.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:59 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:59.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:59 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:59.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:59 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:23:59.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:23:59 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:59.735 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T16:23:59.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:59 smithi107 ceph-mon[34139]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:23:59.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:59 smithi107 ceph-mon[34139]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-13T16:23:59.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:59 smithi107 ceph-mon[34139]: osdmap e12: 2 total, 1 up, 2 in 2024-09-13T16:23:59.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:59 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T16:23:59.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:59 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T16:23:59.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:59 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:59.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:59 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:59.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:59 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:23:59.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:59 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:59.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:59 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:59.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:59 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:23:59.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:23:59 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:59.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:59 smithi016 ceph-mon[31187]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:23:59.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:59 smithi016 ceph-mon[31187]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-13T16:23:59.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:59 smithi016 ceph-mon[31187]: osdmap e12: 2 total, 1 up, 2 in 2024-09-13T16:23:59.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:59 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T16:23:59.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:59 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T16:23:59.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:59 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:59.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:59 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:59.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:59 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:23:59.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:59 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:23:59.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:59 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:23:59.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:59 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:23:59.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:23:59 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:00.681 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:00 smithi111 ceph-mon[34600]: purged_snaps scrub starts 2024-09-13T16:24:00.681 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:00 smithi111 ceph-mon[34600]: purged_snaps scrub ok 2024-09-13T16:24:00.681 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:00 smithi111 ceph-mon[34600]: Detected new or changed devices on smithi107 2024-09-13T16:24:00.681 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:00 smithi111 ceph-mon[34600]: Adjusting osd_memory_target on smithi107 to 17087M 2024-09-13T16:24:00.681 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:00 smithi111 ceph-mon[34600]: osd.1 [v2:172.21.15.107:6800/4021360958,v1:172.21.15.107:6801/4021360958] boot 2024-09-13T16:24:00.681 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:00 smithi111 ceph-mon[34600]: osdmap e13: 2 total, 2 up, 2 in 2024-09-13T16:24:00.681 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:00 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T16:24:00.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:00 smithi107 ceph-mon[34139]: purged_snaps scrub starts 2024-09-13T16:24:00.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:00 smithi107 ceph-mon[34139]: purged_snaps scrub ok 2024-09-13T16:24:00.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:00 smithi107 ceph-mon[34139]: Detected new or changed devices on smithi107 2024-09-13T16:24:00.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:00 smithi107 ceph-mon[34139]: Adjusting osd_memory_target on smithi107 to 17087M 2024-09-13T16:24:00.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:00 smithi107 ceph-mon[34139]: osd.1 [v2:172.21.15.107:6800/4021360958,v1:172.21.15.107:6801/4021360958] boot 2024-09-13T16:24:00.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:00 smithi107 ceph-mon[34139]: osdmap e13: 2 total, 2 up, 2 in 2024-09-13T16:24:00.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:00 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T16:24:00.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:00 smithi016 ceph-mon[31187]: purged_snaps scrub starts 2024-09-13T16:24:00.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:00 smithi016 ceph-mon[31187]: purged_snaps scrub ok 2024-09-13T16:24:00.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:00 smithi016 ceph-mon[31187]: Detected new or changed devices on smithi107 2024-09-13T16:24:00.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:00 smithi016 ceph-mon[31187]: Adjusting osd_memory_target on smithi107 to 17087M 2024-09-13T16:24:00.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:00 smithi016 ceph-mon[31187]: osd.1 [v2:172.21.15.107:6800/4021360958,v1:172.21.15.107:6801/4021360958] boot 2024-09-13T16:24:00.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:00 smithi016 ceph-mon[31187]: osdmap e13: 2 total, 2 up, 2 in 2024-09-13T16:24:00.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:00 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T16:24:01.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.165+0000 7f77e4857640 1 -- 172.21.15.111:0/3475822330 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e0100890 msgr2=0x7f77e0102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:24:01.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.165+0000 7f77e4857640 1 --2- 172.21.15.111:0/3475822330 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e0100890 0x7f77e0102c80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f77d0009900 tx=0x7f77d002a410 comp rx=0 tx=0).stop 2024-09-13T16:24:01.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.165+0000 7f77e4857640 1 -- 172.21.15.111:0/3475822330 shutdown_connections 2024-09-13T16:24:01.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.165+0000 7f77e4857640 1 --2- 172.21.15.111:0/3475822330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77e0105af0 0x7f77e0107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:24:01.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.165+0000 7f77e4857640 1 --2- 172.21.15.111:0/3475822330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77e01031c0 0x7f77e01055b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:24:01.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.165+0000 7f77e4857640 1 --2- 172.21.15.111:0/3475822330 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e0100890 0x7f77e0102c80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:24:01.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.165+0000 7f77e4857640 1 -- 172.21.15.111:0/3475822330 >> 172.21.15.111:0/3475822330 conn(0x7f77e00fa490 msgr2=0x7f77e00fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:24:01.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.165+0000 7f77e4857640 1 -- 172.21.15.111:0/3475822330 shutdown_connections 2024-09-13T16:24:01.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.165+0000 7f77e4857640 1 -- 172.21.15.111:0/3475822330 wait complete. 2024-09-13T16:24:01.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.166+0000 7f77e4857640 1 Processor -- start 2024-09-13T16:24:01.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.166+0000 7f77e4857640 1 -- start start 2024-09-13T16:24:01.169 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.166+0000 7f77e4857640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e0100890 0x7f77e0197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:24:01.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.166+0000 7f77e4857640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77e01031c0 0x7f77e01983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:24:01.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.166+0000 7f77e4857640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77e0105af0 0x7f77e019f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:24:01.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.166+0000 7f77e4857640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e019f9b0 con 0x7f77e0105af0 2024-09-13T16:24:01.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.166+0000 7f77e4857640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e019fb20 con 0x7f77e01031c0 2024-09-13T16:24:01.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.166+0000 7f77e4857640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e019fce0 con 0x7f77e0100890 2024-09-13T16:24:01.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.166+0000 7f77de59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e0100890 0x7f77e0197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:24:01.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.167+0000 7f77de59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e0100890 0x7f77e0197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44904/0 (socket says 172.21.15.111:44904) 2024-09-13T16:24:01.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.167+0000 7f77de59c640 1 -- 172.21.15.111:0/516861867 learned_addr learned my addr 172.21.15.111:0/516861867 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T16:24:01.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.167+0000 7f77ded9d640 1 --2- 172.21.15.111:0/516861867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77e0105af0 0x7f77e019f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:24:01.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.167+0000 7f77ddd9b640 1 --2- 172.21.15.111:0/516861867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77e01031c0 0x7f77e01983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:24:01.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.167+0000 7f77de59c640 1 -- 172.21.15.111:0/516861867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77e01031c0 msgr2=0x7f77e01983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:24:01.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.167+0000 7f77de59c640 1 --2- 172.21.15.111:0/516861867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77e01031c0 0x7f77e01983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:24:01.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.167+0000 7f77de59c640 1 -- 172.21.15.111:0/516861867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77e0105af0 msgr2=0x7f77e019f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:24:01.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.167+0000 7f77de59c640 1 --2- 172.21.15.111:0/516861867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77e0105af0 0x7f77e019f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:24:01.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.167+0000 7f77de59c640 1 -- 172.21.15.111:0/516861867 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d0009560 con 0x7f77e0100890 2024-09-13T16:24:01.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.167+0000 7f77de59c640 1 --2- 172.21.15.111:0/516861867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e0100890 0x7f77e0197eb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f77d0009230 tx=0x7f77d0004380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:24:01.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.167+0000 7f77cf7fe640 1 -- 172.21.15.111:0/516861867 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d0031d70 con 0x7f77e0100890 2024-09-13T16:24:01.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.167+0000 7f77cf7fe640 1 -- 172.21.15.111:0/516861867 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f77d0004b50 con 0x7f77e0100890 2024-09-13T16:24:01.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.167+0000 7f77e4857640 1 -- 172.21.15.111:0/516861867 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e019ff60 con 0x7f77e0100890 2024-09-13T16:24:01.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.167+0000 7f77cf7fe640 1 -- 172.21.15.111:0/516861867 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d0003bf0 con 0x7f77e0100890 2024-09-13T16:24:01.171 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.167+0000 7f77e4857640 1 -- 172.21.15.111:0/516861867 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e01a0370 con 0x7f77e0100890 2024-09-13T16:24:01.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.168+0000 7f77e4857640 1 -- 172.21.15.111:0/516861867 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77e0107640 con 0x7f77e0100890 2024-09-13T16:24:01.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.169+0000 7f77cf7fe640 1 -- 172.21.15.111:0/516861867 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 15) v1 ==== 135817+0+0 (secure 0 0 0) 0x7f77d0003d50 con 0x7f77e0100890 2024-09-13T16:24:01.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.170+0000 7f77cf7fe640 1 --2- 172.21.15.111:0/516861867 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f77c80a10a0 0x7f77c80a3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:24:01.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.171+0000 7f77cf7fe640 1 -- 172.21.15.111:0/516861867 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f77d00ec640 con 0x7f77e0100890 2024-09-13T16:24:01.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.171+0000 7f77ddd9b640 1 --2- 172.21.15.111:0/516861867 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f77c80a10a0 0x7f77c80a3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:24:01.172 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.171+0000 7f77ddd9b640 1 --2- 172.21.15.111:0/516861867 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f77c80a10a0 0x7f77c80a3560 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f77d4005e10 tx=0x7f77d4005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:24:01.175 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.174+0000 7f77cf7fe640 1 -- 172.21.15.111:0/516861867 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f77d00b7580 con 0x7f77e0100890 2024-09-13T16:24:01.348 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:01.347+0000 7f77e4857640 1 -- 172.21.15.111:0/516861867 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f77e002c520 con 0x7f77c80a10a0 2024-09-13T16:24:01.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:01 smithi111 ceph-mon[34600]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:24:01.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:01 smithi111 ceph-mon[34600]: osdmap e14: 2 total, 2 up, 2 in 2024-09-13T16:24:01.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:01 smithi111 ceph-mon[34600]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:24:01.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:01 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:24:01.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:01 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:24:01.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:01 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:24:01.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:01 smithi107 ceph-mon[34139]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:24:01.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:01 smithi107 ceph-mon[34139]: osdmap e14: 2 total, 2 up, 2 in 2024-09-13T16:24:01.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:01 smithi107 ceph-mon[34139]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:24:01.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:01 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:24:01.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:01 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:24:01.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:01 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:24:01.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:01 smithi016 ceph-mon[31187]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-13T16:24:01.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:01 smithi016 ceph-mon[31187]: osdmap e14: 2 total, 2 up, 2 in 2024-09-13T16:24:01.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:01 smithi016 ceph-mon[31187]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:24:01.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:01 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:24:01.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:01 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:24:01.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:01 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:24:02.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:02 smithi107 ceph-mon[34139]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:02.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:02 smithi016 ceph-mon[31187]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:02.903 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:02 smithi111 ceph-mon[34600]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:03.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:03 smithi107 ceph-mon[34139]: from='client.? 172.21.15.111:0/4115151069' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5bbf2eb-6599-44d5-ad96-68bb89ee19f4"}]: dispatch 2024-09-13T16:24:03.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:03 smithi107 ceph-mon[34139]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5bbf2eb-6599-44d5-ad96-68bb89ee19f4"}]: dispatch 2024-09-13T16:24:03.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:03 smithi107 ceph-mon[34139]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5bbf2eb-6599-44d5-ad96-68bb89ee19f4"}]': finished 2024-09-13T16:24:03.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:03 smithi107 ceph-mon[34139]: osdmap e15: 3 total, 2 up, 3 in 2024-09-13T16:24:03.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:03 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T16:24:03.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:03 smithi016 ceph-mon[31187]: from='client.? 172.21.15.111:0/4115151069' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5bbf2eb-6599-44d5-ad96-68bb89ee19f4"}]: dispatch 2024-09-13T16:24:03.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:03 smithi016 ceph-mon[31187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5bbf2eb-6599-44d5-ad96-68bb89ee19f4"}]: dispatch 2024-09-13T16:24:03.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:03 smithi016 ceph-mon[31187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5bbf2eb-6599-44d5-ad96-68bb89ee19f4"}]': finished 2024-09-13T16:24:03.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:03 smithi016 ceph-mon[31187]: osdmap e15: 3 total, 2 up, 3 in 2024-09-13T16:24:03.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:03 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T16:24:03.860 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:03 smithi111 ceph-mon[34600]: from='client.? 172.21.15.111:0/4115151069' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5bbf2eb-6599-44d5-ad96-68bb89ee19f4"}]: dispatch 2024-09-13T16:24:03.860 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:03 smithi111 ceph-mon[34600]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5bbf2eb-6599-44d5-ad96-68bb89ee19f4"}]: dispatch 2024-09-13T16:24:03.860 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:03 smithi111 ceph-mon[34600]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5bbf2eb-6599-44d5-ad96-68bb89ee19f4"}]': finished 2024-09-13T16:24:03.860 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:03 smithi111 ceph-mon[34600]: osdmap e15: 3 total, 2 up, 3 in 2024-09-13T16:24:03.861 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:03 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T16:24:04.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:04 smithi107 ceph-mon[34139]: from='client.? 172.21.15.111:0/1267250162' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:24:04.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:04 smithi107 ceph-mon[34139]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:04.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:04 smithi016 ceph-mon[31187]: from='client.? 172.21.15.111:0/1267250162' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:24:04.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:04 smithi016 ceph-mon[31187]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:04.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:04 smithi111 ceph-mon[34600]: from='client.? 172.21.15.111:0/1267250162' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:24:04.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:04 smithi111 ceph-mon[34600]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:06.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:06 smithi111 ceph-mon[34600]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:07.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:06 smithi107 ceph-mon[34139]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:07.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:06 smithi016 ceph-mon[31187]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:08.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:08 smithi111 ceph-mon[34600]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:09.101 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:08 smithi107 ceph-mon[34139]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:09.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:08 smithi016 ceph-mon[31187]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:10.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:10 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-13T16:24:10.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:10 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:24:10.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:10 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-13T16:24:10.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:10 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:24:10.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:10 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-13T16:24:10.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:10 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:24:11.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:11 smithi111 ceph-mon[34600]: Deploying daemon osd.2 on smithi111 2024-09-13T16:24:11.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:11 smithi111 ceph-mon[34600]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:11.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:11 smithi107 ceph-mon[34139]: Deploying daemon osd.2 on smithi111 2024-09-13T16:24:11.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:11 smithi107 ceph-mon[34139]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:11.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:11 smithi016 ceph-mon[31187]: Deploying daemon osd.2 on smithi111 2024-09-13T16:24:11.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:11 smithi016 ceph-mon[31187]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:12.796 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:12 smithi111 ceph-mon[34600]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:13.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:12 smithi107 ceph-mon[34139]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:13.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:12 smithi016 ceph-mon[31187]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:14.721 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:14 smithi111 ceph-mon[34600]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:14.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:14 smithi107 ceph-mon[34139]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:15.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:14 smithi016 ceph-mon[31187]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:15.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:15 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:24:15.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:15 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:15.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:15 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:16.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:15 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:24:16.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:15 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:16.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:15 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:16.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:15 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:24:16.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:15 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:16.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:15 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:16.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:16 smithi111 ceph-mon[34600]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:17.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:16 smithi107 ceph-mon[34139]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:17.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:16 smithi016 ceph-mon[31187]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:18.014 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 2 on host 'smithi111' 2024-09-13T16:24:18.014 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:18.012+0000 7f77cf7fe640 1 -- 172.21.15.111:0/516861867 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f77e002c520 con 0x7f77c80a10a0 2024-09-13T16:24:18.016 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:18.015+0000 7f77e4857640 1 -- 172.21.15.111:0/516861867 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f77c80a10a0 msgr2=0x7f77c80a3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:24:18.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:18.015+0000 7f77e4857640 1 --2- 172.21.15.111:0/516861867 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f77c80a10a0 0x7f77c80a3560 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f77d4005e10 tx=0x7f77d4005da0 comp rx=0 tx=0).stop 2024-09-13T16:24:18.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:18.015+0000 7f77e4857640 1 -- 172.21.15.111:0/516861867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e0100890 msgr2=0x7f77e0197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:24:18.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:18.015+0000 7f77e4857640 1 --2- 172.21.15.111:0/516861867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e0100890 0x7f77e0197eb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f77d0009230 tx=0x7f77d0004380 comp rx=0 tx=0).stop 2024-09-13T16:24:18.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:18.015+0000 7f77e4857640 1 -- 172.21.15.111:0/516861867 shutdown_connections 2024-09-13T16:24:18.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:18.015+0000 7f77e4857640 1 --2- 172.21.15.111:0/516861867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77e0105af0 0x7f77e019f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:24:18.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:18.015+0000 7f77e4857640 1 --2- 172.21.15.111:0/516861867 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f77c80a10a0 0x7f77c80a3560 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:24:18.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:18.015+0000 7f77e4857640 1 --2- 172.21.15.111:0/516861867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77e01031c0 0x7f77e01983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:24:18.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:18.015+0000 7f77e4857640 1 --2- 172.21.15.111:0/516861867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e0100890 0x7f77e0197eb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:24:18.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:18.015+0000 7f77e4857640 1 -- 172.21.15.111:0/516861867 >> 172.21.15.111:0/516861867 conn(0x7f77e00fa490 msgr2=0x7f77e00fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:24:18.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:18.015+0000 7f77e4857640 1 -- 172.21.15.111:0/516861867 shutdown_connections 2024-09-13T16:24:18.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:24:18.015+0000 7f77e4857640 1 -- 172.21.15.111:0/516861867 wait complete. 2024-09-13T16:24:18.566 DEBUG:teuthology.orchestra.run.smithi111:osd.2> sudo journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.2.service 2024-09-13T16:24:18.568 INFO:tasks.cephadm:Deploying osd.3 on smithi016 with /dev/vg_nvme/lv_3... 2024-09-13T16:24:18.569 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-13T16:24:18.784 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:24:18.816 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:18 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:18.816 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:18 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:18.816 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:18 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:24:18.816 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:18 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:24:18.816 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:18 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:18.816 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:18 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:24:18.816 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:18 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:18.816 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:18 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:18.816 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:18 smithi111 ceph-mon[34600]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:18.817 INFO:journalctl@ceph.osd.2.smithi111.stdout:Sep 13 16:24:18 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-2[39620]: 2024-09-13T16:24:18.702+0000 7fb4ad0ad640 -1 osd.2 0 log_to_monitors true 2024-09-13T16:24:18.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:18 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:18.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:18 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:18.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:18 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:24:18.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:18 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:24:18.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:18 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:18.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:18 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:24:18.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:18 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:18.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:18 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:18.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:18 smithi107 ceph-mon[34139]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:18.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:18 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:18.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:18 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:18.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:18 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:24:18.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:18 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:24:18.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:18 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:18.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:18 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:24:18.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:18 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:18.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:18 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:18.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:18 smithi016 ceph-mon[31187]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:19.682 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:19 smithi111 ceph-mon[34600]: from='osd.2 [v2:172.21.15.111:6800/3227179825,v1:172.21.15.111:6801/3227179825]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-13T16:24:19.682 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:19 smithi111 ceph-mon[34600]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-13T16:24:19.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:19 smithi107 ceph-mon[34139]: from='osd.2 [v2:172.21.15.111:6800/3227179825,v1:172.21.15.111:6801/3227179825]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-13T16:24:19.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:19 smithi107 ceph-mon[34139]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-13T16:24:19.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:19 smithi016 ceph-mon[31187]: from='osd.2 [v2:172.21.15.111:6800/3227179825,v1:172.21.15.111:6801/3227179825]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-13T16:24:19.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:19 smithi016 ceph-mon[31187]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-13T16:24:20.627 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-13T16:24:20.706 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-13T16:24:20.718 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-09-13T16:24:20.718 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-09-13T16:24:20.718 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00967058 s, 1.1 GB/s 2024-09-13T16:24:20.718 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-09-13T16:24:20.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:20 smithi107 ceph-mon[34139]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-13T16:24:20.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:20 smithi107 ceph-mon[34139]: osdmap e16: 3 total, 2 up, 3 in 2024-09-13T16:24:20.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:20 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T16:24:20.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:20 smithi107 ceph-mon[34139]: from='osd.2 [v2:172.21.15.111:6800/3227179825,v1:172.21.15.111:6801/3227179825]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-13T16:24:20.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:20 smithi107 ceph-mon[34139]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-13T16:24:20.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:20 smithi107 ceph-mon[34139]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:20.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:20 smithi016 ceph-mon[31187]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-13T16:24:20.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:20 smithi016 ceph-mon[31187]: osdmap e16: 3 total, 2 up, 3 in 2024-09-13T16:24:20.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:20 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T16:24:20.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:20 smithi016 ceph-mon[31187]: from='osd.2 [v2:172.21.15.111:6800/3227179825,v1:172.21.15.111:6801/3227179825]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-13T16:24:20.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:20 smithi016 ceph-mon[31187]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-13T16:24:20.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:20 smithi016 ceph-mon[31187]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:20.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:20 smithi111 ceph-mon[34600]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-13T16:24:20.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:20 smithi111 ceph-mon[34600]: osdmap e16: 3 total, 2 up, 3 in 2024-09-13T16:24:20.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:20 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T16:24:20.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:20 smithi111 ceph-mon[34600]: from='osd.2 [v2:172.21.15.111:6800/3227179825,v1:172.21.15.111:6801/3227179825]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-13T16:24:20.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:20 smithi111 ceph-mon[34600]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-13T16:24:20.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:20 smithi111 ceph-mon[34600]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:21.348 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_3 2024-09-13T16:24:21.365 INFO:journalctl@ceph.osd.2.smithi111.stdout:Sep 13 16:24:21 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-2[39620]: 2024-09-13T16:24:21.011+0000 7fb4a902e640 -1 osd.2 0 waiting for initial osdmap 2024-09-13T16:24:21.365 INFO:journalctl@ceph.osd.2.smithi111.stdout:Sep 13 16:24:21 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-2[39620]: 2024-09-13T16:24:21.015+0000 7fb4a0e4f640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T16:24:21.567 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:24:21.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:21 smithi111 ceph-mon[34600]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-13T16:24:21.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:21 smithi111 ceph-mon[34600]: osdmap e17: 3 total, 2 up, 3 in 2024-09-13T16:24:21.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:21 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T16:24:21.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:21 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T16:24:21.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:21 smithi111 ceph-mon[34600]: from='osd.2 ' entity='osd.2' 2024-09-13T16:24:21.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:21 smithi111 ceph-mon[34600]: Detected new or changed devices on smithi111 2024-09-13T16:24:21.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:21 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:21.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:21 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:21.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:21 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:24:21.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:21 smithi111 ceph-mon[34600]: Adjusting osd_memory_target on smithi111 to 17088M 2024-09-13T16:24:21.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:21 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:21.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:21 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:24:21.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:21 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:24:21.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:21 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:21.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:21 smithi016 ceph-mon[31187]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-13T16:24:21.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:21 smithi016 ceph-mon[31187]: osdmap e17: 3 total, 2 up, 3 in 2024-09-13T16:24:21.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:21 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T16:24:21.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:21 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T16:24:21.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:21 smithi016 ceph-mon[31187]: from='osd.2 ' entity='osd.2' 2024-09-13T16:24:21.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:21 smithi016 ceph-mon[31187]: Detected new or changed devices on smithi111 2024-09-13T16:24:21.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:21 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:21.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:21 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:21.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:21 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:24:21.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:21 smithi016 ceph-mon[31187]: Adjusting osd_memory_target on smithi111 to 17088M 2024-09-13T16:24:21.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:21 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:21.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:21 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:24:21.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:21 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:24:21.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:21 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:22.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:21 smithi107 ceph-mon[34139]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-13T16:24:22.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:21 smithi107 ceph-mon[34139]: osdmap e17: 3 total, 2 up, 3 in 2024-09-13T16:24:22.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:21 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T16:24:22.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:21 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T16:24:22.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:21 smithi107 ceph-mon[34139]: from='osd.2 ' entity='osd.2' 2024-09-13T16:24:22.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:21 smithi107 ceph-mon[34139]: Detected new or changed devices on smithi111 2024-09-13T16:24:22.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:21 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:22.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:21 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:22.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:21 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:24:22.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:21 smithi107 ceph-mon[34139]: Adjusting osd_memory_target on smithi111 to 17088M 2024-09-13T16:24:22.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:21 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:22.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:21 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:24:22.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:21 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:24:22.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:21 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:23.063 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:22 smithi016 ceph-mon[31187]: purged_snaps scrub starts 2024-09-13T16:24:23.063 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:22 smithi016 ceph-mon[31187]: purged_snaps scrub ok 2024-09-13T16:24:23.064 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:22 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T16:24:23.064 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:22 smithi016 ceph-mon[31187]: osd.2 [v2:172.21.15.111:6800/3227179825,v1:172.21.15.111:6801/3227179825] boot 2024-09-13T16:24:23.064 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:22 smithi016 ceph-mon[31187]: osdmap e18: 3 total, 3 up, 3 in 2024-09-13T16:24:23.064 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:22 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T16:24:23.064 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:22 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-13T16:24:23.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.294+0000 7f962075a640 1 -- 172.21.15.16:0/215402644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9618105ad0 msgr2=0x7f9618107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:24:23.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.294+0000 7f962075a640 1 --2- 172.21.15.16:0/215402644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9618105ad0 0x7f9618107f20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9610009900 tx=0x7f961002a410 comp rx=0 tx=0).stop 2024-09-13T16:24:23.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.294+0000 7f962075a640 1 -- 172.21.15.16:0/215402644 shutdown_connections 2024-09-13T16:24:23.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.294+0000 7f962075a640 1 --2- 172.21.15.16:0/215402644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9618105ad0 0x7f9618107f20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:24:23.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.294+0000 7f962075a640 1 --2- 172.21.15.16:0/215402644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96181031a0 0x7f9618105590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:24:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.294+0000 7f962075a640 1 --2- 172.21.15.16:0/215402644 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9618100870 0x7f9618102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:24:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.294+0000 7f962075a640 1 -- 172.21.15.16:0/215402644 >> 172.21.15.16:0/215402644 conn(0x7f96180fa490 msgr2=0x7f96180fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:24:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.294+0000 7f962075a640 1 -- 172.21.15.16:0/215402644 shutdown_connections 2024-09-13T16:24:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.294+0000 7f962075a640 1 -- 172.21.15.16:0/215402644 wait complete. 2024-09-13T16:24:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.295+0000 7f962075a640 1 Processor -- start 2024-09-13T16:24:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.295+0000 7f962075a640 1 -- start start 2024-09-13T16:24:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.295+0000 7f962075a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9618100870 0x7f96181001d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:24:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.295+0000 7f962075a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96181031a0 0x7f96180fe820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:24:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.295+0000 7f962075a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9618105ad0 0x7f96180fed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:24:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.295+0000 7f962075a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96180ff4f0 con 0x7f96181031a0 2024-09-13T16:24:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.295+0000 7f962075a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96180ff660 con 0x7f9618100870 2024-09-13T16:24:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.295+0000 7f962075a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961819fd00 con 0x7f9618105ad0 2024-09-13T16:24:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.295+0000 7f961dcf5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96181031a0 0x7f96180fe820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:24:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.295+0000 7f961dcf5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96181031a0 0x7f96180fe820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37554/0 (socket says 172.21.15.16:37554) 2024-09-13T16:24:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.295+0000 7f961dcf5640 1 -- 172.21.15.16:0/3527902322 learned_addr learned my addr 172.21.15.16:0/3527902322 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:24:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.295+0000 7f961e4f6640 1 --2- 172.21.15.16:0/3527902322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9618100870 0x7f96181001d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:24:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.295+0000 7f961ecf7640 1 --2- 172.21.15.16:0/3527902322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9618105ad0 0x7f96180fed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:24:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.295+0000 7f961dcf5640 1 -- 172.21.15.16:0/3527902322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9618105ad0 msgr2=0x7f96180fed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:24:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.295+0000 7f961dcf5640 1 --2- 172.21.15.16:0/3527902322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9618105ad0 0x7f96180fed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:24:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.295+0000 7f961dcf5640 1 -- 172.21.15.16:0/3527902322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9618100870 msgr2=0x7f96181001d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:24:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.295+0000 7f961dcf5640 1 --2- 172.21.15.16:0/3527902322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9618100870 0x7f96181001d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:24:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.295+0000 7f961dcf5640 1 -- 172.21.15.16:0/3527902322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9610009560 con 0x7f96181031a0 2024-09-13T16:24:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.295+0000 7f961ecf7640 1 --2- 172.21.15.16:0/3527902322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9618105ad0 0x7f96180fed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:24:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.296+0000 7f961dcf5640 1 --2- 172.21.15.16:0/3527902322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96181031a0 0x7f96180fe820 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f960800ea20 tx=0x7f960800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:24:23.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.296+0000 7f960f7fe640 1 -- 172.21.15.16:0/3527902322 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9608009700 con 0x7f96181031a0 2024-09-13T16:24:23.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.296+0000 7f960f7fe640 1 -- 172.21.15.16:0/3527902322 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f960800cd80 con 0x7f96181031a0 2024-09-13T16:24:23.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.296+0000 7f962075a640 1 -- 172.21.15.16:0/3527902322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f961819ffb0 con 0x7f96181031a0 2024-09-13T16:24:23.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.296+0000 7f960f7fe640 1 -- 172.21.15.16:0/3527902322 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9608010430 con 0x7f96181031a0 2024-09-13T16:24:23.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.296+0000 7f962075a640 1 -- 172.21.15.16:0/3527902322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96181a0470 con 0x7f96181031a0 2024-09-13T16:24:23.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.297+0000 7f962075a640 1 -- 172.21.15.16:0/3527902322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96180fbf80 con 0x7f96181031a0 2024-09-13T16:24:23.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.298+0000 7f960f7fe640 1 -- 172.21.15.16:0/3527902322 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 15) v1 ==== 135817+0+0 (secure 0 0 0) 0x7f9608004eb0 con 0x7f96181031a0 2024-09-13T16:24:23.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.299+0000 7f960f7fe640 1 --2- 172.21.15.16:0/3527902322 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f96040a1070 0x7f96040a3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:24:23.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.299+0000 7f961e4f6640 1 --2- 172.21.15.16:0/3527902322 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f96040a1070 0x7f96040a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:24:23.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.299+0000 7f960f7fe640 1 -- 172.21.15.16:0/3527902322 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3226+0+0 (secure 0 0 0) 0x7f9608014070 con 0x7f96181031a0 2024-09-13T16:24:23.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.299+0000 7f961e4f6640 1 --2- 172.21.15.16:0/3527902322 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f96040a1070 0x7f96040a3530 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9614005fd0 tx=0x7f9614009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:24:23.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.302+0000 7f960f7fe640 1 -- 172.21.15.16:0/3527902322 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f96080967d0 con 0x7f96181031a0 2024-09-13T16:24:23.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:22 smithi107 ceph-mon[34139]: purged_snaps scrub starts 2024-09-13T16:24:23.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:22 smithi107 ceph-mon[34139]: purged_snaps scrub ok 2024-09-13T16:24:23.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:22 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T16:24:23.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:22 smithi107 ceph-mon[34139]: osd.2 [v2:172.21.15.111:6800/3227179825,v1:172.21.15.111:6801/3227179825] boot 2024-09-13T16:24:23.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:22 smithi107 ceph-mon[34139]: osdmap e18: 3 total, 3 up, 3 in 2024-09-13T16:24:23.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:22 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T16:24:23.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:22 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-13T16:24:23.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:22 smithi111 ceph-mon[34600]: purged_snaps scrub starts 2024-09-13T16:24:23.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:22 smithi111 ceph-mon[34600]: purged_snaps scrub ok 2024-09-13T16:24:23.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:22 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T16:24:23.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:22 smithi111 ceph-mon[34600]: osd.2 [v2:172.21.15.111:6800/3227179825,v1:172.21.15.111:6801/3227179825] boot 2024-09-13T16:24:23.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:22 smithi111 ceph-mon[34600]: osdmap e18: 3 total, 3 up, 3 in 2024-09-13T16:24:23.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:22 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T16:24:23.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:22 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-13T16:24:23.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:23.485+0000 7f962075a640 1 -- 172.21.15.16:0/3527902322 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f961802c4f0 con 0x7f96040a1070 2024-09-13T16:24:24.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:23 smithi016 ceph-mon[31187]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:24.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:23 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-13T16:24:24.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:23 smithi016 ceph-mon[31187]: osdmap e19: 3 total, 3 up, 3 in 2024-09-13T16:24:24.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:23 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-13T16:24:24.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:23 smithi016 ceph-mon[31187]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:24:24.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:23 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:24:24.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:23 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:24:24.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:23 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:24:24.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:23 smithi107 ceph-mon[34139]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:24.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:23 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-13T16:24:24.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:23 smithi107 ceph-mon[34139]: osdmap e19: 3 total, 3 up, 3 in 2024-09-13T16:24:24.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:23 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-13T16:24:24.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:23 smithi107 ceph-mon[34139]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:24:24.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:23 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:24:24.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:23 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:24:24.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:23 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:24:24.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:23 smithi111 ceph-mon[34600]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-13T16:24:24.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:23 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-13T16:24:24.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:23 smithi111 ceph-mon[34600]: osdmap e19: 3 total, 3 up, 3 in 2024-09-13T16:24:24.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:23 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-13T16:24:24.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:23 smithi111 ceph-mon[34600]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:24:24.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:23 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:24:24.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:23 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:24:24.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:23 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:24:24.767 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 13 16:24:24 smithi016 sudo[50164]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-13T16:24:24.767 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 13 16:24:24 smithi016 sudo[50164]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T16:24:24.768 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 13 16:24:24 smithi016 sudo[50164]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T16:24:24.768 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 13 16:24:24 smithi016 sudo[50164]: pam_unix(sudo:session): session closed for user root 2024-09-13T16:24:25.028 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 13 16:24:25 smithi016 sudo[50192]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-13T16:24:25.028 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 13 16:24:25 smithi016 sudo[50192]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T16:24:25.028 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 13 16:24:25 smithi016 sudo[50192]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T16:24:25.029 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 13 16:24:25 smithi016 sudo[50192]: pam_unix(sudo:session): session closed for user root 2024-09-13T16:24:25.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:25.032+0000 7f960f7fe640 1 -- 172.21.15.16:0/3527902322 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f9608096ba0 con 0x7f96181031a0 2024-09-13T16:24:25.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:25 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-13T16:24:25.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:25 smithi107 ceph-mon[34139]: osdmap e20: 3 total, 3 up, 3 in 2024-09-13T16:24:25.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:25 smithi107 ceph-mon[34139]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-13T16:24:25.351 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:25 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-13T16:24:25.351 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:25 smithi111 ceph-mon[34600]: osdmap e20: 3 total, 3 up, 3 in 2024-09-13T16:24:25.351 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:25 smithi111 ceph-mon[34600]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-13T16:24:25.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:25 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-13T16:24:25.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:25 smithi016 ceph-mon[31187]: osdmap e20: 3 total, 3 up, 3 in 2024-09-13T16:24:25.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:25 smithi016 ceph-mon[31187]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-13T16:24:25.546 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 13 16:24:25 smithi107 sudo[42296]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-13T16:24:25.546 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 13 16:24:25 smithi107 sudo[42296]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T16:24:25.546 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 13 16:24:25 smithi107 sudo[42296]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T16:24:25.546 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 13 16:24:25 smithi107 sudo[42296]: pam_unix(sudo:session): session closed for user root 2024-09-13T16:24:25.849 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 13 16:24:25 smithi107 sudo[42299]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-13T16:24:25.849 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 13 16:24:25 smithi107 sudo[42299]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T16:24:25.849 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 13 16:24:25 smithi107 sudo[42299]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T16:24:25.849 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 13 16:24:25 smithi107 sudo[42299]: pam_unix(sudo:session): session closed for user root 2024-09-13T16:24:26.080 INFO:journalctl@ceph.osd.2.smithi111.stdout:Sep 13 16:24:25 smithi111 sudo[43640]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-13T16:24:26.080 INFO:journalctl@ceph.osd.2.smithi111.stdout:Sep 13 16:24:25 smithi111 sudo[43640]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T16:24:26.080 INFO:journalctl@ceph.osd.2.smithi111.stdout:Sep 13 16:24:25 smithi111 sudo[43640]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T16:24:26.080 INFO:journalctl@ceph.osd.2.smithi111.stdout:Sep 13 16:24:25 smithi111 sudo[43640]: pam_unix(sudo:session): session closed for user root 2024-09-13T16:24:26.080 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:26 smithi111 ceph-mon[34600]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-13T16:24:26.081 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:26 smithi111 ceph-mon[34600]: osdmap e21: 3 total, 3 up, 3 in 2024-09-13T16:24:26.081 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:26 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:26.081 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:26 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:26.081 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:26 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/785603934' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6295ffe-11bc-4964-a250-9fecca870b97"}]: dispatch 2024-09-13T16:24:26.081 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:26 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/785603934' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6295ffe-11bc-4964-a250-9fecca870b97"}]': finished 2024-09-13T16:24:26.081 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:26 smithi111 ceph-mon[34600]: osdmap e22: 4 total, 3 up, 4 in 2024-09-13T16:24:26.081 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:26 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T16:24:26.301 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:26 smithi016 ceph-mon[31187]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-13T16:24:26.302 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:26 smithi016 ceph-mon[31187]: osdmap e21: 3 total, 3 up, 3 in 2024-09-13T16:24:26.302 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:26 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:26.302 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:26 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:26.302 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:26 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/785603934' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6295ffe-11bc-4964-a250-9fecca870b97"}]: dispatch 2024-09-13T16:24:26.302 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:26 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/785603934' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6295ffe-11bc-4964-a250-9fecca870b97"}]': finished 2024-09-13T16:24:26.302 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:26 smithi016 ceph-mon[31187]: osdmap e22: 4 total, 3 up, 4 in 2024-09-13T16:24:26.302 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:26 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T16:24:26.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:26 smithi107 ceph-mon[34139]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-13T16:24:26.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:26 smithi107 ceph-mon[34139]: osdmap e21: 3 total, 3 up, 3 in 2024-09-13T16:24:26.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:26 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:26.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:26 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:26.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:26 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/785603934' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6295ffe-11bc-4964-a250-9fecca870b97"}]: dispatch 2024-09-13T16:24:26.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:26 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/785603934' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6295ffe-11bc-4964-a250-9fecca870b97"}]': finished 2024-09-13T16:24:26.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:26 smithi107 ceph-mon[34139]: osdmap e22: 4 total, 3 up, 4 in 2024-09-13T16:24:26.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:26 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T16:24:26.437 INFO:journalctl@ceph.osd.2.smithi111.stdout:Sep 13 16:24:26 smithi111 sudo[43643]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-13T16:24:26.437 INFO:journalctl@ceph.osd.2.smithi111.stdout:Sep 13 16:24:26 smithi111 sudo[43643]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T16:24:26.437 INFO:journalctl@ceph.osd.2.smithi111.stdout:Sep 13 16:24:26 smithi111 sudo[43643]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T16:24:26.437 INFO:journalctl@ceph.osd.2.smithi111.stdout:Sep 13 16:24:26 smithi111 sudo[43643]: pam_unix(sudo:session): session closed for user root 2024-09-13T16:24:26.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:26 smithi016 sudo[50275]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-13T16:24:26.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:26 smithi016 sudo[50275]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T16:24:26.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:26 smithi016 sudo[50275]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T16:24:26.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:26 smithi016 sudo[50275]: pam_unix(sudo:session): session closed for user root 2024-09-13T16:24:27.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:26 smithi016 sudo[50278]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-13T16:24:27.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:26 smithi016 sudo[50278]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T16:24:27.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:26 smithi016 sudo[50278]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T16:24:27.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:26 smithi016 sudo[50278]: pam_unix(sudo:session): session closed for user root 2024-09-13T16:24:27.493 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:27 smithi111 sudo[43646]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-13T16:24:27.493 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:27 smithi111 sudo[43646]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T16:24:27.493 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:27 smithi111 sudo[43646]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T16:24:27.493 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:27 smithi111 sudo[43646]: pam_unix(sudo:session): session closed for user root 2024-09-13T16:24:27.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:27 smithi111 sudo[43649]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-13T16:24:27.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:27 smithi111 sudo[43649]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T16:24:27.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:27 smithi111 sudo[43649]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T16:24:27.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:27 smithi111 sudo[43649]: pam_unix(sudo:session): session closed for user root 2024-09-13T16:24:27.995 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:27 smithi107 sudo[42302]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-13T16:24:27.995 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:27 smithi107 sudo[42302]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T16:24:27.995 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:27 smithi107 sudo[42302]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T16:24:27.996 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:27 smithi107 sudo[42302]: pam_unix(sudo:session): session closed for user root 2024-09-13T16:24:28.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:27 smithi107 sudo[42305]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-13T16:24:28.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:27 smithi107 sudo[42305]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T16:24:28.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:27 smithi107 sudo[42305]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T16:24:28.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:27 smithi107 sudo[42305]: pam_unix(sudo:session): session closed for user root 2024-09-13T16:24:28.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:28 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:28.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:28 smithi107 ceph-mon[34139]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T16:24:28.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:28 smithi107 ceph-mon[34139]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-13T16:24:28.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:28 smithi107 ceph-mon[34139]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T16:24:28.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:28 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T16:24:28.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:28 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1229839818' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:24:28.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:28 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T16:24:28.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:28 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T16:24:28.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:28 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T16:24:28.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:28 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T16:24:28.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:28 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T16:24:28.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:28 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:28.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:28 smithi016 ceph-mon[31187]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T16:24:28.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:28 smithi016 ceph-mon[31187]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-13T16:24:28.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:28 smithi016 ceph-mon[31187]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T16:24:28.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:28 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T16:24:28.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:28 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1229839818' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:24:28.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:28 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T16:24:28.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:28 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T16:24:28.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:28 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T16:24:28.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:28 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T16:24:28.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:28 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T16:24:28.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:28 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:28.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:28 smithi111 ceph-mon[34600]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T16:24:28.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:28 smithi111 ceph-mon[34600]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-13T16:24:28.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:28 smithi111 ceph-mon[34600]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T16:24:28.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:28 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T16:24:28.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:28 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1229839818' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:24:28.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:28 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T16:24:28.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:28 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T16:24:28.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:28 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T16:24:28.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:28 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T16:24:28.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:28 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T16:24:29.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:29 smithi107 ceph-mon[34139]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T16:24:29.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:29 smithi107 ceph-mon[34139]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T16:24:29.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:29 smithi107 ceph-mon[34139]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T16:24:29.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:29 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T16:24:29.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:29 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T16:24:29.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:29 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T16:24:29.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:29 smithi107 ceph-mon[34139]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T16:24:29.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:29 smithi107 ceph-mon[34139]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-13T16:24:29.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:29 smithi016 ceph-mon[31187]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T16:24:29.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:29 smithi016 ceph-mon[31187]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T16:24:29.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:29 smithi016 ceph-mon[31187]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T16:24:29.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:29 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T16:24:29.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:29 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T16:24:29.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:29 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T16:24:29.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:29 smithi016 ceph-mon[31187]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T16:24:29.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:29 smithi016 ceph-mon[31187]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-13T16:24:29.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:29 smithi111 ceph-mon[34600]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T16:24:29.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:29 smithi111 ceph-mon[34600]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T16:24:29.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:29 smithi111 ceph-mon[34600]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T16:24:29.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:29 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T16:24:29.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:29 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T16:24:29.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:29 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T16:24:29.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:29 smithi111 ceph-mon[34600]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T16:24:29.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:29 smithi111 ceph-mon[34600]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-13T16:24:30.728 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:30 smithi016 ceph-mon[31187]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-13T16:24:30.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:30 smithi107 ceph-mon[34139]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-13T16:24:30.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:30 smithi111 ceph-mon[34600]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-13T16:24:32.619 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:32 smithi016 ceph-mon[31187]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 4.0 KiB/s rd, 192 KiB/s wr, 12 op/s 2024-09-13T16:24:32.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:32 smithi111 ceph-mon[34600]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 4.0 KiB/s rd, 192 KiB/s wr, 12 op/s 2024-09-13T16:24:33.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:32 smithi107 ceph-mon[34139]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 4.0 KiB/s rd, 192 KiB/s wr, 12 op/s 2024-09-13T16:24:33.898 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:33 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-13T16:24:33.898 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:33 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:24:33.898 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:33 smithi016 ceph-mon[31187]: Deploying daemon osd.3 on smithi016 2024-09-13T16:24:33.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:33 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-13T16:24:33.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:33 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:24:33.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:33 smithi111 ceph-mon[34600]: Deploying daemon osd.3 on smithi016 2024-09-13T16:24:34.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:33 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-13T16:24:34.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:33 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:24:34.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:33 smithi107 ceph-mon[34139]: Deploying daemon osd.3 on smithi016 2024-09-13T16:24:34.747 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:34 smithi016 ceph-mon[31187]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.1 KiB/s rd, 149 KiB/s wr, 10 op/s 2024-09-13T16:24:34.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:34 smithi111 ceph-mon[34600]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.1 KiB/s rd, 149 KiB/s wr, 10 op/s 2024-09-13T16:24:35.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:34 smithi107 ceph-mon[34139]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.1 KiB/s rd, 149 KiB/s wr, 10 op/s 2024-09-13T16:24:36.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:36 smithi016 ceph-mon[31187]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 121 KiB/s wr, 8 op/s 2024-09-13T16:24:36.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:36 smithi111 ceph-mon[34600]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 121 KiB/s wr, 8 op/s 2024-09-13T16:24:37.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:36 smithi107 ceph-mon[34139]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 121 KiB/s wr, 8 op/s 2024-09-13T16:24:38.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:38 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:24:38.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:38 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:38.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:38 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:38.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:38 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:24:38.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:38 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:38.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:38 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:38.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:38 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:24:38.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:38 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:38.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:38 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:39.847 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:39 smithi016 ceph-mon[31187]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-09-13T16:24:39.848 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:39 smithi107 ceph-mon[34139]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-09-13T16:24:39.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:39 smithi111 ceph-mon[34600]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-09-13T16:24:41.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:41 smithi016 ceph-mon[31187]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-09-13T16:24:41.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:41.714+0000 7f960f7fe640 1 -- 172.21.15.16:0/3527902322 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f961802c4f0 con 0x7f96040a1070 2024-09-13T16:24:41.716 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 3 on host 'smithi016' 2024-09-13T16:24:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:41.717+0000 7f962075a640 1 -- 172.21.15.16:0/3527902322 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f96040a1070 msgr2=0x7f96040a3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:24:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:41.717+0000 7f962075a640 1 --2- 172.21.15.16:0/3527902322 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f96040a1070 0x7f96040a3530 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9614005fd0 tx=0x7f9614009500 comp rx=0 tx=0).stop 2024-09-13T16:24:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:41.717+0000 7f962075a640 1 -- 172.21.15.16:0/3527902322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96181031a0 msgr2=0x7f96180fe820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:24:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:41.717+0000 7f962075a640 1 --2- 172.21.15.16:0/3527902322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96181031a0 0x7f96180fe820 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f960800ea20 tx=0x7f960800eee0 comp rx=0 tx=0).stop 2024-09-13T16:24:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:41.717+0000 7f962075a640 1 -- 172.21.15.16:0/3527902322 shutdown_connections 2024-09-13T16:24:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:41.717+0000 7f962075a640 1 --2- 172.21.15.16:0/3527902322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9618105ad0 0x7f96180fed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:24:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:41.717+0000 7f962075a640 1 --2- 172.21.15.16:0/3527902322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96181031a0 0x7f96180fe820 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:24:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:41.717+0000 7f962075a640 1 --2- 172.21.15.16:0/3527902322 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f96040a1070 0x7f96040a3530 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:24:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:41.717+0000 7f962075a640 1 --2- 172.21.15.16:0/3527902322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9618100870 0x7f96181001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:24:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:41.717+0000 7f962075a640 1 -- 172.21.15.16:0/3527902322 >> 172.21.15.16:0/3527902322 conn(0x7f96180fa490 msgr2=0x7f9618101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:24:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:41.717+0000 7f962075a640 1 -- 172.21.15.16:0/3527902322 shutdown_connections 2024-09-13T16:24:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:24:41.717+0000 7f962075a640 1 -- 172.21.15.16:0/3527902322 wait complete. 2024-09-13T16:24:41.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:41 smithi107 ceph-mon[34139]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-09-13T16:24:41.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:41 smithi111 ceph-mon[34600]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-09-13T16:24:42.350 DEBUG:teuthology.orchestra.run.smithi016:osd.3> sudo journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.3.service 2024-09-13T16:24:42.352 INFO:tasks.cephadm:Deploying osd.4 on smithi107 with /dev/vg_nvme/lv_3... 2024-09-13T16:24:42.352 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-13T16:24:42.559 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T16:24:42.999 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:42 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:43.000 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:42 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:43.000 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:42 smithi016 ceph-mon[31187]: from='osd.3 [v2:172.21.15.16:6810/2390375937,v1:172.21.15.16:6811/2390375937]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-13T16:24:43.000 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:42 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:43.000 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:42 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:43.000 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:42 smithi016 ceph-mon[31187]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-09-13T16:24:43.016 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:42 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:43.016 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:42 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:43.016 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:42 smithi107 ceph-mon[34139]: from='osd.3 [v2:172.21.15.16:6810/2390375937,v1:172.21.15.16:6811/2390375937]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-13T16:24:43.016 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:42 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:43.016 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:42 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:43.016 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:42 smithi107 ceph-mon[34139]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-09-13T16:24:43.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:42 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:43.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:42 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:43.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:42 smithi111 ceph-mon[34600]: from='osd.3 [v2:172.21.15.16:6810/2390375937,v1:172.21.15.16:6811/2390375937]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-13T16:24:43.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:42 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:43.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:42 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:43.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:42 smithi111 ceph-mon[34600]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-09-13T16:24:43.737 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:43 smithi016 ceph-mon[31187]: from='osd.3 [v2:172.21.15.16:6810/2390375937,v1:172.21.15.16:6811/2390375937]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-13T16:24:43.737 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:43 smithi016 ceph-mon[31187]: osdmap e23: 4 total, 3 up, 4 in 2024-09-13T16:24:43.737 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:43 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T16:24:43.737 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:43 smithi016 ceph-mon[31187]: from='osd.3 [v2:172.21.15.16:6810/2390375937,v1:172.21.15.16:6811/2390375937]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-13T16:24:43.989 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:43 smithi107 ceph-mon[34139]: from='osd.3 [v2:172.21.15.16:6810/2390375937,v1:172.21.15.16:6811/2390375937]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-13T16:24:43.989 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:43 smithi107 ceph-mon[34139]: osdmap e23: 4 total, 3 up, 4 in 2024-09-13T16:24:43.990 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:43 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T16:24:43.990 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:43 smithi107 ceph-mon[34139]: from='osd.3 [v2:172.21.15.16:6810/2390375937,v1:172.21.15.16:6811/2390375937]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-13T16:24:44.088 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:43 smithi111 ceph-mon[34600]: from='osd.3 [v2:172.21.15.16:6810/2390375937,v1:172.21.15.16:6811/2390375937]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-13T16:24:44.088 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:43 smithi111 ceph-mon[34600]: osdmap e23: 4 total, 3 up, 4 in 2024-09-13T16:24:44.089 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:43 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T16:24:44.089 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:43 smithi111 ceph-mon[34600]: from='osd.3 [v2:172.21.15.16:6810/2390375937,v1:172.21.15.16:6811/2390375937]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-13T16:24:44.355 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 13 16:24:44 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-3[52258]: 2024-09-13T16:24:44.180+0000 7f669012d640 -1 osd.3 0 waiting for initial osdmap 2024-09-13T16:24:44.355 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 13 16:24:44 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-3[52258]: 2024-09-13T16:24:44.187+0000 7f6687f4e640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T16:24:44.359 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-13T16:24:44.446 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-13T16:24:44.460 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-09-13T16:24:44.460 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-09-13T16:24:44.460 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0117496 s, 892 MB/s 2024-09-13T16:24:44.460 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-09-13T16:24:45.001 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:44 smithi107 ceph-mon[34139]: from='osd.3 [v2:172.21.15.16:6810/2390375937,v1:172.21.15.16:6811/2390375937]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-13T16:24:45.001 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:44 smithi107 ceph-mon[34139]: osdmap e24: 4 total, 3 up, 4 in 2024-09-13T16:24:45.001 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:44 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T16:24:45.001 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:44 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T16:24:45.001 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:44 smithi107 ceph-mon[34139]: from='osd.3 [v2:172.21.15.16:6810/2390375937,v1:172.21.15.16:6811/2390375937]' entity='osd.3' 2024-09-13T16:24:45.001 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:44 smithi107 ceph-mon[34139]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-13T16:24:45.001 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:44 smithi107 ceph-mon[34139]: osd.3 [v2:172.21.15.16:6810/2390375937,v1:172.21.15.16:6811/2390375937] boot 2024-09-13T16:24:45.001 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:44 smithi107 ceph-mon[34139]: osdmap e25: 4 total, 4 up, 4 in 2024-09-13T16:24:45.002 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:44 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T16:24:45.022 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_3 2024-09-13T16:24:45.085 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:44 smithi016 ceph-mon[31187]: from='osd.3 [v2:172.21.15.16:6810/2390375937,v1:172.21.15.16:6811/2390375937]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-13T16:24:45.085 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:44 smithi016 ceph-mon[31187]: osdmap e24: 4 total, 3 up, 4 in 2024-09-13T16:24:45.085 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:44 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T16:24:45.085 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:44 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T16:24:45.086 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:44 smithi016 ceph-mon[31187]: from='osd.3 [v2:172.21.15.16:6810/2390375937,v1:172.21.15.16:6811/2390375937]' entity='osd.3' 2024-09-13T16:24:45.086 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:44 smithi016 ceph-mon[31187]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-13T16:24:45.086 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:44 smithi016 ceph-mon[31187]: osd.3 [v2:172.21.15.16:6810/2390375937,v1:172.21.15.16:6811/2390375937] boot 2024-09-13T16:24:45.086 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:44 smithi016 ceph-mon[31187]: osdmap e25: 4 total, 4 up, 4 in 2024-09-13T16:24:45.086 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:44 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T16:24:45.186 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:44 smithi111 ceph-mon[34600]: from='osd.3 [v2:172.21.15.16:6810/2390375937,v1:172.21.15.16:6811/2390375937]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-13T16:24:45.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:44 smithi111 ceph-mon[34600]: osdmap e24: 4 total, 3 up, 4 in 2024-09-13T16:24:45.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:44 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T16:24:45.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:44 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T16:24:45.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:44 smithi111 ceph-mon[34600]: from='osd.3 [v2:172.21.15.16:6810/2390375937,v1:172.21.15.16:6811/2390375937]' entity='osd.3' 2024-09-13T16:24:45.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:44 smithi111 ceph-mon[34600]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-13T16:24:45.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:44 smithi111 ceph-mon[34600]: osd.3 [v2:172.21.15.16:6810/2390375937,v1:172.21.15.16:6811/2390375937] boot 2024-09-13T16:24:45.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:44 smithi111 ceph-mon[34600]: osdmap e25: 4 total, 4 up, 4 in 2024-09-13T16:24:45.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:44 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T16:24:45.216 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T16:24:46.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:46 smithi107 ceph-mon[34139]: purged_snaps scrub starts 2024-09-13T16:24:46.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:46 smithi107 ceph-mon[34139]: purged_snaps scrub ok 2024-09-13T16:24:46.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:46 smithi107 ceph-mon[34139]: Detected new or changed devices on smithi016 2024-09-13T16:24:46.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:46 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:46.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:46 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:46.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:46 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:24:46.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:46 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:24:46.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:46 smithi107 ceph-mon[34139]: Adjusting osd_memory_target on smithi016 to 8546M 2024-09-13T16:24:46.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:46 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:46.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:46 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:24:46.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:46 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:24:46.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:46 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:46.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:46 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:24:46.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:46 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:24:46.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:46 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:24:46.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:46 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:46.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:46 smithi107 ceph-mon[34139]: osdmap e26: 4 total, 4 up, 4 in 2024-09-13T16:24:46.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:46 smithi111 ceph-mon[34600]: purged_snaps scrub starts 2024-09-13T16:24:46.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:46 smithi111 ceph-mon[34600]: purged_snaps scrub ok 2024-09-13T16:24:46.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:46 smithi111 ceph-mon[34600]: Detected new or changed devices on smithi016 2024-09-13T16:24:46.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:46 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:46.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:46 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:46.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:46 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:24:46.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:46 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:24:46.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:46 smithi111 ceph-mon[34600]: Adjusting osd_memory_target on smithi016 to 8546M 2024-09-13T16:24:46.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:46 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:46.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:46 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:24:46.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:46 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:24:46.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:46 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:46.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:46 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:24:46.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:46 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:24:46.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:46 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:24:46.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:46 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:46.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:46 smithi111 ceph-mon[34600]: osdmap e26: 4 total, 4 up, 4 in 2024-09-13T16:24:46.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:46 smithi016 ceph-mon[31187]: purged_snaps scrub starts 2024-09-13T16:24:46.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:46 smithi016 ceph-mon[31187]: purged_snaps scrub ok 2024-09-13T16:24:46.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:46 smithi016 ceph-mon[31187]: Detected new or changed devices on smithi016 2024-09-13T16:24:46.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:46 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:46.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:46 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:46.606 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:46 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:24:46.606 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:46 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:24:46.606 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:46 smithi016 ceph-mon[31187]: Adjusting osd_memory_target on smithi016 to 8546M 2024-09-13T16:24:46.606 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:46 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:46.606 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:46 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:24:46.606 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:46 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:24:46.606 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:46 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:46.606 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:46 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:24:46.607 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:46 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:24:46.607 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:46 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:24:46.607 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:46 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:24:46.607 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:46 smithi016 ceph-mon[31187]: osdmap e26: 4 total, 4 up, 4 in 2024-09-13T16:24:46.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.791+0000 7f78d93dd640 1 -- 172.21.15.107:0/2017230592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78d4103250 msgr2=0x7f78d4105640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:24:46.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.791+0000 7f78d93dd640 1 --2- 172.21.15.107:0/2017230592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78d4103250 0x7f78d4105640 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009900 tx=0x7f78c802a410 comp rx=0 tx=0).stop 2024-09-13T16:24:46.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.791+0000 7f78d93dd640 1 -- 172.21.15.107:0/2017230592 shutdown_connections 2024-09-13T16:24:46.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.791+0000 7f78d93dd640 1 --2- 172.21.15.107:0/2017230592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78d4105b80 0x7f78d4107f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:24:46.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.791+0000 7f78d93dd640 1 --2- 172.21.15.107:0/2017230592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78d4103250 0x7f78d4105640 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:24:46.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.791+0000 7f78d93dd640 1 --2- 172.21.15.107:0/2017230592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78d4100920 0x7f78d4102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:24:46.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.791+0000 7f78d93dd640 1 -- 172.21.15.107:0/2017230592 >> 172.21.15.107:0/2017230592 conn(0x7f78d40fa4c0 msgr2=0x7f78d40fc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:24:46.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.791+0000 7f78d93dd640 1 -- 172.21.15.107:0/2017230592 shutdown_connections 2024-09-13T16:24:46.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.791+0000 7f78d93dd640 1 -- 172.21.15.107:0/2017230592 wait complete. 2024-09-13T16:24:46.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.792+0000 7f78d93dd640 1 Processor -- start 2024-09-13T16:24:46.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.792+0000 7f78d93dd640 1 -- start start 2024-09-13T16:24:46.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.792+0000 7f78d93dd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78d4100920 0x7f78d4100280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:24:46.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.792+0000 7f78d93dd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78d4103250 0x7f78d40fe8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:24:46.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.792+0000 7f78d93dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78d4105b80 0x7f78d40fee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:24:46.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.792+0000 7f78d93dd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d40ff5a0 con 0x7f78d4105b80 2024-09-13T16:24:46.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.792+0000 7f78d93dd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d40ff710 con 0x7f78d4103250 2024-09-13T16:24:46.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.792+0000 7f78d93dd640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d419fea0 con 0x7f78d4100920 2024-09-13T16:24:46.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.792+0000 7f78d27fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78d4103250 0x7f78d40fe8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:24:46.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.793+0000 7f78d27fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78d4103250 0x7f78d40fe8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45098/0 (socket says 172.21.15.107:45098) 2024-09-13T16:24:46.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.793+0000 7f78d27fc640 1 -- 172.21.15.107:0/2454620308 learned_addr learned my addr 172.21.15.107:0/2454620308 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-13T16:24:46.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.793+0000 7f78d2ffd640 1 --2- 172.21.15.107:0/2454620308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78d4100920 0x7f78d4100280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:24:46.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.793+0000 7f78d37fe640 1 --2- 172.21.15.107:0/2454620308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78d4105b80 0x7f78d40fee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:24:46.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.793+0000 7f78d27fc640 1 -- 172.21.15.107:0/2454620308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78d4100920 msgr2=0x7f78d4100280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:24:46.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.793+0000 7f78d27fc640 1 --2- 172.21.15.107:0/2454620308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78d4100920 0x7f78d4100280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:24:46.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.793+0000 7f78d27fc640 1 -- 172.21.15.107:0/2454620308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78d4105b80 msgr2=0x7f78d40fee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:24:46.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.793+0000 7f78d27fc640 1 --2- 172.21.15.107:0/2454620308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78d4105b80 0x7f78d40fee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:24:46.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.793+0000 7f78d27fc640 1 -- 172.21.15.107:0/2454620308 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c8009560 con 0x7f78d4103250 2024-09-13T16:24:46.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.793+0000 7f78d37fe640 1 --2- 172.21.15.107:0/2454620308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78d4105b80 0x7f78d40fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:24:46.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.793+0000 7f78d27fc640 1 --2- 172.21.15.107:0/2454620308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78d4103250 0x7f78d40fe8d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f78c80055e0 tx=0x7f78c80048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:24:46.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.793+0000 7f78bbfff640 1 -- 172.21.15.107:0/2454620308 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c80035f0 con 0x7f78d4103250 2024-09-13T16:24:46.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.793+0000 7f78d93dd640 1 -- 172.21.15.107:0/2454620308 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d41a00f0 con 0x7f78d4103250 2024-09-13T16:24:46.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.793+0000 7f78bbfff640 1 -- 172.21.15.107:0/2454620308 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f78c8003be0 con 0x7f78d4103250 2024-09-13T16:24:46.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.793+0000 7f78bbfff640 1 -- 172.21.15.107:0/2454620308 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c803cbd0 con 0x7f78d4103250 2024-09-13T16:24:46.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.793+0000 7f78d93dd640 1 -- 172.21.15.107:0/2454620308 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d41a0660 con 0x7f78d4103250 2024-09-13T16:24:46.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.794+0000 7f78d93dd640 1 -- 172.21.15.107:0/2454620308 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78d40fbfb0 con 0x7f78d4103250 2024-09-13T16:24:46.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.795+0000 7f78bbfff640 1 -- 172.21.15.107:0/2454620308 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f78c8003750 con 0x7f78d4103250 2024-09-13T16:24:46.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.796+0000 7f78bbfff640 1 --2- 172.21.15.107:0/2454620308 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f78bc0a1080 0x7f78bc0a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:24:46.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.796+0000 7f78bbfff640 1 -- 172.21.15.107:0/2454620308 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f78c80eca20 con 0x7f78d4103250 2024-09-13T16:24:46.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.796+0000 7f78d2ffd640 1 --2- 172.21.15.107:0/2454620308 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f78bc0a1080 0x7f78bc0a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:24:46.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.796+0000 7f78d2ffd640 1 --2- 172.21.15.107:0/2454620308 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f78bc0a1080 0x7f78bc0a3540 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f78c4009890 tx=0x7f78c4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:24:46.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.800+0000 7f78bbfff640 1 -- 172.21.15.107:0/2454620308 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f78c80b73b0 con 0x7f78d4103250 2024-09-13T16:24:46.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:24:46.972+0000 7f78d93dd640 1 -- 172.21.15.107:0/2454620308 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f78d40608f0 con 0x7f78bc0a1080 2024-09-13T16:24:47.328 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:47 smithi107 ceph-mon[34139]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-13T16:24:47.328 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:47 smithi107 ceph-mon[34139]: osdmap e27: 4 total, 4 up, 4 in 2024-09-13T16:24:47.328 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:47 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:24:47.328 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:47 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:24:47.328 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:47 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:24:47.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:47 smithi016 ceph-mon[31187]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-13T16:24:47.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:47 smithi016 ceph-mon[31187]: osdmap e27: 4 total, 4 up, 4 in 2024-09-13T16:24:47.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:47 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:24:47.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:47 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:24:47.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:47 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:24:47.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:47 smithi111 ceph-mon[34600]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-13T16:24:47.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:47 smithi111 ceph-mon[34600]: osdmap e27: 4 total, 4 up, 4 in 2024-09-13T16:24:47.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:47 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:24:47.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:47 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:24:47.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:47 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:24:48.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:48 smithi107 ceph-mon[34139]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:24:48.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:48 smithi107 ceph-mon[34139]: osdmap e28: 4 total, 4 up, 4 in 2024-09-13T16:24:48.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:48 smithi016 ceph-mon[31187]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:24:48.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:48 smithi016 ceph-mon[31187]: osdmap e28: 4 total, 4 up, 4 in 2024-09-13T16:24:48.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:48 smithi111 ceph-mon[34600]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:24:48.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:48 smithi111 ceph-mon[34600]: osdmap e28: 4 total, 4 up, 4 in 2024-09-13T16:24:49.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:49 smithi107 ceph-mon[34139]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-13T16:24:49.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:49 smithi107 ceph-mon[34139]: osdmap e29: 4 total, 4 up, 4 in 2024-09-13T16:24:49.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:49 smithi016 ceph-mon[31187]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-13T16:24:49.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:49 smithi016 ceph-mon[31187]: osdmap e29: 4 total, 4 up, 4 in 2024-09-13T16:24:49.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:49 smithi111 ceph-mon[34600]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-13T16:24:49.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:49 smithi111 ceph-mon[34600]: osdmap e29: 4 total, 4 up, 4 in 2024-09-13T16:24:50.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:50 smithi107 ceph-mon[34139]: from='client.? 172.21.15.107:0/215581620' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a8446e0-fc86-4646-beb6-a7b9d433093d"}]: dispatch 2024-09-13T16:24:50.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:50 smithi107 ceph-mon[34139]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a8446e0-fc86-4646-beb6-a7b9d433093d"}]: dispatch 2024-09-13T16:24:50.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:50 smithi107 ceph-mon[34139]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a8446e0-fc86-4646-beb6-a7b9d433093d"}]': finished 2024-09-13T16:24:50.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:50 smithi107 ceph-mon[34139]: osdmap e30: 5 total, 4 up, 5 in 2024-09-13T16:24:50.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:50 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T16:24:50.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:50 smithi107 ceph-mon[34139]: from='client.? 172.21.15.107:0/319209669' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:24:50.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:50 smithi016 ceph-mon[31187]: from='client.? 172.21.15.107:0/215581620' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a8446e0-fc86-4646-beb6-a7b9d433093d"}]: dispatch 2024-09-13T16:24:50.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:50 smithi016 ceph-mon[31187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a8446e0-fc86-4646-beb6-a7b9d433093d"}]: dispatch 2024-09-13T16:24:50.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:50 smithi016 ceph-mon[31187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a8446e0-fc86-4646-beb6-a7b9d433093d"}]': finished 2024-09-13T16:24:50.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:50 smithi016 ceph-mon[31187]: osdmap e30: 5 total, 4 up, 5 in 2024-09-13T16:24:50.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:50 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T16:24:50.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:50 smithi016 ceph-mon[31187]: from='client.? 172.21.15.107:0/319209669' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:24:50.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:50 smithi111 ceph-mon[34600]: from='client.? 172.21.15.107:0/215581620' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a8446e0-fc86-4646-beb6-a7b9d433093d"}]: dispatch 2024-09-13T16:24:50.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:50 smithi111 ceph-mon[34600]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a8446e0-fc86-4646-beb6-a7b9d433093d"}]: dispatch 2024-09-13T16:24:50.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:50 smithi111 ceph-mon[34600]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a8446e0-fc86-4646-beb6-a7b9d433093d"}]': finished 2024-09-13T16:24:50.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:50 smithi111 ceph-mon[34600]: osdmap e30: 5 total, 4 up, 5 in 2024-09-13T16:24:50.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:50 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T16:24:50.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:50 smithi111 ceph-mon[34600]: from='client.? 172.21.15.107:0/319209669' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:24:51.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:51 smithi107 ceph-mon[34139]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-13T16:24:51.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:51 smithi016 ceph-mon[31187]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-13T16:24:51.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:51 smithi111 ceph-mon[34600]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-13T16:24:52.895 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:52 smithi107 ceph-mon[34139]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-13T16:24:52.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:52 smithi111 ceph-mon[34600]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-13T16:24:53.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:52 smithi016 ceph-mon[31187]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-13T16:24:54.823 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:54 smithi107 ceph-mon[34139]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-13T16:24:54.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:54 smithi111 ceph-mon[34600]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-13T16:24:55.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:54 smithi016 ceph-mon[31187]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-13T16:24:56.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:56 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-13T16:24:56.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:56 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:24:56.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:56 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-13T16:24:56.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:56 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:24:56.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:56 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-13T16:24:56.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:56 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:24:57.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:57 smithi111 ceph-mon[34600]: Deploying daemon osd.4 on smithi107 2024-09-13T16:24:57.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:57 smithi111 ceph-mon[34600]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-13T16:24:57.458 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:57 smithi107 ceph-mon[34139]: Deploying daemon osd.4 on smithi107 2024-09-13T16:24:57.458 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:57 smithi107 ceph-mon[34139]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-13T16:24:57.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:57 smithi016 ceph-mon[31187]: Deploying daemon osd.4 on smithi107 2024-09-13T16:24:57.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:57 smithi016 ceph-mon[31187]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-13T16:24:58.749 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:24:58 smithi107 ceph-mon[34139]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-13T16:24:58.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:24:58 smithi111 ceph-mon[34600]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-13T16:24:59.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:24:58 smithi016 ceph-mon[31187]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-13T16:25:00.841 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:00 smithi107 ceph-mon[34139]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-13T16:25:00.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:00 smithi111 ceph-mon[34600]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-13T16:25:01.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:00 smithi016 ceph-mon[31187]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-13T16:25:01.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:01 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:25:01.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:01 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:01.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:01 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:01.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:01 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:25:01.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:01 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:01.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:01 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:02.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:01 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:25:02.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:01 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:02.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:01 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:02.824 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:02 smithi107 ceph-mon[34139]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-13T16:25:02.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:02 smithi111 ceph-mon[34600]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-13T16:25:03.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:02 smithi016 ceph-mon[31187]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-13T16:25:04.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:04.506+0000 7f78bbfff640 1 -- 172.21.15.107:0/2454620308 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f78d40608f0 con 0x7f78bc0a1080 2024-09-13T16:25:04.507 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 4 on host 'smithi107' 2024-09-13T16:25:04.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:04.508+0000 7f78d93dd640 1 -- 172.21.15.107:0/2454620308 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f78bc0a1080 msgr2=0x7f78bc0a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:25:04.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:04.508+0000 7f78d93dd640 1 --2- 172.21.15.107:0/2454620308 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f78bc0a1080 0x7f78bc0a3540 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f78c4009890 tx=0x7f78c4008040 comp rx=0 tx=0).stop 2024-09-13T16:25:04.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:04.508+0000 7f78d93dd640 1 -- 172.21.15.107:0/2454620308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78d4103250 msgr2=0x7f78d40fe8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:25:04.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:04.508+0000 7f78d93dd640 1 --2- 172.21.15.107:0/2454620308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78d4103250 0x7f78d40fe8d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f78c80055e0 tx=0x7f78c80048f0 comp rx=0 tx=0).stop 2024-09-13T16:25:04.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:04.508+0000 7f78d93dd640 1 -- 172.21.15.107:0/2454620308 shutdown_connections 2024-09-13T16:25:04.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:04.508+0000 7f78d93dd640 1 --2- 172.21.15.107:0/2454620308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78d4105b80 0x7f78d40fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:25:04.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:04.508+0000 7f78d93dd640 1 --2- 172.21.15.107:0/2454620308 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f78bc0a1080 0x7f78bc0a3540 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:25:04.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:04.508+0000 7f78d93dd640 1 --2- 172.21.15.107:0/2454620308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78d4103250 0x7f78d40fe8d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:25:04.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:04.508+0000 7f78d93dd640 1 --2- 172.21.15.107:0/2454620308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78d4100920 0x7f78d4100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:25:04.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:04.508+0000 7f78d93dd640 1 -- 172.21.15.107:0/2454620308 >> 172.21.15.107:0/2454620308 conn(0x7f78d40fa4c0 msgr2=0x7f78d41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:25:04.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:04.508+0000 7f78d93dd640 1 -- 172.21.15.107:0/2454620308 shutdown_connections 2024-09-13T16:25:04.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:04.508+0000 7f78d93dd640 1 -- 172.21.15.107:0/2454620308 wait complete. 2024-09-13T16:25:04.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:04.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:04.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:04.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:25:04.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:04.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:04.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:04.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:04.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:25:04.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:04.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:04.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:04.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:04.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:25:04.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:05.147 DEBUG:teuthology.orchestra.run.smithi107:osd.4> sudo journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.4.service 2024-09-13T16:25:05.149 INFO:tasks.cephadm:Deploying osd.5 on smithi111 with /dev/vg_nvme/lv_3... 2024-09-13T16:25:05.150 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-13T16:25:05.353 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T16:25:05.428 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:05 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:25:05.428 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:05 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:05.428 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:05 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:05.428 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:05 smithi111 ceph-mon[34600]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-13T16:25:05.428 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:05 smithi111 ceph-mon[34600]: from='osd.4 [v2:172.21.15.107:6808/1936582048,v1:172.21.15.107:6809/1936582048]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-13T16:25:05.428 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:05 smithi111 ceph-mon[34600]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-13T16:25:05.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:05 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:25:05.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:05 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:05.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:05 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:05.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:05 smithi107 ceph-mon[34139]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-13T16:25:05.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:05 smithi107 ceph-mon[34139]: from='osd.4 [v2:172.21.15.107:6808/1936582048,v1:172.21.15.107:6809/1936582048]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-13T16:25:05.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:05 smithi107 ceph-mon[34139]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-13T16:25:05.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:05 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:25:05.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:05 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:05.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:05 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:05.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:05 smithi016 ceph-mon[31187]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-13T16:25:05.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:05 smithi016 ceph-mon[31187]: from='osd.4 [v2:172.21.15.107:6808/1936582048,v1:172.21.15.107:6809/1936582048]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-13T16:25:05.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:05 smithi016 ceph-mon[31187]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-13T16:25:06.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:06 smithi107 ceph-mon[34139]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-13T16:25:06.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:06 smithi107 ceph-mon[34139]: osdmap e31: 5 total, 4 up, 5 in 2024-09-13T16:25:06.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:06 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T16:25:06.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:06 smithi107 ceph-mon[34139]: from='osd.4 [v2:172.21.15.107:6808/1936582048,v1:172.21.15.107:6809/1936582048]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-13T16:25:06.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:06 smithi107 ceph-mon[34139]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-13T16:25:06.600 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 13 16:25:06 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-4[45062]: 2024-09-13T16:25:06.519+0000 7f4073fad640 -1 osd.4 0 waiting for initial osdmap 2024-09-13T16:25:06.600 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 13 16:25:06 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-4[45062]: 2024-09-13T16:25:06.524+0000 7f406fdd6640 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T16:25:06.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:06 smithi111 ceph-mon[34600]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-13T16:25:06.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:06 smithi111 ceph-mon[34600]: osdmap e31: 5 total, 4 up, 5 in 2024-09-13T16:25:06.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:06 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T16:25:06.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:06 smithi111 ceph-mon[34600]: from='osd.4 [v2:172.21.15.107:6808/1936582048,v1:172.21.15.107:6809/1936582048]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-13T16:25:06.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:06 smithi111 ceph-mon[34600]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-13T16:25:06.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:06 smithi016 ceph-mon[31187]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-13T16:25:06.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:06 smithi016 ceph-mon[31187]: osdmap e31: 5 total, 4 up, 5 in 2024-09-13T16:25:06.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:06 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T16:25:06.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:06 smithi016 ceph-mon[31187]: from='osd.4 [v2:172.21.15.107:6808/1936582048,v1:172.21.15.107:6809/1936582048]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-13T16:25:06.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:06 smithi016 ceph-mon[31187]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-13T16:25:07.361 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-13T16:25:07.457 INFO:teuthology.orchestra.run.smithi111.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-13T16:25:07.470 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10+0 records in 2024-09-13T16:25:07.470 INFO:teuthology.orchestra.run.smithi111.stderr:10+0 records out 2024-09-13T16:25:07.470 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112852 s, 929 MB/s 2024-09-13T16:25:07.470 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping successful for: 2024-09-13T16:25:07.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:07 smithi107 ceph-mon[34139]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-13T16:25:07.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:07 smithi107 ceph-mon[34139]: osdmap e32: 5 total, 4 up, 5 in 2024-09-13T16:25:07.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:07 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T16:25:07.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:07 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T16:25:07.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:07 smithi107 ceph-mon[34139]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-13T16:25:07.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:07 smithi111 ceph-mon[34600]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-13T16:25:07.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:07 smithi111 ceph-mon[34600]: osdmap e32: 5 total, 4 up, 5 in 2024-09-13T16:25:07.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:07 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T16:25:07.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:07 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T16:25:07.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:07 smithi111 ceph-mon[34600]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-13T16:25:07.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:07 smithi016 ceph-mon[31187]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-13T16:25:07.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:07 smithi016 ceph-mon[31187]: osdmap e32: 5 total, 4 up, 5 in 2024-09-13T16:25:07.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:07 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T16:25:07.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:07 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T16:25:07.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:07 smithi016 ceph-mon[31187]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-13T16:25:08.116 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi111:vg_nvme/lv_3 2024-09-13T16:25:08.316 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T16:25:08.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:08 smithi111 ceph-mon[34600]: purged_snaps scrub starts 2024-09-13T16:25:08.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:08 smithi111 ceph-mon[34600]: purged_snaps scrub ok 2024-09-13T16:25:08.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:08 smithi111 ceph-mon[34600]: osd.4 [v2:172.21.15.107:6808/1936582048,v1:172.21.15.107:6809/1936582048] boot 2024-09-13T16:25:08.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:08 smithi111 ceph-mon[34600]: osdmap e33: 5 total, 5 up, 5 in 2024-09-13T16:25:08.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:08 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T16:25:08.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:08 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:08.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:08 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:08.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:08 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:25:08.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:08 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:25:08.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:08 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:08.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:08 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:08.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:08 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:25:08.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:08 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:08.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:08 smithi107 ceph-mon[34139]: purged_snaps scrub starts 2024-09-13T16:25:08.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:08 smithi107 ceph-mon[34139]: purged_snaps scrub ok 2024-09-13T16:25:08.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:08 smithi107 ceph-mon[34139]: osd.4 [v2:172.21.15.107:6808/1936582048,v1:172.21.15.107:6809/1936582048] boot 2024-09-13T16:25:08.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:08 smithi107 ceph-mon[34139]: osdmap e33: 5 total, 5 up, 5 in 2024-09-13T16:25:08.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:08 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T16:25:08.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:08 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:08.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:08 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:08.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:08 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:25:08.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:08 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:25:08.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:08 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:08.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:08 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:08.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:08 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:25:08.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:08 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:08.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:08 smithi016 ceph-mon[31187]: purged_snaps scrub starts 2024-09-13T16:25:08.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:08 smithi016 ceph-mon[31187]: purged_snaps scrub ok 2024-09-13T16:25:08.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:08 smithi016 ceph-mon[31187]: osd.4 [v2:172.21.15.107:6808/1936582048,v1:172.21.15.107:6809/1936582048] boot 2024-09-13T16:25:08.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:08 smithi016 ceph-mon[31187]: osdmap e33: 5 total, 5 up, 5 in 2024-09-13T16:25:08.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:08 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T16:25:08.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:08 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:08.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:08 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:08.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:08 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:25:08.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:08 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:25:08.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:08 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:08.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:08 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:08.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:08 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:25:08.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:08 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:09.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:09 smithi111 ceph-mon[34600]: Detected new or changed devices on smithi107 2024-09-13T16:25:09.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:09 smithi111 ceph-mon[34600]: Adjusting osd_memory_target on smithi107 to 8543M 2024-09-13T16:25:09.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:09 smithi111 ceph-mon[34600]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 145 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:09.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:09 smithi111 ceph-mon[34600]: osdmap e34: 5 total, 5 up, 5 in 2024-09-13T16:25:09.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:09 smithi107 ceph-mon[34139]: Detected new or changed devices on smithi107 2024-09-13T16:25:09.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:09 smithi107 ceph-mon[34139]: Adjusting osd_memory_target on smithi107 to 8543M 2024-09-13T16:25:09.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:09 smithi107 ceph-mon[34139]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 145 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:09.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:09 smithi107 ceph-mon[34139]: osdmap e34: 5 total, 5 up, 5 in 2024-09-13T16:25:09.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:09 smithi016 ceph-mon[31187]: Detected new or changed devices on smithi107 2024-09-13T16:25:09.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:09 smithi016 ceph-mon[31187]: Adjusting osd_memory_target on smithi107 to 8543M 2024-09-13T16:25:09.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:09 smithi016 ceph-mon[31187]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 145 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:09.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:09 smithi016 ceph-mon[31187]: osdmap e34: 5 total, 5 up, 5 in 2024-09-13T16:25:10.030 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.000+0000 7f5108f55640 1 -- 172.21.15.111:0/3200243893 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51040ffdf0 msgr2=0x7f51041001f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:25:10.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.000+0000 7f5108f55640 1 --2- 172.21.15.111:0/3200243893 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51040ffdf0 0x7f51041001f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009900 tx=0x7f50f402a410 comp rx=0 tx=0).stop 2024-09-13T16:25:10.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.000+0000 7f5108f55640 1 -- 172.21.15.111:0/3200243893 shutdown_connections 2024-09-13T16:25:10.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.000+0000 7f5108f55640 1 --2- 172.21.15.111:0/3200243893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51041019b0 0x7f5104108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:25:10.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.000+0000 7f5108f55640 1 --2- 172.21.15.111:0/3200243893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5104100ff0 0x7f5104101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:25:10.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.000+0000 7f5108f55640 1 --2- 172.21.15.111:0/3200243893 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51040ffdf0 0x7f51041001f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:25:10.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.000+0000 7f5108f55640 1 -- 172.21.15.111:0/3200243893 >> 172.21.15.111:0/3200243893 conn(0x7f51040fb5a0 msgr2=0x7f51040fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:25:10.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.000+0000 7f5108f55640 1 -- 172.21.15.111:0/3200243893 shutdown_connections 2024-09-13T16:25:10.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.000+0000 7f5108f55640 1 -- 172.21.15.111:0/3200243893 wait complete. 2024-09-13T16:25:10.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.001+0000 7f5108f55640 1 Processor -- start 2024-09-13T16:25:10.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.001+0000 7f5108f55640 1 -- start start 2024-09-13T16:25:10.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.001+0000 7f5108f55640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51040ffdf0 0x7f5104074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:25:10.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.001+0000 7f5108f55640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5104100ff0 0x7f51040728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:25:10.031 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.001+0000 7f5108f55640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51041019b0 0x7f5104072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:25:10.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.001+0000 7f5108f55640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5104073570 con 0x7f51040ffdf0 2024-09-13T16:25:10.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.001+0000 7f5108f55640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51040736e0 con 0x7f51041019b0 2024-09-13T16:25:10.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.001+0000 7f5108f55640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510419fcb0 con 0x7f5104100ff0 2024-09-13T16:25:10.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.001+0000 7f5101d9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5104100ff0 0x7f51040728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:25:10.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.001+0000 7f5101d9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5104100ff0 0x7f51040728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41546/0 (socket says 172.21.15.111:41546) 2024-09-13T16:25:10.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.001+0000 7f5101d9b640 1 -- 172.21.15.111:0/3449151816 learned_addr learned my addr 172.21.15.111:0/3449151816 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T16:25:10.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.002+0000 7f510259c640 1 --2- 172.21.15.111:0/3449151816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51040ffdf0 0x7f5104074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:25:10.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.002+0000 7f5102d9d640 1 --2- 172.21.15.111:0/3449151816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51041019b0 0x7f5104072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:25:10.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.002+0000 7f5101d9b640 1 -- 172.21.15.111:0/3449151816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51041019b0 msgr2=0x7f5104072de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:25:10.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.002+0000 7f5101d9b640 1 --2- 172.21.15.111:0/3449151816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51041019b0 0x7f5104072de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:25:10.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.002+0000 7f5101d9b640 1 -- 172.21.15.111:0/3449151816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51040ffdf0 msgr2=0x7f5104074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:25:10.032 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.002+0000 7f5101d9b640 1 --2- 172.21.15.111:0/3449151816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51040ffdf0 0x7f5104074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:25:10.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.002+0000 7f5101d9b640 1 -- 172.21.15.111:0/3449151816 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f4009560 con 0x7f5104100ff0 2024-09-13T16:25:10.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.002+0000 7f5101d9b640 1 --2- 172.21.15.111:0/3449151816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5104100ff0 0x7f51040728a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f50f800c950 tx=0x7f50f800ce10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:25:10.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.002+0000 7f50f37fe640 1 -- 172.21.15.111:0/3449151816 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f8007c70 con 0x7f5104100ff0 2024-09-13T16:25:10.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.002+0000 7f50f37fe640 1 -- 172.21.15.111:0/3449151816 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f50f8007dd0 con 0x7f5104100ff0 2024-09-13T16:25:10.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.002+0000 7f50f37fe640 1 -- 172.21.15.111:0/3449151816 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f8018780 con 0x7f5104100ff0 2024-09-13T16:25:10.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.002+0000 7f5108f55640 1 -- 172.21.15.111:0/3449151816 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f510419ff60 con 0x7f5104100ff0 2024-09-13T16:25:10.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.002+0000 7f5108f55640 1 -- 172.21.15.111:0/3449151816 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51041a0380 con 0x7f5104100ff0 2024-09-13T16:25:10.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.003+0000 7f5108f55640 1 -- 172.21.15.111:0/3449151816 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5104107280 con 0x7f5104100ff0 2024-09-13T16:25:10.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.005+0000 7f50f37fe640 1 -- 172.21.15.111:0/3449151816 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f50f800f450 con 0x7f5104100ff0 2024-09-13T16:25:10.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.005+0000 7f50f37fe640 1 --2- 172.21.15.111:0/3449151816 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f50ec0a1080 0x7f50ec0a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:25:10.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.006+0000 7f50f37fe640 1 -- 172.21.15.111:0/3449151816 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f50f80cbd00 con 0x7f5104100ff0 2024-09-13T16:25:10.033 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.006+0000 7f510259c640 1 --2- 172.21.15.111:0/3449151816 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f50ec0a1080 0x7f50ec0a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:25:10.034 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.006+0000 7f510259c640 1 --2- 172.21.15.111:0/3449151816 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f50ec0a1080 0x7f50ec0a3540 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f50f40057c0 tx=0x7f50f4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:25:10.034 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.009+0000 7f50f37fe640 1 -- 172.21.15.111:0/3449151816 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f50f8096530 con 0x7f5104100ff0 2024-09-13T16:25:10.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:10.183+0000 7f5108f55640 1 -- 172.21.15.111:0/3449151816 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f510402c550 con 0x7f50ec0a1080 2024-09-13T16:25:10.825 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:10 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:25:10.825 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:10 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:25:10.825 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:10 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:10.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:10 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:25:10.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:10 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:25:10.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:10 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:10.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:10 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:25:10.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:10 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:25:10.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:10 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:11.778 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:11 smithi111 ceph-mon[34600]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:25:11.778 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:11 smithi111 ceph-mon[34600]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 145 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:11.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:11 smithi107 ceph-mon[34139]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:25:11.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:11 smithi107 ceph-mon[34139]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 145 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:11.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:11 smithi016 ceph-mon[31187]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:25:11.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:11 smithi016 ceph-mon[31187]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 145 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:12.804 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:12 smithi111 ceph-mon[34600]: from='client.? 172.21.15.111:0/618913280' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c97a446-357e-4975-9e51-9c95fe84a663"}]: dispatch 2024-09-13T16:25:12.804 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:12 smithi111 ceph-mon[34600]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c97a446-357e-4975-9e51-9c95fe84a663"}]: dispatch 2024-09-13T16:25:12.804 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:12 smithi111 ceph-mon[34600]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c97a446-357e-4975-9e51-9c95fe84a663"}]': finished 2024-09-13T16:25:12.804 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:12 smithi111 ceph-mon[34600]: osdmap e35: 6 total, 5 up, 6 in 2024-09-13T16:25:12.804 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:12 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T16:25:12.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:12 smithi107 ceph-mon[34139]: from='client.? 172.21.15.111:0/618913280' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c97a446-357e-4975-9e51-9c95fe84a663"}]: dispatch 2024-09-13T16:25:12.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:12 smithi107 ceph-mon[34139]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c97a446-357e-4975-9e51-9c95fe84a663"}]: dispatch 2024-09-13T16:25:12.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:12 smithi107 ceph-mon[34139]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c97a446-357e-4975-9e51-9c95fe84a663"}]': finished 2024-09-13T16:25:12.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:12 smithi107 ceph-mon[34139]: osdmap e35: 6 total, 5 up, 6 in 2024-09-13T16:25:12.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:12 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T16:25:12.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:12 smithi016 ceph-mon[31187]: from='client.? 172.21.15.111:0/618913280' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c97a446-357e-4975-9e51-9c95fe84a663"}]: dispatch 2024-09-13T16:25:12.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:12 smithi016 ceph-mon[31187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c97a446-357e-4975-9e51-9c95fe84a663"}]: dispatch 2024-09-13T16:25:12.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:12 smithi016 ceph-mon[31187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c97a446-357e-4975-9e51-9c95fe84a663"}]': finished 2024-09-13T16:25:12.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:12 smithi016 ceph-mon[31187]: osdmap e35: 6 total, 5 up, 6 in 2024-09-13T16:25:12.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:12 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T16:25:13.750 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:13 smithi107 ceph-mon[34139]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 145 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:13.750 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:13 smithi107 ceph-mon[34139]: from='client.? 172.21.15.111:0/182034961' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:25:13.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:13 smithi016 ceph-mon[31187]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 145 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:13.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:13 smithi016 ceph-mon[31187]: from='client.? 172.21.15.111:0/182034961' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:25:13.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:13 smithi111 ceph-mon[34600]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 145 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:13.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:13 smithi111 ceph-mon[34600]: from='client.? 172.21.15.111:0/182034961' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:25:15.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:15 smithi107 ceph-mon[34139]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 145 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:15.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:15 smithi016 ceph-mon[31187]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 145 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:15.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:15 smithi111 ceph-mon[34600]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 145 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:17.689 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:17 smithi111 ceph-mon[34600]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:17.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:17 smithi107 ceph-mon[34139]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:17.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:17 smithi016 ceph-mon[31187]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:19.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:19 smithi107 ceph-mon[34139]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:19.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:19 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-13T16:25:19.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:19 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:19.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:19 smithi016 ceph-mon[31187]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:19.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:19 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-13T16:25:19.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:19 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:19.897 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:19 smithi111 ceph-mon[34600]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:19.897 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:19 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-13T16:25:19.897 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:19 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:20.804 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:20 smithi111 ceph-mon[34600]: Deploying daemon osd.5 on smithi111 2024-09-13T16:25:20.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:20 smithi107 ceph-mon[34139]: Deploying daemon osd.5 on smithi111 2024-09-13T16:25:20.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:20 smithi016 ceph-mon[31187]: Deploying daemon osd.5 on smithi111 2024-09-13T16:25:21.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:21 smithi107 ceph-mon[34139]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:21.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:21 smithi016 ceph-mon[31187]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:21.875 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:21 smithi111 ceph-mon[34600]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:23.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:23 smithi107 ceph-mon[34139]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:23.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:23 smithi016 ceph-mon[31187]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:23.858 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:23 smithi111 ceph-mon[34600]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:24.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:24 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:25:24.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:24 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:24.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:24 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:24.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:24 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:25:24.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:24 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:24.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:24 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:24.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:24 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:25:24.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:24 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:24.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:24 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:25.786 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:25 smithi111 ceph-mon[34600]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:25.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:25 smithi107 ceph-mon[34139]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:25.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:25 smithi016 ceph-mon[31187]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:27.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:27.444+0000 7f50f37fe640 1 -- 172.21.15.111:0/3449151816 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f510402c550 con 0x7f50ec0a1080 2024-09-13T16:25:27.445 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 5 on host 'smithi111' 2024-09-13T16:25:27.447 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:27.446+0000 7f5108f55640 1 -- 172.21.15.111:0/3449151816 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f50ec0a1080 msgr2=0x7f50ec0a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:25:27.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:27.446+0000 7f5108f55640 1 --2- 172.21.15.111:0/3449151816 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f50ec0a1080 0x7f50ec0a3540 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f50f40057c0 tx=0x7f50f4005a20 comp rx=0 tx=0).stop 2024-09-13T16:25:27.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:27.446+0000 7f5108f55640 1 -- 172.21.15.111:0/3449151816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5104100ff0 msgr2=0x7f51040728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:25:27.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:27.446+0000 7f5108f55640 1 --2- 172.21.15.111:0/3449151816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5104100ff0 0x7f51040728a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f50f800c950 tx=0x7f50f800ce10 comp rx=0 tx=0).stop 2024-09-13T16:25:27.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:27.446+0000 7f5108f55640 1 -- 172.21.15.111:0/3449151816 shutdown_connections 2024-09-13T16:25:27.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:27.446+0000 7f5108f55640 1 --2- 172.21.15.111:0/3449151816 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f50ec0a1080 0x7f50ec0a3540 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:25:27.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:27.446+0000 7f5108f55640 1 --2- 172.21.15.111:0/3449151816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51041019b0 0x7f5104072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:25:27.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:27.446+0000 7f5108f55640 1 --2- 172.21.15.111:0/3449151816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5104100ff0 0x7f51040728a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:25:27.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:27.446+0000 7f5108f55640 1 --2- 172.21.15.111:0/3449151816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51040ffdf0 0x7f5104074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:25:27.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:27.446+0000 7f5108f55640 1 -- 172.21.15.111:0/3449151816 >> 172.21.15.111:0/3449151816 conn(0x7f51040fb5a0 msgr2=0x7f5104104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:25:27.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:27.446+0000 7f5108f55640 1 -- 172.21.15.111:0/3449151816 shutdown_connections 2024-09-13T16:25:27.448 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:25:27.446+0000 7f5108f55640 1 -- 172.21.15.111:0/3449151816 wait complete. 2024-09-13T16:25:27.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:27 smithi111 ceph-mon[34600]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:27.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:27 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:27.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:27 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:27.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:27 smithi107 ceph-mon[34139]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:27.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:27 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:27.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:27 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:27.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:27 smithi016 ceph-mon[31187]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:27.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:27 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:27.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:27 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:28.095 DEBUG:teuthology.orchestra.run.smithi111:osd.5> sudo journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.5.service 2024-09-13T16:25:28.097 INFO:tasks.cephadm:Deploying osd.6 on smithi016 with /dev/vg_nvme/lv_2... 2024-09-13T16:25:28.097 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-13T16:25:28.312 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:25:28.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:28 smithi111 ceph-mon[34600]: from='osd.5 [v2:172.21.15.111:6808/483858448,v1:172.21.15.111:6809/483858448]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-13T16:25:28.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:28 smithi111 ceph-mon[34600]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-13T16:25:28.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:28 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:28.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:28 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:28.687 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:25:28 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:25:28.329+0000 7fc88f13c640 -1 osd.5 0 log_to_monitors true 2024-09-13T16:25:28.704 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:28 smithi016 ceph-mon[31187]: from='osd.5 [v2:172.21.15.111:6808/483858448,v1:172.21.15.111:6809/483858448]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-13T16:25:28.705 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:28 smithi016 ceph-mon[31187]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-13T16:25:28.705 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:28 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:28.705 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:28 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:28.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:28 smithi107 ceph-mon[34139]: from='osd.5 [v2:172.21.15.111:6808/483858448,v1:172.21.15.111:6809/483858448]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-13T16:25:28.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:28 smithi107 ceph-mon[34139]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-13T16:25:28.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:28 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:28.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:28 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:29.827 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:29 smithi016 ceph-mon[31187]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:29.827 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:29 smithi016 ceph-mon[31187]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-13T16:25:29.827 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:29 smithi016 ceph-mon[31187]: osdmap e36: 6 total, 5 up, 6 in 2024-09-13T16:25:29.827 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:29 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T16:25:29.827 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:29 smithi016 ceph-mon[31187]: from='osd.5 [v2:172.21.15.111:6808/483858448,v1:172.21.15.111:6809/483858448]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-13T16:25:29.827 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:29 smithi016 ceph-mon[31187]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-13T16:25:29.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:29 smithi107 ceph-mon[34139]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:29.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:29 smithi107 ceph-mon[34139]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-13T16:25:29.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:29 smithi107 ceph-mon[34139]: osdmap e36: 6 total, 5 up, 6 in 2024-09-13T16:25:29.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:29 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T16:25:29.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:29 smithi107 ceph-mon[34139]: from='osd.5 [v2:172.21.15.111:6808/483858448,v1:172.21.15.111:6809/483858448]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-13T16:25:29.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:29 smithi107 ceph-mon[34139]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-13T16:25:29.937 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:25:29 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:25:29.573+0000 7fc88b0bd640 -1 osd.5 0 waiting for initial osdmap 2024-09-13T16:25:29.937 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:25:29 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:25:29.580+0000 7fc886ee6640 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T16:25:29.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:29 smithi111 ceph-mon[34600]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:29.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:29 smithi111 ceph-mon[34600]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-13T16:25:29.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:29 smithi111 ceph-mon[34600]: osdmap e36: 6 total, 5 up, 6 in 2024-09-13T16:25:29.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:29 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T16:25:29.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:29 smithi111 ceph-mon[34600]: from='osd.5 [v2:172.21.15.111:6808/483858448,v1:172.21.15.111:6809/483858448]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-13T16:25:29.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:29 smithi111 ceph-mon[34600]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-13T16:25:30.218 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-13T16:25:30.318 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-13T16:25:30.330 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-09-13T16:25:30.330 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-09-13T16:25:30.330 INFO:teuthology.orchestra.run.smithi016.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0101214 s, 1.0 GB/s 2024-09-13T16:25:30.330 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-09-13T16:25:30.649 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:30 smithi016 ceph-mon[31187]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-13T16:25:30.650 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:30 smithi016 ceph-mon[31187]: osdmap e37: 6 total, 5 up, 6 in 2024-09-13T16:25:30.650 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:30 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T16:25:30.650 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:30 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T16:25:30.650 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:30 smithi016 ceph-mon[31187]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:30.833 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:30 smithi111 ceph-mon[34600]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-13T16:25:30.833 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:30 smithi111 ceph-mon[34600]: osdmap e37: 6 total, 5 up, 6 in 2024-09-13T16:25:30.833 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:30 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T16:25:30.833 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:30 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T16:25:30.833 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:30 smithi111 ceph-mon[34600]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:30.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:30 smithi107 ceph-mon[34139]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-13T16:25:30.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:30 smithi107 ceph-mon[34139]: osdmap e37: 6 total, 5 up, 6 in 2024-09-13T16:25:30.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:30 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T16:25:30.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:30 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T16:25:30.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:30 smithi107 ceph-mon[34139]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-13T16:25:30.956 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_2 2024-09-13T16:25:31.158 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:25:31.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:31 smithi107 ceph-mon[34139]: purged_snaps scrub starts 2024-09-13T16:25:31.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:31 smithi107 ceph-mon[34139]: purged_snaps scrub ok 2024-09-13T16:25:31.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:31 smithi107 ceph-mon[34139]: osd.5 [v2:172.21.15.111:6808/483858448,v1:172.21.15.111:6809/483858448] boot 2024-09-13T16:25:31.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:31 smithi107 ceph-mon[34139]: osdmap e38: 6 total, 6 up, 6 in 2024-09-13T16:25:31.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:31 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T16:25:31.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:31 smithi107 ceph-mon[34139]: Detected new or changed devices on smithi111 2024-09-13T16:25:31.851 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:31 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:31.851 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:31 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:31.851 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:31 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:25:31.851 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:31 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:25:31.851 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:31 smithi107 ceph-mon[34139]: Adjusting osd_memory_target on smithi111 to 8544M 2024-09-13T16:25:31.851 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:31 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:31.852 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:31 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:31.852 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:31 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:25:31.852 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:31 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:31.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:31 smithi016 ceph-mon[31187]: purged_snaps scrub starts 2024-09-13T16:25:31.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:31 smithi016 ceph-mon[31187]: purged_snaps scrub ok 2024-09-13T16:25:31.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:31 smithi016 ceph-mon[31187]: osd.5 [v2:172.21.15.111:6808/483858448,v1:172.21.15.111:6809/483858448] boot 2024-09-13T16:25:31.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:31 smithi016 ceph-mon[31187]: osdmap e38: 6 total, 6 up, 6 in 2024-09-13T16:25:31.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:31 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T16:25:31.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:31 smithi016 ceph-mon[31187]: Detected new or changed devices on smithi111 2024-09-13T16:25:31.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:31 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:31.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:31 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:31.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:31 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:25:31.856 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:31 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:25:31.856 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:31 smithi016 ceph-mon[31187]: Adjusting osd_memory_target on smithi111 to 8544M 2024-09-13T16:25:31.856 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:31 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:31.856 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:31 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:31.856 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:31 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:25:31.856 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:31 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:31.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:31 smithi111 ceph-mon[34600]: purged_snaps scrub starts 2024-09-13T16:25:31.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:31 smithi111 ceph-mon[34600]: purged_snaps scrub ok 2024-09-13T16:25:31.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:31 smithi111 ceph-mon[34600]: osd.5 [v2:172.21.15.111:6808/483858448,v1:172.21.15.111:6809/483858448] boot 2024-09-13T16:25:31.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:31 smithi111 ceph-mon[34600]: osdmap e38: 6 total, 6 up, 6 in 2024-09-13T16:25:31.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:31 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T16:25:31.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:31 smithi111 ceph-mon[34600]: Detected new or changed devices on smithi111 2024-09-13T16:25:31.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:31 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:31.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:31 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:31.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:31 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:25:31.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:31 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:25:31.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:31 smithi111 ceph-mon[34600]: Adjusting osd_memory_target on smithi111 to 8544M 2024-09-13T16:25:31.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:31 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:31.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:31 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:31.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:31 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:25:31.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:31 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:32.838 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:32 smithi016 ceph-mon[31187]: osdmap e39: 6 total, 6 up, 6 in 2024-09-13T16:25:32.839 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:32 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:25:32.839 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:32 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:32.839 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:32 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:25:32.839 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:32 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:32.839 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:32 smithi016 ceph-mon[31187]: pgmap v133: 1 pgs: 1 peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-09-13T16:25:32.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:32 smithi107 ceph-mon[34139]: osdmap e39: 6 total, 6 up, 6 in 2024-09-13T16:25:32.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:32 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:25:32.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:32 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:32.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:32 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:25:32.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:32 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:32.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:32 smithi107 ceph-mon[34139]: pgmap v133: 1 pgs: 1 peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-09-13T16:25:32.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.835+0000 7fad3a27b640 1 -- 172.21.15.16:0/435385838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad340ffe00 msgr2=0x7fad34100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:25:32.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.835+0000 7fad3a27b640 1 --2- 172.21.15.16:0/435385838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad340ffe00 0x7fad34100200 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fad24009900 tx=0x7fad2402a410 comp rx=0 tx=0).stop 2024-09-13T16:25:32.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.835+0000 7fad3a27b640 1 -- 172.21.15.16:0/435385838 shutdown_connections 2024-09-13T16:25:32.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.835+0000 7fad3a27b640 1 --2- 172.21.15.16:0/435385838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad34101920 0x7fad34108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:25:32.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.835+0000 7fad3a27b640 1 --2- 172.21.15.16:0/435385838 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad34100f60 0x7fad341013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:25:32.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.835+0000 7fad3a27b640 1 --2- 172.21.15.16:0/435385838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad340ffe00 0x7fad34100200 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:25:32.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.835+0000 7fad3a27b640 1 -- 172.21.15.16:0/435385838 >> 172.21.15.16:0/435385838 conn(0x7fad340fb5d0 msgr2=0x7fad340fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:25:32.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.835+0000 7fad3a27b640 1 -- 172.21.15.16:0/435385838 shutdown_connections 2024-09-13T16:25:32.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.835+0000 7fad3a27b640 1 -- 172.21.15.16:0/435385838 wait complete. 2024-09-13T16:25:32.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.836+0000 7fad3a27b640 1 Processor -- start 2024-09-13T16:25:32.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.836+0000 7fad3a27b640 1 -- start start 2024-09-13T16:25:32.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.836+0000 7fad3a27b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad340ffe00 0x7fad34195c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:25:32.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.836+0000 7fad3a27b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad34100f60 0x7fad341961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:25:32.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.836+0000 7fad3a27b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad34101920 0x7fad3419d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:25:32.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.836+0000 7fad3a27b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3419d760 con 0x7fad34100f60 2024-09-13T16:25:32.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.836+0000 7fad3a27b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3419d8d0 con 0x7fad340ffe00 2024-09-13T16:25:32.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.836+0000 7fad3a27b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3419da90 con 0x7fad34101920 2024-09-13T16:25:32.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.837+0000 7fad337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad34100f60 0x7fad341961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:25:32.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.837+0000 7fad337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad34100f60 0x7fad341961b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39558/0 (socket says 172.21.15.16:39558) 2024-09-13T16:25:32.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.837+0000 7fad337fe640 1 -- 172.21.15.16:0/3914154092 learned_addr learned my addr 172.21.15.16:0/3914154092 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:25:32.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.837+0000 7fad33fff640 1 --2- 172.21.15.16:0/3914154092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad340ffe00 0x7fad34195c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:25:32.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.837+0000 7fad38818640 1 --2- 172.21.15.16:0/3914154092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad34101920 0x7fad3419d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:25:32.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.837+0000 7fad337fe640 1 -- 172.21.15.16:0/3914154092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad34101920 msgr2=0x7fad3419d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:25:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.837+0000 7fad337fe640 1 --2- 172.21.15.16:0/3914154092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad34101920 0x7fad3419d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:25:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.837+0000 7fad337fe640 1 -- 172.21.15.16:0/3914154092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad340ffe00 msgr2=0x7fad34195c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:25:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.837+0000 7fad337fe640 1 --2- 172.21.15.16:0/3914154092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad340ffe00 0x7fad34195c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:25:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.837+0000 7fad337fe640 1 -- 172.21.15.16:0/3914154092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad24009560 con 0x7fad34100f60 2024-09-13T16:25:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.837+0000 7fad38818640 1 --2- 172.21.15.16:0/3914154092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad34101920 0x7fad3419d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:25:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.837+0000 7fad337fe640 1 --2- 172.21.15.16:0/3914154092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad34100f60 0x7fad341961b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fad2800d770 tx=0x7fad2800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:25:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.837+0000 7fad317fa640 1 -- 172.21.15.16:0/3914154092 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad28009c30 con 0x7fad34100f60 2024-09-13T16:25:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.837+0000 7fad3a27b640 1 -- 172.21.15.16:0/3914154092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad3419dd70 con 0x7fad34100f60 2024-09-13T16:25:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.837+0000 7fad317fa640 1 -- 172.21.15.16:0/3914154092 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fad2800de80 con 0x7fad34100f60 2024-09-13T16:25:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.837+0000 7fad3a27b640 1 -- 172.21.15.16:0/3914154092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad3419e200 con 0x7fad34100f60 2024-09-13T16:25:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.837+0000 7fad317fa640 1 -- 172.21.15.16:0/3914154092 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad28003ce0 con 0x7fad34100f60 2024-09-13T16:25:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.839+0000 7fad3a27b640 1 -- 172.21.15.16:0/3914154092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad34107290 con 0x7fad34100f60 2024-09-13T16:25:32.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.840+0000 7fad317fa640 1 -- 172.21.15.16:0/3914154092 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fad28003e40 con 0x7fad34100f60 2024-09-13T16:25:32.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.840+0000 7fad317fa640 1 --2- 172.21.15.16:0/3914154092 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fad1c0a1080 0x7fad1c0a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:25:32.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.840+0000 7fad33fff640 1 --2- 172.21.15.16:0/3914154092 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fad1c0a1080 0x7fad1c0a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:25:32.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.841+0000 7fad317fa640 1 -- 172.21.15.16:0/3914154092 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4562+0+0 (secure 0 0 0) 0x7fad280cb540 con 0x7fad34100f60 2024-09-13T16:25:32.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.841+0000 7fad33fff640 1 --2- 172.21.15.16:0/3914154092 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fad1c0a1080 0x7fad1c0a3540 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fad34197150 tx=0x7fad24005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:25:32.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:32.845+0000 7fad317fa640 1 -- 172.21.15.16:0/3914154092 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fad28095b70 con 0x7fad34100f60 2024-09-13T16:25:32.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:32 smithi111 ceph-mon[34600]: osdmap e39: 6 total, 6 up, 6 in 2024-09-13T16:25:32.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:32 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:25:32.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:32 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:32.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:32 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:25:32.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:32 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:32.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:32 smithi111 ceph-mon[34600]: pgmap v133: 1 pgs: 1 peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-09-13T16:25:33.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:33.023+0000 7fad3a27b640 1 -- 172.21.15.16:0/3914154092 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fad3402c550 con 0x7fad1c0a1080 2024-09-13T16:25:33.681 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:33 smithi016 ceph-mon[31187]: osdmap e40: 6 total, 6 up, 6 in 2024-09-13T16:25:33.681 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:33 smithi016 ceph-mon[31187]: from='client.14337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:25:33.681 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:33 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:25:33.681 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:33 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:25:33.681 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:33 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:33.681 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:33 smithi016 ceph-mon[31187]: osdmap e41: 6 total, 6 up, 6 in 2024-09-13T16:25:33.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:33 smithi107 ceph-mon[34139]: osdmap e40: 6 total, 6 up, 6 in 2024-09-13T16:25:33.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:33 smithi107 ceph-mon[34139]: from='client.14337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:25:33.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:33 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:25:33.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:33 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:25:33.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:33 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:33.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:33 smithi107 ceph-mon[34139]: osdmap e41: 6 total, 6 up, 6 in 2024-09-13T16:25:33.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:33 smithi111 ceph-mon[34600]: osdmap e40: 6 total, 6 up, 6 in 2024-09-13T16:25:33.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:33 smithi111 ceph-mon[34600]: from='client.14337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:25:33.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:33 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:25:33.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:33 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:25:33.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:33 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:33.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:33 smithi111 ceph-mon[34600]: osdmap e41: 6 total, 6 up, 6 in 2024-09-13T16:25:35.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:35 smithi016 ceph-mon[31187]: osdmap e42: 6 total, 6 up, 6 in 2024-09-13T16:25:35.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:35 smithi016 ceph-mon[31187]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-09-13T16:25:35.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:35 smithi107 ceph-mon[34139]: osdmap e42: 6 total, 6 up, 6 in 2024-09-13T16:25:35.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:35 smithi107 ceph-mon[34139]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-09-13T16:25:35.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:35 smithi111 ceph-mon[34600]: osdmap e42: 6 total, 6 up, 6 in 2024-09-13T16:25:35.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:35 smithi111 ceph-mon[34600]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-09-13T16:25:36.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:36 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3167896979' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "890239e6-e6fb-4f9c-b973-05cc66d199bf"}]: dispatch 2024-09-13T16:25:36.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:36 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3167896979' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "890239e6-e6fb-4f9c-b973-05cc66d199bf"}]': finished 2024-09-13T16:25:36.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:36 smithi107 ceph-mon[34139]: osdmap e43: 7 total, 6 up, 7 in 2024-09-13T16:25:36.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:36 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T16:25:36.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:36 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/904443681' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:25:36.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:36 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3167896979' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "890239e6-e6fb-4f9c-b973-05cc66d199bf"}]: dispatch 2024-09-13T16:25:36.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:36 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3167896979' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "890239e6-e6fb-4f9c-b973-05cc66d199bf"}]': finished 2024-09-13T16:25:36.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:36 smithi016 ceph-mon[31187]: osdmap e43: 7 total, 6 up, 7 in 2024-09-13T16:25:36.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:36 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T16:25:36.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:36 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/904443681' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:25:36.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:36 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3167896979' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "890239e6-e6fb-4f9c-b973-05cc66d199bf"}]: dispatch 2024-09-13T16:25:36.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:36 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3167896979' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "890239e6-e6fb-4f9c-b973-05cc66d199bf"}]': finished 2024-09-13T16:25:36.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:36 smithi111 ceph-mon[34600]: osdmap e43: 7 total, 6 up, 7 in 2024-09-13T16:25:36.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:36 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T16:25:36.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:36 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/904443681' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:25:37.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:37 smithi107 ceph-mon[34139]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-13T16:25:37.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:37 smithi016 ceph-mon[31187]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-13T16:25:37.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:37 smithi111 ceph-mon[34600]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-13T16:25:38.869 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:38 smithi016 ceph-mon[31187]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-13T16:25:38.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:38 smithi111 ceph-mon[34600]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-13T16:25:39.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:38 smithi107 ceph-mon[34139]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-13T16:25:40.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:40 smithi111 ceph-mon[34600]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-13T16:25:41.027 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:40 smithi016 ceph-mon[31187]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-13T16:25:41.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:40 smithi107 ceph-mon[34139]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-13T16:25:42.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:42 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-13T16:25:42.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:42 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:42.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:42 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-13T16:25:42.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:42 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:42.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:42 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-13T16:25:42.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:42 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:43.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:43 smithi111 ceph-mon[34600]: Deploying daemon osd.6 on smithi016 2024-09-13T16:25:43.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:43 smithi111 ceph-mon[34600]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-13T16:25:43.750 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:43 smithi107 ceph-mon[34139]: Deploying daemon osd.6 on smithi016 2024-09-13T16:25:43.774 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:43 smithi107 ceph-mon[34139]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-13T16:25:43.775 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:43 smithi016 ceph-mon[31187]: Deploying daemon osd.6 on smithi016 2024-09-13T16:25:43.775 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:43 smithi016 ceph-mon[31187]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-13T16:25:44.772 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:44 smithi016 ceph-mon[31187]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-13T16:25:44.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:44 smithi111 ceph-mon[34600]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-13T16:25:44.974 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:44 smithi107 ceph-mon[34139]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-13T16:25:46.784 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:46 smithi016 ceph-mon[31187]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-13T16:25:46.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:46 smithi111 ceph-mon[34600]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-13T16:25:47.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:46 smithi107 ceph-mon[34139]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-13T16:25:47.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:47 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:25:47.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:47 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:47.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:47 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:48.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:47 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:25:48.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:47 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:48.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:47 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:48.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:47 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:25:48.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:47 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:48.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:47 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:49.040 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:48 smithi016 ceph-mon[31187]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-13T16:25:49.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:48 smithi107 ceph-mon[34139]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-13T16:25:49.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:48 smithi111 ceph-mon[34600]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-13T16:25:51.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:51.241+0000 7fad317fa640 1 -- 172.21.15.16:0/3914154092 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fad3402c550 con 0x7fad1c0a1080 2024-09-13T16:25:51.242 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 6 on host 'smithi016' 2024-09-13T16:25:51.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:51.243+0000 7fad3a27b640 1 -- 172.21.15.16:0/3914154092 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fad1c0a1080 msgr2=0x7fad1c0a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:25:51.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:51.243+0000 7fad3a27b640 1 --2- 172.21.15.16:0/3914154092 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fad1c0a1080 0x7fad1c0a3540 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fad34197150 tx=0x7fad24005a20 comp rx=0 tx=0).stop 2024-09-13T16:25:51.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:51.243+0000 7fad3a27b640 1 -- 172.21.15.16:0/3914154092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad34100f60 msgr2=0x7fad341961b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:25:51.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:51.243+0000 7fad3a27b640 1 --2- 172.21.15.16:0/3914154092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad34100f60 0x7fad341961b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fad2800d770 tx=0x7fad2800dc30 comp rx=0 tx=0).stop 2024-09-13T16:25:51.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:51.243+0000 7fad3a27b640 1 -- 172.21.15.16:0/3914154092 shutdown_connections 2024-09-13T16:25:51.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:51.243+0000 7fad3a27b640 1 --2- 172.21.15.16:0/3914154092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad34101920 0x7fad3419d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:25:51.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:51.243+0000 7fad3a27b640 1 --2- 172.21.15.16:0/3914154092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad34100f60 0x7fad341961b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:25:51.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:51.243+0000 7fad3a27b640 1 --2- 172.21.15.16:0/3914154092 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fad1c0a1080 0x7fad1c0a3540 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:25:51.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:51.243+0000 7fad3a27b640 1 --2- 172.21.15.16:0/3914154092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad340ffe00 0x7fad34195c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:25:51.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:51.243+0000 7fad3a27b640 1 -- 172.21.15.16:0/3914154092 >> 172.21.15.16:0/3914154092 conn(0x7fad340fb5d0 msgr2=0x7fad34104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:25:51.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:51.243+0000 7fad3a27b640 1 -- 172.21.15.16:0/3914154092 shutdown_connections 2024-09-13T16:25:51.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:25:51.243+0000 7fad3a27b640 1 -- 172.21.15.16:0/3914154092 wait complete. 2024-09-13T16:25:51.593 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:51 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:51.593 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:51 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:51.593 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:51 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:51.593 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:51 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:25:51.593 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:51 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:51.594 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:51 smithi016 ceph-mon[31187]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-13T16:25:51.594 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:51 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:25:51.594 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:51 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:51.594 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:51 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:51.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:51 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:51.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:51 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:51.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:51 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:51.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:51 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:25:51.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:51 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:51.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:51 smithi107 ceph-mon[34139]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-13T16:25:51.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:51 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:25:51.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:51 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:51.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:51 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:51.903 DEBUG:teuthology.orchestra.run.smithi016:osd.6> sudo journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.6.service 2024-09-13T16:25:51.905 INFO:tasks.cephadm:Deploying osd.7 on smithi107 with /dev/vg_nvme/lv_2... 2024-09-13T16:25:51.906 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-13T16:25:51.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:51 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:51.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:51 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:51.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:51 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:51.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:51 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:25:51.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:51 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:51.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:51 smithi111 ceph-mon[34600]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-13T16:25:51.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:51 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:25:51.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:51 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:51.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:51 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:52.109 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T16:25:52.555 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:52 smithi107 ceph-mon[34139]: from='osd.6 [v2:172.21.15.16:6818/1330214141,v1:172.21.15.16:6819/1330214141]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-13T16:25:52.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:52 smithi016 ceph-mon[31187]: from='osd.6 [v2:172.21.15.16:6818/1330214141,v1:172.21.15.16:6819/1330214141]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-13T16:25:52.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:52 smithi111 ceph-mon[34600]: from='osd.6 [v2:172.21.15.16:6818/1330214141,v1:172.21.15.16:6819/1330214141]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-13T16:25:53.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:53 smithi107 ceph-mon[34139]: from='osd.6 [v2:172.21.15.16:6818/1330214141,v1:172.21.15.16:6819/1330214141]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-13T16:25:53.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:53 smithi107 ceph-mon[34139]: osdmap e44: 7 total, 6 up, 7 in 2024-09-13T16:25:53.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:53 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T16:25:53.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:53 smithi107 ceph-mon[34139]: from='osd.6 [v2:172.21.15.16:6818/1330214141,v1:172.21.15.16:6819/1330214141]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-13T16:25:53.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:53 smithi107 ceph-mon[34139]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-13T16:25:53.850 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-13T16:25:53.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:53 smithi016 ceph-mon[31187]: from='osd.6 [v2:172.21.15.16:6818/1330214141,v1:172.21.15.16:6819/1330214141]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-13T16:25:53.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:53 smithi016 ceph-mon[31187]: osdmap e44: 7 total, 6 up, 7 in 2024-09-13T16:25:53.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:53 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T16:25:53.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:53 smithi016 ceph-mon[31187]: from='osd.6 [v2:172.21.15.16:6818/1330214141,v1:172.21.15.16:6819/1330214141]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-13T16:25:53.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:53 smithi016 ceph-mon[31187]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-13T16:25:53.932 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-13T16:25:53.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:53 smithi111 ceph-mon[34600]: from='osd.6 [v2:172.21.15.16:6818/1330214141,v1:172.21.15.16:6819/1330214141]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-13T16:25:53.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:53 smithi111 ceph-mon[34600]: osdmap e44: 7 total, 6 up, 7 in 2024-09-13T16:25:53.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:53 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T16:25:53.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:53 smithi111 ceph-mon[34600]: from='osd.6 [v2:172.21.15.16:6818/1330214141,v1:172.21.15.16:6819/1330214141]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-13T16:25:53.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:53 smithi111 ceph-mon[34600]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-13T16:25:53.945 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-09-13T16:25:53.946 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-09-13T16:25:53.946 INFO:teuthology.orchestra.run.smithi107.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0116297 s, 902 MB/s 2024-09-13T16:25:53.946 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 2024-09-13T16:25:53.946 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-09-13T16:25:54.201 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:25:53 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:25:53.918+0000 7f8b1864e640 -1 osd.6 0 waiting for initial osdmap 2024-09-13T16:25:54.201 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:25:53 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:25:53.929+0000 7f8b0fc58640 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T16:25:54.603 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_2 2024-09-13T16:25:54.799 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T16:25:54.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:54 smithi107 ceph-mon[34139]: from='osd.6 [v2:172.21.15.16:6818/1330214141,v1:172.21.15.16:6819/1330214141]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-13T16:25:54.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:54 smithi107 ceph-mon[34139]: osdmap e45: 7 total, 6 up, 7 in 2024-09-13T16:25:54.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:54 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T16:25:54.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:54 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T16:25:54.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:54 smithi107 ceph-mon[34139]: from='osd.6 [v2:172.21.15.16:6818/1330214141,v1:172.21.15.16:6819/1330214141]' entity='osd.6' 2024-09-13T16:25:54.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:54 smithi016 ceph-mon[31187]: from='osd.6 [v2:172.21.15.16:6818/1330214141,v1:172.21.15.16:6819/1330214141]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-13T16:25:54.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:54 smithi016 ceph-mon[31187]: osdmap e45: 7 total, 6 up, 7 in 2024-09-13T16:25:54.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:54 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T16:25:54.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:54 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T16:25:54.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:54 smithi016 ceph-mon[31187]: from='osd.6 [v2:172.21.15.16:6818/1330214141,v1:172.21.15.16:6819/1330214141]' entity='osd.6' 2024-09-13T16:25:54.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:54 smithi111 ceph-mon[34600]: from='osd.6 [v2:172.21.15.16:6818/1330214141,v1:172.21.15.16:6819/1330214141]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-13T16:25:54.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:54 smithi111 ceph-mon[34600]: osdmap e45: 7 total, 6 up, 7 in 2024-09-13T16:25:54.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:54 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T16:25:54.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:54 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T16:25:54.941 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:54 smithi111 ceph-mon[34600]: from='osd.6 [v2:172.21.15.16:6818/1330214141,v1:172.21.15.16:6819/1330214141]' entity='osd.6' 2024-09-13T16:25:55.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:55 smithi107 ceph-mon[34139]: purged_snaps scrub starts 2024-09-13T16:25:55.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:55 smithi107 ceph-mon[34139]: purged_snaps scrub ok 2024-09-13T16:25:55.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:55 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T16:25:55.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:55 smithi107 ceph-mon[34139]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-13T16:25:55.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:55 smithi107 ceph-mon[34139]: osd.6 [v2:172.21.15.16:6818/1330214141,v1:172.21.15.16:6819/1330214141] boot 2024-09-13T16:25:55.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:55 smithi107 ceph-mon[34139]: osdmap e46: 7 total, 7 up, 7 in 2024-09-13T16:25:55.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:55 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T16:25:55.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:55 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:55.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:55 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:55.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:55 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:25:55.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:55 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:25:55.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:55 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:25:55.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:55 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:55.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:55 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:55.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:55 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:25:55.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:55 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:55.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:55 smithi016 ceph-mon[31187]: purged_snaps scrub starts 2024-09-13T16:25:55.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:55 smithi016 ceph-mon[31187]: purged_snaps scrub ok 2024-09-13T16:25:55.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:55 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T16:25:55.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:55 smithi016 ceph-mon[31187]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-13T16:25:55.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:55 smithi016 ceph-mon[31187]: osd.6 [v2:172.21.15.16:6818/1330214141,v1:172.21.15.16:6819/1330214141] boot 2024-09-13T16:25:55.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:55 smithi016 ceph-mon[31187]: osdmap e46: 7 total, 7 up, 7 in 2024-09-13T16:25:55.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:55 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T16:25:55.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:55 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:55.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:55 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:55.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:55 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:25:55.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:55 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:25:55.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:55 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:25:55.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:55 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:55.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:55 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:55.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:55 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:25:55.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:55 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:55.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:55 smithi111 ceph-mon[34600]: purged_snaps scrub starts 2024-09-13T16:25:55.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:55 smithi111 ceph-mon[34600]: purged_snaps scrub ok 2024-09-13T16:25:55.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:55 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T16:25:55.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:55 smithi111 ceph-mon[34600]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-13T16:25:55.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:55 smithi111 ceph-mon[34600]: osd.6 [v2:172.21.15.16:6818/1330214141,v1:172.21.15.16:6819/1330214141] boot 2024-09-13T16:25:55.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:55 smithi111 ceph-mon[34600]: osdmap e46: 7 total, 7 up, 7 in 2024-09-13T16:25:55.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:55 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T16:25:55.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:55 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:55.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:55 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:55.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:55 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:25:55.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:55 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:25:55.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:55 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:25:55.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:55 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:55.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:55 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:55.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:55 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:25:55.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:55 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:25:56.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.492+0000 7f7d75407640 1 -- 172.21.15.107:0/3982939262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d70105ae0 msgr2=0x7f7d70107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:25:56.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.492+0000 7f7d75407640 1 --2- 172.21.15.107:0/3982939262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d70105ae0 0x7f7d70107f30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7d68009ca0 tx=0x7f7d6802a8c0 comp rx=0 tx=0).stop 2024-09-13T16:25:56.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.492+0000 7f7d75407640 1 -- 172.21.15.107:0/3982939262 shutdown_connections 2024-09-13T16:25:56.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.492+0000 7f7d75407640 1 --2- 172.21.15.107:0/3982939262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d70105ae0 0x7f7d70107f30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:25:56.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.492+0000 7f7d75407640 1 --2- 172.21.15.107:0/3982939262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d701031b0 0x7f7d701055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:25:56.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.492+0000 7f7d75407640 1 --2- 172.21.15.107:0/3982939262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d70100880 0x7f7d70102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:25:56.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.492+0000 7f7d75407640 1 -- 172.21.15.107:0/3982939262 >> 172.21.15.107:0/3982939262 conn(0x7f7d700fa480 msgr2=0x7f7d700fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:25:56.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.492+0000 7f7d75407640 1 -- 172.21.15.107:0/3982939262 shutdown_connections 2024-09-13T16:25:56.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.492+0000 7f7d75407640 1 -- 172.21.15.107:0/3982939262 wait complete. 2024-09-13T16:25:56.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.493+0000 7f7d75407640 1 Processor -- start 2024-09-13T16:25:56.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.493+0000 7f7d75407640 1 -- start start 2024-09-13T16:25:56.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.493+0000 7f7d75407640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d70100880 0x7f7d70195ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:25:56.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.493+0000 7f7d75407640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d701031b0 0x7f7d70196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:25:56.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.493+0000 7f7d75407640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d70105ae0 0x7f7d7019d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:25:56.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.493+0000 7f7d75407640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7019d7d0 con 0x7f7d70100880 2024-09-13T16:25:56.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.493+0000 7f7d75407640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7019d940 con 0x7f7d701031b0 2024-09-13T16:25:56.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.493+0000 7f7d75407640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7019db00 con 0x7f7d70105ae0 2024-09-13T16:25:56.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.493+0000 7f7d67fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d701031b0 0x7f7d70196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:25:56.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.493+0000 7f7d67fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d701031b0 0x7f7d70196220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50342/0 (socket says 172.21.15.107:50342) 2024-09-13T16:25:56.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.493+0000 7f7d67fff640 1 -- 172.21.15.107:0/2747692442 learned_addr learned my addr 172.21.15.107:0/2747692442 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-13T16:25:56.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.493+0000 7f7d6effd640 1 --2- 172.21.15.107:0/2747692442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d70100880 0x7f7d70195ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:25:56.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.493+0000 7f7d6f7fe640 1 --2- 172.21.15.107:0/2747692442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d70105ae0 0x7f7d7019d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:25:56.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.493+0000 7f7d67fff640 1 -- 172.21.15.107:0/2747692442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d70105ae0 msgr2=0x7f7d7019d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:25:56.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.493+0000 7f7d67fff640 1 --2- 172.21.15.107:0/2747692442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d70105ae0 0x7f7d7019d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:25:56.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.493+0000 7f7d67fff640 1 -- 172.21.15.107:0/2747692442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d70100880 msgr2=0x7f7d70195ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:25:56.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.493+0000 7f7d67fff640 1 --2- 172.21.15.107:0/2747692442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d70100880 0x7f7d70195ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:25:56.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.493+0000 7f7d67fff640 1 -- 172.21.15.107:0/2747692442 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d68009900 con 0x7f7d701031b0 2024-09-13T16:25:56.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.493+0000 7f7d67fff640 1 --2- 172.21.15.107:0/2747692442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d701031b0 0x7f7d70196220 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800e9f0 tx=0x7f7d5800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:25:56.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.494+0000 7f7d6cff9640 1 -- 172.21.15.107:0/2747692442 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d580096c0 con 0x7f7d701031b0 2024-09-13T16:25:56.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.494+0000 7f7d6cff9640 1 -- 172.21.15.107:0/2747692442 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7d5800ccc0 con 0x7f7d701031b0 2024-09-13T16:25:56.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.494+0000 7f7d75407640 1 -- 172.21.15.107:0/2747692442 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d7019dde0 con 0x7f7d701031b0 2024-09-13T16:25:56.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.494+0000 7f7d6cff9640 1 -- 172.21.15.107:0/2747692442 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d58004e60 con 0x7f7d701031b0 2024-09-13T16:25:56.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.494+0000 7f7d75407640 1 -- 172.21.15.107:0/2747692442 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d7019e200 con 0x7f7d701031b0 2024-09-13T16:25:56.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.495+0000 7f7d75407640 1 -- 172.21.15.107:0/2747692442 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d700fbf70 con 0x7f7d701031b0 2024-09-13T16:25:56.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.496+0000 7f7d6cff9640 1 -- 172.21.15.107:0/2747692442 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7d58005000 con 0x7f7d701031b0 2024-09-13T16:25:56.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.497+0000 7f7d6cff9640 1 --2- 172.21.15.107:0/2747692442 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7d5c0a1080 0x7f7d5c0a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:25:56.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.497+0000 7f7d6cff9640 1 -- 172.21.15.107:0/2747692442 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f7d58014070 con 0x7f7d701031b0 2024-09-13T16:25:56.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.497+0000 7f7d6effd640 1 --2- 172.21.15.107:0/2747692442 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7d5c0a1080 0x7f7d5c0a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:25:56.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.497+0000 7f7d6effd640 1 --2- 172.21.15.107:0/2747692442 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7d5c0a1080 0x7f7d5c0a3540 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7d701971c0 tx=0x7f7d60005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:25:56.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.501+0000 7f7d6cff9640 1 -- 172.21.15.107:0/2747692442 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7d58096450 con 0x7f7d701031b0 2024-09-13T16:25:56.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:56 smithi107 ceph-mon[34139]: Detected new or changed devices on smithi016 2024-09-13T16:25:56.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:56 smithi107 ceph-mon[34139]: Adjusting osd_memory_target on smithi016 to 5697M 2024-09-13T16:25:56.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:25:56.672+0000 7f7d75407640 1 -- 172.21.15.107:0/2747692442 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d7002c4f0 con 0x7f7d5c0a1080 2024-09-13T16:25:56.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:56 smithi016 ceph-mon[31187]: Detected new or changed devices on smithi016 2024-09-13T16:25:56.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:56 smithi016 ceph-mon[31187]: Adjusting osd_memory_target on smithi016 to 5697M 2024-09-13T16:25:56.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:56 smithi111 ceph-mon[34600]: Detected new or changed devices on smithi016 2024-09-13T16:25:56.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:56 smithi111 ceph-mon[34600]: Adjusting osd_memory_target on smithi016 to 5697M 2024-09-13T16:25:57.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:57 smithi107 ceph-mon[34139]: osdmap e47: 7 total, 7 up, 7 in 2024-09-13T16:25:57.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:57 smithi107 ceph-mon[34139]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:25:57.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:57 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:25:57.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:57 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:25:57.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:57 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:57.755 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:57 smithi016 ceph-mon[31187]: osdmap e47: 7 total, 7 up, 7 in 2024-09-13T16:25:57.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:57 smithi016 ceph-mon[31187]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:25:57.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:57 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:25:57.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:57 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:25:57.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:57 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:57.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:57 smithi111 ceph-mon[34600]: osdmap e47: 7 total, 7 up, 7 in 2024-09-13T16:25:57.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:57 smithi111 ceph-mon[34600]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:25:57.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:57 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:25:57.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:57 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:25:57.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:57 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:25:58.687 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:58 smithi107 ceph-mon[34139]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:25:58.768 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:58 smithi016 ceph-mon[31187]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:25:58.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:58 smithi111 ceph-mon[34600]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:25:59.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:59 smithi107 ceph-mon[34139]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:25:59.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:59 smithi107 ceph-mon[34139]: from='client.? 172.21.15.107:0/675946227' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee826a9c-f441-4903-9b1b-66b627f7b712"}]: dispatch 2024-09-13T16:25:59.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:59 smithi107 ceph-mon[34139]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee826a9c-f441-4903-9b1b-66b627f7b712"}]: dispatch 2024-09-13T16:25:59.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:59 smithi107 ceph-mon[34139]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee826a9c-f441-4903-9b1b-66b627f7b712"}]': finished 2024-09-13T16:25:59.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:59 smithi107 ceph-mon[34139]: osdmap e48: 8 total, 7 up, 8 in 2024-09-13T16:25:59.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:25:59 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T16:25:59.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:59 smithi016 ceph-mon[31187]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:25:59.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:59 smithi016 ceph-mon[31187]: from='client.? 172.21.15.107:0/675946227' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee826a9c-f441-4903-9b1b-66b627f7b712"}]: dispatch 2024-09-13T16:25:59.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:59 smithi016 ceph-mon[31187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee826a9c-f441-4903-9b1b-66b627f7b712"}]: dispatch 2024-09-13T16:25:59.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:59 smithi016 ceph-mon[31187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee826a9c-f441-4903-9b1b-66b627f7b712"}]': finished 2024-09-13T16:25:59.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:59 smithi016 ceph-mon[31187]: osdmap e48: 8 total, 7 up, 8 in 2024-09-13T16:25:59.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:25:59 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T16:25:59.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:59 smithi111 ceph-mon[34600]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:25:59.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:59 smithi111 ceph-mon[34600]: from='client.? 172.21.15.107:0/675946227' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee826a9c-f441-4903-9b1b-66b627f7b712"}]: dispatch 2024-09-13T16:25:59.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:59 smithi111 ceph-mon[34600]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee826a9c-f441-4903-9b1b-66b627f7b712"}]: dispatch 2024-09-13T16:25:59.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:59 smithi111 ceph-mon[34600]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee826a9c-f441-4903-9b1b-66b627f7b712"}]': finished 2024-09-13T16:25:59.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:59 smithi111 ceph-mon[34600]: osdmap e48: 8 total, 7 up, 8 in 2024-09-13T16:25:59.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:25:59 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T16:26:00.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:00 smithi107 ceph-mon[34139]: from='client.? 172.21.15.107:0/211580148' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:26:00.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:00 smithi016 ceph-mon[31187]: from='client.? 172.21.15.107:0/211580148' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:26:00.936 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:00 smithi111 ceph-mon[34600]: from='client.? 172.21.15.107:0/211580148' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:26:01.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:01 smithi107 ceph-mon[34139]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:26:01.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:01 smithi016 ceph-mon[31187]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:26:01.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:01 smithi111 ceph-mon[34600]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:26:02.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:02 smithi107 ceph-mon[34139]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:26:02.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:02 smithi111 ceph-mon[34600]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:26:03.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:02 smithi016 ceph-mon[31187]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:26:04.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:04 smithi111 ceph-mon[34600]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:26:05.003 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:04 smithi107 ceph-mon[34139]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:26:05.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:04 smithi016 ceph-mon[31187]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:26:06.314 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:06 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-13T16:26:06.314 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:06 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:26:06.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:06 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-13T16:26:06.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:06 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:26:06.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:06 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-13T16:26:06.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:06 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:26:07.337 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:07 smithi107 ceph-mon[34139]: Deploying daemon osd.7 on smithi107 2024-09-13T16:26:07.337 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:07 smithi107 ceph-mon[34139]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:26:07.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:07 smithi016 ceph-mon[31187]: Deploying daemon osd.7 on smithi107 2024-09-13T16:26:07.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:07 smithi016 ceph-mon[31187]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:26:07.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:07 smithi111 ceph-mon[34600]: Deploying daemon osd.7 on smithi107 2024-09-13T16:26:07.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:07 smithi111 ceph-mon[34600]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:26:08.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:08 smithi111 ceph-mon[34600]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:26:08.969 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:08 smithi107 ceph-mon[34139]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:26:09.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:08 smithi016 ceph-mon[31187]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:26:10.837 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:10 smithi107 ceph-mon[34139]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:26:10.936 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:10 smithi111 ceph-mon[34600]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:26:11.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:10 smithi016 ceph-mon[31187]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:26:11.814 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:11 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:26:11.814 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:11 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:11.814 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:11 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:11.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:11 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:26:11.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:11 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:11.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:11 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:12.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:11 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:26:12.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:11 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:12.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:11 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:12.769 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:12 smithi107 ceph-mon[34139]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:26:12.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:12 smithi111 ceph-mon[34600]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:26:13.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:12 smithi016 ceph-mon[31187]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:26:14.555 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 7 on host 'smithi107' 2024-09-13T16:26:14.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:26:14.553+0000 7f7d6cff9640 1 -- 172.21.15.107:0/2747692442 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7d7002c4f0 con 0x7f7d5c0a1080 2024-09-13T16:26:14.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:26:14.556+0000 7f7d75407640 1 -- 172.21.15.107:0/2747692442 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7d5c0a1080 msgr2=0x7f7d5c0a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:26:14.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:26:14.556+0000 7f7d75407640 1 --2- 172.21.15.107:0/2747692442 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7d5c0a1080 0x7f7d5c0a3540 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7d701971c0 tx=0x7f7d60005f90 comp rx=0 tx=0).stop 2024-09-13T16:26:14.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:26:14.556+0000 7f7d75407640 1 -- 172.21.15.107:0/2747692442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d701031b0 msgr2=0x7f7d70196220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:26:14.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:26:14.556+0000 7f7d75407640 1 --2- 172.21.15.107:0/2747692442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d701031b0 0x7f7d70196220 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800e9f0 tx=0x7f7d5800eeb0 comp rx=0 tx=0).stop 2024-09-13T16:26:14.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:26:14.556+0000 7f7d75407640 1 -- 172.21.15.107:0/2747692442 shutdown_connections 2024-09-13T16:26:14.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:26:14.556+0000 7f7d75407640 1 --2- 172.21.15.107:0/2747692442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d70105ae0 0x7f7d7019d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:26:14.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:26:14.556+0000 7f7d75407640 1 --2- 172.21.15.107:0/2747692442 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7d5c0a1080 0x7f7d5c0a3540 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:26:14.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:26:14.556+0000 7f7d75407640 1 --2- 172.21.15.107:0/2747692442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d701031b0 0x7f7d70196220 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:26:14.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:26:14.556+0000 7f7d75407640 1 --2- 172.21.15.107:0/2747692442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d70100880 0x7f7d70195ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:26:14.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:26:14.556+0000 7f7d75407640 1 -- 172.21.15.107:0/2747692442 >> 172.21.15.107:0/2747692442 conn(0x7f7d700fa480 msgr2=0x7f7d70101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:26:14.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:26:14.556+0000 7f7d75407640 1 -- 172.21.15.107:0/2747692442 shutdown_connections 2024-09-13T16:26:14.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:26:14.556+0000 7f7d75407640 1 -- 172.21.15.107:0/2747692442 wait complete. 2024-09-13T16:26:14.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:14 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:14.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:14 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:14.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:14 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:26:14.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:14 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:26:14.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:14 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:14.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:14 smithi107 ceph-mon[34139]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:26:14.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:14 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:26:14.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:14 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:14.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:14 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:15.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:14 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:15.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:14 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:15.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:14 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:26:15.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:14 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:26:15.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:14 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:15.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:14 smithi016 ceph-mon[31187]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:26:15.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:14 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:26:15.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:14 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:15.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:14 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:15.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:14 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:15.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:14 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:15.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:14 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:26:15.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:14 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:26:15.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:14 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:15.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:14 smithi111 ceph-mon[34600]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:26:15.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:14 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:26:15.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:14 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:15.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:14 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:15.347 DEBUG:teuthology.orchestra.run.smithi107:osd.7> sudo journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.7.service 2024-09-13T16:26:15.349 INFO:tasks.cephadm:Deploying osd.8 on smithi111 with /dev/vg_nvme/lv_2... 2024-09-13T16:26:15.349 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-13T16:26:15.555 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T16:26:15.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:15 smithi111 ceph-mon[34600]: from='osd.7 [v2:172.21.15.107:6816/1133876754,v1:172.21.15.107:6817/1133876754]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-13T16:26:15.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:15 smithi111 ceph-mon[34600]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-13T16:26:16.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:15 smithi107 ceph-mon[34139]: from='osd.7 [v2:172.21.15.107:6816/1133876754,v1:172.21.15.107:6817/1133876754]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-13T16:26:16.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:15 smithi107 ceph-mon[34139]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-13T16:26:16.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:15 smithi016 ceph-mon[31187]: from='osd.7 [v2:172.21.15.107:6816/1133876754,v1:172.21.15.107:6817/1133876754]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-13T16:26:16.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:15 smithi016 ceph-mon[31187]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-13T16:26:16.968 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:16 smithi111 ceph-mon[34600]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-13T16:26:16.968 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:16 smithi111 ceph-mon[34600]: osdmap e49: 8 total, 7 up, 8 in 2024-09-13T16:26:16.968 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:16 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T16:26:16.968 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:16 smithi111 ceph-mon[34600]: from='osd.7 [v2:172.21.15.107:6816/1133876754,v1:172.21.15.107:6817/1133876754]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-13T16:26:16.968 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:16 smithi111 ceph-mon[34600]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-13T16:26:16.969 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:16 smithi111 ceph-mon[34600]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:26:17.079 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:16 smithi107 ceph-mon[34139]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-13T16:26:17.079 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:16 smithi107 ceph-mon[34139]: osdmap e49: 8 total, 7 up, 8 in 2024-09-13T16:26:17.079 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:16 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T16:26:17.079 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:16 smithi107 ceph-mon[34139]: from='osd.7 [v2:172.21.15.107:6816/1133876754,v1:172.21.15.107:6817/1133876754]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-13T16:26:17.079 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:16 smithi107 ceph-mon[34139]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-13T16:26:17.080 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:16 smithi107 ceph-mon[34139]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:26:17.080 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:26:16 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:26:16.796+0000 7fda108f0640 -1 osd.7 0 waiting for initial osdmap 2024-09-13T16:26:17.080 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:26:16 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:26:16.806+0000 7fda0c719640 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T16:26:17.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:16 smithi016 ceph-mon[31187]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-13T16:26:17.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:16 smithi016 ceph-mon[31187]: osdmap e49: 8 total, 7 up, 8 in 2024-09-13T16:26:17.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:16 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T16:26:17.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:16 smithi016 ceph-mon[31187]: from='osd.7 [v2:172.21.15.107:6816/1133876754,v1:172.21.15.107:6817/1133876754]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-13T16:26:17.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:16 smithi016 ceph-mon[31187]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-13T16:26:17.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:16 smithi016 ceph-mon[31187]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-13T16:26:17.484 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-13T16:26:17.572 INFO:teuthology.orchestra.run.smithi111.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-13T16:26:17.586 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10+0 records in 2024-09-13T16:26:17.586 INFO:teuthology.orchestra.run.smithi111.stderr:10+0 records out 2024-09-13T16:26:17.586 INFO:teuthology.orchestra.run.smithi111.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115393 s, 909 MB/s 2024-09-13T16:26:17.586 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 2024-09-13T16:26:17.586 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping successful for: 2024-09-13T16:26:17.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:17 smithi107 ceph-mon[34139]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-13T16:26:17.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:17 smithi107 ceph-mon[34139]: osdmap e50: 8 total, 7 up, 8 in 2024-09-13T16:26:17.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:17 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T16:26:17.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:17 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T16:26:17.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:17 smithi107 ceph-mon[34139]: osd.7 [v2:172.21.15.107:6816/1133876754,v1:172.21.15.107:6817/1133876754] boot 2024-09-13T16:26:17.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:17 smithi107 ceph-mon[34139]: osdmap e51: 8 total, 8 up, 8 in 2024-09-13T16:26:17.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:17 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T16:26:17.895 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:17 smithi111 ceph-mon[34600]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-13T16:26:17.895 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:17 smithi111 ceph-mon[34600]: osdmap e50: 8 total, 7 up, 8 in 2024-09-13T16:26:17.895 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:17 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T16:26:17.895 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:17 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T16:26:17.895 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:17 smithi111 ceph-mon[34600]: osd.7 [v2:172.21.15.107:6816/1133876754,v1:172.21.15.107:6817/1133876754] boot 2024-09-13T16:26:17.896 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:17 smithi111 ceph-mon[34600]: osdmap e51: 8 total, 8 up, 8 in 2024-09-13T16:26:17.896 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:17 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T16:26:18.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:17 smithi016 ceph-mon[31187]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-13T16:26:18.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:17 smithi016 ceph-mon[31187]: osdmap e50: 8 total, 7 up, 8 in 2024-09-13T16:26:18.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:17 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T16:26:18.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:17 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T16:26:18.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:17 smithi016 ceph-mon[31187]: osd.7 [v2:172.21.15.107:6816/1133876754,v1:172.21.15.107:6817/1133876754] boot 2024-09-13T16:26:18.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:17 smithi016 ceph-mon[31187]: osdmap e51: 8 total, 8 up, 8 in 2024-09-13T16:26:18.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:17 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T16:26:18.185 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi111:vg_nvme/lv_2 2024-09-13T16:26:18.386 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T16:26:19.077 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:18 smithi111 ceph-mon[34600]: purged_snaps scrub starts 2024-09-13T16:26:19.077 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:18 smithi111 ceph-mon[34600]: purged_snaps scrub ok 2024-09-13T16:26:19.077 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:18 smithi111 ceph-mon[34600]: osdmap e52: 8 total, 8 up, 8 in 2024-09-13T16:26:19.077 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:18 smithi111 ceph-mon[34600]: Detected new or changed devices on smithi107 2024-09-13T16:26:19.077 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:18 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:19.078 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:18 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:19.078 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:18 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:26:19.078 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:18 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:26:19.078 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:18 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:26:19.078 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:18 smithi111 ceph-mon[34600]: Adjusting osd_memory_target on smithi107 to 5695M 2024-09-13T16:26:19.078 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:18 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:19.078 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:18 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:26:19.078 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:18 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:26:19.078 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:18 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:19.078 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:18 smithi111 ceph-mon[34600]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:19.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:18 smithi107 ceph-mon[34139]: purged_snaps scrub starts 2024-09-13T16:26:19.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:18 smithi107 ceph-mon[34139]: purged_snaps scrub ok 2024-09-13T16:26:19.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:18 smithi107 ceph-mon[34139]: osdmap e52: 8 total, 8 up, 8 in 2024-09-13T16:26:19.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:18 smithi107 ceph-mon[34139]: Detected new or changed devices on smithi107 2024-09-13T16:26:19.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:18 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:19.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:18 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:19.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:18 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:26:19.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:18 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:26:19.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:18 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:26:19.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:18 smithi107 ceph-mon[34139]: Adjusting osd_memory_target on smithi107 to 5695M 2024-09-13T16:26:19.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:18 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:19.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:18 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:26:19.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:18 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:26:19.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:18 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:19.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:18 smithi107 ceph-mon[34139]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:19.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:18 smithi016 ceph-mon[31187]: purged_snaps scrub starts 2024-09-13T16:26:19.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:18 smithi016 ceph-mon[31187]: purged_snaps scrub ok 2024-09-13T16:26:19.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:18 smithi016 ceph-mon[31187]: osdmap e52: 8 total, 8 up, 8 in 2024-09-13T16:26:19.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:18 smithi016 ceph-mon[31187]: Detected new or changed devices on smithi107 2024-09-13T16:26:19.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:18 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:19.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:18 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:19.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:18 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:26:19.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:18 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:26:19.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:18 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:26:19.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:18 smithi016 ceph-mon[31187]: Adjusting osd_memory_target on smithi107 to 5695M 2024-09-13T16:26:19.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:18 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:19.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:18 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:26:19.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:18 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:26:19.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:18 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:19.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:18 smithi016 ceph-mon[31187]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:19.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.944+0000 7ff30466e640 1 -- 172.21.15.111:0/643940496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2fc0fecf0 msgr2=0x7ff2fc0ff0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:26:19.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.944+0000 7ff30466e640 1 --2- 172.21.15.111:0/643940496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2fc0fecf0 0x7ff2fc0ff0f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009900 tx=0x7ff2f802a410 comp rx=0 tx=0).stop 2024-09-13T16:26:19.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.944+0000 7ff30466e640 1 -- 172.21.15.111:0/643940496 shutdown_connections 2024-09-13T16:26:19.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.944+0000 7ff30466e640 1 --2- 172.21.15.111:0/643940496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc1008b0 0x7ff2fc107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:26:19.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.944+0000 7ff30466e640 1 --2- 172.21.15.111:0/643940496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2fc0ffef0 0x7ff2fc100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:26:19.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.944+0000 7ff30466e640 1 --2- 172.21.15.111:0/643940496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2fc0fecf0 0x7ff2fc0ff0f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:26:19.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.944+0000 7ff30466e640 1 -- 172.21.15.111:0/643940496 >> 172.21.15.111:0/643940496 conn(0x7ff2fc0fa4c0 msgr2=0x7ff2fc0fc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:26:19.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.944+0000 7ff30466e640 1 -- 172.21.15.111:0/643940496 shutdown_connections 2024-09-13T16:26:19.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.944+0000 7ff30466e640 1 -- 172.21.15.111:0/643940496 wait complete. 2024-09-13T16:26:19.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.945+0000 7ff30466e640 1 Processor -- start 2024-09-13T16:26:19.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.945+0000 7ff30466e640 1 -- start start 2024-09-13T16:26:19.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.945+0000 7ff30466e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2fc0fecf0 0x7ff2fc195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:26:19.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.945+0000 7ff30466e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2fc0ffef0 0x7ff2fc196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:26:19.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.945+0000 7ff30466e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc1008b0 0x7ff2fc19d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:26:19.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.945+0000 7ff30466e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc19d740 con 0x7ff2fc1008b0 2024-09-13T16:26:19.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.945+0000 7ff30466e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc19d8b0 con 0x7ff2fc0fecf0 2024-09-13T16:26:19.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.945+0000 7ff30466e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc19da70 con 0x7ff2fc0ffef0 2024-09-13T16:26:19.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.945+0000 7ff301c09640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2fc0ffef0 0x7ff2fc196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:26:19.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.945+0000 7ff301c09640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2fc0ffef0 0x7ff2fc196190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52818/0 (socket says 172.21.15.111:52818) 2024-09-13T16:26:19.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.945+0000 7ff301c09640 1 -- 172.21.15.111:0/3624294278 learned_addr learned my addr 172.21.15.111:0/3624294278 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T16:26:19.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.945+0000 7ff302c0b640 1 --2- 172.21.15.111:0/3624294278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc1008b0 0x7ff2fc19d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:26:19.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.945+0000 7ff30240a640 1 --2- 172.21.15.111:0/3624294278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2fc0fecf0 0x7ff2fc195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:26:19.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.945+0000 7ff301c09640 1 -- 172.21.15.111:0/3624294278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2fc0fecf0 msgr2=0x7ff2fc195c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:26:19.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.945+0000 7ff301c09640 1 --2- 172.21.15.111:0/3624294278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2fc0fecf0 0x7ff2fc195c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:26:19.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.945+0000 7ff301c09640 1 -- 172.21.15.111:0/3624294278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc1008b0 msgr2=0x7ff2fc19d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:26:19.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.945+0000 7ff301c09640 1 --2- 172.21.15.111:0/3624294278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc1008b0 0x7ff2fc19d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:26:19.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.945+0000 7ff301c09640 1 -- 172.21.15.111:0/3624294278 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f8009560 con 0x7ff2fc0ffef0 2024-09-13T16:26:19.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.945+0000 7ff302c0b640 1 --2- 172.21.15.111:0/3624294278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc1008b0 0x7ff2fc19d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:26:19.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.945+0000 7ff30240a640 1 --2- 172.21.15.111:0/3624294278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2fc0fecf0 0x7ff2fc195c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:26:19.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.945+0000 7ff301c09640 1 --2- 172.21.15.111:0/3624294278 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2fc0ffef0 0x7ff2fc196190 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00d950 tx=0x7ff2ec00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:26:19.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.945+0000 7ff2f37fe640 1 -- 172.21.15.111:0/3624294278 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec009c30 con 0x7ff2fc0ffef0 2024-09-13T16:26:19.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.946+0000 7ff2f37fe640 1 -- 172.21.15.111:0/3624294278 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff2ec009d90 con 0x7ff2fc0ffef0 2024-09-13T16:26:19.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.946+0000 7ff30466e640 1 -- 172.21.15.111:0/3624294278 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2fc19dd50 con 0x7ff2fc0ffef0 2024-09-13T16:26:19.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.946+0000 7ff2f37fe640 1 -- 172.21.15.111:0/3624294278 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec00f620 con 0x7ff2fc0ffef0 2024-09-13T16:26:19.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.946+0000 7ff30466e640 1 -- 172.21.15.111:0/3624294278 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2fc19e1e0 con 0x7ff2fc0ffef0 2024-09-13T16:26:19.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.947+0000 7ff30466e640 1 -- 172.21.15.111:0/3624294278 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2fc106190 con 0x7ff2fc0ffef0 2024-09-13T16:26:19.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.948+0000 7ff2f37fe640 1 -- 172.21.15.111:0/3624294278 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff2ec0109f0 con 0x7ff2fc0ffef0 2024-09-13T16:26:19.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.948+0000 7ff2f37fe640 1 --2- 172.21.15.111:0/3624294278 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff2e80a1100 0x7ff2e80a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:26:19.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.949+0000 7ff2f37fe640 1 -- 172.21.15.111:0/3624294278 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5426+0+0 (secure 0 0 0) 0x7ff2ec0cb9f0 con 0x7ff2fc0ffef0 2024-09-13T16:26:19.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.949+0000 7ff30240a640 1 --2- 172.21.15.111:0/3624294278 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff2e80a1100 0x7ff2e80a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:26:19.955 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.949+0000 7ff30240a640 1 --2- 172.21.15.111:0/3624294278 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff2e80a1100 0x7ff2e80a35c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8000c00 tx=0x7ff2f8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:26:19.956 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:19.953+0000 7ff2f37fe640 1 -- 172.21.15.111:0/3624294278 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff2ec095cc0 con 0x7ff2fc0ffef0 2024-09-13T16:26:20.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:20.124+0000 7ff30466e640 1 -- 172.21.15.111:0/3624294278 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2fc02c520 con 0x7ff2e80a1100 2024-09-13T16:26:20.510 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:20 smithi111 ceph-mon[34600]: osdmap e53: 8 total, 8 up, 8 in 2024-09-13T16:26:20.510 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:20 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:26:20.510 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:20 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:26:20.510 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:20 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:26:20.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:20 smithi107 ceph-mon[34139]: osdmap e53: 8 total, 8 up, 8 in 2024-09-13T16:26:20.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:20 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:26:20.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:20 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:26:20.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:20 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:26:20.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:20 smithi016 ceph-mon[31187]: osdmap e53: 8 total, 8 up, 8 in 2024-09-13T16:26:20.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:20 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:26:20.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:20 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:26:20.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:20 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:26:21.547 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:21 smithi111 ceph-mon[34600]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:26:21.547 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:21 smithi111 ceph-mon[34600]: osdmap e54: 8 total, 8 up, 8 in 2024-09-13T16:26:21.548 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:21 smithi111 ceph-mon[34600]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:21.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:21 smithi107 ceph-mon[34139]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:26:21.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:21 smithi107 ceph-mon[34139]: osdmap e54: 8 total, 8 up, 8 in 2024-09-13T16:26:21.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:21 smithi107 ceph-mon[34139]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:21.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:21 smithi016 ceph-mon[31187]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:26:21.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:21 smithi016 ceph-mon[31187]: osdmap e54: 8 total, 8 up, 8 in 2024-09-13T16:26:21.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:21 smithi016 ceph-mon[31187]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:22.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:22 smithi107 ceph-mon[34139]: osdmap e55: 8 total, 8 up, 8 in 2024-09-13T16:26:22.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:22 smithi016 ceph-mon[31187]: osdmap e55: 8 total, 8 up, 8 in 2024-09-13T16:26:22.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:22 smithi111 ceph-mon[34600]: osdmap e55: 8 total, 8 up, 8 in 2024-09-13T16:26:23.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:23 smithi111 ceph-mon[34600]: from='client.? 172.21.15.111:0/3033929601' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e19773f2-dec8-44e0-9d33-0252e07d6415"}]: dispatch 2024-09-13T16:26:23.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:23 smithi111 ceph-mon[34600]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e19773f2-dec8-44e0-9d33-0252e07d6415"}]: dispatch 2024-09-13T16:26:23.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:23 smithi111 ceph-mon[34600]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e19773f2-dec8-44e0-9d33-0252e07d6415"}]': finished 2024-09-13T16:26:23.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:23 smithi111 ceph-mon[34600]: osdmap e56: 9 total, 8 up, 9 in 2024-09-13T16:26:23.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:23 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T16:26:23.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:23 smithi111 ceph-mon[34600]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:23.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:23 smithi111 ceph-mon[34600]: from='client.? 172.21.15.111:0/3649375613' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:26:23.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:23 smithi107 ceph-mon[34139]: from='client.? 172.21.15.111:0/3033929601' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e19773f2-dec8-44e0-9d33-0252e07d6415"}]: dispatch 2024-09-13T16:26:23.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:23 smithi107 ceph-mon[34139]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e19773f2-dec8-44e0-9d33-0252e07d6415"}]: dispatch 2024-09-13T16:26:23.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:23 smithi107 ceph-mon[34139]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e19773f2-dec8-44e0-9d33-0252e07d6415"}]': finished 2024-09-13T16:26:23.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:23 smithi107 ceph-mon[34139]: osdmap e56: 9 total, 8 up, 9 in 2024-09-13T16:26:23.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:23 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T16:26:23.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:23 smithi107 ceph-mon[34139]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:23.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:23 smithi107 ceph-mon[34139]: from='client.? 172.21.15.111:0/3649375613' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:26:23.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:23 smithi016 ceph-mon[31187]: from='client.? 172.21.15.111:0/3033929601' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e19773f2-dec8-44e0-9d33-0252e07d6415"}]: dispatch 2024-09-13T16:26:23.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:23 smithi016 ceph-mon[31187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e19773f2-dec8-44e0-9d33-0252e07d6415"}]: dispatch 2024-09-13T16:26:23.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:23 smithi016 ceph-mon[31187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e19773f2-dec8-44e0-9d33-0252e07d6415"}]': finished 2024-09-13T16:26:23.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:23 smithi016 ceph-mon[31187]: osdmap e56: 9 total, 8 up, 9 in 2024-09-13T16:26:23.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:23 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T16:26:23.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:23 smithi016 ceph-mon[31187]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:23.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:23 smithi016 ceph-mon[31187]: from='client.? 172.21.15.111:0/3649375613' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:26:24.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:24 smithi107 ceph-mon[34139]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:24.936 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:24 smithi111 ceph-mon[34600]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:25.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:24 smithi016 ceph-mon[31187]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:26.757 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:26 smithi111 ceph-mon[34600]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:27.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:26 smithi107 ceph-mon[34139]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:27.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:26 smithi016 ceph-mon[31187]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:28.813 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:28 smithi111 ceph-mon[34600]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:28.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:28 smithi016 ceph-mon[31187]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:29.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:28 smithi107 ceph-mon[34139]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:29.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:29 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-13T16:26:29.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:29 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:26:29.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:29 smithi111 ceph-mon[34600]: Deploying daemon osd.8 on smithi111 2024-09-13T16:26:30.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:29 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-13T16:26:30.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:29 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:26:30.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:29 smithi107 ceph-mon[34139]: Deploying daemon osd.8 on smithi111 2024-09-13T16:26:30.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:29 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-13T16:26:30.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:29 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:26:30.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:29 smithi016 ceph-mon[31187]: Deploying daemon osd.8 on smithi111 2024-09-13T16:26:30.830 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:30 smithi111 ceph-mon[34600]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:31.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:30 smithi107 ceph-mon[34139]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:31.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:30 smithi016 ceph-mon[31187]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:32.829 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:32 smithi111 ceph-mon[34600]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:33.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:32 smithi107 ceph-mon[34139]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:33.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:32 smithi016 ceph-mon[31187]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:34.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:34 smithi111 ceph-mon[34600]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:35.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:34 smithi107 ceph-mon[34139]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:35.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:34 smithi016 ceph-mon[31187]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:35.883 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:35 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:26:35.883 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:35 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:35.883 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:35 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:36.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:35 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:26:36.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:35 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:36.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:35 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:36.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:35 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:26:36.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:35 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:36.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:35 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:36.787 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:36 smithi111 ceph-mon[34600]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:37.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:36 smithi107 ceph-mon[34139]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:37.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:36 smithi016 ceph-mon[31187]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:38.155 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 8 on host 'smithi111' 2024-09-13T16:26:38.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:38.153+0000 7ff2f37fe640 1 -- 172.21.15.111:0/3624294278 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff2fc02c520 con 0x7ff2e80a1100 2024-09-13T16:26:38.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:38.155+0000 7ff30466e640 1 -- 172.21.15.111:0/3624294278 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff2e80a1100 msgr2=0x7ff2e80a35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:26:38.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:38.155+0000 7ff30466e640 1 --2- 172.21.15.111:0/3624294278 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff2e80a1100 0x7ff2e80a35c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8000c00 tx=0x7ff2f8005a20 comp rx=0 tx=0).stop 2024-09-13T16:26:38.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:38.155+0000 7ff30466e640 1 -- 172.21.15.111:0/3624294278 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2fc0ffef0 msgr2=0x7ff2fc196190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:26:38.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:38.155+0000 7ff30466e640 1 --2- 172.21.15.111:0/3624294278 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2fc0ffef0 0x7ff2fc196190 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00d950 tx=0x7ff2ec00de10 comp rx=0 tx=0).stop 2024-09-13T16:26:38.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:38.155+0000 7ff30466e640 1 -- 172.21.15.111:0/3624294278 shutdown_connections 2024-09-13T16:26:38.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:38.155+0000 7ff30466e640 1 --2- 172.21.15.111:0/3624294278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc1008b0 0x7ff2fc19d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:26:38.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:38.155+0000 7ff30466e640 1 --2- 172.21.15.111:0/3624294278 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2fc0ffef0 0x7ff2fc196190 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:26:38.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:38.155+0000 7ff30466e640 1 --2- 172.21.15.111:0/3624294278 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff2e80a1100 0x7ff2e80a35c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:26:38.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:38.155+0000 7ff30466e640 1 --2- 172.21.15.111:0/3624294278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2fc0fecf0 0x7ff2fc195c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:26:38.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:38.155+0000 7ff30466e640 1 -- 172.21.15.111:0/3624294278 >> 172.21.15.111:0/3624294278 conn(0x7ff2fc0fa4c0 msgr2=0x7ff2fc103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:26:38.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:38.155+0000 7ff30466e640 1 -- 172.21.15.111:0/3624294278 shutdown_connections 2024-09-13T16:26:38.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:26:38.155+0000 7ff30466e640 1 -- 172.21.15.111:0/3624294278 wait complete. 2024-09-13T16:26:38.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:38 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:38.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:38 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:38.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:38 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:26:38.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:38 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:26:38.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:38 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:38.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:38 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:26:38.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:38 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:38.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:38 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:38.830 DEBUG:teuthology.orchestra.run.smithi111:osd.8> sudo journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.8.service 2024-09-13T16:26:38.832 INFO:tasks.cephadm:Deploying osd.9 on smithi016 with /dev/vg_nvme/lv_1... 2024-09-13T16:26:38.832 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-13T16:26:38.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:38 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:38.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:38 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:38.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:38 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:26:38.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:38 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:26:38.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:38 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:38.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:38 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:26:38.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:38 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:38.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:38 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:38.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:38 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:38.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:38 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:38.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:38 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:26:38.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:38 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:26:38.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:38 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:38.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:38 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:26:38.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:38 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:38.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:38 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:39.048 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:26:39.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:39 smithi111 ceph-mon[34600]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:39.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:39 smithi111 ceph-mon[34600]: from='osd.8 [v2:172.21.15.111:6816/837548190,v1:172.21.15.111:6817/837548190]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-13T16:26:39.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:39 smithi111 ceph-mon[34600]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-13T16:26:39.491 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:39 smithi016 ceph-mon[31187]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:39.491 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:39 smithi016 ceph-mon[31187]: from='osd.8 [v2:172.21.15.111:6816/837548190,v1:172.21.15.111:6817/837548190]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-13T16:26:39.491 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:39 smithi016 ceph-mon[31187]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-13T16:26:39.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:39 smithi107 ceph-mon[34139]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:39.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:39 smithi107 ceph-mon[34139]: from='osd.8 [v2:172.21.15.111:6816/837548190,v1:172.21.15.111:6817/837548190]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-13T16:26:39.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:39 smithi107 ceph-mon[34139]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-13T16:26:40.577 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:40 smithi016 ceph-mon[31187]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-13T16:26:40.577 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:40 smithi016 ceph-mon[31187]: osdmap e57: 9 total, 8 up, 9 in 2024-09-13T16:26:40.577 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:40 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T16:26:40.577 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:40 smithi016 ceph-mon[31187]: from='osd.8 [v2:172.21.15.111:6816/837548190,v1:172.21.15.111:6817/837548190]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-13T16:26:40.577 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:40 smithi016 ceph-mon[31187]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-13T16:26:40.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:40 smithi111 ceph-mon[34600]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-13T16:26:40.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:40 smithi111 ceph-mon[34600]: osdmap e57: 9 total, 8 up, 9 in 2024-09-13T16:26:40.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:40 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T16:26:40.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:40 smithi111 ceph-mon[34600]: from='osd.8 [v2:172.21.15.111:6816/837548190,v1:172.21.15.111:6817/837548190]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-13T16:26:40.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:40 smithi111 ceph-mon[34600]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-13T16:26:40.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:40 smithi107 ceph-mon[34139]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-13T16:26:40.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:40 smithi107 ceph-mon[34139]: osdmap e57: 9 total, 8 up, 9 in 2024-09-13T16:26:40.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:40 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T16:26:40.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:40 smithi107 ceph-mon[34139]: from='osd.8 [v2:172.21.15.111:6816/837548190,v1:172.21.15.111:6817/837548190]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-13T16:26:40.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:40 smithi107 ceph-mon[34139]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-13T16:26:41.086 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:26:40 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:26:40.814+0000 7ff3c3b96640 -1 osd.8 0 waiting for initial osdmap 2024-09-13T16:26:41.086 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:26:40 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:26:40.825+0000 7ff3bb1a0640 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T16:26:41.120 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-13T16:26:41.211 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-13T16:26:41.223 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-09-13T16:26:41.223 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-09-13T16:26:41.223 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.009607 s, 1.1 GB/s 2024-09-13T16:26:41.223 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-09-13T16:26:41.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:41 smithi111 ceph-mon[34600]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-13T16:26:41.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:41 smithi111 ceph-mon[34600]: osdmap e58: 9 total, 8 up, 9 in 2024-09-13T16:26:41.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:41 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T16:26:41.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:41 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T16:26:41.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:41 smithi111 ceph-mon[34600]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:41.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:41 smithi111 ceph-mon[34600]: from='osd.8 ' entity='osd.8' 2024-09-13T16:26:41.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:41 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T16:26:41.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:41 smithi107 ceph-mon[34139]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-13T16:26:41.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:41 smithi107 ceph-mon[34139]: osdmap e58: 9 total, 8 up, 9 in 2024-09-13T16:26:41.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:41 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T16:26:41.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:41 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T16:26:41.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:41 smithi107 ceph-mon[34139]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:41.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:41 smithi107 ceph-mon[34139]: from='osd.8 ' entity='osd.8' 2024-09-13T16:26:41.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:41 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T16:26:41.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:41 smithi016 ceph-mon[31187]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-13T16:26:41.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:41 smithi016 ceph-mon[31187]: osdmap e58: 9 total, 8 up, 9 in 2024-09-13T16:26:41.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:41 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T16:26:41.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:41 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T16:26:41.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:41 smithi016 ceph-mon[31187]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T16:26:41.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:41 smithi016 ceph-mon[31187]: from='osd.8 ' entity='osd.8' 2024-09-13T16:26:41.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:41 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T16:26:42.011 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_1 2024-09-13T16:26:42.253 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:26:42.626 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:42 smithi016 ceph-mon[31187]: purged_snaps scrub starts 2024-09-13T16:26:42.626 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:42 smithi016 ceph-mon[31187]: purged_snaps scrub ok 2024-09-13T16:26:42.626 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:42 smithi016 ceph-mon[31187]: osd.8 [v2:172.21.15.111:6816/837548190,v1:172.21.15.111:6817/837548190] boot 2024-09-13T16:26:42.626 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:42 smithi016 ceph-mon[31187]: osdmap e59: 9 total, 9 up, 9 in 2024-09-13T16:26:42.626 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:42 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T16:26:42.626 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:42 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:42.626 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:42 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:42.626 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:42 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:26:42.627 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:42 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:26:42.627 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:42 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:26:42.627 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:42 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:42.627 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:42 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:26:42.627 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:42 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:26:42.627 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:42 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:42.693 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:42 smithi111 ceph-mon[34600]: purged_snaps scrub starts 2024-09-13T16:26:42.693 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:42 smithi111 ceph-mon[34600]: purged_snaps scrub ok 2024-09-13T16:26:42.693 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:42 smithi111 ceph-mon[34600]: osd.8 [v2:172.21.15.111:6816/837548190,v1:172.21.15.111:6817/837548190] boot 2024-09-13T16:26:42.693 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:42 smithi111 ceph-mon[34600]: osdmap e59: 9 total, 9 up, 9 in 2024-09-13T16:26:42.693 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:42 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T16:26:42.694 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:42 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:42.694 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:42 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:42.694 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:42 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:26:42.694 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:42 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:26:42.694 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:42 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:26:42.694 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:42 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:42.694 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:42 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:26:42.694 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:42 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:26:42.694 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:42 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:42.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:42 smithi107 ceph-mon[34139]: purged_snaps scrub starts 2024-09-13T16:26:42.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:42 smithi107 ceph-mon[34139]: purged_snaps scrub ok 2024-09-13T16:26:42.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:42 smithi107 ceph-mon[34139]: osd.8 [v2:172.21.15.111:6816/837548190,v1:172.21.15.111:6817/837548190] boot 2024-09-13T16:26:42.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:42 smithi107 ceph-mon[34139]: osdmap e59: 9 total, 9 up, 9 in 2024-09-13T16:26:42.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:42 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T16:26:42.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:42 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:42.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:42 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:42.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:42 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:26:42.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:42 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:26:42.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:42 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:26:42.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:42 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:42.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:42 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:26:42.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:42 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:26:42.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:42 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:26:43.669 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:43 smithi016 ceph-mon[31187]: Detected new or changed devices on smithi111 2024-09-13T16:26:43.669 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:43 smithi016 ceph-mon[31187]: Adjusting osd_memory_target on smithi111 to 5696M 2024-09-13T16:26:43.669 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:43 smithi016 ceph-mon[31187]: osdmap e60: 9 total, 9 up, 9 in 2024-09-13T16:26:43.669 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:43 smithi016 ceph-mon[31187]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T16:26:43.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:43 smithi111 ceph-mon[34600]: Detected new or changed devices on smithi111 2024-09-13T16:26:43.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:43 smithi111 ceph-mon[34600]: Adjusting osd_memory_target on smithi111 to 5696M 2024-09-13T16:26:43.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:43 smithi111 ceph-mon[34600]: osdmap e60: 9 total, 9 up, 9 in 2024-09-13T16:26:43.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:43 smithi111 ceph-mon[34600]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T16:26:43.750 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:43 smithi107 ceph-mon[34139]: Detected new or changed devices on smithi111 2024-09-13T16:26:43.750 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:43 smithi107 ceph-mon[34139]: Adjusting osd_memory_target on smithi111 to 5696M 2024-09-13T16:26:43.751 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:43 smithi107 ceph-mon[34139]: osdmap e60: 9 total, 9 up, 9 in 2024-09-13T16:26:43.751 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:43 smithi107 ceph-mon[34139]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T16:26:44.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.047+0000 7fe237c78640 1 -- 172.21.15.16:0/465883480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2301008a0 msgr2=0x7fe230107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:26:44.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.047+0000 7fe237c78640 1 --2- 172.21.15.16:0/465883480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2301008a0 0x7fe230107130 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe220009ca0 tx=0x7fe22002a8c0 comp rx=0 tx=0).stop 2024-09-13T16:26:44.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.047+0000 7fe237c78640 1 -- 172.21.15.16:0/465883480 shutdown_connections 2024-09-13T16:26:44.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.047+0000 7fe237c78640 1 --2- 172.21.15.16:0/465883480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2301008a0 0x7fe230107130 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:26:44.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.047+0000 7fe237c78640 1 --2- 172.21.15.16:0/465883480 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2300ffee0 0x7fe230100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:26:44.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.047+0000 7fe237c78640 1 --2- 172.21.15.16:0/465883480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2300fece0 0x7fe2300ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:26:44.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.047+0000 7fe237c78640 1 -- 172.21.15.16:0/465883480 >> 172.21.15.16:0/465883480 conn(0x7fe2300fa490 msgr2=0x7fe2300fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:26:44.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.047+0000 7fe237c78640 1 -- 172.21.15.16:0/465883480 shutdown_connections 2024-09-13T16:26:44.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.047+0000 7fe237c78640 1 -- 172.21.15.16:0/465883480 wait complete. 2024-09-13T16:26:44.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.047+0000 7fe237c78640 1 Processor -- start 2024-09-13T16:26:44.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.047+0000 7fe237c78640 1 -- start start 2024-09-13T16:26:44.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.048+0000 7fe237c78640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2300fece0 0x7fe230197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:26:44.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.048+0000 7fe237c78640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2300ffee0 0x7fe2301983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:26:44.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.048+0000 7fe237c78640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2301008a0 0x7fe23019f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:26:44.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.048+0000 7fe237c78640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23019f9b0 con 0x7fe2300ffee0 2024-09-13T16:26:44.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.048+0000 7fe237c78640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23019fb20 con 0x7fe2301008a0 2024-09-13T16:26:44.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.048+0000 7fe237c78640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23019fce0 con 0x7fe2300fece0 2024-09-13T16:26:44.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.048+0000 7fe235213640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2300ffee0 0x7fe2301983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:26:44.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.048+0000 7fe235213640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2300ffee0 0x7fe2301983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49888/0 (socket says 172.21.15.16:49888) 2024-09-13T16:26:44.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.048+0000 7fe235213640 1 -- 172.21.15.16:0/3285721100 learned_addr learned my addr 172.21.15.16:0/3285721100 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:26:44.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.048+0000 7fe235a14640 1 --2- 172.21.15.16:0/3285721100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2300fece0 0x7fe230197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:26:44.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.048+0000 7fe236215640 1 --2- 172.21.15.16:0/3285721100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2301008a0 0x7fe23019f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:26:44.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.048+0000 7fe235213640 1 -- 172.21.15.16:0/3285721100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2300fece0 msgr2=0x7fe230197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:26:44.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.048+0000 7fe235213640 1 --2- 172.21.15.16:0/3285721100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2300fece0 0x7fe230197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:26:44.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.048+0000 7fe235213640 1 -- 172.21.15.16:0/3285721100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2301008a0 msgr2=0x7fe23019f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:26:44.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.048+0000 7fe235213640 1 --2- 172.21.15.16:0/3285721100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2301008a0 0x7fe23019f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:26:44.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.048+0000 7fe235213640 1 -- 172.21.15.16:0/3285721100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe220009900 con 0x7fe2300ffee0 2024-09-13T16:26:44.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.048+0000 7fe235213640 1 --2- 172.21.15.16:0/3285721100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2300ffee0 0x7fe2301983f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe22400d770 tx=0x7fe22400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:26:44.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.048+0000 7fe22effd640 1 -- 172.21.15.16:0/3285721100 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe224009c30 con 0x7fe2300ffee0 2024-09-13T16:26:44.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.049+0000 7fe237c78640 1 -- 172.21.15.16:0/3285721100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe23019ffc0 con 0x7fe2300ffee0 2024-09-13T16:26:44.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.049+0000 7fe22effd640 1 -- 172.21.15.16:0/3285721100 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe22400de80 con 0x7fe2300ffee0 2024-09-13T16:26:44.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.049+0000 7fe22effd640 1 -- 172.21.15.16:0/3285721100 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe224004d40 con 0x7fe2300ffee0 2024-09-13T16:26:44.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.049+0000 7fe237c78640 1 -- 172.21.15.16:0/3285721100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2301a0430 con 0x7fe2300ffee0 2024-09-13T16:26:44.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.050+0000 7fe237c78640 1 -- 172.21.15.16:0/3285721100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe23010b590 con 0x7fe2300ffee0 2024-09-13T16:26:44.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.051+0000 7fe22effd640 1 -- 172.21.15.16:0/3285721100 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe224004ea0 con 0x7fe2300ffee0 2024-09-13T16:26:44.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.051+0000 7fe22effd640 1 --2- 172.21.15.16:0/3285721100 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe2180a10b0 0x7fe2180a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:26:44.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.052+0000 7fe235a14640 1 --2- 172.21.15.16:0/3285721100 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe2180a10b0 0x7fe2180a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:26:44.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.052+0000 7fe22effd640 1 -- 172.21.15.16:0/3285721100 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5858+0+0 (secure 0 0 0) 0x7fe2240cb8c0 con 0x7fe2300ffee0 2024-09-13T16:26:44.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.052+0000 7fe235a14640 1 --2- 172.21.15.16:0/3285721100 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe2180a10b0 0x7fe2180a3570 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe2300ffd40 tx=0x7fe22800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:26:44.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.056+0000 7fe22effd640 1 -- 172.21.15.16:0/3285721100 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fe224095a90 con 0x7fe2300ffee0 2024-09-13T16:26:44.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:26:44.235+0000 7fe237c78640 1 -- 172.21.15.16:0/3285721100 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe230106e40 con 0x7fe2180a10b0 2024-09-13T16:26:44.603 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:44 smithi016 ceph-mon[31187]: osdmap e61: 9 total, 9 up, 9 in 2024-09-13T16:26:44.603 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:44 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:26:44.603 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:44 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:26:44.603 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:44 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:26:44.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:44 smithi111 ceph-mon[34600]: osdmap e61: 9 total, 9 up, 9 in 2024-09-13T16:26:44.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:44 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:26:44.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:44 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:26:44.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:44 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:26:44.762 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:44 smithi107 ceph-mon[34139]: osdmap e61: 9 total, 9 up, 9 in 2024-09-13T16:26:44.762 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:44 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:26:44.762 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:44 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:26:44.762 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:44 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:26:45.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:45 smithi111 ceph-mon[34600]: from='client.14400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:26:45.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:45 smithi111 ceph-mon[34600]: osdmap e62: 9 total, 9 up, 9 in 2024-09-13T16:26:45.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:45 smithi111 ceph-mon[34600]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T16:26:45.751 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:45 smithi016 ceph-mon[31187]: from='client.14400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:26:45.752 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:45 smithi016 ceph-mon[31187]: osdmap e62: 9 total, 9 up, 9 in 2024-09-13T16:26:45.752 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:45 smithi016 ceph-mon[31187]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T16:26:45.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:45 smithi107 ceph-mon[34139]: from='client.14400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:26:45.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:45 smithi107 ceph-mon[34139]: osdmap e62: 9 total, 9 up, 9 in 2024-09-13T16:26:45.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:45 smithi107 ceph-mon[34139]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T16:26:46.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:46 smithi111 ceph-mon[34600]: osdmap e63: 9 total, 9 up, 9 in 2024-09-13T16:26:46.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:46 smithi107 ceph-mon[34139]: osdmap e63: 9 total, 9 up, 9 in 2024-09-13T16:26:46.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:46 smithi016 ceph-mon[31187]: osdmap e63: 9 total, 9 up, 9 in 2024-09-13T16:26:47.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:47 smithi111 ceph-mon[34600]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T16:26:47.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:47 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/4182955159' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bae4d6db-8b90-4d02-8e83-7d8f2a6f8fdb"}]: dispatch 2024-09-13T16:26:47.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:47 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/4182955159' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bae4d6db-8b90-4d02-8e83-7d8f2a6f8fdb"}]': finished 2024-09-13T16:26:47.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:47 smithi111 ceph-mon[34600]: osdmap e64: 10 total, 9 up, 10 in 2024-09-13T16:26:47.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:47 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T16:26:47.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:47 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3816300080' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:26:47.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:47 smithi107 ceph-mon[34139]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T16:26:47.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:47 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/4182955159' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bae4d6db-8b90-4d02-8e83-7d8f2a6f8fdb"}]: dispatch 2024-09-13T16:26:47.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:47 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/4182955159' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bae4d6db-8b90-4d02-8e83-7d8f2a6f8fdb"}]': finished 2024-09-13T16:26:47.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:47 smithi107 ceph-mon[34139]: osdmap e64: 10 total, 9 up, 10 in 2024-09-13T16:26:47.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:47 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T16:26:47.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:47 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3816300080' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:26:47.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:47 smithi016 ceph-mon[31187]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T16:26:47.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:47 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/4182955159' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bae4d6db-8b90-4d02-8e83-7d8f2a6f8fdb"}]: dispatch 2024-09-13T16:26:47.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:47 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/4182955159' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bae4d6db-8b90-4d02-8e83-7d8f2a6f8fdb"}]': finished 2024-09-13T16:26:47.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:47 smithi016 ceph-mon[31187]: osdmap e64: 10 total, 9 up, 10 in 2024-09-13T16:26:47.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:47 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T16:26:47.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:47 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3816300080' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:26:48.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:48 smithi111 ceph-mon[34600]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-09-13T16:26:49.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:48 smithi107 ceph-mon[34139]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-09-13T16:26:49.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:48 smithi016 ceph-mon[31187]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-09-13T16:26:50.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:50 smithi111 ceph-mon[34600]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-13T16:26:51.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:50 smithi107 ceph-mon[34139]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-13T16:26:51.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:50 smithi016 ceph-mon[31187]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-13T16:26:52.909 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:52 smithi016 ceph-mon[31187]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-13T16:26:52.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:52 smithi111 ceph-mon[34600]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-13T16:26:53.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:52 smithi107 ceph-mon[34139]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-13T16:26:54.588 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:54 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-13T16:26:54.589 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:54 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:26:54.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:54 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-13T16:26:54.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:54 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:26:54.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:54 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-13T16:26:54.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:54 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:26:55.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:55 smithi107 ceph-mon[34139]: Deploying daemon osd.9 on smithi016 2024-09-13T16:26:55.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:55 smithi107 ceph-mon[34139]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-13T16:26:55.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:55 smithi016 ceph-mon[31187]: Deploying daemon osd.9 on smithi016 2024-09-13T16:26:55.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:55 smithi016 ceph-mon[31187]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-13T16:26:55.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:55 smithi111 ceph-mon[34600]: Deploying daemon osd.9 on smithi016 2024-09-13T16:26:55.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:55 smithi111 ceph-mon[34600]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-13T16:26:56.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:56 smithi111 ceph-mon[34600]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-13T16:26:57.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:56 smithi107 ceph-mon[34139]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-13T16:26:57.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:56 smithi016 ceph-mon[31187]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-13T16:26:58.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:26:58 smithi016 ceph-mon[31187]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-13T16:26:58.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:26:58 smithi111 ceph-mon[34600]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-13T16:26:59.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:26:58 smithi107 ceph-mon[34139]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-13T16:27:00.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:00 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:27:00.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:00 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:00.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:00 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:00.356 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:00 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:27:00.356 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:00 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:00.356 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:00 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:00.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:00 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:27:00.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:00 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:00.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:00 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:01.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:01 smithi016 ceph-mon[31187]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T16:27:01.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:01 smithi107 ceph-mon[34139]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T16:27:01.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:01 smithi111 ceph-mon[34600]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T16:27:02.798 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:02 smithi016 ceph-mon[31187]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T16:27:02.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:02 smithi111 ceph-mon[34600]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T16:27:03.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:02 smithi107 ceph-mon[34139]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T16:27:03.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:03.961+0000 7fe22effd640 1 -- 172.21.15.16:0/3285721100 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe230106e40 con 0x7fe2180a10b0 2024-09-13T16:27:03.963 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 9 on host 'smithi016' 2024-09-13T16:27:03.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:03.964+0000 7fe237c78640 1 -- 172.21.15.16:0/3285721100 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe2180a10b0 msgr2=0x7fe2180a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:27:03.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:03.964+0000 7fe237c78640 1 --2- 172.21.15.16:0/3285721100 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe2180a10b0 0x7fe2180a3570 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe2300ffd40 tx=0x7fe22800b390 comp rx=0 tx=0).stop 2024-09-13T16:27:03.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:03.964+0000 7fe237c78640 1 -- 172.21.15.16:0/3285721100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2300ffee0 msgr2=0x7fe2301983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:27:03.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:03.964+0000 7fe237c78640 1 --2- 172.21.15.16:0/3285721100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2300ffee0 0x7fe2301983f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe22400d770 tx=0x7fe22400dc30 comp rx=0 tx=0).stop 2024-09-13T16:27:03.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:03.964+0000 7fe237c78640 1 -- 172.21.15.16:0/3285721100 shutdown_connections 2024-09-13T16:27:03.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:03.964+0000 7fe237c78640 1 --2- 172.21.15.16:0/3285721100 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe2180a10b0 0x7fe2180a3570 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:03.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:03.964+0000 7fe237c78640 1 --2- 172.21.15.16:0/3285721100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2301008a0 0x7fe23019f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:03.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:03.964+0000 7fe237c78640 1 --2- 172.21.15.16:0/3285721100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2300ffee0 0x7fe2301983f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:03.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:03.964+0000 7fe237c78640 1 --2- 172.21.15.16:0/3285721100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2300fece0 0x7fe230197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:03.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:03.964+0000 7fe237c78640 1 -- 172.21.15.16:0/3285721100 >> 172.21.15.16:0/3285721100 conn(0x7fe2300fa490 msgr2=0x7fe230105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:27:03.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:03.964+0000 7fe237c78640 1 -- 172.21.15.16:0/3285721100 shutdown_connections 2024-09-13T16:27:03.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:03.964+0000 7fe237c78640 1 -- 172.21.15.16:0/3285721100 wait complete. 2024-09-13T16:27:04.336 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:04.336 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:04.336 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:27:04.336 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:27:04.336 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:04.337 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:04 smithi016 ceph-mon[31187]: from='osd.9 [v2:172.21.15.16:6826/4175383686,v1:172.21.15.16:6827/4175383686]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-13T16:27:04.337 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:27:04.337 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:04.337 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:04.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:04.440 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:04.440 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:27:04.440 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:27:04.440 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:04.440 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:04 smithi111 ceph-mon[34600]: from='osd.9 [v2:172.21.15.16:6826/4175383686,v1:172.21.15.16:6827/4175383686]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-13T16:27:04.440 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:27:04.440 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:04.440 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:04.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:04.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:04.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:27:04.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:27:04.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:04.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:04 smithi107 ceph-mon[34139]: from='osd.9 [v2:172.21.15.16:6826/4175383686,v1:172.21.15.16:6827/4175383686]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-13T16:27:04.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:27:04.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:04.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:04.685 DEBUG:teuthology.orchestra.run.smithi016:osd.9> sudo journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.9.service 2024-09-13T16:27:04.687 INFO:tasks.cephadm:Deploying osd.10 on smithi107 with /dev/vg_nvme/lv_1... 2024-09-13T16:27:04.688 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-13T16:27:04.895 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T16:27:05.216 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:05 smithi016 ceph-mon[31187]: from='osd.9 [v2:172.21.15.16:6826/4175383686,v1:172.21.15.16:6827/4175383686]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-13T16:27:05.216 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:05 smithi016 ceph-mon[31187]: osdmap e65: 10 total, 9 up, 10 in 2024-09-13T16:27:05.216 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:05 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T16:27:05.216 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:05 smithi016 ceph-mon[31187]: from='osd.9 [v2:172.21.15.16:6826/4175383686,v1:172.21.15.16:6827/4175383686]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-13T16:27:05.216 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:05 smithi016 ceph-mon[31187]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T16:27:05.216 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:27:05 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:27:05.199+0000 7f4ab34a5640 -1 osd.9 0 waiting for initial osdmap 2024-09-13T16:27:05.217 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:27:05 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:27:05.214+0000 7f4aaf2ce640 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T16:27:05.319 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:05 smithi107 ceph-mon[34139]: from='osd.9 [v2:172.21.15.16:6826/4175383686,v1:172.21.15.16:6827/4175383686]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-13T16:27:05.320 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:05 smithi107 ceph-mon[34139]: osdmap e65: 10 total, 9 up, 10 in 2024-09-13T16:27:05.320 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:05 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T16:27:05.320 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:05 smithi107 ceph-mon[34139]: from='osd.9 [v2:172.21.15.16:6826/4175383686,v1:172.21.15.16:6827/4175383686]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-13T16:27:05.320 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:05 smithi107 ceph-mon[34139]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T16:27:05.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:05 smithi111 ceph-mon[34600]: from='osd.9 [v2:172.21.15.16:6826/4175383686,v1:172.21.15.16:6827/4175383686]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-13T16:27:05.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:05 smithi111 ceph-mon[34600]: osdmap e65: 10 total, 9 up, 10 in 2024-09-13T16:27:05.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:05 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T16:27:05.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:05 smithi111 ceph-mon[34600]: from='osd.9 [v2:172.21.15.16:6826/4175383686,v1:172.21.15.16:6827/4175383686]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-13T16:27:05.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:05 smithi111 ceph-mon[34600]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T16:27:06.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:06 smithi107 ceph-mon[34139]: from='osd.9 [v2:172.21.15.16:6826/4175383686,v1:172.21.15.16:6827/4175383686]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-13T16:27:06.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:06 smithi107 ceph-mon[34139]: osdmap e66: 10 total, 9 up, 10 in 2024-09-13T16:27:06.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:06 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T16:27:06.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:06 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T16:27:06.476 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:06 smithi016 ceph-mon[31187]: from='osd.9 [v2:172.21.15.16:6826/4175383686,v1:172.21.15.16:6827/4175383686]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-13T16:27:06.476 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:06 smithi016 ceph-mon[31187]: osdmap e66: 10 total, 9 up, 10 in 2024-09-13T16:27:06.476 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:06 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T16:27:06.476 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:06 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T16:27:06.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:06 smithi111 ceph-mon[34600]: from='osd.9 [v2:172.21.15.16:6826/4175383686,v1:172.21.15.16:6827/4175383686]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-13T16:27:06.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:06 smithi111 ceph-mon[34600]: osdmap e66: 10 total, 9 up, 10 in 2024-09-13T16:27:06.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:06 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T16:27:06.689 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:06 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T16:27:06.777 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-13T16:27:06.864 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-13T16:27:06.878 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-09-13T16:27:06.878 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-09-13T16:27:06.878 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0116888 s, 897 MB/s 2024-09-13T16:27:06.878 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-09-13T16:27:07.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:07 smithi016 ceph-mon[31187]: purged_snaps scrub starts 2024-09-13T16:27:07.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:07 smithi016 ceph-mon[31187]: purged_snaps scrub ok 2024-09-13T16:27:07.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:07 smithi016 ceph-mon[31187]: osd.9 [v2:172.21.15.16:6826/4175383686,v1:172.21.15.16:6827/4175383686] boot 2024-09-13T16:27:07.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:07 smithi016 ceph-mon[31187]: osdmap e67: 10 total, 10 up, 10 in 2024-09-13T16:27:07.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:07 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T16:27:07.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:07 smithi016 ceph-mon[31187]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T16:27:07.462 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:07 smithi107 ceph-mon[34139]: purged_snaps scrub starts 2024-09-13T16:27:07.462 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:07 smithi107 ceph-mon[34139]: purged_snaps scrub ok 2024-09-13T16:27:07.462 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:07 smithi107 ceph-mon[34139]: osd.9 [v2:172.21.15.16:6826/4175383686,v1:172.21.15.16:6827/4175383686] boot 2024-09-13T16:27:07.462 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:07 smithi107 ceph-mon[34139]: osdmap e67: 10 total, 10 up, 10 in 2024-09-13T16:27:07.462 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:07 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T16:27:07.462 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:07 smithi107 ceph-mon[34139]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T16:27:07.481 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_1 2024-09-13T16:27:07.684 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T16:27:07.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:07 smithi111 ceph-mon[34600]: purged_snaps scrub starts 2024-09-13T16:27:07.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:07 smithi111 ceph-mon[34600]: purged_snaps scrub ok 2024-09-13T16:27:07.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:07 smithi111 ceph-mon[34600]: osd.9 [v2:172.21.15.16:6826/4175383686,v1:172.21.15.16:6827/4175383686] boot 2024-09-13T16:27:07.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:07 smithi111 ceph-mon[34600]: osdmap e67: 10 total, 10 up, 10 in 2024-09-13T16:27:07.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:07 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T16:27:07.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:07 smithi111 ceph-mon[34600]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T16:27:08.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:08 smithi107 ceph-mon[34139]: osdmap e68: 10 total, 10 up, 10 in 2024-09-13T16:27:08.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:08 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:08.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:08 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:08.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:08 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:08.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:08 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:08.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:08 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:08.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:08 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:08.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:08 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:08.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:08 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:27:08.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:08 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:27:08.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:08 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:08.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:08 smithi016 ceph-mon[31187]: osdmap e68: 10 total, 10 up, 10 in 2024-09-13T16:27:08.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:08 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:08.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:08 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:08.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:08 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:08.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:08 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:08.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:08 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:08.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:08 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:08.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:08 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:08.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:08 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:27:08.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:08 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:27:08.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:08 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:08.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:08 smithi111 ceph-mon[34600]: osdmap e68: 10 total, 10 up, 10 in 2024-09-13T16:27:08.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:08 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:08.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:08 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:08.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:08 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:08.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:08 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:08.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:08 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:08.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:08 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:08.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:08 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:08.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:08 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:27:08.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:08 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:27:08.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:08 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:09.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.246+0000 7ffb64717640 1 -- 172.21.15.107:0/392091384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb5c0fed10 msgr2=0x7ffb5c0ff110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:27:09.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.246+0000 7ffb64717640 1 --2- 172.21.15.107:0/392091384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb5c0fed10 0x7ffb5c0ff110 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffb58009900 tx=0x7ffb5802a410 comp rx=0 tx=0).stop 2024-09-13T16:27:09.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.246+0000 7ffb64717640 1 -- 172.21.15.107:0/392091384 shutdown_connections 2024-09-13T16:27:09.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.246+0000 7ffb64717640 1 --2- 172.21.15.107:0/392091384 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb5c1008d0 0x7ffb5c107160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:09.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.246+0000 7ffb64717640 1 --2- 172.21.15.107:0/392091384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb5c0fff10 0x7ffb5c100390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:09.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.246+0000 7ffb64717640 1 --2- 172.21.15.107:0/392091384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb5c0fed10 0x7ffb5c0ff110 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:09.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.246+0000 7ffb64717640 1 -- 172.21.15.107:0/392091384 >> 172.21.15.107:0/392091384 conn(0x7ffb5c0fa4c0 msgr2=0x7ffb5c0fc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:27:09.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.246+0000 7ffb64717640 1 -- 172.21.15.107:0/392091384 shutdown_connections 2024-09-13T16:27:09.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.246+0000 7ffb64717640 1 -- 172.21.15.107:0/392091384 wait complete. 2024-09-13T16:27:09.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.247+0000 7ffb64717640 1 Processor -- start 2024-09-13T16:27:09.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.247+0000 7ffb64717640 1 -- start start 2024-09-13T16:27:09.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.247+0000 7ffb64717640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb5c0fed10 0x7ffb5c195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:27:09.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.247+0000 7ffb64717640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb5c0fff10 0x7ffb5c196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:27:09.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.247+0000 7ffb64717640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb5c1008d0 0x7ffb5c19d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:27:09.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.247+0000 7ffb64717640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c19d8c0 con 0x7ffb5c0fed10 2024-09-13T16:27:09.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.247+0000 7ffb64717640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c19da30 con 0x7ffb5c1008d0 2024-09-13T16:27:09.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.247+0000 7ffb64717640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c19dbf0 con 0x7ffb5c0fff10 2024-09-13T16:27:09.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.247+0000 7ffb62cb4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb5c1008d0 0x7ffb5c19d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:27:09.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.247+0000 7ffb62cb4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb5c1008d0 0x7ffb5c19d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42476/0 (socket says 172.21.15.107:42476) 2024-09-13T16:27:09.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.247+0000 7ffb62cb4640 1 -- 172.21.15.107:0/2058882741 learned_addr learned my addr 172.21.15.107:0/2058882741 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-13T16:27:09.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.247+0000 7ffb61cb2640 1 --2- 172.21.15.107:0/2058882741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb5c0fff10 0x7ffb5c196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:27:09.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.247+0000 7ffb624b3640 1 --2- 172.21.15.107:0/2058882741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb5c0fed10 0x7ffb5c195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:27:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.247+0000 7ffb62cb4640 1 -- 172.21.15.107:0/2058882741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb5c0fff10 msgr2=0x7ffb5c196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:27:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.247+0000 7ffb62cb4640 1 --2- 172.21.15.107:0/2058882741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb5c0fff10 0x7ffb5c196300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.247+0000 7ffb62cb4640 1 -- 172.21.15.107:0/2058882741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb5c0fed10 msgr2=0x7ffb5c195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:27:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.247+0000 7ffb62cb4640 1 --2- 172.21.15.107:0/2058882741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb5c0fed10 0x7ffb5c195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.247+0000 7ffb62cb4640 1 -- 172.21.15.107:0/2058882741 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb58009560 con 0x7ffb5c1008d0 2024-09-13T16:27:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.247+0000 7ffb61cb2640 1 --2- 172.21.15.107:0/2058882741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb5c0fff10 0x7ffb5c196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:27:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.247+0000 7ffb624b3640 1 --2- 172.21.15.107:0/2058882741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb5c0fed10 0x7ffb5c195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:27:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.248+0000 7ffb62cb4640 1 --2- 172.21.15.107:0/2058882741 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb5c1008d0 0x7ffb5c19d380 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400b970 tx=0x7ffb5400be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:27:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.248+0000 7ffb537fe640 1 -- 172.21.15.107:0/2058882741 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5400cb40 con 0x7ffb5c1008d0 2024-09-13T16:27:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.248+0000 7ffb537fe640 1 -- 172.21.15.107:0/2058882741 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb5400cca0 con 0x7ffb5c1008d0 2024-09-13T16:27:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.248+0000 7ffb64717640 1 -- 172.21.15.107:0/2058882741 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb5c19ded0 con 0x7ffb5c1008d0 2024-09-13T16:27:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.248+0000 7ffb537fe640 1 -- 172.21.15.107:0/2058882741 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb54007610 con 0x7ffb5c1008d0 2024-09-13T16:27:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.248+0000 7ffb64717640 1 -- 172.21.15.107:0/2058882741 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb5c19e3c0 con 0x7ffb5c1008d0 2024-09-13T16:27:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.249+0000 7ffb64717640 1 -- 172.21.15.107:0/2058882741 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb5c1062a0 con 0x7ffb5c1008d0 2024-09-13T16:27:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.250+0000 7ffb537fe640 1 -- 172.21.15.107:0/2058882741 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ffb54014440 con 0x7ffb5c1008d0 2024-09-13T16:27:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.250+0000 7ffb537fe640 1 --2- 172.21.15.107:0/2058882741 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ffb480a10b0 0x7ffb480a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:27:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.251+0000 7ffb537fe640 1 -- 172.21.15.107:0/2058882741 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6261+0+0 (secure 0 0 0) 0x7ffb540cb9e0 con 0x7ffb5c1008d0 2024-09-13T16:27:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.251+0000 7ffb624b3640 1 --2- 172.21.15.107:0/2058882741 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ffb480a10b0 0x7ffb480a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:27:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.251+0000 7ffb624b3640 1 --2- 172.21.15.107:0/2058882741 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ffb480a10b0 0x7ffb480a3570 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffb58005ab0 tx=0x7ffb58005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:27:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.255+0000 7ffb537fe640 1 -- 172.21.15.107:0/2058882741 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ffb54095970 con 0x7ffb5c1008d0 2024-09-13T16:27:09.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:09 smithi107 ceph-mon[34139]: Detected new or changed devices on smithi016 2024-09-13T16:27:09.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:09 smithi107 ceph-mon[34139]: Adjusting osd_memory_target on smithi016 to 4273M 2024-09-13T16:27:09.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:09 smithi107 ceph-mon[34139]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:09.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:09.428+0000 7ffb64717640 1 -- 172.21.15.107:0/2058882741 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb5c02c4f0 con 0x7ffb480a10b0 2024-09-13T16:27:09.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:09 smithi016 ceph-mon[31187]: Detected new or changed devices on smithi016 2024-09-13T16:27:09.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:09 smithi016 ceph-mon[31187]: Adjusting osd_memory_target on smithi016 to 4273M 2024-09-13T16:27:09.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:09 smithi016 ceph-mon[31187]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:09.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:09 smithi111 ceph-mon[34600]: Detected new or changed devices on smithi016 2024-09-13T16:27:09.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:09 smithi111 ceph-mon[34600]: Adjusting osd_memory_target on smithi016 to 4273M 2024-09-13T16:27:09.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:09 smithi111 ceph-mon[34600]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:10.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:10 smithi107 ceph-mon[34139]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:27:10.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:10 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:27:10.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:10 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:27:10.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:10 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:27:10.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:10 smithi016 ceph-mon[31187]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:27:10.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:10 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:27:10.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:10 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:27:10.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:10 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:27:10.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:10 smithi111 ceph-mon[34600]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:27:10.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:10 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:27:10.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:10 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:27:10.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:10 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:27:11.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:11 smithi016 ceph-mon[31187]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:11.506 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:11 smithi107 ceph-mon[34139]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:11.686 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:11 smithi111 ceph-mon[34600]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:12.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:12 smithi107 ceph-mon[34139]: from='client.? 172.21.15.107:0/1469362179' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd0e7740-76c8-4c99-a349-7077b4905143"}]: dispatch 2024-09-13T16:27:12.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:12 smithi107 ceph-mon[34139]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd0e7740-76c8-4c99-a349-7077b4905143"}]: dispatch 2024-09-13T16:27:12.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:12 smithi107 ceph-mon[34139]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd0e7740-76c8-4c99-a349-7077b4905143"}]': finished 2024-09-13T16:27:12.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:12 smithi107 ceph-mon[34139]: osdmap e69: 11 total, 10 up, 11 in 2024-09-13T16:27:12.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:12 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T16:27:12.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:12 smithi016 ceph-mon[31187]: from='client.? 172.21.15.107:0/1469362179' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd0e7740-76c8-4c99-a349-7077b4905143"}]: dispatch 2024-09-13T16:27:12.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:12 smithi016 ceph-mon[31187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd0e7740-76c8-4c99-a349-7077b4905143"}]: dispatch 2024-09-13T16:27:12.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:12 smithi016 ceph-mon[31187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd0e7740-76c8-4c99-a349-7077b4905143"}]': finished 2024-09-13T16:27:12.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:12 smithi016 ceph-mon[31187]: osdmap e69: 11 total, 10 up, 11 in 2024-09-13T16:27:12.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:12 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T16:27:12.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:12 smithi111 ceph-mon[34600]: from='client.? 172.21.15.107:0/1469362179' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd0e7740-76c8-4c99-a349-7077b4905143"}]: dispatch 2024-09-13T16:27:12.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:12 smithi111 ceph-mon[34600]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd0e7740-76c8-4c99-a349-7077b4905143"}]: dispatch 2024-09-13T16:27:12.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:12 smithi111 ceph-mon[34600]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd0e7740-76c8-4c99-a349-7077b4905143"}]': finished 2024-09-13T16:27:12.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:12 smithi111 ceph-mon[34600]: osdmap e69: 11 total, 10 up, 11 in 2024-09-13T16:27:12.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:12 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T16:27:13.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:13 smithi107 ceph-mon[34139]: from='client.? 172.21.15.107:0/403554322' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:27:13.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:13 smithi107 ceph-mon[34139]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:13.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:13 smithi016 ceph-mon[31187]: from='client.? 172.21.15.107:0/403554322' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:27:13.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:13 smithi016 ceph-mon[31187]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:13.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:13 smithi111 ceph-mon[34600]: from='client.? 172.21.15.107:0/403554322' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:27:13.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:13 smithi111 ceph-mon[34600]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:14.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:14 smithi111 ceph-mon[34600]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:15.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:14 smithi107 ceph-mon[34139]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:15.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:14 smithi016 ceph-mon[31187]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:16.758 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:16 smithi107 ceph-mon[34139]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:16.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:16 smithi111 ceph-mon[34600]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:17.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:16 smithi016 ceph-mon[31187]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:18.829 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:18 smithi107 ceph-mon[34139]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:18.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:18 smithi111 ceph-mon[34600]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:19.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:18 smithi016 ceph-mon[31187]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:19.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:19 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-13T16:27:19.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:19 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:27:19.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:19 smithi107 ceph-mon[34139]: Deploying daemon osd.10 on smithi107 2024-09-13T16:27:19.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:19 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-13T16:27:19.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:19 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:27:19.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:19 smithi111 ceph-mon[34600]: Deploying daemon osd.10 on smithi107 2024-09-13T16:27:20.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:19 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-13T16:27:20.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:19 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:27:20.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:19 smithi016 ceph-mon[31187]: Deploying daemon osd.10 on smithi107 2024-09-13T16:27:20.830 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:20 smithi107 ceph-mon[34139]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:20.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:20 smithi111 ceph-mon[34600]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:21.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:20 smithi016 ceph-mon[31187]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:22.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:22 smithi111 ceph-mon[34600]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:23.009 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:22 smithi107 ceph-mon[34139]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:23.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:22 smithi016 ceph-mon[31187]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:24.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:24 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:27:24.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:24 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:24.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:24 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:24.356 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:24 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:27:24.356 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:24 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:24.356 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:24 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:24.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:24 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:27:24.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:24 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:24.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:24 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:25.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:25 smithi111 ceph-mon[34600]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:25.482 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:25 smithi107 ceph-mon[34139]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:25.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:25 smithi016 ceph-mon[31187]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:26.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:26 smithi107 ceph-mon[34139]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:26.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:26 smithi111 ceph-mon[34600]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:27.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:26 smithi016 ceph-mon[31187]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:27.732 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 10 on host 'smithi107' 2024-09-13T16:27:27.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:27.729+0000 7ffb537fe640 1 -- 172.21.15.107:0/2058882741 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7ffb5c02c4f0 con 0x7ffb480a10b0 2024-09-13T16:27:27.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:27.732+0000 7ffb64717640 1 -- 172.21.15.107:0/2058882741 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ffb480a10b0 msgr2=0x7ffb480a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:27:27.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:27.732+0000 7ffb64717640 1 --2- 172.21.15.107:0/2058882741 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ffb480a10b0 0x7ffb480a3570 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffb58005ab0 tx=0x7ffb58005a20 comp rx=0 tx=0).stop 2024-09-13T16:27:27.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:27.732+0000 7ffb64717640 1 -- 172.21.15.107:0/2058882741 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb5c1008d0 msgr2=0x7ffb5c19d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:27:27.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:27.732+0000 7ffb64717640 1 --2- 172.21.15.107:0/2058882741 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb5c1008d0 0x7ffb5c19d380 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400b970 tx=0x7ffb5400be30 comp rx=0 tx=0).stop 2024-09-13T16:27:27.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:27.732+0000 7ffb64717640 1 -- 172.21.15.107:0/2058882741 shutdown_connections 2024-09-13T16:27:27.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:27.732+0000 7ffb64717640 1 --2- 172.21.15.107:0/2058882741 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ffb480a10b0 0x7ffb480a3570 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:27.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:27.732+0000 7ffb64717640 1 --2- 172.21.15.107:0/2058882741 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb5c1008d0 0x7ffb5c19d380 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:27.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:27.732+0000 7ffb64717640 1 --2- 172.21.15.107:0/2058882741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb5c0fff10 0x7ffb5c196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:27.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:27.732+0000 7ffb64717640 1 --2- 172.21.15.107:0/2058882741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb5c0fed10 0x7ffb5c195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:27.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:27.732+0000 7ffb64717640 1 -- 172.21.15.107:0/2058882741 >> 172.21.15.107:0/2058882741 conn(0x7ffb5c0fa4c0 msgr2=0x7ffb5c103470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:27:27.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:27.732+0000 7ffb64717640 1 -- 172.21.15.107:0/2058882741 shutdown_connections 2024-09-13T16:27:27.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-13T16:27:27.732+0000 7ffb64717640 1 -- 172.21.15.107:0/2058882741 wait complete. 2024-09-13T16:27:28.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:27 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:28.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:27 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:28.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:27 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:27:28.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:27 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:27:28.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:27 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:28.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:27 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:27:28.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:27 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:28.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:27 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:28.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:27 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:28.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:27 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:28.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:27 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:27:28.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:27 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:27:28.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:27 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:28.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:27 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:27:28.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:27 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:28.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:27 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:28.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:27 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:28.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:27 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:28.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:27 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:27:28.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:27 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:27:28.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:27 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:28.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:27 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:27:28.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:27 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:28.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:27 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:28.450 DEBUG:teuthology.orchestra.run.smithi107:osd.10> sudo journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.10.service 2024-09-13T16:27:28.452 INFO:tasks.cephadm:Deploying osd.11 on smithi111 with /dev/vg_nvme/lv_1... 2024-09-13T16:27:28.452 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-13T16:27:28.655 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T16:27:29.050 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:28 smithi111 ceph-mon[34600]: from='osd.10 [v2:172.21.15.107:6824/2659550426,v1:172.21.15.107:6825/2659550426]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-13T16:27:29.050 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:28 smithi111 ceph-mon[34600]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-13T16:27:29.050 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:28 smithi111 ceph-mon[34600]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:29.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:28 smithi107 ceph-mon[34139]: from='osd.10 [v2:172.21.15.107:6824/2659550426,v1:172.21.15.107:6825/2659550426]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-13T16:27:29.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:28 smithi107 ceph-mon[34139]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-13T16:27:29.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:28 smithi107 ceph-mon[34139]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:29.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:28 smithi016 ceph-mon[31187]: from='osd.10 [v2:172.21.15.107:6824/2659550426,v1:172.21.15.107:6825/2659550426]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-13T16:27:29.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:28 smithi016 ceph-mon[31187]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-13T16:27:29.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:28 smithi016 ceph-mon[31187]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:29.998 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:29 smithi107 ceph-mon[34139]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-13T16:27:29.998 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:29 smithi107 ceph-mon[34139]: osdmap e70: 11 total, 10 up, 11 in 2024-09-13T16:27:29.999 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:29 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T16:27:29.999 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:29 smithi107 ceph-mon[34139]: from='osd.10 [v2:172.21.15.107:6824/2659550426,v1:172.21.15.107:6825/2659550426]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-13T16:27:29.999 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:29 smithi107 ceph-mon[34139]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-13T16:27:30.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:29 smithi016 ceph-mon[31187]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-13T16:27:30.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:29 smithi016 ceph-mon[31187]: osdmap e70: 11 total, 10 up, 11 in 2024-09-13T16:27:30.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:29 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T16:27:30.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:29 smithi016 ceph-mon[31187]: from='osd.10 [v2:172.21.15.107:6824/2659550426,v1:172.21.15.107:6825/2659550426]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-13T16:27:30.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:29 smithi016 ceph-mon[31187]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-13T16:27:30.123 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:29 smithi111 ceph-mon[34600]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-13T16:27:30.123 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:29 smithi111 ceph-mon[34600]: osdmap e70: 11 total, 10 up, 11 in 2024-09-13T16:27:30.123 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:29 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T16:27:30.123 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:29 smithi111 ceph-mon[34600]: from='osd.10 [v2:172.21.15.107:6824/2659550426,v1:172.21.15.107:6825/2659550426]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-13T16:27:30.123 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:29 smithi111 ceph-mon[34600]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-13T16:27:30.521 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-13T16:27:30.599 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:27:30 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:27:30.263+0000 7ff82aaf0640 -1 osd.10 0 waiting for initial osdmap 2024-09-13T16:27:30.599 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:27:30 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:27:30.277+0000 7ff822911640 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T16:27:30.610 INFO:teuthology.orchestra.run.smithi111.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-13T16:27:30.623 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10+0 records in 2024-09-13T16:27:30.623 INFO:teuthology.orchestra.run.smithi111.stderr:10+0 records out 2024-09-13T16:27:30.624 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0116159 s, 903 MB/s 2024-09-13T16:27:30.624 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping successful for: 2024-09-13T16:27:30.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:30 smithi111 ceph-mon[34600]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-13T16:27:30.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:30 smithi111 ceph-mon[34600]: osdmap e71: 11 total, 10 up, 11 in 2024-09-13T16:27:30.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:30 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T16:27:30.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:30 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T16:27:30.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:30 smithi111 ceph-mon[34600]: from='osd.10 ' entity='osd.10' 2024-09-13T16:27:30.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:30 smithi111 ceph-mon[34600]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:31.018 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:30 smithi107 ceph-mon[34139]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-13T16:27:31.018 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:30 smithi107 ceph-mon[34139]: osdmap e71: 11 total, 10 up, 11 in 2024-09-13T16:27:31.018 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:30 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T16:27:31.018 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:30 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T16:27:31.018 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:30 smithi107 ceph-mon[34139]: from='osd.10 ' entity='osd.10' 2024-09-13T16:27:31.019 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:30 smithi107 ceph-mon[34139]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:31.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:30 smithi016 ceph-mon[31187]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-13T16:27:31.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:30 smithi016 ceph-mon[31187]: osdmap e71: 11 total, 10 up, 11 in 2024-09-13T16:27:31.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:30 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T16:27:31.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:30 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T16:27:31.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:30 smithi016 ceph-mon[31187]: from='osd.10 ' entity='osd.10' 2024-09-13T16:27:31.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:30 smithi016 ceph-mon[31187]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T16:27:31.333 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi111:vg_nvme/lv_1 2024-09-13T16:27:31.544 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T16:27:31.899 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:31 smithi111 ceph-mon[34600]: purged_snaps scrub starts 2024-09-13T16:27:31.900 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:31 smithi111 ceph-mon[34600]: purged_snaps scrub ok 2024-09-13T16:27:31.900 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:31 smithi111 ceph-mon[34600]: osd.10 [v2:172.21.15.107:6824/2659550426,v1:172.21.15.107:6825/2659550426] boot 2024-09-13T16:27:31.900 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:31 smithi111 ceph-mon[34600]: osdmap e72: 11 total, 11 up, 11 in 2024-09-13T16:27:31.900 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:31 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T16:27:31.900 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:31 smithi111 ceph-mon[34600]: Detected new or changed devices on smithi107 2024-09-13T16:27:31.900 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:31 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:31.900 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:31 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:31.900 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:31 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:31.900 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:31 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:31.900 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:31 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:31.900 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:31 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:31.900 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:31 smithi111 ceph-mon[34600]: Adjusting osd_memory_target on smithi107 to 4271M 2024-09-13T16:27:31.901 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:31 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:31.901 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:31 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:27:31.901 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:31 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:27:31.901 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:31 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:32.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:31 smithi107 ceph-mon[34139]: purged_snaps scrub starts 2024-09-13T16:27:32.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:31 smithi107 ceph-mon[34139]: purged_snaps scrub ok 2024-09-13T16:27:32.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:31 smithi107 ceph-mon[34139]: osd.10 [v2:172.21.15.107:6824/2659550426,v1:172.21.15.107:6825/2659550426] boot 2024-09-13T16:27:32.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:31 smithi107 ceph-mon[34139]: osdmap e72: 11 total, 11 up, 11 in 2024-09-13T16:27:32.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:31 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T16:27:32.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:31 smithi107 ceph-mon[34139]: Detected new or changed devices on smithi107 2024-09-13T16:27:32.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:31 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:32.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:31 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:32.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:31 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:32.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:31 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:32.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:31 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:32.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:31 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:32.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:31 smithi107 ceph-mon[34139]: Adjusting osd_memory_target on smithi107 to 4271M 2024-09-13T16:27:32.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:31 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:32.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:31 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:27:32.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:31 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:27:32.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:31 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:32.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:31 smithi016 ceph-mon[31187]: purged_snaps scrub starts 2024-09-13T16:27:32.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:31 smithi016 ceph-mon[31187]: purged_snaps scrub ok 2024-09-13T16:27:32.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:31 smithi016 ceph-mon[31187]: osd.10 [v2:172.21.15.107:6824/2659550426,v1:172.21.15.107:6825/2659550426] boot 2024-09-13T16:27:32.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:31 smithi016 ceph-mon[31187]: osdmap e72: 11 total, 11 up, 11 in 2024-09-13T16:27:32.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:31 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T16:27:32.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:31 smithi016 ceph-mon[31187]: Detected new or changed devices on smithi107 2024-09-13T16:27:32.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:31 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:32.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:31 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:32.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:31 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:32.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:31 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:32.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:31 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:32.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:31 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:32.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:31 smithi016 ceph-mon[31187]: Adjusting osd_memory_target on smithi107 to 4271M 2024-09-13T16:27:32.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:31 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:32.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:31 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:27:32.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:31 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:27:32.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:31 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:32.936 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:32 smithi111 ceph-mon[34600]: osdmap e73: 11 total, 11 up, 11 in 2024-09-13T16:27:32.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:32 smithi111 ceph-mon[34600]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-13T16:27:33.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:32 smithi107 ceph-mon[34139]: osdmap e73: 11 total, 11 up, 11 in 2024-09-13T16:27:33.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:32 smithi107 ceph-mon[34139]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-13T16:27:33.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:32 smithi016 ceph-mon[31187]: osdmap e73: 11 total, 11 up, 11 in 2024-09-13T16:27:33.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:32 smithi016 ceph-mon[31187]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-13T16:27:33.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.139+0000 7fc5adad9640 1 -- 172.21.15.111:0/2260075663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5a81008d0 msgr2=0x7fc5a8107160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:27:33.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.139+0000 7fc5adad9640 1 --2- 172.21.15.111:0/2260075663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5a81008d0 0x7fc5a8107160 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0009ca0 tx=0x7fc5a002a8c0 comp rx=0 tx=0).stop 2024-09-13T16:27:33.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.139+0000 7fc5adad9640 1 -- 172.21.15.111:0/2260075663 shutdown_connections 2024-09-13T16:27:33.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.139+0000 7fc5adad9640 1 --2- 172.21.15.111:0/2260075663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5a81008d0 0x7fc5a8107160 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:33.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.139+0000 7fc5adad9640 1 --2- 172.21.15.111:0/2260075663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5a80fff10 0x7fc5a8100390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:33.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.139+0000 7fc5adad9640 1 --2- 172.21.15.111:0/2260075663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5a80fed10 0x7fc5a80ff110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:33.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.139+0000 7fc5adad9640 1 -- 172.21.15.111:0/2260075663 >> 172.21.15.111:0/2260075663 conn(0x7fc5a80fa4c0 msgr2=0x7fc5a80fc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:27:33.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.139+0000 7fc5adad9640 1 -- 172.21.15.111:0/2260075663 shutdown_connections 2024-09-13T16:27:33.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.139+0000 7fc5adad9640 1 -- 172.21.15.111:0/2260075663 wait complete. 2024-09-13T16:27:33.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.140+0000 7fc5adad9640 1 Processor -- start 2024-09-13T16:27:33.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.140+0000 7fc5adad9640 1 -- start start 2024-09-13T16:27:33.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.140+0000 7fc5adad9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5a80fed10 0x7fc5a8194be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:27:33.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.140+0000 7fc5adad9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5a80fff10 0x7fc5a8195120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:27:33.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.140+0000 7fc5adad9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5a81008d0 0x7fc5a819c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:27:33.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.140+0000 7fc5adad9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a819c6e0 con 0x7fc5a81008d0 2024-09-13T16:27:33.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.140+0000 7fc5adad9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a819c850 con 0x7fc5a80fff10 2024-09-13T16:27:33.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.140+0000 7fc5adad9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a819ca10 con 0x7fc5a80fed10 2024-09-13T16:27:33.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.140+0000 7fc5a77fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5a80fed10 0x7fc5a8194be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:27:33.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.140+0000 7fc5a77fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5a80fed10 0x7fc5a8194be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43088/0 (socket says 172.21.15.111:43088) 2024-09-13T16:27:33.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.140+0000 7fc5a77fe640 1 -- 172.21.15.111:0/2691768290 learned_addr learned my addr 172.21.15.111:0/2691768290 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T16:27:33.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.141+0000 7fc5a7fff640 1 --2- 172.21.15.111:0/2691768290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5a81008d0 0x7fc5a819c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:27:33.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.141+0000 7fc5a77fe640 1 -- 172.21.15.111:0/2691768290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5a80fff10 msgr2=0x7fc5a8195120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:27:33.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.141+0000 7fc5a77fe640 1 --2- 172.21.15.111:0/2691768290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5a80fff10 0x7fc5a8195120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:33.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.141+0000 7fc5a77fe640 1 -- 172.21.15.111:0/2691768290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5a81008d0 msgr2=0x7fc5a819c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:27:33.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.141+0000 7fc5a77fe640 1 --2- 172.21.15.111:0/2691768290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5a81008d0 0x7fc5a819c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:33.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.141+0000 7fc5a77fe640 1 -- 172.21.15.111:0/2691768290 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a0009900 con 0x7fc5a80fed10 2024-09-13T16:27:33.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.141+0000 7fc5a7fff640 1 --2- 172.21.15.111:0/2691768290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5a81008d0 0x7fc5a819c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:27:33.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.141+0000 7fc5a77fe640 1 --2- 172.21.15.111:0/2691768290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5a80fed10 0x7fc5a8194be0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc59800b450 tx=0x7fc59800b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:27:33.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.141+0000 7fc5a4ff9640 1 -- 172.21.15.111:0/2691768290 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598011890 con 0x7fc5a80fed10 2024-09-13T16:27:33.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.141+0000 7fc5adad9640 1 -- 172.21.15.111:0/2691768290 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a819ccf0 con 0x7fc5a80fed10 2024-09-13T16:27:33.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.141+0000 7fc5a4ff9640 1 -- 172.21.15.111:0/2691768290 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc598011e80 con 0x7fc5a80fed10 2024-09-13T16:27:33.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.141+0000 7fc5a4ff9640 1 -- 172.21.15.111:0/2691768290 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59800f340 con 0x7fc5a80fed10 2024-09-13T16:27:33.145 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.141+0000 7fc5adad9640 1 -- 172.21.15.111:0/2691768290 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a80687a0 con 0x7fc5a80fed10 2024-09-13T16:27:33.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.142+0000 7fc5adad9640 1 -- 172.21.15.111:0/2691768290 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc594005280 con 0x7fc5a80fed10 2024-09-13T16:27:33.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.143+0000 7fc5a4ff9640 1 -- 172.21.15.111:0/2691768290 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fc5980119f0 con 0x7fc5a80fed10 2024-09-13T16:27:33.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.144+0000 7fc5a4ff9640 1 --2- 172.21.15.111:0/2691768290 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc5900a32c0 0x7fc5900a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:27:33.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.144+0000 7fc5a4ff9640 1 -- 172.21.15.111:0/2691768290 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6723+0+0 (secure 0 0 0) 0x7fc5980cc070 con 0x7fc5a80fed10 2024-09-13T16:27:33.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.144+0000 7fc5a6ffd640 1 --2- 172.21.15.111:0/2691768290 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc5900a32c0 0x7fc5900a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:27:33.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.145+0000 7fc5a6ffd640 1 --2- 172.21.15.111:0/2691768290 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc5900a32c0 0x7fc5900a5780 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc59c005fd0 tx=0x7fc59c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:27:33.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.148+0000 7fc5a4ff9640 1 -- 172.21.15.111:0/2691768290 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc598095e30 con 0x7fc5a80fed10 2024-09-13T16:27:33.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:33.320+0000 7fc5adad9640 1 -- 172.21.15.111:0/2691768290 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc594000bf0 con 0x7fc5900a32c0 2024-09-13T16:27:33.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:33 smithi111 ceph-mon[34600]: osdmap e74: 11 total, 11 up, 11 in 2024-09-13T16:27:33.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:33 smithi111 ceph-mon[34600]: osdmap e75: 11 total, 11 up, 11 in 2024-09-13T16:27:33.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:33 smithi111 ceph-mon[34600]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:27:33.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:33 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:27:33.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:33 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:27:33.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:33 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:27:34.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:33 smithi107 ceph-mon[34139]: osdmap e74: 11 total, 11 up, 11 in 2024-09-13T16:27:34.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:33 smithi107 ceph-mon[34139]: osdmap e75: 11 total, 11 up, 11 in 2024-09-13T16:27:34.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:33 smithi107 ceph-mon[34139]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:27:34.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:33 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:27:34.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:33 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:27:34.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:33 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:27:34.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:33 smithi016 ceph-mon[31187]: osdmap e74: 11 total, 11 up, 11 in 2024-09-13T16:27:34.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:33 smithi016 ceph-mon[31187]: osdmap e75: 11 total, 11 up, 11 in 2024-09-13T16:27:34.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:33 smithi016 ceph-mon[31187]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:27:34.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:33 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T16:27:34.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:33 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T16:27:34.106 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:33 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:27:35.416 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:35 smithi111 ceph-mon[34600]: osdmap e76: 11 total, 11 up, 11 in 2024-09-13T16:27:35.416 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:35 smithi111 ceph-mon[34600]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-13T16:27:35.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:35 smithi107 ceph-mon[34139]: osdmap e76: 11 total, 11 up, 11 in 2024-09-13T16:27:35.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:35 smithi107 ceph-mon[34139]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-13T16:27:35.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:35 smithi016 ceph-mon[31187]: osdmap e76: 11 total, 11 up, 11 in 2024-09-13T16:27:35.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:35 smithi016 ceph-mon[31187]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-13T16:27:36.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:36 smithi107 ceph-mon[34139]: from='client.? 172.21.15.111:0/4149062899' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94a1751d-b00c-4e23-815d-3a8bc3dea146"}]: dispatch 2024-09-13T16:27:36.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:36 smithi107 ceph-mon[34139]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94a1751d-b00c-4e23-815d-3a8bc3dea146"}]: dispatch 2024-09-13T16:27:36.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:36 smithi107 ceph-mon[34139]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94a1751d-b00c-4e23-815d-3a8bc3dea146"}]': finished 2024-09-13T16:27:36.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:36 smithi107 ceph-mon[34139]: osdmap e77: 12 total, 11 up, 12 in 2024-09-13T16:27:36.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:36 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T16:27:36.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:36 smithi016 ceph-mon[31187]: from='client.? 172.21.15.111:0/4149062899' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94a1751d-b00c-4e23-815d-3a8bc3dea146"}]: dispatch 2024-09-13T16:27:36.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:36 smithi016 ceph-mon[31187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94a1751d-b00c-4e23-815d-3a8bc3dea146"}]: dispatch 2024-09-13T16:27:36.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:36 smithi016 ceph-mon[31187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94a1751d-b00c-4e23-815d-3a8bc3dea146"}]': finished 2024-09-13T16:27:36.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:36 smithi016 ceph-mon[31187]: osdmap e77: 12 total, 11 up, 12 in 2024-09-13T16:27:36.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:36 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T16:27:36.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:36 smithi111 ceph-mon[34600]: from='client.? 172.21.15.111:0/4149062899' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94a1751d-b00c-4e23-815d-3a8bc3dea146"}]: dispatch 2024-09-13T16:27:36.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:36 smithi111 ceph-mon[34600]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94a1751d-b00c-4e23-815d-3a8bc3dea146"}]: dispatch 2024-09-13T16:27:36.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:36 smithi111 ceph-mon[34600]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94a1751d-b00c-4e23-815d-3a8bc3dea146"}]': finished 2024-09-13T16:27:36.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:36 smithi111 ceph-mon[34600]: osdmap e77: 12 total, 11 up, 12 in 2024-09-13T16:27:36.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:36 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T16:27:37.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:37 smithi107 ceph-mon[34139]: from='client.? 172.21.15.111:0/1496293117' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:27:37.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:37 smithi107 ceph-mon[34139]: pgmap v233: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-13T16:27:37.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:37 smithi016 ceph-mon[31187]: from='client.? 172.21.15.111:0/1496293117' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:27:37.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:37 smithi016 ceph-mon[31187]: pgmap v233: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-13T16:27:37.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:37 smithi111 ceph-mon[34600]: from='client.? 172.21.15.111:0/1496293117' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T16:27:37.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:37 smithi111 ceph-mon[34600]: pgmap v233: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-13T16:27:38.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:38 smithi111 ceph-mon[34600]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-13T16:27:39.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:38 smithi107 ceph-mon[34139]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-13T16:27:39.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:38 smithi016 ceph-mon[31187]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-13T16:27:40.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:40 smithi111 ceph-mon[34600]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-13T16:27:41.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:40 smithi107 ceph-mon[34139]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-13T16:27:41.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:40 smithi016 ceph-mon[31187]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-13T16:27:42.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:42 smithi111 ceph-mon[34600]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-13T16:27:43.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:42 smithi107 ceph-mon[34139]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-13T16:27:43.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:42 smithi016 ceph-mon[31187]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-13T16:27:43.681 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:43 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-13T16:27:43.681 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:43 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:27:43.681 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:43 smithi111 ceph-mon[34600]: Deploying daemon osd.11 on smithi111 2024-09-13T16:27:44.000 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:43 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-13T16:27:44.000 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:43 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:27:44.000 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:43 smithi107 ceph-mon[34139]: Deploying daemon osd.11 on smithi111 2024-09-13T16:27:44.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:43 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-13T16:27:44.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:43 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:27:44.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:43 smithi016 ceph-mon[31187]: Deploying daemon osd.11 on smithi111 2024-09-13T16:27:44.936 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:44 smithi111 ceph-mon[34600]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-13T16:27:45.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:44 smithi107 ceph-mon[34139]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-13T16:27:45.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:44 smithi016 ceph-mon[31187]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-13T16:27:46.917 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:46 smithi111 ceph-mon[34600]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-13T16:27:47.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:46 smithi107 ceph-mon[34139]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-13T16:27:47.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:46 smithi016 ceph-mon[31187]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-13T16:27:48.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:48 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:27:48.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:48 smithi111 ceph-mon[34600]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-13T16:27:48.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:48 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:48.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:48 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:48.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:48 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:27:48.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:48 smithi016 ceph-mon[31187]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-13T16:27:48.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:48 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:48.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:48 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:49.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:48 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:27:49.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:48 smithi107 ceph-mon[34139]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-13T16:27:49.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:48 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:49.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:48 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:50.773 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:50 smithi111 ceph-mon[34600]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-13T16:27:51.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:50 smithi107 ceph-mon[34139]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-13T16:27:51.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:50 smithi016 ceph-mon[31187]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-13T16:27:51.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:51.669+0000 7fc5a4ff9640 1 -- 172.21.15.111:0/2691768290 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fc594000bf0 con 0x7fc5900a32c0 2024-09-13T16:27:51.671 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 11 on host 'smithi111' 2024-09-13T16:27:51.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:51.671+0000 7fc5adad9640 1 -- 172.21.15.111:0/2691768290 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc5900a32c0 msgr2=0x7fc5900a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:27:51.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:51.671+0000 7fc5adad9640 1 --2- 172.21.15.111:0/2691768290 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc5900a32c0 0x7fc5900a5780 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc59c005fd0 tx=0x7fc59c005850 comp rx=0 tx=0).stop 2024-09-13T16:27:51.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:51.671+0000 7fc5adad9640 1 -- 172.21.15.111:0/2691768290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5a80fed10 msgr2=0x7fc5a8194be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:27:51.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:51.671+0000 7fc5adad9640 1 --2- 172.21.15.111:0/2691768290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5a80fed10 0x7fc5a8194be0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc59800b450 tx=0x7fc59800b910 comp rx=0 tx=0).stop 2024-09-13T16:27:51.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:51.672+0000 7fc5adad9640 1 -- 172.21.15.111:0/2691768290 shutdown_connections 2024-09-13T16:27:51.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:51.672+0000 7fc5adad9640 1 --2- 172.21.15.111:0/2691768290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5a81008d0 0x7fc5a819c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:51.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:51.672+0000 7fc5adad9640 1 --2- 172.21.15.111:0/2691768290 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc5900a32c0 0x7fc5900a5780 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:51.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:51.672+0000 7fc5adad9640 1 --2- 172.21.15.111:0/2691768290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5a80fff10 0x7fc5a8195120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:51.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:51.672+0000 7fc5adad9640 1 --2- 172.21.15.111:0/2691768290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5a80fed10 0x7fc5a8194be0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:51.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:51.672+0000 7fc5adad9640 1 -- 172.21.15.111:0/2691768290 >> 172.21.15.111:0/2691768290 conn(0x7fc5a80fa4c0 msgr2=0x7fc5a81033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:27:51.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:51.672+0000 7fc5adad9640 1 -- 172.21.15.111:0/2691768290 shutdown_connections 2024-09-13T16:27:51.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:27:51.672+0000 7fc5adad9640 1 -- 172.21.15.111:0/2691768290 wait complete. 2024-09-13T16:27:52.332 DEBUG:teuthology.orchestra.run.smithi111:osd.11> sudo journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.11.service 2024-09-13T16:27:52.335 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-13T16:27:52.335 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-13T16:27:52.554 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:27:52.664 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:27:52 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:27:52.457+0000 7fa9d0aa0640 -1 osd.11 0 log_to_monitors true 2024-09-13T16:27:52.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:52 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:52.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:52 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:52.933 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:52 smithi016 ceph-mon[31187]: from='osd.11 [v2:172.21.15.111:6824/1633548683,v1:172.21.15.111:6825/1633548683]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-13T16:27:52.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:52 smithi016 ceph-mon[31187]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-13T16:27:52.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:52 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:52.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:52 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:52.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:52 smithi016 ceph-mon[31187]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-13T16:27:52.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:52 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:52.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:52 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:52.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:52 smithi111 ceph-mon[34600]: from='osd.11 [v2:172.21.15.111:6824/1633548683,v1:172.21.15.111:6825/1633548683]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-13T16:27:52.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:52 smithi111 ceph-mon[34600]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-13T16:27:52.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:52 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:52.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:52 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:52.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:52 smithi111 ceph-mon[34600]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-13T16:27:53.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:52 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:53.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:52 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:53.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:52 smithi107 ceph-mon[34139]: from='osd.11 [v2:172.21.15.111:6824/1633548683,v1:172.21.15.111:6825/1633548683]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-13T16:27:53.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:52 smithi107 ceph-mon[34139]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-13T16:27:53.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:52 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:53.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:52 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:53.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:52 smithi107 ceph-mon[34139]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-13T16:27:53.937 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:27:53 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:27:53.674+0000 7fa9cca21640 -1 osd.11 0 waiting for initial osdmap 2024-09-13T16:27:53.937 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:27:53 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:27:53.688+0000 7fa9c884a640 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T16:27:53.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:53 smithi111 ceph-mon[34600]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-13T16:27:53.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:53 smithi111 ceph-mon[34600]: osdmap e78: 12 total, 11 up, 12 in 2024-09-13T16:27:53.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:53 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T16:27:53.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:53 smithi111 ceph-mon[34600]: from='osd.11 [v2:172.21.15.111:6824/1633548683,v1:172.21.15.111:6825/1633548683]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-13T16:27:53.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:53 smithi111 ceph-mon[34600]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-13T16:27:54.013 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:53 smithi016 ceph-mon[31187]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-13T16:27:54.013 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:53 smithi016 ceph-mon[31187]: osdmap e78: 12 total, 11 up, 12 in 2024-09-13T16:27:54.013 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:53 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T16:27:54.014 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:53 smithi016 ceph-mon[31187]: from='osd.11 [v2:172.21.15.111:6824/1633548683,v1:172.21.15.111:6825/1633548683]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-13T16:27:54.014 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:53 smithi016 ceph-mon[31187]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-13T16:27:54.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:53 smithi107 ceph-mon[34139]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-13T16:27:54.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:53 smithi107 ceph-mon[34139]: osdmap e78: 12 total, 11 up, 12 in 2024-09-13T16:27:54.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:53 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T16:27:54.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:53 smithi107 ceph-mon[34139]: from='osd.11 [v2:172.21.15.111:6824/1633548683,v1:172.21.15.111:6825/1633548683]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-13T16:27:54.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:53 smithi107 ceph-mon[34139]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-13T16:27:54.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.541+0000 7f6f79801640 1 -- 172.21.15.16:0/3045899408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f740feb60 msgr2=0x7f6f740fef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:27:54.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.541+0000 7f6f79801640 1 --2- 172.21.15.16:0/3045899408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f740feb60 0x7f6f740fef60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6f64009900 tx=0x7f6f6402a410 comp rx=0 tx=0).stop 2024-09-13T16:27:54.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.541+0000 7f6f79801640 1 -- 172.21.15.16:0/3045899408 shutdown_connections 2024-09-13T16:27:54.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.541+0000 7f6f79801640 1 --2- 172.21.15.16:0/3045899408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f74100720 0x7f6f74106fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:54.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.541+0000 7f6f79801640 1 --2- 172.21.15.16:0/3045899408 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f740ffd60 0x7f6f741001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:54.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.541+0000 7f6f79801640 1 --2- 172.21.15.16:0/3045899408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f740feb60 0x7f6f740fef60 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:54.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.541+0000 7f6f79801640 1 -- 172.21.15.16:0/3045899408 >> 172.21.15.16:0/3045899408 conn(0x7f6f740fa2f0 msgr2=0x7f6f740fc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:27:54.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.541+0000 7f6f79801640 1 -- 172.21.15.16:0/3045899408 shutdown_connections 2024-09-13T16:27:54.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.541+0000 7f6f79801640 1 -- 172.21.15.16:0/3045899408 wait complete. 2024-09-13T16:27:54.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.542+0000 7f6f79801640 1 Processor -- start 2024-09-13T16:27:54.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.542+0000 7f6f79801640 1 -- start start 2024-09-13T16:27:54.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.542+0000 7f6f79801640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f740feb60 0x7f6f74195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:27:54.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.542+0000 7f6f79801640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f740ffd60 0x7f6f741960c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:27:54.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.542+0000 7f6f79801640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f74100720 0x7f6f7419d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:27:54.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.542+0000 7f6f79801640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7419d630 con 0x7f6f74100720 2024-09-13T16:27:54.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.542+0000 7f6f79801640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7419d7a0 con 0x7f6f740feb60 2024-09-13T16:27:54.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.542+0000 7f6f79801640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7419d960 con 0x7f6f740ffd60 2024-09-13T16:27:54.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.542+0000 7f6f737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f74100720 0x7f6f7419d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:27:54.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.542+0000 7f6f737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f74100720 0x7f6f7419d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48136/0 (socket says 172.21.15.16:48136) 2024-09-13T16:27:54.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.542+0000 7f6f737fe640 1 -- 172.21.15.16:0/1097892181 learned_addr learned my addr 172.21.15.16:0/1097892181 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:27:54.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.542+0000 7f6f727fc640 1 --2- 172.21.15.16:0/1097892181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f740ffd60 0x7f6f741960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:27:54.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.543+0000 7f6f72ffd640 1 --2- 172.21.15.16:0/1097892181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f740feb60 0x7f6f74195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:27:54.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.543+0000 7f6f727fc640 1 -- 172.21.15.16:0/1097892181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f740feb60 msgr2=0x7f6f74195b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:27:54.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.543+0000 7f6f727fc640 1 --2- 172.21.15.16:0/1097892181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f740feb60 0x7f6f74195b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:54.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.543+0000 7f6f727fc640 1 -- 172.21.15.16:0/1097892181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f74100720 msgr2=0x7f6f7419d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:27:54.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.543+0000 7f6f727fc640 1 --2- 172.21.15.16:0/1097892181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f74100720 0x7f6f7419d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:54.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.543+0000 7f6f727fc640 1 -- 172.21.15.16:0/1097892181 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f64009560 con 0x7f6f740ffd60 2024-09-13T16:27:54.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.543+0000 7f6f737fe640 1 --2- 172.21.15.16:0/1097892181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f74100720 0x7f6f7419d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:27:54.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.543+0000 7f6f727fc640 1 --2- 172.21.15.16:0/1097892181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f740ffd60 0x7f6f741960c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6f6800e9f0 tx=0x7f6f6800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:27:54.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.543+0000 7f6f5bfff640 1 -- 172.21.15.16:0/1097892181 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6800ccc0 con 0x7f6f740ffd60 2024-09-13T16:27:54.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.543+0000 7f6f79801640 1 -- 172.21.15.16:0/1097892181 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f7419dc40 con 0x7f6f740ffd60 2024-09-13T16:27:54.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.543+0000 7f6f5bfff640 1 -- 172.21.15.16:0/1097892181 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6f680044b0 con 0x7f6f740ffd60 2024-09-13T16:27:54.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.543+0000 7f6f5bfff640 1 -- 172.21.15.16:0/1097892181 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f68004a00 con 0x7f6f740ffd60 2024-09-13T16:27:54.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.543+0000 7f6f79801640 1 -- 172.21.15.16:0/1097892181 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f7419e140 con 0x7f6f740ffd60 2024-09-13T16:27:54.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.544+0000 7f6f79801640 1 -- 172.21.15.16:0/1097892181 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f7410a2d0 con 0x7f6f740ffd60 2024-09-13T16:27:54.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.545+0000 7f6f5bfff640 1 -- 172.21.15.16:0/1097892181 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f6f68005020 con 0x7f6f740ffd60 2024-09-13T16:27:54.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.546+0000 7f6f5bfff640 1 --2- 172.21.15.16:0/1097892181 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f6f5c0a10a0 0x7f6f5c0a3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:27:54.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.546+0000 7f6f72ffd640 1 --2- 172.21.15.16:0/1097892181 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f6f5c0a10a0 0x7f6f5c0a3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:27:54.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.546+0000 7f6f5bfff640 1 -- 172.21.15.16:0/1097892181 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f6f68014070 con 0x7f6f740ffd60 2024-09-13T16:27:54.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.546+0000 7f6f72ffd640 1 --2- 172.21.15.16:0/1097892181 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f6f5c0a10a0 0x7f6f5c0a3560 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6f640057c0 tx=0x7f6f64005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:27:54.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.549+0000 7f6f5bfff640 1 -- 172.21.15.16:0/1097892181 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f6f68096690 con 0x7f6f740ffd60 2024-09-13T16:27:54.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.726+0000 7f6f79801640 1 -- 172.21.15.16:0/1097892181 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f6f7419e500 con 0x7f6f740ffd60 2024-09-13T16:27:54.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.727+0000 7f6f5bfff640 1 -- 172.21.15.16:0/1097892181 <== mon.1 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) v1 ==== 74+0+133 (secure 0 0 0) 0x7f6f68099cb0 con 0x7f6f740ffd60 2024-09-13T16:27:54.729 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:27:54.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.728+0000 7f6f79801640 1 -- 172.21.15.16:0/1097892181 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f6f5c0a10a0 msgr2=0x7f6f5c0a3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:27:54.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.728+0000 7f6f79801640 1 --2- 172.21.15.16:0/1097892181 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f6f5c0a10a0 0x7f6f5c0a3560 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6f640057c0 tx=0x7f6f64005a20 comp rx=0 tx=0).stop 2024-09-13T16:27:54.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.729+0000 7f6f79801640 1 -- 172.21.15.16:0/1097892181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f740ffd60 msgr2=0x7f6f741960c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:27:54.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.729+0000 7f6f79801640 1 --2- 172.21.15.16:0/1097892181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f740ffd60 0x7f6f741960c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6f6800e9f0 tx=0x7f6f6800eeb0 comp rx=0 tx=0).stop 2024-09-13T16:27:54.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.729+0000 7f6f79801640 1 -- 172.21.15.16:0/1097892181 shutdown_connections 2024-09-13T16:27:54.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.729+0000 7f6f79801640 1 --2- 172.21.15.16:0/1097892181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f74100720 0x7f6f7419d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:54.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.729+0000 7f6f79801640 1 --2- 172.21.15.16:0/1097892181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f740ffd60 0x7f6f741960c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:54.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.729+0000 7f6f79801640 1 --2- 172.21.15.16:0/1097892181 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f6f5c0a10a0 0x7f6f5c0a3560 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:54.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.729+0000 7f6f79801640 1 --2- 172.21.15.16:0/1097892181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f740feb60 0x7f6f74195b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:54.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.729+0000 7f6f79801640 1 -- 172.21.15.16:0/1097892181 >> 172.21.15.16:0/1097892181 conn(0x7f6f740fa2f0 msgr2=0x7f6f74067f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:27:54.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.729+0000 7f6f79801640 1 -- 172.21.15.16:0/1097892181 shutdown_connections 2024-09-13T16:27:54.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:54.729+0000 7f6f79801640 1 -- 172.21.15.16:0/1097892181 wait complete. 2024-09-13T16:27:54.932 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:54 smithi111 ceph-mon[34600]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-13T16:27:54.933 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:54 smithi111 ceph-mon[34600]: osdmap e79: 12 total, 11 up, 12 in 2024-09-13T16:27:54.933 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:54 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T16:27:54.933 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:54 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T16:27:54.933 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:54 smithi111 ceph-mon[34600]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-13T16:27:55.022 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:54 smithi016 ceph-mon[31187]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-13T16:27:55.023 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:54 smithi016 ceph-mon[31187]: osdmap e79: 12 total, 11 up, 12 in 2024-09-13T16:27:55.023 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:54 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T16:27:55.023 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:54 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T16:27:55.023 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:54 smithi016 ceph-mon[31187]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-13T16:27:55.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:54 smithi107 ceph-mon[34139]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-13T16:27:55.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:54 smithi107 ceph-mon[34139]: osdmap e79: 12 total, 11 up, 12 in 2024-09-13T16:27:55.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:54 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T16:27:55.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:54 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T16:27:55.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:54 smithi107 ceph-mon[34139]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-13T16:27:55.558 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1726244874,"num_in_osds":12,"osd_in_since":1726244855,"num_remapped_pgs":0} 2024-09-13T16:27:55.558 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-13T16:27:55.770 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:27:55.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:55 smithi016 ceph-mon[31187]: purged_snaps scrub starts 2024-09-13T16:27:55.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:55 smithi016 ceph-mon[31187]: purged_snaps scrub ok 2024-09-13T16:27:55.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:55 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T16:27:55.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:55 smithi016 ceph-mon[31187]: osd.11 [v2:172.21.15.111:6824/1633548683,v1:172.21.15.111:6825/1633548683] boot 2024-09-13T16:27:55.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:55 smithi016 ceph-mon[31187]: osdmap e80: 12 total, 12 up, 12 in 2024-09-13T16:27:55.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:55 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T16:27:55.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:55 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1097892181' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-13T16:27:55.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:55 smithi111 ceph-mon[34600]: purged_snaps scrub starts 2024-09-13T16:27:55.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:55 smithi111 ceph-mon[34600]: purged_snaps scrub ok 2024-09-13T16:27:55.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:55 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T16:27:55.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:55 smithi111 ceph-mon[34600]: osd.11 [v2:172.21.15.111:6824/1633548683,v1:172.21.15.111:6825/1633548683] boot 2024-09-13T16:27:55.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:55 smithi111 ceph-mon[34600]: osdmap e80: 12 total, 12 up, 12 in 2024-09-13T16:27:55.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:55 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T16:27:55.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:55 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1097892181' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-13T16:27:56.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:55 smithi107 ceph-mon[34139]: purged_snaps scrub starts 2024-09-13T16:27:56.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:55 smithi107 ceph-mon[34139]: purged_snaps scrub ok 2024-09-13T16:27:56.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:55 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T16:27:56.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:55 smithi107 ceph-mon[34139]: osd.11 [v2:172.21.15.111:6824/1633548683,v1:172.21.15.111:6825/1633548683] boot 2024-09-13T16:27:56.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:55 smithi107 ceph-mon[34139]: osdmap e80: 12 total, 12 up, 12 in 2024-09-13T16:27:56.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:55 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T16:27:56.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:55 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1097892181' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-13T16:27:56.957 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:56 smithi016 ceph-mon[31187]: osdmap e81: 12 total, 12 up, 12 in 2024-09-13T16:27:56.957 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:56 smithi016 ceph-mon[31187]: Detected new or changed devices on smithi111 2024-09-13T16:27:56.957 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:56 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:56.957 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:56 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:56.957 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:56 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:56.957 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:56 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:56.957 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:56 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:56.957 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:56 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:56.957 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:56 smithi016 ceph-mon[31187]: Adjusting osd_memory_target on smithi111 to 4272M 2024-09-13T16:27:56.957 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:56 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:56.957 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:56 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:27:56.958 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:56 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:27:56.958 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:56 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:56.958 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:56 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:27:56.958 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:56 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:27:56.958 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:56 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:27:56.958 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:56 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:56.958 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:56 smithi016 ceph-mon[31187]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:27:57.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:56 smithi107 ceph-mon[34139]: osdmap e81: 12 total, 12 up, 12 in 2024-09-13T16:27:57.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:56 smithi107 ceph-mon[34139]: Detected new or changed devices on smithi111 2024-09-13T16:27:57.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:56 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:57.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:56 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:57.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:56 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:57.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:56 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:57.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:56 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:57.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:56 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:57.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:56 smithi107 ceph-mon[34139]: Adjusting osd_memory_target on smithi111 to 4272M 2024-09-13T16:27:57.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:56 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:57.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:56 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:27:57.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:56 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:27:57.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:56 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:57.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:56 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:27:57.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:56 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:27:57.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:56 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:27:57.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:56 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:57.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:56 smithi107 ceph-mon[34139]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:27:57.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:56 smithi111 ceph-mon[34600]: osdmap e81: 12 total, 12 up, 12 in 2024-09-13T16:27:57.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:56 smithi111 ceph-mon[34600]: Detected new or changed devices on smithi111 2024-09-13T16:27:57.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:56 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:57.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:56 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:57.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:56 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:57.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:56 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:57.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:56 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:57.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:56 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T16:27:57.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:56 smithi111 ceph-mon[34600]: Adjusting osd_memory_target on smithi111 to 4272M 2024-09-13T16:27:57.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:56 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:57.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:56 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:27:57.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:56 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:27:57.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:56 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:57.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:56 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:27:57.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:56 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:27:57.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:56 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:27:57.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:56 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:27:57.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:56 smithi111 ceph-mon[34600]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:27:57.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.519+0000 7ff7a4ffe640 1 -- 172.21.15.16:0/3948703411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7a010a3a0 msgr2=0x7ff7a010c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:27:57.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.519+0000 7ff7a4ffe640 1 --2- 172.21.15.16:0/3948703411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7a010a3a0 0x7ff7a010c840 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff798009ca0 tx=0x7ff79802a8c0 comp rx=0 tx=0).stop 2024-09-13T16:27:57.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.519+0000 7ff7a4ffe640 1 -- 172.21.15.16:0/3948703411 shutdown_connections 2024-09-13T16:27:57.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.519+0000 7ff7a4ffe640 1 --2- 172.21.15.16:0/3948703411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7a010a3a0 0x7ff7a010c840 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:57.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.520+0000 7ff7a4ffe640 1 --2- 172.21.15.16:0/3948703411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7a00ff540 0x7ff7a0109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:57.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.520+0000 7ff7a4ffe640 1 --2- 172.21.15.16:0/3948703411 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7a00feb90 0x7ff7a00fef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:57.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.520+0000 7ff7a4ffe640 1 -- 172.21.15.16:0/3948703411 >> 172.21.15.16:0/3948703411 conn(0x7ff7a0074130 msgr2=0x7ff7a0074540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:27:57.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.520+0000 7ff7a4ffe640 1 -- 172.21.15.16:0/3948703411 shutdown_connections 2024-09-13T16:27:57.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.520+0000 7ff7a4ffe640 1 -- 172.21.15.16:0/3948703411 wait complete. 2024-09-13T16:27:57.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.520+0000 7ff7a4ffe640 1 Processor -- start 2024-09-13T16:27:57.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.520+0000 7ff7a4ffe640 1 -- start start 2024-09-13T16:27:57.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.521+0000 7ff7a4ffe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7a00feb90 0x7ff7a01960c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:27:57.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.521+0000 7ff7a4ffe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7a00ff540 0x7ff7a0196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:27:57.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.521+0000 7ff7a4ffe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7a010a3a0 0x7ff7a019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:27:57.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.521+0000 7ff7a4ffe640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a019b000 con 0x7ff7a00ff540 2024-09-13T16:27:57.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.521+0000 7ff7a4ffe640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a019b170 con 0x7ff7a010a3a0 2024-09-13T16:27:57.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.521+0000 7ff7a4ffe640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a019b2e0 con 0x7ff7a00feb90 2024-09-13T16:27:57.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.521+0000 7ff79dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7a00ff540 0x7ff7a0196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:27:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.521+0000 7ff79dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7a00ff540 0x7ff7a0196600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48152/0 (socket says 172.21.15.16:48152) 2024-09-13T16:27:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.521+0000 7ff79dd9b640 1 -- 172.21.15.16:0/526718824 learned_addr learned my addr 172.21.15.16:0/526718824 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:27:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.521+0000 7ff79e59c640 1 --2- 172.21.15.16:0/526718824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7a00feb90 0x7ff7a01960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:27:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.521+0000 7ff79ed9d640 1 --2- 172.21.15.16:0/526718824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7a010a3a0 0x7ff7a019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:27:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.521+0000 7ff79dd9b640 1 -- 172.21.15.16:0/526718824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7a00feb90 msgr2=0x7ff7a01960c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:27:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.521+0000 7ff79dd9b640 1 --2- 172.21.15.16:0/526718824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7a00feb90 0x7ff7a01960c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.521+0000 7ff79dd9b640 1 -- 172.21.15.16:0/526718824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7a010a3a0 msgr2=0x7ff7a019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:27:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.521+0000 7ff79dd9b640 1 --2- 172.21.15.16:0/526718824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7a010a3a0 0x7ff7a019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.521+0000 7ff79dd9b640 1 -- 172.21.15.16:0/526718824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff798009900 con 0x7ff7a00ff540 2024-09-13T16:27:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.521+0000 7ff79ed9d640 1 --2- 172.21.15.16:0/526718824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7a010a3a0 0x7ff7a019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:27:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.521+0000 7ff79dd9b640 1 --2- 172.21.15.16:0/526718824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7a00ff540 0x7ff7a0196600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff79400c970 tx=0x7ff79400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:27:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.521+0000 7ff78f7fe640 1 -- 172.21.15.16:0/526718824 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff794007c10 con 0x7ff7a00ff540 2024-09-13T16:27:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.521+0000 7ff78f7fe640 1 -- 172.21.15.16:0/526718824 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff794007d70 con 0x7ff7a00ff540 2024-09-13T16:27:57.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.521+0000 7ff7a4ffe640 1 -- 172.21.15.16:0/526718824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a019b5c0 con 0x7ff7a00ff540 2024-09-13T16:27:57.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.521+0000 7ff78f7fe640 1 -- 172.21.15.16:0/526718824 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79400f720 con 0x7ff7a00ff540 2024-09-13T16:27:57.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.521+0000 7ff7a4ffe640 1 -- 172.21.15.16:0/526718824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a01a2e80 con 0x7ff7a00ff540 2024-09-13T16:27:57.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.522+0000 7ff7a4ffe640 1 -- 172.21.15.16:0/526718824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a00680a0 con 0x7ff7a00ff540 2024-09-13T16:27:57.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.524+0000 7ff78f7fe640 1 -- 172.21.15.16:0/526718824 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff794018480 con 0x7ff7a00ff540 2024-09-13T16:27:57.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.524+0000 7ff78f7fe640 1 --2- 172.21.15.16:0/526718824 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff7880a1100 0x7ff7880a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:27:57.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.524+0000 7ff79e59c640 1 --2- 172.21.15.16:0/526718824 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff7880a1100 0x7ff7880a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:27:57.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.524+0000 7ff78f7fe640 1 -- 172.21.15.16:0/526718824 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff794013070 con 0x7ff7a00ff540 2024-09-13T16:27:57.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.525+0000 7ff79e59c640 1 --2- 172.21.15.16:0/526718824 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff7880a1100 0x7ff7880a35c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff790005fd0 tx=0x7ff79000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:27:57.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.528+0000 7ff78f7fe640 1 -- 172.21.15.16:0/526718824 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff794096660 con 0x7ff7a00ff540 2024-09-13T16:27:57.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.699+0000 7ff7a4ffe640 1 -- 172.21.15.16:0/526718824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7a00fef70 con 0x7ff7a00ff540 2024-09-13T16:27:57.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.700+0000 7ff78f7fe640 1 -- 172.21.15.16:0/526718824 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18433 (secure 0 0 0) 0x7ff794099c80 con 0x7ff7a00ff540 2024-09-13T16:27:57.702 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:27:57.702 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"fsid":"0d241f6a-71ec-11ef-bceb-c7b262605968","created":"2024-09-13T16:21:17.971738+0000","modified":"2024-09-13T16:27:55.677869+0000","last_up_change":"2024-09-13T16:27:54.673413+0000","last_in_change":"2024-09-13T16:27:35.718390+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:24:22.981019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e11ef8f-08d6-4e74-afd6-f49f4bd40032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6803","nonce":2079394186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6805","nonce":2079394186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6809","nonce":2079394186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6807","nonce":2079394186}]},"public_addr":"172.21.15.16:6803/2079394186","cluster_addr":"172.21.15.16:6805/2079394186","heartbeat_back_addr":"172.21.15.16:6809/2079394186","heartbeat_front_addr":"172.21.15.16:6807/2079394186","state":["exists","up"]},{"osd":1,"uuid":"f2cd3694-9c9e-4da7-976e-f80a8df120f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6801","nonce":4021360958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6803","nonce":4021360958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6807","nonce":4021360958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6805","nonce":4021360958}]},"public_addr":"172.21.15.107:6801/4021360958","cluster_addr":"172.21.15.107:6803/4021360958","heartbeat_back_addr":"172.21.15.107:6807/4021360958","heartbeat_front_addr":"172.21.15.107:6805/4021360958","state":["exists","up"]},{"osd":2,"uuid":"a5bbf2eb-6599-44d5-ad96-68bb89ee19f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6801","nonce":3227179825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6803","nonce":3227179825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6807","nonce":3227179825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6805","nonce":3227179825}]},"public_addr":"172.21.15.111:6801/3227179825","cluster_addr":"172.21.15.111:6803/3227179825","heartbeat_back_addr":"172.21.15.111:6807/3227179825","heartbeat_front_addr":"172.21.15.111:6805/3227179825","state":["exists","up"]},{"osd":3,"uuid":"d6295ffe-11bc-4964-a250-9fecca870b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6811","nonce":2390375937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6813","nonce":2390375937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6817","nonce":2390375937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6815","nonce":2390375937}]},"public_addr":"172.21.15.16:6811/2390375937","cluster_addr":"172.21.15.16:6813/2390375937","heartbeat_back_addr":"172.21.15.16:6817/2390375937","heartbeat_front_addr":"172.21.15.16:6815/2390375937","state":["exists","up"]},{"osd":4,"uuid":"0a8446e0-fc86-4646-beb6-a7b9d433093d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6809","nonce":1936582048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6811","nonce":1936582048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6815","nonce":1936582048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6813","nonce":1936582048}]},"public_addr":"172.21.15.107:6809/1936582048","cluster_addr":"172.21.15.107:6811/1936582048","heartbeat_back_addr":"172.21.15.107:6815/1936582048","heartbeat_front_addr":"172.21.15.107:6813/1936582048","state":["exists","up"]},{"osd":5,"uuid":"2c97a446-357e-4975-9e51-9c95fe84a663","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6809","nonce":483858448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6811","nonce":483858448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6815","nonce":483858448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6813","nonce":483858448}]},"public_addr":"172.21.15.111:6809/483858448","cluster_addr":"172.21.15.111:6811/483858448","heartbeat_back_addr":"172.21.15.111:6815/483858448","heartbeat_front_addr":"172.21.15.111:6813/483858448","state":["exists","up"]},{"osd":6,"uuid":"890239e6-e6fb-4f9c-b973-05cc66d199bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6819","nonce":1330214141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6821","nonce":1330214141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6825","nonce":1330214141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6823","nonce":1330214141}]},"public_addr":"172.21.15.16:6819/1330214141","cluster_addr":"172.21.15.16:6821/1330214141","heartbeat_back_addr":"172.21.15.16:6825/1330214141","heartbeat_front_addr":"172.21.15.16:6823/1330214141","state":["exists","up"]},{"osd":7,"uuid":"ee826a9c-f441-4903-9b1b-66b627f7b712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6817","nonce":1133876754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6819","nonce":1133876754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6823","nonce":1133876754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6821","nonce":1133876754}]},"public_addr":"172.21.15.107:6817/1133876754","cluster_addr":"172.21.15.107:6819/1133876754","heartbeat_back_addr":"172.21.15.107:6823/1133876754","heartbeat_front_addr":"172.21.15.107:6821/1133876754","state":["exists","up"]},{"osd":8,"uuid":"e19773f2-dec8-44e0-9d33-0252e07d6415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6817","nonce":837548190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6819","nonce":837548190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6823","nonce":837548190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6821","nonce":837548190}]},"public_addr":"172.21.15.111:6817/837548190","cluster_addr":"172.21.15.111:6819/837548190","heartbeat_back_addr":"172.21.15.111:6823/837548190","heartbeat_front_addr":"172.21.15.111:6821/837548190","state":["exists","up"]},{"osd":9,"uuid":"bae4d6db-8b90-4d02-8e83-7d8f2a6f8fdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6827","nonce":4175383686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6829","nonce":4175383686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6833","nonce":4175383686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6831","nonce":4175383686}]},"public_addr":"172.21.15.16:6827/4175383686","cluster_addr":"172.21.15.16:6829/4175383686","heartbeat_back_addr":"172.21.15.16:6833/4175383686","heartbeat_front_addr":"172.21.15.16:6831/4175383686","state":["exists","up"]},{"osd":10,"uuid":"fd0e7740-76c8-4c99-a349-7077b4905143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6825","nonce":2659550426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6827","nonce":2659550426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6831","nonce":2659550426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6829","nonce":2659550426}]},"public_addr":"172.21.15.107:6825/2659550426","cluster_addr":"172.21.15.107:6827/2659550426","heartbeat_back_addr":"172.21.15.107:6831/2659550426","heartbeat_front_addr":"172.21.15.107:6829/2659550426","state":["exists","up"]},{"osd":11,"uuid":"94a1751d-b00c-4e23-815d-3a8bc3dea146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6825","nonce":1633548683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6827","nonce":1633548683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6831","nonce":1633548683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6829","nonce":1633548683}]},"public_addr":"172.21.15.111:6825/1633548683","cluster_addr":"172.21.15.111:6827/1633548683","heartbeat_back_addr":"172.21.15.111:6831/1633548683","heartbeat_front_addr":"172.21.15.111:6829/1633548683","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:36.359852+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:58.014202+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:19.753672+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:43.404817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:06.001677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:29.361587+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:52.554847+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:15.957553+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:39.581093+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:04.911389+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:28.971280+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:53.473561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/2098378667":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/3536229016":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6801/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/2684687547":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6800/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/368410612":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/12158160":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6800/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6801/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/3754423379":"2024-09-14T16:21:40.126459+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:27:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.702+0000 7ff7a4ffe640 1 -- 172.21.15.16:0/526718824 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff7880a1100 msgr2=0x7ff7880a35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:27:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.702+0000 7ff7a4ffe640 1 --2- 172.21.15.16:0/526718824 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff7880a1100 0x7ff7880a35c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff790005fd0 tx=0x7ff79000b390 comp rx=0 tx=0).stop 2024-09-13T16:27:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.702+0000 7ff7a4ffe640 1 -- 172.21.15.16:0/526718824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7a00ff540 msgr2=0x7ff7a0196600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:27:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.702+0000 7ff7a4ffe640 1 --2- 172.21.15.16:0/526718824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7a00ff540 0x7ff7a0196600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff79400c970 tx=0x7ff79400ce30 comp rx=0 tx=0).stop 2024-09-13T16:27:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.703+0000 7ff7a4ffe640 1 -- 172.21.15.16:0/526718824 shutdown_connections 2024-09-13T16:27:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.703+0000 7ff7a4ffe640 1 --2- 172.21.15.16:0/526718824 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff7880a1100 0x7ff7880a35c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.703+0000 7ff7a4ffe640 1 --2- 172.21.15.16:0/526718824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7a010a3a0 0x7ff7a019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.703+0000 7ff7a4ffe640 1 --2- 172.21.15.16:0/526718824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7a00ff540 0x7ff7a0196600 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.703+0000 7ff7a4ffe640 1 --2- 172.21.15.16:0/526718824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7a00feb90 0x7ff7a01960c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:27:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.703+0000 7ff7a4ffe640 1 -- 172.21.15.16:0/526718824 >> 172.21.15.16:0/526718824 conn(0x7ff7a0074130 msgr2=0x7ff7a00fbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:27:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.703+0000 7ff7a4ffe640 1 -- 172.21.15.16:0/526718824 shutdown_connections 2024-09-13T16:27:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:27:57.703+0000 7ff7a4ffe640 1 -- 172.21.15.16:0/526718824 wait complete. 2024-09-13T16:27:58.080 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:57 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/526718824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:27:58.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:57 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/526718824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:27:58.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:57 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/526718824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:27:58.416 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:24:22.981019+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T16:27:58.417 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-13T16:27:58.623 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:27:58.997 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:27:58 smithi016 ceph-mon[31187]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:27:59.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:27:58 smithi111 ceph-mon[34600]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:27:59.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:27:58 smithi107 ceph-mon[34139]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:00.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.297+0000 7feeb778c640 1 -- 172.21.15.16:0/599279990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeb010a3a0 msgr2=0x7feeb010c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:00.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.297+0000 7feeb778c640 1 --2- 172.21.15.16:0/599279990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeb010a3a0 0x7feeb010c820 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7feea8009ca0 tx=0x7feea802a950 comp rx=0 tx=0).stop 2024-09-13T16:28:00.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.297+0000 7feeb778c640 1 -- 172.21.15.16:0/599279990 shutdown_connections 2024-09-13T16:28:00.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.297+0000 7feeb778c640 1 --2- 172.21.15.16:0/599279990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeb010a3a0 0x7feeb010c820 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:00.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.297+0000 7feeb778c640 1 --2- 172.21.15.16:0/599279990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feeb01018e0 0x7feeb0109dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:00.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.297+0000 7feeb778c640 1 --2- 172.21.15.16:0/599279990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feeb0100fc0 0x7feeb01013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:00.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.297+0000 7feeb778c640 1 -- 172.21.15.16:0/599279990 >> 172.21.15.16:0/599279990 conn(0x7feeb00fa9a0 msgr2=0x7feeb00fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:00.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.297+0000 7feeb778c640 1 -- 172.21.15.16:0/599279990 shutdown_connections 2024-09-13T16:28:00.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.297+0000 7feeb778c640 1 -- 172.21.15.16:0/599279990 wait complete. 2024-09-13T16:28:00.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.298+0000 7feeb778c640 1 Processor -- start 2024-09-13T16:28:00.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.298+0000 7feeb778c640 1 -- start start 2024-09-13T16:28:00.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.298+0000 7feeb778c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeb0100fc0 0x7feeb00fec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:00.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.298+0000 7feeb778c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feeb01018e0 0x7feeb00ff160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:00.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.298+0000 7feeb778c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feeb010a3a0 0x7feeb00ff6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:00.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.298+0000 7feeb778c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb01a4550 con 0x7feeb0100fc0 2024-09-13T16:28:00.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.298+0000 7feeb778c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb01a46c0 con 0x7feeb01018e0 2024-09-13T16:28:00.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.298+0000 7feeb778c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb01a4880 con 0x7feeb010a3a0 2024-09-13T16:28:00.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.298+0000 7feeb5528640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeb0100fc0 0x7feeb00fec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:00.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.298+0000 7feeb5528640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeb0100fc0 0x7feeb00fec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48172/0 (socket says 172.21.15.16:48172) 2024-09-13T16:28:00.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.298+0000 7feeb5528640 1 -- 172.21.15.16:0/2525280051 learned_addr learned my addr 172.21.15.16:0/2525280051 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:00.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.298+0000 7feeb5d29640 1 --2- 172.21.15.16:0/2525280051 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feeb010a3a0 0x7feeb00ff6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:00.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.298+0000 7feeb5528640 1 -- 172.21.15.16:0/2525280051 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feeb010a3a0 msgr2=0x7feeb00ff6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:00.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.298+0000 7feeb5528640 1 --2- 172.21.15.16:0/2525280051 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feeb010a3a0 0x7feeb00ff6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:00.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.298+0000 7feeb4d27640 1 --2- 172.21.15.16:0/2525280051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feeb01018e0 0x7feeb00ff160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:00.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.298+0000 7feeb5528640 1 -- 172.21.15.16:0/2525280051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feeb01018e0 msgr2=0x7feeb00ff160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:00.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.298+0000 7feeb5528640 1 --2- 172.21.15.16:0/2525280051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feeb01018e0 0x7feeb00ff160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:00.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.298+0000 7feeb5528640 1 -- 172.21.15.16:0/2525280051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea8009900 con 0x7feeb0100fc0 2024-09-13T16:28:00.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.298+0000 7feeb5d29640 1 --2- 172.21.15.16:0/2525280051 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feeb010a3a0 0x7feeb00ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:28:00.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.299+0000 7feeb5528640 1 --2- 172.21.15.16:0/2525280051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeb0100fc0 0x7feeb00fec20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feeac00c890 tx=0x7feeac00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:00.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.299+0000 7feea67fc640 1 -- 172.21.15.16:0/2525280051 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac004220 con 0x7feeb0100fc0 2024-09-13T16:28:00.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.299+0000 7feea67fc640 1 -- 172.21.15.16:0/2525280051 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feeac007e40 con 0x7feeb0100fc0 2024-09-13T16:28:00.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.299+0000 7feea67fc640 1 -- 172.21.15.16:0/2525280051 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac004a00 con 0x7feeb0100fc0 2024-09-13T16:28:00.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.299+0000 7feeb778c640 1 -- 172.21.15.16:0/2525280051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeb01a4b60 con 0x7feeb0100fc0 2024-09-13T16:28:00.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.299+0000 7feeb778c640 1 -- 172.21.15.16:0/2525280051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeb01a5060 con 0x7feeb0100fc0 2024-09-13T16:28:00.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.300+0000 7feeb778c640 1 -- 172.21.15.16:0/2525280051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeb0068fa0 con 0x7feeb0100fc0 2024-09-13T16:28:00.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.301+0000 7feea67fc640 1 -- 172.21.15.16:0/2525280051 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7feeac0079b0 con 0x7feeb0100fc0 2024-09-13T16:28:00.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.302+0000 7feea67fc640 1 --2- 172.21.15.16:0/2525280051 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fee9c0a1100 0x7fee9c0a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:00.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.302+0000 7feeb4d27640 1 --2- 172.21.15.16:0/2525280051 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fee9c0a1100 0x7fee9c0a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:00.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.302+0000 7feea67fc640 1 -- 172.21.15.16:0/2525280051 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7feeac0cc4a0 con 0x7feeb0100fc0 2024-09-13T16:28:00.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.303+0000 7feeb4d27640 1 --2- 172.21.15.16:0/2525280051 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fee9c0a1100 0x7fee9c0a35c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7feea0005d80 tx=0x7feea0005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:00.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.306+0000 7feea67fc640 1 -- 172.21.15.16:0/2525280051 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7feeac0960d0 con 0x7feeb0100fc0 2024-09-13T16:28:00.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.481+0000 7feeb778c640 1 -- 172.21.15.16:0/2525280051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feeb01054a0 con 0x7feeb0100fc0 2024-09-13T16:28:00.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.481+0000 7feea67fc640 1 -- 172.21.15.16:0/2525280051 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7feeac0996f0 con 0x7feeb0100fc0 2024-09-13T16:28:00.483 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-13T16:28:00.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.484+0000 7feeb778c640 1 -- 172.21.15.16:0/2525280051 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fee9c0a1100 msgr2=0x7fee9c0a35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:00.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.484+0000 7feeb778c640 1 --2- 172.21.15.16:0/2525280051 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fee9c0a1100 0x7fee9c0a35c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7feea0005d80 tx=0x7feea0005c90 comp rx=0 tx=0).stop 2024-09-13T16:28:00.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.484+0000 7feeb778c640 1 -- 172.21.15.16:0/2525280051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeb0100fc0 msgr2=0x7feeb00fec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:00.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.484+0000 7feeb778c640 1 --2- 172.21.15.16:0/2525280051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeb0100fc0 0x7feeb00fec20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feeac00c890 tx=0x7feeac00cd50 comp rx=0 tx=0).stop 2024-09-13T16:28:00.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.484+0000 7feeb778c640 1 -- 172.21.15.16:0/2525280051 shutdown_connections 2024-09-13T16:28:00.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.484+0000 7feeb778c640 1 --2- 172.21.15.16:0/2525280051 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feeb010a3a0 0x7feeb00ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:00.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.484+0000 7feeb778c640 1 --2- 172.21.15.16:0/2525280051 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fee9c0a1100 0x7fee9c0a35c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:00.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.484+0000 7feeb778c640 1 --2- 172.21.15.16:0/2525280051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feeb01018e0 0x7feeb00ff160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:00.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.484+0000 7feeb778c640 1 --2- 172.21.15.16:0/2525280051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeb0100fc0 0x7feeb00fec20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:00.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.484+0000 7feeb778c640 1 -- 172.21.15.16:0/2525280051 >> 172.21.15.16:0/2525280051 conn(0x7feeb00fa9a0 msgr2=0x7feeb00fc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:00.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.484+0000 7feeb778c640 1 -- 172.21.15.16:0/2525280051 shutdown_connections 2024-09-13T16:28:00.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:00.484+0000 7feeb778c640 1 -- 172.21.15.16:0/2525280051 wait complete. 2024-09-13T16:28:00.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:00 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/2525280051' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:28:00.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:00 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2525280051' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:28:00.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:00 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/2525280051' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:28:01.209 INFO:tasks.cephadm:Adding mds.a on smithi016 2024-09-13T16:28:01.209 INFO:tasks.cephadm:Adding mds.d on smithi016 2024-09-13T16:28:01.209 INFO:tasks.cephadm:Adding mds.g on smithi016 2024-09-13T16:28:01.209 INFO:tasks.cephadm:Adding mds.j on smithi016 2024-09-13T16:28:01.209 INFO:tasks.cephadm:Adding mds.b on smithi107 2024-09-13T16:28:01.209 INFO:tasks.cephadm:Adding mds.e on smithi107 2024-09-13T16:28:01.209 INFO:tasks.cephadm:Adding mds.h on smithi107 2024-09-13T16:28:01.209 INFO:tasks.cephadm:Adding mds.k on smithi107 2024-09-13T16:28:01.209 INFO:tasks.cephadm:Adding mds.c on smithi111 2024-09-13T16:28:01.210 INFO:tasks.cephadm:Adding mds.f on smithi111 2024-09-13T16:28:01.210 INFO:tasks.cephadm:Adding mds.i on smithi111 2024-09-13T16:28:01.210 INFO:tasks.cephadm:Adding mds.l on smithi111 2024-09-13T16:28:01.210 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph orch apply mds all '12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi111=c;smithi111=f;smithi111=i;smithi111=l' 2024-09-13T16:28:01.419 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T16:28:01.845 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:01 smithi111 ceph-mon[34600]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:01.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:01 smithi107 ceph-mon[34139]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:01.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:01 smithi016 ceph-mon[31187]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:02.876 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:02 smithi111 ceph-mon[34600]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:03.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:02 smithi107 ceph-mon[34139]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:03.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:02 smithi016 ceph-mon[31187]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:03.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.228+0000 7fd2c7e4b640 1 -- 172.21.15.111:0/2009320709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2c01008b0 msgr2=0x7fd2c0107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:03.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.228+0000 7fd2c7e4b640 1 --2- 172.21.15.111:0/2009320709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2c01008b0 0x7fd2c0107140 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8009cc0 tx=0x7fd2b802caf0 comp rx=0 tx=0).stop 2024-09-13T16:28:03.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.228+0000 7fd2c7e4b640 1 -- 172.21.15.111:0/2009320709 shutdown_connections 2024-09-13T16:28:03.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.228+0000 7fd2c7e4b640 1 --2- 172.21.15.111:0/2009320709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2c01008b0 0x7fd2c0107140 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:03.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.228+0000 7fd2c7e4b640 1 --2- 172.21.15.111:0/2009320709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c00ffef0 0x7fd2c0100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:03.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.228+0000 7fd2c7e4b640 1 --2- 172.21.15.111:0/2009320709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2c00fecf0 0x7fd2c00ff0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:03.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.228+0000 7fd2c7e4b640 1 -- 172.21.15.111:0/2009320709 >> 172.21.15.111:0/2009320709 conn(0x7fd2c00fa480 msgr2=0x7fd2c00fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:03.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.228+0000 7fd2c7e4b640 1 -- 172.21.15.111:0/2009320709 shutdown_connections 2024-09-13T16:28:03.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.228+0000 7fd2c7e4b640 1 -- 172.21.15.111:0/2009320709 wait complete. 2024-09-13T16:28:03.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.229+0000 7fd2c7e4b640 1 Processor -- start 2024-09-13T16:28:03.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.229+0000 7fd2c7e4b640 1 -- start start 2024-09-13T16:28:03.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.229+0000 7fd2c7e4b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c00fecf0 0x7fd2c0197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:03.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.229+0000 7fd2c7e4b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2c00ffef0 0x7fd2c0198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:03.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.229+0000 7fd2c7e4b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2c01008b0 0x7fd2c019f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:03.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.229+0000 7fd2c7e4b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c019f930 con 0x7fd2c00fecf0 2024-09-13T16:28:03.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.229+0000 7fd2c7e4b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c019faa0 con 0x7fd2c00ffef0 2024-09-13T16:28:03.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.229+0000 7fd2c7e4b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c019fc60 con 0x7fd2c01008b0 2024-09-13T16:28:03.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.229+0000 7fd2c63e8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2c01008b0 0x7fd2c019f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:03.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.229+0000 7fd2c63e8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2c01008b0 0x7fd2c019f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44310/0 (socket says 172.21.15.111:44310) 2024-09-13T16:28:03.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.229+0000 7fd2c63e8640 1 -- 172.21.15.111:0/2493323822 learned_addr learned my addr 172.21.15.111:0/2493323822 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-13T16:28:03.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.229+0000 7fd2c5be7640 1 --2- 172.21.15.111:0/2493323822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c00fecf0 0x7fd2c0197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:03.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.229+0000 7fd2c53e6640 1 --2- 172.21.15.111:0/2493323822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2c00ffef0 0x7fd2c0198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:03.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.229+0000 7fd2c63e8640 1 -- 172.21.15.111:0/2493323822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2c00ffef0 msgr2=0x7fd2c0198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:03.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.229+0000 7fd2c63e8640 1 --2- 172.21.15.111:0/2493323822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2c00ffef0 0x7fd2c0198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:03.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.229+0000 7fd2c63e8640 1 -- 172.21.15.111:0/2493323822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c00fecf0 msgr2=0x7fd2c0197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:03.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.229+0000 7fd2c63e8640 1 --2- 172.21.15.111:0/2493323822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c00fecf0 0x7fd2c0197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:03.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.229+0000 7fd2c63e8640 1 -- 172.21.15.111:0/2493323822 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b8009900 con 0x7fd2c01008b0 2024-09-13T16:28:03.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.230+0000 7fd2c63e8640 1 --2- 172.21.15.111:0/2493323822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2c01008b0 0x7fd2c019f3f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8009c10 tx=0x7fd2b8003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:03.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.230+0000 7fd2b6ffd640 1 -- 172.21.15.111:0/2493323822 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8003e30 con 0x7fd2c01008b0 2024-09-13T16:28:03.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.230+0000 7fd2b6ffd640 1 -- 172.21.15.111:0/2493323822 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd2b802e650 con 0x7fd2c01008b0 2024-09-13T16:28:03.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.230+0000 7fd2c7e4b640 1 -- 172.21.15.111:0/2493323822 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c019fee0 con 0x7fd2c01008b0 2024-09-13T16:28:03.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.230+0000 7fd2b6ffd640 1 -- 172.21.15.111:0/2493323822 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b80043a0 con 0x7fd2c01008b0 2024-09-13T16:28:03.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.230+0000 7fd2c7e4b640 1 -- 172.21.15.111:0/2493323822 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c01a0350 con 0x7fd2c01008b0 2024-09-13T16:28:03.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.231+0000 7fd2c7e4b640 1 -- 172.21.15.111:0/2493323822 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c0106230 con 0x7fd2c01008b0 2024-09-13T16:28:03.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.232+0000 7fd2b6ffd640 1 -- 172.21.15.111:0/2493323822 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd2b800a040 con 0x7fd2c01008b0 2024-09-13T16:28:03.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.232+0000 7fd2b6ffd640 1 --2- 172.21.15.111:0/2493323822 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd2ac0a1130 0x7fd2ac0a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:03.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.233+0000 7fd2b6ffd640 1 -- 172.21.15.111:0/2493323822 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd2b80ecac0 con 0x7fd2c01008b0 2024-09-13T16:28:03.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.233+0000 7fd2c5be7640 1 --2- 172.21.15.111:0/2493323822 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd2ac0a1130 0x7fd2ac0a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:03.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.233+0000 7fd2c5be7640 1 --2- 172.21.15.111:0/2493323822 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd2ac0a1130 0x7fd2ac0a35f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0005fd0 tx=0x7fd2b0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:03.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.237+0000 7fd2b6ffd640 1 -- 172.21.15.111:0/2493323822 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd2b80b66f0 con 0x7fd2c01008b0 2024-09-13T16:28:03.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.413+0000 7fd2c7e4b640 1 -- 172.21.15.111:0/2493323822 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi111=c;smithi111=f;smithi111=i;smithi111=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fd2c002c4f0 con 0x7fd2ac0a1130 2024-09-13T16:28:03.422 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.421+0000 7fd2b6ffd640 1 -- 172.21.15.111:0/2493323822 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fd2c002c4f0 con 0x7fd2ac0a1130 2024-09-13T16:28:03.422 INFO:teuthology.orchestra.run.smithi111.stdout:Scheduled mds.all update... 2024-09-13T16:28:03.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.423+0000 7fd2c7e4b640 1 -- 172.21.15.111:0/2493323822 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd2ac0a1130 msgr2=0x7fd2ac0a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:03.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.423+0000 7fd2c7e4b640 1 --2- 172.21.15.111:0/2493323822 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd2ac0a1130 0x7fd2ac0a35f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0005fd0 tx=0x7fd2b0005850 comp rx=0 tx=0).stop 2024-09-13T16:28:03.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.423+0000 7fd2c7e4b640 1 -- 172.21.15.111:0/2493323822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2c01008b0 msgr2=0x7fd2c019f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:03.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.423+0000 7fd2c7e4b640 1 --2- 172.21.15.111:0/2493323822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2c01008b0 0x7fd2c019f3f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8009c10 tx=0x7fd2b8003c10 comp rx=0 tx=0).stop 2024-09-13T16:28:03.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.423+0000 7fd2c7e4b640 1 -- 172.21.15.111:0/2493323822 shutdown_connections 2024-09-13T16:28:03.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.423+0000 7fd2c7e4b640 1 --2- 172.21.15.111:0/2493323822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2c01008b0 0x7fd2c019f3f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:03.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.423+0000 7fd2c7e4b640 1 --2- 172.21.15.111:0/2493323822 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd2ac0a1130 0x7fd2ac0a35f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:03.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.423+0000 7fd2c7e4b640 1 --2- 172.21.15.111:0/2493323822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2c00ffef0 0x7fd2c0198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:03.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.423+0000 7fd2c7e4b640 1 --2- 172.21.15.111:0/2493323822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c00fecf0 0x7fd2c0197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:03.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.423+0000 7fd2c7e4b640 1 -- 172.21.15.111:0/2493323822 >> 172.21.15.111:0/2493323822 conn(0x7fd2c00fa480 msgr2=0x7fd2c0103400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:03.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.423+0000 7fd2c7e4b640 1 -- 172.21.15.111:0/2493323822 shutdown_connections 2024-09-13T16:28:03.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-13T16:28:03.423+0000 7fd2c7e4b640 1 -- 172.21.15.111:0/2493323822 wait complete. 2024-09-13T16:28:04.134 DEBUG:teuthology.orchestra.run.smithi016:mds.a> sudo journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.a.service 2024-09-13T16:28:04.137 DEBUG:teuthology.orchestra.run.smithi016:mds.d> sudo journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.d.service 2024-09-13T16:28:04.180 DEBUG:teuthology.orchestra.run.smithi016:mds.g> sudo journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.g.service 2024-09-13T16:28:04.224 DEBUG:teuthology.orchestra.run.smithi016:mds.j> sudo journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.j.service 2024-09-13T16:28:04.267 DEBUG:teuthology.orchestra.run.smithi107:mds.b> sudo journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.b.service 2024-09-13T16:28:04.269 DEBUG:teuthology.orchestra.run.smithi107:mds.e> sudo journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.e.service 2024-09-13T16:28:04.314 DEBUG:teuthology.orchestra.run.smithi107:mds.h> sudo journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.h.service 2024-09-13T16:28:04.358 DEBUG:teuthology.orchestra.run.smithi107:mds.k> sudo journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.k.service 2024-09-13T16:28:04.401 DEBUG:teuthology.orchestra.run.smithi111:mds.c> sudo journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.c.service 2024-09-13T16:28:04.403 DEBUG:teuthology.orchestra.run.smithi111:mds.f> sudo journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.f.service 2024-09-13T16:28:04.447 DEBUG:teuthology.orchestra.run.smithi111:mds.i> sudo journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.i.service 2024-09-13T16:28:04.490 DEBUG:teuthology.orchestra.run.smithi111:mds.l> sudo journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.l.service 2024-09-13T16:28:04.533 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-13T16:28:04.534 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:28:04.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:04 smithi107 ceph-mon[34139]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi111=c;smithi111=f;smithi111=i;smithi111=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:28:04.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:04 smithi107 ceph-mon[34139]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi111=c;smithi111=f;smithi111=i;smithi111=l;count:12 2024-09-13T16:28:04.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:04.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:28:04.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:04.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:28:04.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:04.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:04.601 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:04.601 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:04.637 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:04 smithi111 ceph-mon[34600]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi111=c;smithi111=f;smithi111=i;smithi111=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:28:04.637 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:04 smithi111 ceph-mon[34600]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi111=c;smithi111=f;smithi111=i;smithi111=l;count:12 2024-09-13T16:28:04.637 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:04.638 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:28:04.638 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:04.638 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:28:04.638 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:04.638 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:04.638 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:04.639 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:04.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.649+0000 7fdb3a2fe640 1 -- 172.21.15.16:0/3065872500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb340ffe90 msgr2=0x7fdb34100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:04.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.649+0000 7fdb3a2fe640 1 --2- 172.21.15.16:0/3065872500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb340ffe90 0x7fdb34100310 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdb28009900 tx=0x7fdb2802a410 comp rx=0 tx=0).stop 2024-09-13T16:28:04.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.649+0000 7fdb3a2fe640 1 -- 172.21.15.16:0/3065872500 shutdown_connections 2024-09-13T16:28:04.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.649+0000 7fdb3a2fe640 1 --2- 172.21.15.16:0/3065872500 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb34100850 0x7fdb341070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:04.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.649+0000 7fdb3a2fe640 1 --2- 172.21.15.16:0/3065872500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb340ffe90 0x7fdb34100310 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:04.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.649+0000 7fdb3a2fe640 1 --2- 172.21.15.16:0/3065872500 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb340feca0 0x7fdb340ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:04.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.649+0000 7fdb3a2fe640 1 -- 172.21.15.16:0/3065872500 >> 172.21.15.16:0/3065872500 conn(0x7fdb340fa440 msgr2=0x7fdb340fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:04.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.649+0000 7fdb3a2fe640 1 -- 172.21.15.16:0/3065872500 shutdown_connections 2024-09-13T16:28:04.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.649+0000 7fdb3a2fe640 1 -- 172.21.15.16:0/3065872500 wait complete. 2024-09-13T16:28:04.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.650+0000 7fdb3a2fe640 1 Processor -- start 2024-09-13T16:28:04.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.650+0000 7fdb3a2fe640 1 -- start start 2024-09-13T16:28:04.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.650+0000 7fdb3a2fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb340feca0 0x7fdb34194c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:04.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.650+0000 7fdb3a2fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb340ffe90 0x7fdb34195190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:04.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.650+0000 7fdb3a2fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb34100850 0x7fdb3419c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:04.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.650+0000 7fdb3a2fe640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3419c730 con 0x7fdb340ffe90 2024-09-13T16:28:04.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.650+0000 7fdb3a2fe640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3419c8a0 con 0x7fdb34100850 2024-09-13T16:28:04.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.650+0000 7fdb3a2fe640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3419ca60 con 0x7fdb340feca0 2024-09-13T16:28:04.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.651+0000 7fdb337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb340ffe90 0x7fdb34195190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:04.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.651+0000 7fdb337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb340ffe90 0x7fdb34195190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59308/0 (socket says 172.21.15.16:59308) 2024-09-13T16:28:04.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.651+0000 7fdb337fe640 1 -- 172.21.15.16:0/4090045914 learned_addr learned my addr 172.21.15.16:0/4090045914 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:04.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.651+0000 7fdb33fff640 1 --2- 172.21.15.16:0/4090045914 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb340feca0 0x7fdb34194c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:04.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.651+0000 7fdb3889b640 1 --2- 172.21.15.16:0/4090045914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb34100850 0x7fdb3419c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:04.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.651+0000 7fdb337fe640 1 -- 172.21.15.16:0/4090045914 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb340feca0 msgr2=0x7fdb34194c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:04.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.651+0000 7fdb337fe640 1 --2- 172.21.15.16:0/4090045914 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb340feca0 0x7fdb34194c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:04.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.651+0000 7fdb337fe640 1 -- 172.21.15.16:0/4090045914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb34100850 msgr2=0x7fdb3419c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:04.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.651+0000 7fdb337fe640 1 --2- 172.21.15.16:0/4090045914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb34100850 0x7fdb3419c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:04.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.651+0000 7fdb337fe640 1 -- 172.21.15.16:0/4090045914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb28009560 con 0x7fdb340ffe90 2024-09-13T16:28:04.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.651+0000 7fdb337fe640 1 --2- 172.21.15.16:0/4090045914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb340ffe90 0x7fdb34195190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdb28006010 tx=0x7fdb280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:04.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.651+0000 7fdb317fa640 1 -- 172.21.15.16:0/4090045914 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb28004cf0 con 0x7fdb340ffe90 2024-09-13T16:28:04.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.651+0000 7fdb317fa640 1 -- 172.21.15.16:0/4090045914 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdb28003a80 con 0x7fdb340ffe90 2024-09-13T16:28:04.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.651+0000 7fdb3a2fe640 1 -- 172.21.15.16:0/4090045914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb3419cce0 con 0x7fdb340ffe90 2024-09-13T16:28:04.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.651+0000 7fdb317fa640 1 -- 172.21.15.16:0/4090045914 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2803ca30 con 0x7fdb340ffe90 2024-09-13T16:28:04.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.651+0000 7fdb3a2fe640 1 -- 172.21.15.16:0/4090045914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb3419d190 con 0x7fdb340ffe90 2024-09-13T16:28:04.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.652+0000 7fdb3a2fe640 1 -- 172.21.15.16:0/4090045914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb34106220 con 0x7fdb340ffe90 2024-09-13T16:28:04.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.654+0000 7fdb317fa640 1 -- 172.21.15.16:0/4090045914 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fdb28003bf0 con 0x7fdb340ffe90 2024-09-13T16:28:04.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.654+0000 7fdb317fa640 1 --2- 172.21.15.16:0/4090045914 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdb1c0a1110 0x7fdb1c0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:04.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.654+0000 7fdb33fff640 1 --2- 172.21.15.16:0/4090045914 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdb1c0a1110 0x7fdb1c0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:04.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.654+0000 7fdb317fa640 1 -- 172.21.15.16:0/4090045914 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdb280ed3b0 con 0x7fdb340ffe90 2024-09-13T16:28:04.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.655+0000 7fdb33fff640 1 --2- 172.21.15.16:0/4090045914 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdb1c0a1110 0x7fdb1c0a35d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdb34196120 tx=0x7fdb24005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:04.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.658+0000 7fdb317fa640 1 -- 172.21.15.16:0/4090045914 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fdb280b7000 con 0x7fdb340ffe90 2024-09-13T16:28:04.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.838+0000 7fdb3a2fe640 1 -- 172.21.15.16:0/4090045914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb3410a440 con 0x7fdb340ffe90 2024-09-13T16:28:04.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.839+0000 7fdb317fa640 1 -- 172.21.15.16:0/4090045914 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18433 (secure 0 0 0) 0x7fdb280b8f30 con 0x7fdb340ffe90 2024-09-13T16:28:04.841 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:28:04.841 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"fsid":"0d241f6a-71ec-11ef-bceb-c7b262605968","created":"2024-09-13T16:21:17.971738+0000","modified":"2024-09-13T16:27:55.677869+0000","last_up_change":"2024-09-13T16:27:54.673413+0000","last_in_change":"2024-09-13T16:27:35.718390+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:24:22.981019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e11ef8f-08d6-4e74-afd6-f49f4bd40032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6803","nonce":2079394186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6805","nonce":2079394186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6809","nonce":2079394186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6807","nonce":2079394186}]},"public_addr":"172.21.15.16:6803/2079394186","cluster_addr":"172.21.15.16:6805/2079394186","heartbeat_back_addr":"172.21.15.16:6809/2079394186","heartbeat_front_addr":"172.21.15.16:6807/2079394186","state":["exists","up"]},{"osd":1,"uuid":"f2cd3694-9c9e-4da7-976e-f80a8df120f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6801","nonce":4021360958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6803","nonce":4021360958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6807","nonce":4021360958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6805","nonce":4021360958}]},"public_addr":"172.21.15.107:6801/4021360958","cluster_addr":"172.21.15.107:6803/4021360958","heartbeat_back_addr":"172.21.15.107:6807/4021360958","heartbeat_front_addr":"172.21.15.107:6805/4021360958","state":["exists","up"]},{"osd":2,"uuid":"a5bbf2eb-6599-44d5-ad96-68bb89ee19f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6801","nonce":3227179825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6803","nonce":3227179825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6807","nonce":3227179825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6805","nonce":3227179825}]},"public_addr":"172.21.15.111:6801/3227179825","cluster_addr":"172.21.15.111:6803/3227179825","heartbeat_back_addr":"172.21.15.111:6807/3227179825","heartbeat_front_addr":"172.21.15.111:6805/3227179825","state":["exists","up"]},{"osd":3,"uuid":"d6295ffe-11bc-4964-a250-9fecca870b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6811","nonce":2390375937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6813","nonce":2390375937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6817","nonce":2390375937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6815","nonce":2390375937}]},"public_addr":"172.21.15.16:6811/2390375937","cluster_addr":"172.21.15.16:6813/2390375937","heartbeat_back_addr":"172.21.15.16:6817/2390375937","heartbeat_front_addr":"172.21.15.16:6815/2390375937","state":["exists","up"]},{"osd":4,"uuid":"0a8446e0-fc86-4646-beb6-a7b9d433093d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6809","nonce":1936582048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6811","nonce":1936582048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6815","nonce":1936582048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6813","nonce":1936582048}]},"public_addr":"172.21.15.107:6809/1936582048","cluster_addr":"172.21.15.107:6811/1936582048","heartbeat_back_addr":"172.21.15.107:6815/1936582048","heartbeat_front_addr":"172.21.15.107:6813/1936582048","state":["exists","up"]},{"osd":5,"uuid":"2c97a446-357e-4975-9e51-9c95fe84a663","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6809","nonce":483858448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6811","nonce":483858448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6815","nonce":483858448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6813","nonce":483858448}]},"public_addr":"172.21.15.111:6809/483858448","cluster_addr":"172.21.15.111:6811/483858448","heartbeat_back_addr":"172.21.15.111:6815/483858448","heartbeat_front_addr":"172.21.15.111:6813/483858448","state":["exists","up"]},{"osd":6,"uuid":"890239e6-e6fb-4f9c-b973-05cc66d199bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6819","nonce":1330214141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6821","nonce":1330214141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6825","nonce":1330214141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6823","nonce":1330214141}]},"public_addr":"172.21.15.16:6819/1330214141","cluster_addr":"172.21.15.16:6821/1330214141","heartbeat_back_addr":"172.21.15.16:6825/1330214141","heartbeat_front_addr":"172.21.15.16:6823/1330214141","state":["exists","up"]},{"osd":7,"uuid":"ee826a9c-f441-4903-9b1b-66b627f7b712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6817","nonce":1133876754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6819","nonce":1133876754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6823","nonce":1133876754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6821","nonce":1133876754}]},"public_addr":"172.21.15.107:6817/1133876754","cluster_addr":"172.21.15.107:6819/1133876754","heartbeat_back_addr":"172.21.15.107:6823/1133876754","heartbeat_front_addr":"172.21.15.107:6821/1133876754","state":["exists","up"]},{"osd":8,"uuid":"e19773f2-dec8-44e0-9d33-0252e07d6415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6817","nonce":837548190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6819","nonce":837548190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6823","nonce":837548190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6821","nonce":837548190}]},"public_addr":"172.21.15.111:6817/837548190","cluster_addr":"172.21.15.111:6819/837548190","heartbeat_back_addr":"172.21.15.111:6823/837548190","heartbeat_front_addr":"172.21.15.111:6821/837548190","state":["exists","up"]},{"osd":9,"uuid":"bae4d6db-8b90-4d02-8e83-7d8f2a6f8fdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6827","nonce":4175383686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6829","nonce":4175383686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6833","nonce":4175383686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6831","nonce":4175383686}]},"public_addr":"172.21.15.16:6827/4175383686","cluster_addr":"172.21.15.16:6829/4175383686","heartbeat_back_addr":"172.21.15.16:6833/4175383686","heartbeat_front_addr":"172.21.15.16:6831/4175383686","state":["exists","up"]},{"osd":10,"uuid":"fd0e7740-76c8-4c99-a349-7077b4905143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6825","nonce":2659550426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6827","nonce":2659550426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6831","nonce":2659550426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6829","nonce":2659550426}]},"public_addr":"172.21.15.107:6825/2659550426","cluster_addr":"172.21.15.107:6827/2659550426","heartbeat_back_addr":"172.21.15.107:6831/2659550426","heartbeat_front_addr":"172.21.15.107:6829/2659550426","state":["exists","up"]},{"osd":11,"uuid":"94a1751d-b00c-4e23-815d-3a8bc3dea146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6825","nonce":1633548683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6827","nonce":1633548683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6831","nonce":1633548683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6829","nonce":1633548683}]},"public_addr":"172.21.15.111:6825/1633548683","cluster_addr":"172.21.15.111:6827/1633548683","heartbeat_back_addr":"172.21.15.111:6831/1633548683","heartbeat_front_addr":"172.21.15.111:6829/1633548683","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:36.359852+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:58.014202+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:19.753672+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:43.404817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:06.001677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:29.361587+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:52.554847+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:15.957553+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:39.581093+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:04.911389+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:28.971280+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:53.473561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/2098378667":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/3536229016":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6801/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/2684687547":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6800/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/368410612":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/12158160":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6800/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6801/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/3754423379":"2024-09-14T16:21:40.126459+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:28:04.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.841+0000 7fdb3a2fe640 1 -- 172.21.15.16:0/4090045914 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdb1c0a1110 msgr2=0x7fdb1c0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:04.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.841+0000 7fdb3a2fe640 1 --2- 172.21.15.16:0/4090045914 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdb1c0a1110 0x7fdb1c0a35d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdb34196120 tx=0x7fdb24005f90 comp rx=0 tx=0).stop 2024-09-13T16:28:04.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.841+0000 7fdb3a2fe640 1 -- 172.21.15.16:0/4090045914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb340ffe90 msgr2=0x7fdb34195190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:04.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.841+0000 7fdb3a2fe640 1 --2- 172.21.15.16:0/4090045914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb340ffe90 0x7fdb34195190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdb28006010 tx=0x7fdb280046c0 comp rx=0 tx=0).stop 2024-09-13T16:28:04.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.841+0000 7fdb3a2fe640 1 -- 172.21.15.16:0/4090045914 shutdown_connections 2024-09-13T16:28:04.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.841+0000 7fdb3a2fe640 1 --2- 172.21.15.16:0/4090045914 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdb1c0a1110 0x7fdb1c0a35d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:04.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.841+0000 7fdb3a2fe640 1 --2- 172.21.15.16:0/4090045914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb34100850 0x7fdb3419c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:04.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.842+0000 7fdb3a2fe640 1 --2- 172.21.15.16:0/4090045914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb340ffe90 0x7fdb34195190 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:04.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.842+0000 7fdb3a2fe640 1 --2- 172.21.15.16:0/4090045914 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb340feca0 0x7fdb34194c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:04.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.842+0000 7fdb3a2fe640 1 -- 172.21.15.16:0/4090045914 >> 172.21.15.16:0/4090045914 conn(0x7fdb340fa440 msgr2=0x7fdb341033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:04.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.842+0000 7fdb3a2fe640 1 -- 172.21.15.16:0/4090045914 shutdown_connections 2024-09-13T16:28:04.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.842+0000 7fdb3a2fe640 1 -- 172.21.15.16:0/4090045914 wait complete. 2024-09-13T16:28:04.852 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:04 smithi016 ceph-mon[31187]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi111=c;smithi111=f;smithi111=i;smithi111=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:28:04.852 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:04 smithi016 ceph-mon[31187]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi111=c;smithi111=f;smithi111=i;smithi111=l;count:12 2024-09-13T16:28:04.852 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:04.852 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:28:04.853 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:04.853 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:28:04.853 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:04.853 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:04.853 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:04.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:04.855 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:24:22.981019+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T16:28:04.856 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:28:04.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.960+0000 7f5736308640 1 -- 172.21.15.16:0/2064551907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57300ff1c0 msgr2=0x7f57300ff5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:04.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.960+0000 7f5736308640 1 --2- 172.21.15.16:0/2064551907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57300ff1c0 0x7f57300ff5c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5720009900 tx=0x7f572002a480 comp rx=0 tx=0).stop 2024-09-13T16:28:04.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.960+0000 7f5736308640 1 -- 172.21.15.16:0/2064551907 shutdown_connections 2024-09-13T16:28:04.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.960+0000 7f5736308640 1 --2- 172.21.15.16:0/2064551907 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5730100890 0x7f5730107120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:04.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.960+0000 7f5736308640 1 --2- 172.21.15.16:0/2064551907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57300ffed0 0x7f5730100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:04.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.960+0000 7f5736308640 1 --2- 172.21.15.16:0/2064551907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57300ff1c0 0x7f57300ff5c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:04.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.960+0000 7f5736308640 1 -- 172.21.15.16:0/2064551907 >> 172.21.15.16:0/2064551907 conn(0x7f57300f9b80 msgr2=0x7f57300fbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:04.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.960+0000 7f5736308640 1 -- 172.21.15.16:0/2064551907 shutdown_connections 2024-09-13T16:28:04.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.960+0000 7f5736308640 1 -- 172.21.15.16:0/2064551907 wait complete. 2024-09-13T16:28:04.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.960+0000 7f5736308640 1 Processor -- start 2024-09-13T16:28:04.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.960+0000 7f5736308640 1 -- start start 2024-09-13T16:28:04.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.961+0000 7f5736308640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57300ff1c0 0x7f5730194b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:04.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.961+0000 7f5736308640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57300ffed0 0x7f5730195050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:04.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.961+0000 7f5736308640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5730100890 0x7f573019c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:04.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.961+0000 7f5736308640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573019c600 con 0x7f5730100890 2024-09-13T16:28:04.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.961+0000 7f5736308640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573019c770 con 0x7f57300ff1c0 2024-09-13T16:28:04.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.961+0000 7f5736308640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573019c930 con 0x7f57300ffed0 2024-09-13T16:28:04.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.961+0000 7f57348a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5730100890 0x7f573019c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:04.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.961+0000 7f57348a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5730100890 0x7f573019c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59334/0 (socket says 172.21.15.16:59334) 2024-09-13T16:28:04.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.961+0000 7f57348a5640 1 -- 172.21.15.16:0/2933236391 learned_addr learned my addr 172.21.15.16:0/2933236391 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:04.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.961+0000 7f572f7fe640 1 --2- 172.21.15.16:0/2933236391 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57300ffed0 0x7f5730195050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:04.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.961+0000 7f572ffff640 1 --2- 172.21.15.16:0/2933236391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57300ff1c0 0x7f5730194b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:04.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.961+0000 7f57348a5640 1 -- 172.21.15.16:0/2933236391 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57300ffed0 msgr2=0x7f5730195050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:04.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.961+0000 7f57348a5640 1 --2- 172.21.15.16:0/2933236391 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57300ffed0 0x7f5730195050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:04.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.961+0000 7f57348a5640 1 -- 172.21.15.16:0/2933236391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57300ff1c0 msgr2=0x7f5730194b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:04.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.961+0000 7f57348a5640 1 --2- 172.21.15.16:0/2933236391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57300ff1c0 0x7f5730194b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:04.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.961+0000 7f57348a5640 1 -- 172.21.15.16:0/2933236391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5720009560 con 0x7f5730100890 2024-09-13T16:28:04.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.961+0000 7f572ffff640 1 --2- 172.21.15.16:0/2933236391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57300ff1c0 0x7f5730194b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:28:04.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.961+0000 7f57348a5640 1 --2- 172.21.15.16:0/2933236391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5730100890 0x7f573019c0c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f572800eca0 tx=0x7f572800b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:04.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.962+0000 7f572d7fa640 1 -- 172.21.15.16:0/2933236391 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5728013070 con 0x7f5730100890 2024-09-13T16:28:04.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.962+0000 7f572d7fa640 1 -- 172.21.15.16:0/2933236391 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5728005020 con 0x7f5730100890 2024-09-13T16:28:04.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.962+0000 7f5736308640 1 -- 172.21.15.16:0/2933236391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57300fdd70 con 0x7f5730100890 2024-09-13T16:28:04.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.962+0000 7f572d7fa640 1 -- 172.21.15.16:0/2933236391 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57280099b0 con 0x7f5730100890 2024-09-13T16:28:04.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.962+0000 7f5736308640 1 -- 172.21.15.16:0/2933236391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57300fe230 con 0x7f5730100890 2024-09-13T16:28:04.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.963+0000 7f5736308640 1 -- 172.21.15.16:0/2933236391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57300fec10 con 0x7f5730100890 2024-09-13T16:28:04.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.964+0000 7f572d7fa640 1 -- 172.21.15.16:0/2933236391 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f5728005190 con 0x7f5730100890 2024-09-13T16:28:04.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.965+0000 7f572d7fa640 1 --2- 172.21.15.16:0/2933236391 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f57180a1090 0x7f57180a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:04.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.965+0000 7f572ffff640 1 --2- 172.21.15.16:0/2933236391 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f57180a1090 0x7f57180a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:04.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.965+0000 7f572d7fa640 1 -- 172.21.15.16:0/2933236391 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f57280cdde0 con 0x7f5730100890 2024-09-13T16:28:04.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.965+0000 7f572ffff640 1 --2- 172.21.15.16:0/2933236391 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f57180a1090 0x7f57180a3550 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5720005ab0 tx=0x7f5720005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:04.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:04.969+0000 7f572d7fa640 1 -- 172.21.15.16:0/2933236391 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f5728097a10 con 0x7f5730100890 2024-09-13T16:28:05.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.140+0000 7f5736308640 1 -- 172.21.15.16:0/2933236391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5730105c20 con 0x7f5730100890 2024-09-13T16:28:05.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.140+0000 7f572d7fa640 1 -- 172.21.15.16:0/2933236391 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f572809b030 con 0x7f5730100890 2024-09-13T16:28:05.141 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-13T16:28:05.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.142+0000 7f5736308640 1 -- 172.21.15.16:0/2933236391 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f57180a1090 msgr2=0x7f57180a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.142+0000 7f5736308640 1 --2- 172.21.15.16:0/2933236391 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f57180a1090 0x7f57180a3550 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5720005ab0 tx=0x7f5720005a20 comp rx=0 tx=0).stop 2024-09-13T16:28:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.142+0000 7f5736308640 1 -- 172.21.15.16:0/2933236391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5730100890 msgr2=0x7f573019c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.142+0000 7f5736308640 1 --2- 172.21.15.16:0/2933236391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5730100890 0x7f573019c0c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f572800eca0 tx=0x7f572800b690 comp rx=0 tx=0).stop 2024-09-13T16:28:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.142+0000 7f5736308640 1 -- 172.21.15.16:0/2933236391 shutdown_connections 2024-09-13T16:28:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.142+0000 7f5736308640 1 --2- 172.21.15.16:0/2933236391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5730100890 0x7f573019c0c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.142+0000 7f5736308640 1 --2- 172.21.15.16:0/2933236391 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f57300ffed0 0x7f5730195050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.142+0000 7f5736308640 1 --2- 172.21.15.16:0/2933236391 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f57180a1090 0x7f57180a3550 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.142+0000 7f5736308640 1 --2- 172.21.15.16:0/2933236391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57300ff1c0 0x7f5730194b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.142+0000 7f5736308640 1 -- 172.21.15.16:0/2933236391 >> 172.21.15.16:0/2933236391 conn(0x7f57300f9b80 msgr2=0x7f5730103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.142+0000 7f5736308640 1 -- 172.21.15.16:0/2933236391 shutdown_connections 2024-09-13T16:28:05.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.142+0000 7f5736308640 1 -- 172.21.15.16:0/2933236391 wait complete. 2024-09-13T16:28:05.154 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:28:05.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.255+0000 7f8e95ee9640 1 -- 172.21.15.16:0/971527818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e900feca0 msgr2=0x7f8e900ff0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:05.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.255+0000 7f8e95ee9640 1 --2- 172.21.15.16:0/971527818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e900feca0 0x7f8e900ff0a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009900 tx=0x7f8e8002a410 comp rx=0 tx=0).stop 2024-09-13T16:28:05.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.255+0000 7f8e95ee9640 1 -- 172.21.15.16:0/971527818 shutdown_connections 2024-09-13T16:28:05.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.255+0000 7f8e95ee9640 1 --2- 172.21.15.16:0/971527818 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e90100850 0x7f8e901070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:05.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.255+0000 7f8e95ee9640 1 --2- 172.21.15.16:0/971527818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e900ffe90 0x7f8e90100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:05.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.255+0000 7f8e95ee9640 1 --2- 172.21.15.16:0/971527818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e900feca0 0x7f8e900ff0a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:05.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.255+0000 7f8e95ee9640 1 -- 172.21.15.16:0/971527818 >> 172.21.15.16:0/971527818 conn(0x7f8e900fa420 msgr2=0x7f8e900fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:05.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.255+0000 7f8e95ee9640 1 -- 172.21.15.16:0/971527818 shutdown_connections 2024-09-13T16:28:05.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.255+0000 7f8e95ee9640 1 -- 172.21.15.16:0/971527818 wait complete. 2024-09-13T16:28:05.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.256+0000 7f8e95ee9640 1 Processor -- start 2024-09-13T16:28:05.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.256+0000 7f8e95ee9640 1 -- start start 2024-09-13T16:28:05.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.256+0000 7f8e95ee9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e900feca0 0x7f8e90069180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:05.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.256+0000 7f8e95ee9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e900ffe90 0x7f8e9019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:05.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.256+0000 7f8e95ee9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e90100850 0x7f8e9019d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:05.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.256+0000 7f8e95ee9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9019d7f0 con 0x7f8e900ffe90 2024-09-13T16:28:05.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.256+0000 7f8e95ee9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9019d960 con 0x7f8e900feca0 2024-09-13T16:28:05.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.256+0000 7f8e95ee9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9019db20 con 0x7f8e90100850 2024-09-13T16:28:05.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.256+0000 7f8e8effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e900ffe90 0x7f8e9019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:05.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.256+0000 7f8e8effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e900ffe90 0x7f8e9019a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59344/0 (socket says 172.21.15.16:59344) 2024-09-13T16:28:05.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.256+0000 7f8e8effd640 1 -- 172.21.15.16:0/2546319208 learned_addr learned my addr 172.21.15.16:0/2546319208 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:05.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.256+0000 7f8e8f7fe640 1 --2- 172.21.15.16:0/2546319208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e900feca0 0x7f8e90069180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:05.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.256+0000 7f8e8effd640 1 -- 172.21.15.16:0/2546319208 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e90100850 msgr2=0x7f8e9019d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:05.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.256+0000 7f8e8effd640 1 --2- 172.21.15.16:0/2546319208 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e90100850 0x7f8e9019d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:05.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.256+0000 7f8e8effd640 1 -- 172.21.15.16:0/2546319208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e900feca0 msgr2=0x7f8e90069180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:05.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.256+0000 7f8e8effd640 1 --2- 172.21.15.16:0/2546319208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e900feca0 0x7f8e90069180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:05.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.256+0000 7f8e8effd640 1 -- 172.21.15.16:0/2546319208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e80009560 con 0x7f8e900ffe90 2024-09-13T16:28:05.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.256+0000 7f8e8f7fe640 1 --2- 172.21.15.16:0/2546319208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e900feca0 0x7f8e90069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:28:05.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.257+0000 7f8e8effd640 1 --2- 172.21.15.16:0/2546319208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e900ffe90 0x7f8e9019a9f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400c920 tx=0x7f8e8400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:05.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.257+0000 7f8e8cff9640 1 -- 172.21.15.16:0/2546319208 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e84007c70 con 0x7f8e900ffe90 2024-09-13T16:28:05.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.257+0000 7f8e8cff9640 1 -- 172.21.15.16:0/2546319208 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8e84007dd0 con 0x7f8e900ffe90 2024-09-13T16:28:05.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.257+0000 7f8e95ee9640 1 -- 172.21.15.16:0/2546319208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e9019dda0 con 0x7f8e900ffe90 2024-09-13T16:28:05.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.257+0000 7f8e8cff9640 1 -- 172.21.15.16:0/2546319208 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e84018730 con 0x7f8e900ffe90 2024-09-13T16:28:05.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.257+0000 7f8e95ee9640 1 -- 172.21.15.16:0/2546319208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e9019e270 con 0x7f8e900ffe90 2024-09-13T16:28:05.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.258+0000 7f8e95ee9640 1 -- 172.21.15.16:0/2546319208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e90106220 con 0x7f8e900ffe90 2024-09-13T16:28:05.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.259+0000 7f8e8cff9640 1 -- 172.21.15.16:0/2546319208 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8e8400f450 con 0x7f8e900ffe90 2024-09-13T16:28:05.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.260+0000 7f8e8cff9640 1 --2- 172.21.15.16:0/2546319208 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8e780a1110 0x7f8e780a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:05.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.260+0000 7f8e8f7fe640 1 --2- 172.21.15.16:0/2546319208 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8e780a1110 0x7f8e780a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:05.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.260+0000 7f8e8cff9640 1 -- 172.21.15.16:0/2546319208 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8e840cc5d0 con 0x7f8e900ffe90 2024-09-13T16:28:05.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.260+0000 7f8e8f7fe640 1 --2- 172.21.15.16:0/2546319208 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8e780a1110 0x7f8e780a35d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8e80006010 tx=0x7f8e80031470 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:05.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.264+0000 7f8e8cff9640 1 -- 172.21.15.16:0/2546319208 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f8e84096200 con 0x7f8e900ffe90 2024-09-13T16:28:05.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.471+0000 7f8e95ee9640 1 -- 172.21.15.16:0/2546319208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e9002c480 con 0x7f8e900ffe90 2024-09-13T16:28:05.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.471+0000 7f8e8cff9640 1 -- 172.21.15.16:0/2546319208 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7f8e84099820 con 0x7f8e900ffe90 2024-09-13T16:28:05.472 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:28:05.472 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-13T16:28:05.473 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 1 2024-09-13T16:28:05.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.473+0000 7f8e95ee9640 1 -- 172.21.15.16:0/2546319208 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8e780a1110 msgr2=0x7f8e780a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:05.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.473+0000 7f8e95ee9640 1 --2- 172.21.15.16:0/2546319208 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8e780a1110 0x7f8e780a35d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8e80006010 tx=0x7f8e80031470 comp rx=0 tx=0).stop 2024-09-13T16:28:05.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.473+0000 7f8e95ee9640 1 -- 172.21.15.16:0/2546319208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e900ffe90 msgr2=0x7f8e9019a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:05.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.473+0000 7f8e95ee9640 1 --2- 172.21.15.16:0/2546319208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e900ffe90 0x7f8e9019a9f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400c920 tx=0x7f8e8400cde0 comp rx=0 tx=0).stop 2024-09-13T16:28:05.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.473+0000 7f8e95ee9640 1 -- 172.21.15.16:0/2546319208 shutdown_connections 2024-09-13T16:28:05.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.473+0000 7f8e95ee9640 1 --2- 172.21.15.16:0/2546319208 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e90100850 0x7f8e9019d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:05.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.473+0000 7f8e95ee9640 1 --2- 172.21.15.16:0/2546319208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e900ffe90 0x7f8e9019a9f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:05.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.473+0000 7f8e95ee9640 1 --2- 172.21.15.16:0/2546319208 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8e780a1110 0x7f8e780a35d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:05.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.473+0000 7f8e95ee9640 1 --2- 172.21.15.16:0/2546319208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e900feca0 0x7f8e90069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:05.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.473+0000 7f8e95ee9640 1 -- 172.21.15.16:0/2546319208 >> 172.21.15.16:0/2546319208 conn(0x7f8e900fa420 msgr2=0x7f8e901033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:05.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.473+0000 7f8e95ee9640 1 -- 172.21.15.16:0/2546319208 shutdown_connections 2024-09-13T16:28:05.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:05.473+0000 7f8e95ee9640 1 -- 172.21.15.16:0/2546319208 wait complete. 2024-09-13T16:28:05.676 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:05 smithi111 ceph-mon[34600]: Deploying daemon mds.l on smithi111 2024-09-13T16:28:05.676 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:05 smithi111 ceph-mon[34600]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:05.676 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:05 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/4090045914' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:28:05.676 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:05 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/2933236391' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:28:05.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:05 smithi107 ceph-mon[34139]: Deploying daemon mds.l on smithi111 2024-09-13T16:28:05.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:05 smithi107 ceph-mon[34139]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:05.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:05 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/4090045914' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:28:05.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:05 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/2933236391' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:28:05.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:05 smithi016 ceph-mon[31187]: Deploying daemon mds.l on smithi111 2024-09-13T16:28:05.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:05 smithi016 ceph-mon[31187]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:05.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:05 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/4090045914' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:28:05.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:05 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2933236391' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:28:05.937 INFO:journalctl@ceph.mds.l.smithi111.stdout:Sep 13 16:28:05 smithi111 systemd[1]: Starting Ceph mds.l for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:28:06.364 INFO:journalctl@ceph.mds.l.smithi111.stdout:Sep 13 16:28:06 smithi111 podman[64845]: 2024-09-13 16:28:05.979456661 +0000 UTC m=+0.015709163 image pull 152d8bc6b19e354515e8ba109d4619445de0e09b4d4a747c2f16eb5c0edb670c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a 2024-09-13T16:28:06.364 INFO:journalctl@ceph.mds.l.smithi111.stdout:Sep 13 16:28:06 smithi111 podman[64845]: 2024-09-13 16:28:06.102373971 +0000 UTC m=+0.138626460 container create aad66e58c1dccfadeecef3774715ac4de96bcaf827395fb6edb6e9618971a840 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-l, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-13T16:28:06.364 INFO:journalctl@ceph.mds.l.smithi111.stdout:Sep 13 16:28:06 smithi111 podman[64845]: 2024-09-13 16:28:06.363498168 +0000 UTC m=+0.399750666 container init aad66e58c1dccfadeecef3774715ac4de96bcaf827395fb6edb6e9618971a840 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-l, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T16:28:06.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:06 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/2546319208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:06.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:06 smithi111 ceph-mon[34600]: mds.? [v2:172.21.15.111:6832/1984275936,v1:172.21.15.111:6833/1984275936] up:boot 2024-09-13T16:28:06.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:06 smithi111 ceph-mon[34600]: fsmap 1 up:standby 2024-09-13T16:28:06.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:06 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-13T16:28:06.687 INFO:journalctl@ceph.mds.l.smithi111.stdout:Sep 13 16:28:06 smithi111 podman[64845]: 2024-09-13 16:28:06.366319569 +0000 UTC m=+0.402572064 container start aad66e58c1dccfadeecef3774715ac4de96bcaf827395fb6edb6e9618971a840 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-l, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-13T16:28:06.687 INFO:journalctl@ceph.mds.l.smithi111.stdout:Sep 13 16:28:06 smithi111 ceph-mds[64864]: -- 172.21.15.111:0/1755517630 <== mon.1 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5622ce6f8d00 con 0x5622ce6cd400 2024-09-13T16:28:06.688 INFO:journalctl@ceph.mds.l.smithi111.stdout:Sep 13 16:28:06 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-l[64860]: starting mds.l at 2024-09-13T16:28:06.688 INFO:journalctl@ceph.mds.l.smithi111.stdout:Sep 13 16:28:06 smithi111 bash[64845]: aad66e58c1dccfadeecef3774715ac4de96bcaf827395fb6edb6e9618971a840 2024-09-13T16:28:06.688 INFO:journalctl@ceph.mds.l.smithi111.stdout:Sep 13 16:28:06 smithi111 systemd[1]: Started Ceph mds.l for 0d241f6a-71ec-11ef-bceb-c7b262605968. 2024-09-13T16:28:06.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:06 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/2546319208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:06.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:06 smithi107 ceph-mon[34139]: mds.? [v2:172.21.15.111:6832/1984275936,v1:172.21.15.111:6833/1984275936] up:boot 2024-09-13T16:28:06.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:06 smithi107 ceph-mon[34139]: fsmap 1 up:standby 2024-09-13T16:28:06.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:06 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-13T16:28:06.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:06 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2546319208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:06.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:06 smithi016 ceph-mon[31187]: mds.? [v2:172.21.15.111:6832/1984275936,v1:172.21.15.111:6833/1984275936] up:boot 2024-09-13T16:28:06.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:06 smithi016 ceph-mon[31187]: fsmap 1 up:standby 2024-09-13T16:28:06.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:06 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-13T16:28:07.486 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:28:07.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.602+0000 7fe9260d7640 1 -- 172.21.15.16:0/1602833600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9201008e0 msgr2=0x7fe920102cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:07.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.602+0000 7fe9260d7640 1 --2- 172.21.15.16:0/1602833600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9201008e0 0x7fe920102cd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe910009900 tx=0x7fe91002a410 comp rx=0 tx=0).stop 2024-09-13T16:28:07.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.602+0000 7fe9260d7640 1 -- 172.21.15.16:0/1602833600 shutdown_connections 2024-09-13T16:28:07.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.602+0000 7fe9260d7640 1 --2- 172.21.15.16:0/1602833600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe920105b40 0x7fe920107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:07.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.602+0000 7fe9260d7640 1 --2- 172.21.15.16:0/1602833600 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe920103210 0x7fe920105600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:07.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.602+0000 7fe9260d7640 1 --2- 172.21.15.16:0/1602833600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9201008e0 0x7fe920102cd0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:07.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.602+0000 7fe9260d7640 1 -- 172.21.15.16:0/1602833600 >> 172.21.15.16:0/1602833600 conn(0x7fe9200fa440 msgr2=0x7fe9200fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:07.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.602+0000 7fe9260d7640 1 -- 172.21.15.16:0/1602833600 shutdown_connections 2024-09-13T16:28:07.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.602+0000 7fe9260d7640 1 -- 172.21.15.16:0/1602833600 wait complete. 2024-09-13T16:28:07.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.603+0000 7fe9260d7640 1 Processor -- start 2024-09-13T16:28:07.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.603+0000 7fe9260d7640 1 -- start start 2024-09-13T16:28:07.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.603+0000 7fe9260d7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9201008e0 0x7fe920195c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:07.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.603+0000 7fe9260d7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe920103210 0x7fe9201961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:07.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.603+0000 7fe9260d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe920105b40 0x7fe92019d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:07.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.603+0000 7fe9260d7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92019d750 con 0x7fe920105b40 2024-09-13T16:28:07.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.603+0000 7fe9260d7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92019d8c0 con 0x7fe9201008e0 2024-09-13T16:28:07.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.603+0000 7fe9260d7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92019da80 con 0x7fe920103210 2024-09-13T16:28:07.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.603+0000 7fe91ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe920105b40 0x7fe92019d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:07.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.603+0000 7fe91ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe920105b40 0x7fe92019d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59358/0 (socket says 172.21.15.16:59358) 2024-09-13T16:28:07.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.603+0000 7fe91ffff640 1 -- 172.21.15.16:0/519292397 learned_addr learned my addr 172.21.15.16:0/519292397 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:07.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.603+0000 7fe91f7fe640 1 --2- 172.21.15.16:0/519292397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9201008e0 0x7fe920195c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:07.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.603+0000 7fe91effd640 1 --2- 172.21.15.16:0/519292397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe920103210 0x7fe9201961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:07.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.603+0000 7fe91ffff640 1 -- 172.21.15.16:0/519292397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe920103210 msgr2=0x7fe9201961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:07.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.603+0000 7fe91ffff640 1 --2- 172.21.15.16:0/519292397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe920103210 0x7fe9201961b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:07.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.603+0000 7fe91ffff640 1 -- 172.21.15.16:0/519292397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9201008e0 msgr2=0x7fe920195c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:07.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.603+0000 7fe91ffff640 1 --2- 172.21.15.16:0/519292397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9201008e0 0x7fe920195c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:07.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.603+0000 7fe91ffff640 1 -- 172.21.15.16:0/519292397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe910009560 con 0x7fe920105b40 2024-09-13T16:28:07.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.604+0000 7fe91ffff640 1 --2- 172.21.15.16:0/519292397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe920105b40 0x7fe92019d1e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe91800eaa0 tx=0x7fe91800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:07.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.604+0000 7fe91cff9640 1 -- 172.21.15.16:0/519292397 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91800cdb0 con 0x7fe920105b40 2024-09-13T16:28:07.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.604+0000 7fe91cff9640 1 -- 172.21.15.16:0/519292397 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe918005020 con 0x7fe920105b40 2024-09-13T16:28:07.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.604+0000 7fe9260d7640 1 -- 172.21.15.16:0/519292397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe92019dd60 con 0x7fe920105b40 2024-09-13T16:28:07.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.604+0000 7fe91cff9640 1 -- 172.21.15.16:0/519292397 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9180116d0 con 0x7fe920105b40 2024-09-13T16:28:07.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.604+0000 7fe9260d7640 1 -- 172.21.15.16:0/519292397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe92019e1c0 con 0x7fe920105b40 2024-09-13T16:28:07.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.605+0000 7fe9260d7640 1 -- 172.21.15.16:0/519292397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9200fbf30 con 0x7fe920105b40 2024-09-13T16:28:07.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.606+0000 7fe91cff9640 1 -- 172.21.15.16:0/519292397 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe918005190 con 0x7fe920105b40 2024-09-13T16:28:07.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.607+0000 7fe91cff9640 1 --2- 172.21.15.16:0/519292397 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe9080a1090 0x7fe9080a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:07.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.607+0000 7fe91f7fe640 1 --2- 172.21.15.16:0/519292397 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe9080a1090 0x7fe9080a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:07.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.607+0000 7fe91cff9640 1 -- 172.21.15.16:0/519292397 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe9180cc7b0 con 0x7fe920105b40 2024-09-13T16:28:07.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.608+0000 7fe91f7fe640 1 --2- 172.21.15.16:0/519292397 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe9080a1090 0x7fe9080a3550 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe920197140 tx=0x7fe910005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:07.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.611+0000 7fe91cff9640 1 -- 172.21.15.16:0/519292397 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fe9180962b0 con 0x7fe920105b40 2024-09-13T16:28:07.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.823+0000 7fe9260d7640 1 -- 172.21.15.16:0/519292397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe92002c480 con 0x7fe920105b40 2024-09-13T16:28:07.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.823+0000 7fe91cff9640 1 -- 172.21.15.16:0/519292397 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1260 (secure 0 0 0) 0x7fe9180981e0 con 0x7fe920105b40 2024-09-13T16:28:07.825 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:28:07.825 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1984275936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1984275936},{"type":"v1","addr":"172.21.15.111:6833","nonce":1984275936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-09-13T16:28:07.825 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 2 2024-09-13T16:28:07.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.826+0000 7fe9260d7640 1 -- 172.21.15.16:0/519292397 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe9080a1090 msgr2=0x7fe9080a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:07.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.826+0000 7fe9260d7640 1 --2- 172.21.15.16:0/519292397 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe9080a1090 0x7fe9080a3550 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe920197140 tx=0x7fe910005a20 comp rx=0 tx=0).stop 2024-09-13T16:28:07.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.826+0000 7fe9260d7640 1 -- 172.21.15.16:0/519292397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe920105b40 msgr2=0x7fe92019d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:07.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.826+0000 7fe9260d7640 1 --2- 172.21.15.16:0/519292397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe920105b40 0x7fe92019d1e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe91800eaa0 tx=0x7fe91800ef60 comp rx=0 tx=0).stop 2024-09-13T16:28:07.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.826+0000 7fe9260d7640 1 -- 172.21.15.16:0/519292397 shutdown_connections 2024-09-13T16:28:07.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.826+0000 7fe9260d7640 1 --2- 172.21.15.16:0/519292397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe920105b40 0x7fe92019d1e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:07.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.826+0000 7fe9260d7640 1 --2- 172.21.15.16:0/519292397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe920103210 0x7fe9201961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:07.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.826+0000 7fe9260d7640 1 --2- 172.21.15.16:0/519292397 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe9080a1090 0x7fe9080a3550 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:07.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.826+0000 7fe9260d7640 1 --2- 172.21.15.16:0/519292397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9201008e0 0x7fe920195c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:07.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.826+0000 7fe9260d7640 1 -- 172.21.15.16:0/519292397 >> 172.21.15.16:0/519292397 conn(0x7fe9200fa440 msgr2=0x7fe920101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:07.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.826+0000 7fe9260d7640 1 -- 172.21.15.16:0/519292397 shutdown_connections 2024-09-13T16:28:07.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:07.826+0000 7fe9260d7640 1 -- 172.21.15.16:0/519292397 wait complete. 2024-09-13T16:28:07.836 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:07 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:07.837 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:07 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:07.837 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:07 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:07.837 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:07 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:07.837 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:07 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:07.837 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:07 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:07.837 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:07 smithi016 ceph-mon[31187]: Deploying daemon mds.k on smithi107 2024-09-13T16:28:07.837 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:07 smithi016 ceph-mon[31187]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:07.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:07 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:07.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:07 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:07.851 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:07 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:07.851 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:07 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:07.851 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:07 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:07.852 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:07 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:07.852 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:07 smithi107 ceph-mon[34139]: Deploying daemon mds.k on smithi107 2024-09-13T16:28:07.852 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:07 smithi107 ceph-mon[34139]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:07.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:07 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:07.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:07 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:07.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:07 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:07.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:07 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:07.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:07 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:07.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:07 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:07.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:07 smithi111 ceph-mon[34600]: Deploying daemon mds.k on smithi107 2024-09-13T16:28:07.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:07 smithi111 ceph-mon[34600]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:08.531 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 13 16:28:08 smithi107 systemd[1]: Starting Ceph mds.k for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:28:08.823 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:08 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/519292397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:08.823 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 13 16:28:08 smithi107 podman[62544]: 2024-09-13 16:28:08.571642532 +0000 UTC m=+0.015607734 image pull 152d8bc6b19e354515e8ba109d4619445de0e09b4d4a747c2f16eb5c0edb670c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a 2024-09-13T16:28:08.823 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 13 16:28:08 smithi107 podman[62544]: 2024-09-13 16:28:08.693217399 +0000 UTC m=+0.137182595 container create 9fb9b76f26adca92252d49b8495bc90340a302ae885df9c66001958c1afb0a07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-k, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-13T16:28:08.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:08 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/519292397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:08.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:08 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/519292397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:09.099 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 13 16:28:08 smithi107 podman[62544]: 2024-09-13 16:28:08.946146908 +0000 UTC m=+0.390112109 container init 9fb9b76f26adca92252d49b8495bc90340a302ae885df9c66001958c1afb0a07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-09-13T16:28:09.099 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 13 16:28:08 smithi107 podman[62544]: 2024-09-13 16:28:08.949261959 +0000 UTC m=+0.393227158 container start 9fb9b76f26adca92252d49b8495bc90340a302ae885df9c66001958c1afb0a07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-k, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-13T16:28:09.099 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 13 16:28:08 smithi107 ceph-mds[62563]: -- 172.21.15.107:0/2712453193 <== mon.2 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556cf3c08d00 con 0x556cf3bdd400 2024-09-13T16:28:09.100 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 13 16:28:08 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-k[62559]: starting mds.k at 2024-09-13T16:28:09.100 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 13 16:28:09 smithi107 bash[62544]: 9fb9b76f26adca92252d49b8495bc90340a302ae885df9c66001958c1afb0a07 2024-09-13T16:28:09.100 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 13 16:28:09 smithi107 systemd[1]: Started Ceph mds.k for 0d241f6a-71ec-11ef-bceb-c7b262605968. 2024-09-13T16:28:09.736 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:09 smithi016 ceph-mon[31187]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:09.736 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:09 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:09.736 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:09 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:09.736 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:09 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:09.737 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:09 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:09.737 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:09 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:09.737 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:09 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:09.839 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:28:09.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:09 smithi107 ceph-mon[34139]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:09.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:09 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:09.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:09 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:09.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:09 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:09.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:09 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:09.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:09 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:09.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:09 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:09.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:09 smithi111 ceph-mon[34600]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:09.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:09 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:09.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:09 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:09.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:09 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:09.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:09 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:09.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:09 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:09.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:09 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:09.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.943+0000 7f00f01ed640 1 -- 172.21.15.16:0/1442333985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e8100850 msgr2=0x7f00e8102c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:09.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.943+0000 7f00f01ed640 1 --2- 172.21.15.16:0/1442333985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e8100850 0x7f00e8102c40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f00e4009900 tx=0x7f00e402a410 comp rx=0 tx=0).stop 2024-09-13T16:28:09.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.943+0000 7f00f01ed640 1 -- 172.21.15.16:0/1442333985 shutdown_connections 2024-09-13T16:28:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.943+0000 7f00f01ed640 1 --2- 172.21.15.16:0/1442333985 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00e8105ab0 0x7f00e8107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.943+0000 7f00f01ed640 1 --2- 172.21.15.16:0/1442333985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00e8103180 0x7f00e8105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.943+0000 7f00f01ed640 1 --2- 172.21.15.16:0/1442333985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e8100850 0x7f00e8102c40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.943+0000 7f00f01ed640 1 -- 172.21.15.16:0/1442333985 >> 172.21.15.16:0/1442333985 conn(0x7f00e80fa430 msgr2=0x7f00e80fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.943+0000 7f00f01ed640 1 -- 172.21.15.16:0/1442333985 shutdown_connections 2024-09-13T16:28:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.943+0000 7f00f01ed640 1 -- 172.21.15.16:0/1442333985 wait complete. 2024-09-13T16:28:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.944+0000 7f00f01ed640 1 Processor -- start 2024-09-13T16:28:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.944+0000 7f00f01ed640 1 -- start start 2024-09-13T16:28:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.944+0000 7f00f01ed640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00e8100850 0x7f00e8195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.944+0000 7f00f01ed640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00e8103180 0x7f00e81962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.944+0000 7f00f01ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e8105ab0 0x7f00e819d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.944+0000 7f00f01ed640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e819d890 con 0x7f00e8105ab0 2024-09-13T16:28:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.944+0000 7f00f01ed640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e819da00 con 0x7f00e8103180 2024-09-13T16:28:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.944+0000 7f00f01ed640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e819dbc0 con 0x7f00e8100850 2024-09-13T16:28:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.944+0000 7f00ee78a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e8105ab0 0x7f00e819d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.944+0000 7f00edf89640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00e8100850 0x7f00e8195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.944+0000 7f00ee78a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e8105ab0 0x7f00e819d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59384/0 (socket says 172.21.15.16:59384) 2024-09-13T16:28:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.944+0000 7f00ee78a640 1 -- 172.21.15.16:0/294447238 learned_addr learned my addr 172.21.15.16:0/294447238 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.944+0000 7f00ed788640 1 --2- 172.21.15.16:0/294447238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00e8103180 0x7f00e81962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.944+0000 7f00ee78a640 1 -- 172.21.15.16:0/294447238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00e8100850 msgr2=0x7f00e8195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.944+0000 7f00ee78a640 1 --2- 172.21.15.16:0/294447238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00e8100850 0x7f00e8195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.944+0000 7f00ee78a640 1 -- 172.21.15.16:0/294447238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00e8103180 msgr2=0x7f00e81962f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.944+0000 7f00ee78a640 1 --2- 172.21.15.16:0/294447238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00e8103180 0x7f00e81962f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.944+0000 7f00ee78a640 1 -- 172.21.15.16:0/294447238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e4009560 con 0x7f00e8105ab0 2024-09-13T16:28:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.944+0000 7f00ee78a640 1 --2- 172.21.15.16:0/294447238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e8105ab0 0x7f00e819d320 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f00e000dac0 tx=0x7f00e000df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.945+0000 7f00deffd640 1 -- 172.21.15.16:0/294447238 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e000be30 con 0x7f00e8105ab0 2024-09-13T16:28:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.945+0000 7f00deffd640 1 -- 172.21.15.16:0/294447238 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f00e0003e00 con 0x7f00e8105ab0 2024-09-13T16:28:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.945+0000 7f00deffd640 1 -- 172.21.15.16:0/294447238 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e00106c0 con 0x7f00e8105ab0 2024-09-13T16:28:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.945+0000 7f00f01ed640 1 -- 172.21.15.16:0/294447238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00e819dea0 con 0x7f00e8105ab0 2024-09-13T16:28:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.945+0000 7f00f01ed640 1 -- 172.21.15.16:0/294447238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e819e270 con 0x7f00e8105ab0 2024-09-13T16:28:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.946+0000 7f00f01ed640 1 -- 172.21.15.16:0/294447238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00e8103560 con 0x7f00e8105ab0 2024-09-13T16:28:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.947+0000 7f00deffd640 1 -- 172.21.15.16:0/294447238 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f00e0003970 con 0x7f00e8105ab0 2024-09-13T16:28:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.947+0000 7f00deffd640 1 --2- 172.21.15.16:0/294447238 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f00d40a1110 0x7f00d40a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.948+0000 7f00edf89640 1 --2- 172.21.15.16:0/294447238 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f00d40a1110 0x7f00d40a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.948+0000 7f00deffd640 1 -- 172.21.15.16:0/294447238 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f00e00cbff0 con 0x7f00e8105ab0 2024-09-13T16:28:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.948+0000 7f00edf89640 1 --2- 172.21.15.16:0/294447238 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f00d40a1110 0x7f00d40a35d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f00e8197280 tx=0x7f00e4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:09.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:09.952+0000 7f00deffd640 1 -- 172.21.15.16:0/294447238 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f00e0095c40 con 0x7f00e8105ab0 2024-09-13T16:28:10.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:10.163+0000 7f00f01ed640 1 -- 172.21.15.16:0/294447238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00e80fbf20 con 0x7f00e8105ab0 2024-09-13T16:28:10.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:10.164+0000 7f00deffd640 1 -- 172.21.15.16:0/294447238 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2020 (secure 0 0 0) 0x7f00e0097b70 con 0x7f00e8105ab0 2024-09-13T16:28:10.165 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:28:10.165 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1984275936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1984275936},{"type":"v1","addr":"172.21.15.111:6833","nonce":1984275936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3797712690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3797712690},{"type":"v1","addr":"172.21.15.107:6833","nonce":3797712690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-09-13T16:28:10.166 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 3 2024-09-13T16:28:10.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:10.166+0000 7f00f01ed640 1 -- 172.21.15.16:0/294447238 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f00d40a1110 msgr2=0x7f00d40a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:10.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:10.166+0000 7f00f01ed640 1 --2- 172.21.15.16:0/294447238 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f00d40a1110 0x7f00d40a35d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f00e8197280 tx=0x7f00e4005a20 comp rx=0 tx=0).stop 2024-09-13T16:28:10.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:10.166+0000 7f00f01ed640 1 -- 172.21.15.16:0/294447238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e8105ab0 msgr2=0x7f00e819d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:10.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:10.166+0000 7f00f01ed640 1 --2- 172.21.15.16:0/294447238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e8105ab0 0x7f00e819d320 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f00e000dac0 tx=0x7f00e000df80 comp rx=0 tx=0).stop 2024-09-13T16:28:10.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:10.166+0000 7f00f01ed640 1 -- 172.21.15.16:0/294447238 shutdown_connections 2024-09-13T16:28:10.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:10.166+0000 7f00f01ed640 1 --2- 172.21.15.16:0/294447238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00e8105ab0 0x7f00e819d320 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:10.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:10.166+0000 7f00f01ed640 1 --2- 172.21.15.16:0/294447238 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f00d40a1110 0x7f00d40a35d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:10.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:10.166+0000 7f00f01ed640 1 --2- 172.21.15.16:0/294447238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00e8103180 0x7f00e81962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:10.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:10.166+0000 7f00f01ed640 1 --2- 172.21.15.16:0/294447238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00e8100850 0x7f00e8195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:10.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:10.166+0000 7f00f01ed640 1 -- 172.21.15.16:0/294447238 >> 172.21.15.16:0/294447238 conn(0x7f00e80fa430 msgr2=0x7f00e81012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:10.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:10.166+0000 7f00f01ed640 1 -- 172.21.15.16:0/294447238 shutdown_connections 2024-09-13T16:28:10.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:10.166+0000 7f00f01ed640 1 -- 172.21.15.16:0/294447238 wait complete. 2024-09-13T16:28:10.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:10 smithi107 ceph-mon[34139]: Deploying daemon mds.j on smithi016 2024-09-13T16:28:10.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:10 smithi107 ceph-mon[34139]: mds.? [v2:172.21.15.107:6832/3797712690,v1:172.21.15.107:6833/3797712690] up:boot 2024-09-13T16:28:10.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:10 smithi107 ceph-mon[34139]: fsmap 2 up:standby 2024-09-13T16:28:10.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:10 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-13T16:28:10.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:10 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/294447238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:10.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:10 smithi016 ceph-mon[31187]: Deploying daemon mds.j on smithi016 2024-09-13T16:28:10.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:10 smithi016 ceph-mon[31187]: mds.? [v2:172.21.15.107:6832/3797712690,v1:172.21.15.107:6833/3797712690] up:boot 2024-09-13T16:28:10.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:10 smithi016 ceph-mon[31187]: fsmap 2 up:standby 2024-09-13T16:28:10.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:10 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-13T16:28:10.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:10 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/294447238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:10.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:10 smithi111 ceph-mon[34600]: Deploying daemon mds.j on smithi016 2024-09-13T16:28:10.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:10 smithi111 ceph-mon[34600]: mds.? [v2:172.21.15.107:6832/3797712690,v1:172.21.15.107:6833/3797712690] up:boot 2024-09-13T16:28:10.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:10 smithi111 ceph-mon[34600]: fsmap 2 up:standby 2024-09-13T16:28:10.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:10 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-13T16:28:10.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:10 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/294447238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:11.292 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 13 16:28:11 smithi016 systemd[1]: Starting Ceph mds.j for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:28:11.552 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 13 16:28:11 smithi016 podman[71509]: 2024-09-13 16:28:11.193941658 +0000 UTC m=+0.016667575 image pull 152d8bc6b19e354515e8ba109d4619445de0e09b4d4a747c2f16eb5c0edb670c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a 2024-09-13T16:28:11.552 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 13 16:28:11 smithi016 podman[71509]: 2024-09-13 16:28:11.345049525 +0000 UTC m=+0.167775486 container create 216caf59eb38152ceaf1f71f153c5268526fcd3a91e1c6a95ade5cb10d2d0588 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-j, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-13T16:28:11.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:11 smithi107 ceph-mon[34139]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:11.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:11 smithi016 ceph-mon[31187]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:11.855 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 13 16:28:11 smithi016 podman[71509]: 2024-09-13 16:28:11.613915013 +0000 UTC m=+0.436640916 container init 216caf59eb38152ceaf1f71f153c5268526fcd3a91e1c6a95ade5cb10d2d0588 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-j, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-13T16:28:11.855 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 13 16:28:11 smithi016 podman[71509]: 2024-09-13 16:28:11.616988779 +0000 UTC m=+0.439714682 container start 216caf59eb38152ceaf1f71f153c5268526fcd3a91e1c6a95ade5cb10d2d0588 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-j, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-13T16:28:11.855 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 13 16:28:11 smithi016 ceph-mds[71528]: -- 172.21.15.16:0/1842101478 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5598578bcd00 con 0x559857891000 2024-09-13T16:28:11.855 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 13 16:28:11 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-j[71524]: starting mds.j at 2024-09-13T16:28:11.856 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 13 16:28:11 smithi016 bash[71509]: 216caf59eb38152ceaf1f71f153c5268526fcd3a91e1c6a95ade5cb10d2d0588 2024-09-13T16:28:11.856 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 13 16:28:11 smithi016 systemd[1]: Started Ceph mds.j for 0d241f6a-71ec-11ef-bceb-c7b262605968. 2024-09-13T16:28:11.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:11 smithi111 ceph-mon[34600]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:12.179 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:28:12.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.289+0000 7f1fb7fc7640 1 -- 172.21.15.16:0/944147341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb00ffd90 msgr2=0x7f1fb0100190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:12.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.289+0000 7f1fb7fc7640 1 --2- 172.21.15.16:0/944147341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb00ffd90 0x7f1fb0100190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009900 tx=0x7f1fac02a410 comp rx=0 tx=0).stop 2024-09-13T16:28:12.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.289+0000 7f1fb7fc7640 1 -- 172.21.15.16:0/944147341 shutdown_connections 2024-09-13T16:28:12.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.289+0000 7f1fb7fc7640 1 --2- 172.21.15.16:0/944147341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1fb0101940 0x7f1fb01081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:12.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.289+0000 7f1fb7fc7640 1 --2- 172.21.15.16:0/944147341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fb0100f80 0x7f1fb0101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:12.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.289+0000 7f1fb7fc7640 1 --2- 172.21.15.16:0/944147341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb00ffd90 0x7f1fb0100190 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:12.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.289+0000 7f1fb7fc7640 1 -- 172.21.15.16:0/944147341 >> 172.21.15.16:0/944147341 conn(0x7f1fb00fb510 msgr2=0x7f1fb00fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:12.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.289+0000 7f1fb7fc7640 1 -- 172.21.15.16:0/944147341 shutdown_connections 2024-09-13T16:28:12.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.289+0000 7f1fb7fc7640 1 -- 172.21.15.16:0/944147341 wait complete. 2024-09-13T16:28:12.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.290+0000 7f1fb7fc7640 1 Processor -- start 2024-09-13T16:28:12.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.290+0000 7f1fb7fc7640 1 -- start start 2024-09-13T16:28:12.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.290+0000 7f1fb7fc7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1fb00ffd90 0x7f1fb0195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:12.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.290+0000 7f1fb7fc7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb0100f80 0x7f1fb0196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:12.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.290+0000 7f1fb7fc7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fb0101940 0x7f1fb019d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:12.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.290+0000 7f1fb7fc7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb019d7d0 con 0x7f1fb0100f80 2024-09-13T16:28:12.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.290+0000 7f1fb7fc7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb019d940 con 0x7f1fb0101940 2024-09-13T16:28:12.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.290+0000 7f1fb7fc7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb019db00 con 0x7f1fb00ffd90 2024-09-13T16:28:12.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.290+0000 7f1fb5562640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb0100f80 0x7f1fb0196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:12.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.290+0000 7f1fb5562640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb0100f80 0x7f1fb0196200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59430/0 (socket says 172.21.15.16:59430) 2024-09-13T16:28:12.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.290+0000 7f1fb5562640 1 -- 172.21.15.16:0/611341912 learned_addr learned my addr 172.21.15.16:0/611341912 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:12.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.290+0000 7f1fb5d63640 1 --2- 172.21.15.16:0/611341912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1fb00ffd90 0x7f1fb0195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:12.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.290+0000 7f1fb6564640 1 --2- 172.21.15.16:0/611341912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fb0101940 0x7f1fb019d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:12.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.290+0000 7f1fb5562640 1 -- 172.21.15.16:0/611341912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1fb00ffd90 msgr2=0x7f1fb0195cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:12.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.290+0000 7f1fb5562640 1 --2- 172.21.15.16:0/611341912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1fb00ffd90 0x7f1fb0195cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:12.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.290+0000 7f1fb5562640 1 -- 172.21.15.16:0/611341912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fb0101940 msgr2=0x7f1fb019d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:12.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.290+0000 7f1fb5562640 1 --2- 172.21.15.16:0/611341912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fb0101940 0x7f1fb019d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:12.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.290+0000 7f1fb5562640 1 -- 172.21.15.16:0/611341912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fac009560 con 0x7f1fb0100f80 2024-09-13T16:28:12.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.291+0000 7f1fb5562640 1 --2- 172.21.15.16:0/611341912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb0100f80 0x7f1fb0196200 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000c890 tx=0x7f1fa000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:12.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.291+0000 7f1fa6ffd640 1 -- 172.21.15.16:0/611341912 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0004220 con 0x7f1fb0100f80 2024-09-13T16:28:12.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.291+0000 7f1fa6ffd640 1 -- 172.21.15.16:0/611341912 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1fa0007e40 con 0x7f1fb0100f80 2024-09-13T16:28:12.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.291+0000 7f1fb7fc7640 1 -- 172.21.15.16:0/611341912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb019dde0 con 0x7f1fb0100f80 2024-09-13T16:28:12.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.291+0000 7f1fa6ffd640 1 -- 172.21.15.16:0/611341912 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0004a00 con 0x7f1fb0100f80 2024-09-13T16:28:12.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.291+0000 7f1fb7fc7640 1 -- 172.21.15.16:0/611341912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb019e260 con 0x7f1fb0100f80 2024-09-13T16:28:12.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.292+0000 7f1fb7fc7640 1 -- 172.21.15.16:0/611341912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fb01072c0 con 0x7f1fb0100f80 2024-09-13T16:28:12.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.293+0000 7f1fa6ffd640 1 -- 172.21.15.16:0/611341912 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1fa00079b0 con 0x7f1fb0100f80 2024-09-13T16:28:12.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.294+0000 7f1fa6ffd640 1 --2- 172.21.15.16:0/611341912 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f1f9c0a1090 0x7f1f9c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:12.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.294+0000 7f1fb5d63640 1 --2- 172.21.15.16:0/611341912 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f1f9c0a1090 0x7f1f9c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:12.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.294+0000 7f1fa6ffd640 1 -- 172.21.15.16:0/611341912 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1fa00cc5c0 con 0x7f1fb0100f80 2024-09-13T16:28:12.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.295+0000 7f1fb5d63640 1 --2- 172.21.15.16:0/611341912 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f1f9c0a1090 0x7f1f9c0a3550 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1fac006010 tx=0x7f1fac000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.299+0000 7f1fa6ffd640 1 -- 172.21.15.16:0/611341912 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f1fa0096150 con 0x7f1fb0100f80 2024-09-13T16:28:12.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.511+0000 7f1fb7fc7640 1 -- 172.21.15.16:0/611341912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1fb002c480 con 0x7f1fb0100f80 2024-09-13T16:28:12.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.512+0000 7f1fa6ffd640 1 -- 172.21.15.16:0/611341912 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2020 (secure 0 0 0) 0x7f1fa001d800 con 0x7f1fb0100f80 2024-09-13T16:28:12.514 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:28:12.514 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1984275936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1984275936},{"type":"v1","addr":"172.21.15.111:6833","nonce":1984275936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3797712690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3797712690},{"type":"v1","addr":"172.21.15.107:6833","nonce":3797712690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-09-13T16:28:12.516 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 3 2024-09-13T16:28:12.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.514+0000 7f1fb7fc7640 1 -- 172.21.15.16:0/611341912 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f1f9c0a1090 msgr2=0x7f1f9c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.514+0000 7f1fb7fc7640 1 --2- 172.21.15.16:0/611341912 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f1f9c0a1090 0x7f1f9c0a3550 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1fac006010 tx=0x7f1fac000bc0 comp rx=0 tx=0).stop 2024-09-13T16:28:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.514+0000 7f1fb7fc7640 1 -- 172.21.15.16:0/611341912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb0100f80 msgr2=0x7f1fb0196200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.514+0000 7f1fb7fc7640 1 --2- 172.21.15.16:0/611341912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb0100f80 0x7f1fb0196200 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000c890 tx=0x7f1fa000cd50 comp rx=0 tx=0).stop 2024-09-13T16:28:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.514+0000 7f1fb7fc7640 1 -- 172.21.15.16:0/611341912 shutdown_connections 2024-09-13T16:28:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.514+0000 7f1fb7fc7640 1 --2- 172.21.15.16:0/611341912 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f1f9c0a1090 0x7f1f9c0a3550 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.514+0000 7f1fb7fc7640 1 --2- 172.21.15.16:0/611341912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fb0101940 0x7f1fb019d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.514+0000 7f1fb7fc7640 1 --2- 172.21.15.16:0/611341912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb0100f80 0x7f1fb0196200 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.514+0000 7f1fb7fc7640 1 --2- 172.21.15.16:0/611341912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1fb00ffd90 0x7f1fb0195cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.514+0000 7f1fb7fc7640 1 -- 172.21.15.16:0/611341912 >> 172.21.15.16:0/611341912 conn(0x7f1fb00fb510 msgr2=0x7f1fb0104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.514+0000 7f1fb7fc7640 1 -- 172.21.15.16:0/611341912 shutdown_connections 2024-09-13T16:28:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:12.514+0000 7f1fb7fc7640 1 -- 172.21.15.16:0/611341912 wait complete. 2024-09-13T16:28:13.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:12 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:13.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:12 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:13.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:12 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:13.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:12 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:13.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:12 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:13.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:12 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:13.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:12 smithi107 ceph-mon[34139]: Deploying daemon mds.d on smithi016 2024-09-13T16:28:13.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:12 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/611341912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:13.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:12 smithi107 ceph-mon[34139]: mds.? [v2:172.21.15.16:6834/3292781664,v1:172.21.15.16:6835/3292781664] up:boot 2024-09-13T16:28:13.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:12 smithi107 ceph-mon[34139]: fsmap 3 up:standby 2024-09-13T16:28:13.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:12 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-13T16:28:13.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:12 smithi107 ceph-mon[34139]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:13.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:12 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:13.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:12 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:13.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:12 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:13.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:12 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:13.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:12 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:13.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:12 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:13.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:12 smithi016 ceph-mon[31187]: Deploying daemon mds.d on smithi016 2024-09-13T16:28:13.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:12 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/611341912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:13.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:12 smithi016 ceph-mon[31187]: mds.? [v2:172.21.15.16:6834/3292781664,v1:172.21.15.16:6835/3292781664] up:boot 2024-09-13T16:28:13.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:12 smithi016 ceph-mon[31187]: fsmap 3 up:standby 2024-09-13T16:28:13.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:12 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-13T16:28:13.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:12 smithi016 ceph-mon[31187]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:13.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:12 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:13.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:12 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:13.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:12 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:13.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:12 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:13.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:12 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:13.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:12 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:13.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:12 smithi111 ceph-mon[34600]: Deploying daemon mds.d on smithi016 2024-09-13T16:28:13.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:12 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/611341912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:13.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:12 smithi111 ceph-mon[34600]: mds.? [v2:172.21.15.16:6834/3292781664,v1:172.21.15.16:6835/3292781664] up:boot 2024-09-13T16:28:13.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:12 smithi111 ceph-mon[34600]: fsmap 3 up:standby 2024-09-13T16:28:13.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:12 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-13T16:28:13.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:12 smithi111 ceph-mon[34600]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:13.854 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 13 16:28:13 smithi016 systemd[1]: Starting Ceph mds.d for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:28:14.156 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 13 16:28:13 smithi016 podman[71853]: 2024-09-13 16:28:13.781265035 +0000 UTC m=+0.016716787 image pull 152d8bc6b19e354515e8ba109d4619445de0e09b4d4a747c2f16eb5c0edb670c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a 2024-09-13T16:28:14.156 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 13 16:28:13 smithi016 podman[71853]: 2024-09-13 16:28:13.894781665 +0000 UTC m=+0.130233403 container create 47749514ffd28e2056c7e484e67039385062e5b015fe07ee20a26486f4cfafca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-13T16:28:14.527 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:28:14.553 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 13 16:28:14 smithi016 podman[71853]: 2024-09-13 16:28:14.156016816 +0000 UTC m=+0.391468553 container init 47749514ffd28e2056c7e484e67039385062e5b015fe07ee20a26486f4cfafca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-d, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T16:28:14.553 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 13 16:28:14 smithi016 podman[71853]: 2024-09-13 16:28:14.159074048 +0000 UTC m=+0.394525784 container start 47749514ffd28e2056c7e484e67039385062e5b015fe07ee20a26486f4cfafca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-d, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f) 2024-09-13T16:28:14.553 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 13 16:28:14 smithi016 ceph-mds[71872]: -- 172.21.15.16:0/355751079 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5592bc362d00 con 0x5592bc337000 2024-09-13T16:28:14.553 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 13 16:28:14 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-d[71868]: starting mds.d at 2024-09-13T16:28:14.553 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 13 16:28:14 smithi016 bash[71853]: 47749514ffd28e2056c7e484e67039385062e5b015fe07ee20a26486f4cfafca 2024-09-13T16:28:14.553 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 13 16:28:14 smithi016 systemd[1]: Started Ceph mds.d for 0d241f6a-71ec-11ef-bceb-c7b262605968. 2024-09-13T16:28:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.635+0000 7f02dfc12640 1 -- 172.21.15.16:0/1573376804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02d80ffda0 msgr2=0x7f02d81001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.635+0000 7f02dfc12640 1 --2- 172.21.15.16:0/1573376804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02d80ffda0 0x7f02d81001a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009900 tx=0x7f02d402a410 comp rx=0 tx=0).stop 2024-09-13T16:28:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.635+0000 7f02dfc12640 1 -- 172.21.15.16:0/1573376804 shutdown_connections 2024-09-13T16:28:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.635+0000 7f02dfc12640 1 --2- 172.21.15.16:0/1573376804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02d8101950 0x7f02d81081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.635+0000 7f02dfc12640 1 --2- 172.21.15.16:0/1573376804 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02d8100f90 0x7f02d8101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.635+0000 7f02dfc12640 1 --2- 172.21.15.16:0/1573376804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02d80ffda0 0x7f02d81001a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.635+0000 7f02dfc12640 1 -- 172.21.15.16:0/1573376804 >> 172.21.15.16:0/1573376804 conn(0x7f02d80fb520 msgr2=0x7f02d80fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.635+0000 7f02dfc12640 1 -- 172.21.15.16:0/1573376804 shutdown_connections 2024-09-13T16:28:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.635+0000 7f02dfc12640 1 -- 172.21.15.16:0/1573376804 wait complete. 2024-09-13T16:28:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.636+0000 7f02dfc12640 1 Processor -- start 2024-09-13T16:28:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.636+0000 7f02dfc12640 1 -- start start 2024-09-13T16:28:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.636+0000 7f02dfc12640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02d80ffda0 0x7f02d8074260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.636+0000 7f02dfc12640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02d8100f90 0x7f02d80728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.636+0000 7f02dfc12640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02d8101950 0x7f02d8072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.636+0000 7f02dfc12640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d8073540 con 0x7f02d8101950 2024-09-13T16:28:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.636+0000 7f02dfc12640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d80736b0 con 0x7f02d8100f90 2024-09-13T16:28:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.636+0000 7f02dfc12640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d819fdb0 con 0x7f02d80ffda0 2024-09-13T16:28:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.636+0000 7f02de1af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02d8101950 0x7f02d8072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.636+0000 7f02de1af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02d8101950 0x7f02d8072de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38208/0 (socket says 172.21.15.16:38208) 2024-09-13T16:28:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.636+0000 7f02de1af640 1 -- 172.21.15.16:0/3026665606 learned_addr learned my addr 172.21.15.16:0/3026665606 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.636+0000 7f02dd9ae640 1 --2- 172.21.15.16:0/3026665606 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02d80ffda0 0x7f02d8074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.637+0000 7f02dd1ad640 1 --2- 172.21.15.16:0/3026665606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02d8100f90 0x7f02d80728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.637+0000 7f02de1af640 1 -- 172.21.15.16:0/3026665606 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02d80ffda0 msgr2=0x7f02d8074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:14.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.637+0000 7f02de1af640 1 --2- 172.21.15.16:0/3026665606 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02d80ffda0 0x7f02d8074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:14.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.637+0000 7f02de1af640 1 -- 172.21.15.16:0/3026665606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02d8100f90 msgr2=0x7f02d80728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:14.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.637+0000 7f02de1af640 1 --2- 172.21.15.16:0/3026665606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02d8100f90 0x7f02d80728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:14.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.637+0000 7f02de1af640 1 -- 172.21.15.16:0/3026665606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d4009560 con 0x7f02d8101950 2024-09-13T16:28:14.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.637+0000 7f02dd1ad640 1 --2- 172.21.15.16:0/3026665606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02d8100f90 0x7f02d80728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:28:14.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.637+0000 7f02de1af640 1 --2- 172.21.15.16:0/3026665606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02d8101950 0x7f02d8072de0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f02d000b9a0 tx=0x7f02d000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:14.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.637+0000 7f02ceffd640 1 -- 172.21.15.16:0/3026665606 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d000cba0 con 0x7f02d8101950 2024-09-13T16:28:14.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.637+0000 7f02ceffd640 1 -- 172.21.15.16:0/3026665606 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02d000cd00 con 0x7f02d8101950 2024-09-13T16:28:14.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.637+0000 7f02dfc12640 1 -- 172.21.15.16:0/3026665606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02d81a0060 con 0x7f02d8101950 2024-09-13T16:28:14.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.637+0000 7f02ceffd640 1 -- 172.21.15.16:0/3026665606 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d00075d0 con 0x7f02d8101950 2024-09-13T16:28:14.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.637+0000 7f02dfc12640 1 -- 172.21.15.16:0/3026665606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d81a0570 con 0x7f02d8101950 2024-09-13T16:28:14.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.638+0000 7f02dfc12640 1 -- 172.21.15.16:0/3026665606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02d8107320 con 0x7f02d8101950 2024-09-13T16:28:14.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.639+0000 7f02ceffd640 1 -- 172.21.15.16:0/3026665606 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f02d0014440 con 0x7f02d8101950 2024-09-13T16:28:14.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.640+0000 7f02ceffd640 1 --2- 172.21.15.16:0/3026665606 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f02c40a1110 0x7f02c40a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:14.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.640+0000 7f02dd9ae640 1 --2- 172.21.15.16:0/3026665606 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f02c40a1110 0x7f02c40a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:14.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.640+0000 7f02ceffd640 1 -- 172.21.15.16:0/3026665606 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f02d00cbcb0 con 0x7f02d8101950 2024-09-13T16:28:14.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.640+0000 7f02dd9ae640 1 --2- 172.21.15.16:0/3026665606 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f02c40a1110 0x7f02c40a35d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f02d40057c0 tx=0x7f02d4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:14.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.644+0000 7f02ceffd640 1 -- 172.21.15.16:0/3026665606 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f02d00958f0 con 0x7f02d8101950 2024-09-13T16:28:14.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.852+0000 7f02dfc12640 1 -- 172.21.15.16:0/3026665606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02d810b590 con 0x7f02d8101950 2024-09-13T16:28:14.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.853+0000 7f02ceffd640 1 -- 172.21.15.16:0/3026665606 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3534 (secure 0 0 0) 0x7f02d0019de0 con 0x7f02d8101950 2024-09-13T16:28:14.854 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:28:14.854 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3292781664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3292781664},{"type":"v1","addr":"172.21.15.16:6835","nonce":3292781664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4109039027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4109039027},{"type":"v1","addr":"172.21.15.16:6837","nonce":4109039027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1984275936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1984275936},{"type":"v1","addr":"172.21.15.111:6833","nonce":1984275936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3797712690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3797712690},{"type":"v1","addr":"172.21.15.107:6833","nonce":3797712690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-09-13T16:28:14.855 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 5 2024-09-13T16:28:14.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.855+0000 7f02dfc12640 1 -- 172.21.15.16:0/3026665606 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f02c40a1110 msgr2=0x7f02c40a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:14.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.855+0000 7f02dfc12640 1 --2- 172.21.15.16:0/3026665606 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f02c40a1110 0x7f02c40a35d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f02d40057c0 tx=0x7f02d4005a20 comp rx=0 tx=0).stop 2024-09-13T16:28:14.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.855+0000 7f02dfc12640 1 -- 172.21.15.16:0/3026665606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02d8101950 msgr2=0x7f02d8072de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:14.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.855+0000 7f02dfc12640 1 --2- 172.21.15.16:0/3026665606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02d8101950 0x7f02d8072de0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f02d000b9a0 tx=0x7f02d000be60 comp rx=0 tx=0).stop 2024-09-13T16:28:14.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.855+0000 7f02dfc12640 1 -- 172.21.15.16:0/3026665606 shutdown_connections 2024-09-13T16:28:14.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.855+0000 7f02dfc12640 1 --2- 172.21.15.16:0/3026665606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02d8101950 0x7f02d8072de0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:14.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.855+0000 7f02dfc12640 1 --2- 172.21.15.16:0/3026665606 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f02c40a1110 0x7f02c40a35d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:14.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.855+0000 7f02dfc12640 1 --2- 172.21.15.16:0/3026665606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02d8100f90 0x7f02d80728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:14.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.855+0000 7f02dfc12640 1 --2- 172.21.15.16:0/3026665606 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02d80ffda0 0x7f02d8074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:14.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.855+0000 7f02dfc12640 1 -- 172.21.15.16:0/3026665606 >> 172.21.15.16:0/3026665606 conn(0x7f02d80fb520 msgr2=0x7f02d81044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:14.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.855+0000 7f02dfc12640 1 -- 172.21.15.16:0/3026665606 shutdown_connections 2024-09-13T16:28:14.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:14.855+0000 7f02dfc12640 1 -- 172.21.15.16:0/3026665606 wait complete. 2024-09-13T16:28:15.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:15 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:15.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:15 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:15.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:15 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:15.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:15 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:15.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:15 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:15.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:15 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:15.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:15 smithi107 ceph-mon[34139]: Deploying daemon mds.e on smithi107 2024-09-13T16:28:15.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:15 smithi107 ceph-mon[34139]: mds.? [v2:172.21.15.16:6836/4109039027,v1:172.21.15.16:6837/4109039027] up:boot 2024-09-13T16:28:15.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:15 smithi107 ceph-mon[34139]: fsmap 4 up:standby 2024-09-13T16:28:15.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:15 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-13T16:28:15.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:15 smithi107 ceph-mon[34139]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:15.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:15 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3026665606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:15.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:15 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:15.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:15 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:15.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:15 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:15.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:15 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:15.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:15 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:15.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:15 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:15.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:15 smithi016 ceph-mon[31187]: Deploying daemon mds.e on smithi107 2024-09-13T16:28:15.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:15 smithi016 ceph-mon[31187]: mds.? [v2:172.21.15.16:6836/4109039027,v1:172.21.15.16:6837/4109039027] up:boot 2024-09-13T16:28:15.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:15 smithi016 ceph-mon[31187]: fsmap 4 up:standby 2024-09-13T16:28:15.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:15 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-13T16:28:15.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:15 smithi016 ceph-mon[31187]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:15.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:15 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3026665606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:15.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:15 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:15.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:15 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:15.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:15 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:15.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:15 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:15.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:15 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:15.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:15 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:15.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:15 smithi111 ceph-mon[34600]: Deploying daemon mds.e on smithi107 2024-09-13T16:28:15.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:15 smithi111 ceph-mon[34600]: mds.? [v2:172.21.15.16:6836/4109039027,v1:172.21.15.16:6837/4109039027] up:boot 2024-09-13T16:28:15.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:15 smithi111 ceph-mon[34600]: fsmap 4 up:standby 2024-09-13T16:28:15.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:15 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-13T16:28:15.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:15 smithi111 ceph-mon[34600]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:15.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:15 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3026665606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:16.503 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 13 16:28:16 smithi107 systemd[1]: Starting Ceph mds.e for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:28:16.801 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:16 smithi107 ceph-mon[34139]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:16.802 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 13 16:28:16 smithi107 podman[62858]: 2024-09-13 16:28:16.405299738 +0000 UTC m=+0.016190875 image pull 152d8bc6b19e354515e8ba109d4619445de0e09b4d4a747c2f16eb5c0edb670c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a 2024-09-13T16:28:16.802 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 13 16:28:16 smithi107 podman[62858]: 2024-09-13 16:28:16.531943261 +0000 UTC m=+0.142834444 container create b50793f01f1926f2c81823a85d43b59f612f1e7749a8d4d5875584ebea278c11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-e, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f) 2024-09-13T16:28:16.802 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 13 16:28:16 smithi107 podman[62858]: 2024-09-13 16:28:16.801243199 +0000 UTC m=+0.412134336 container init b50793f01f1926f2c81823a85d43b59f612f1e7749a8d4d5875584ebea278c11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-e, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T16:28:16.867 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:28:16.895 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:16 smithi016 ceph-mon[31187]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:16.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:16 smithi111 ceph-mon[34600]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:16.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.985+0000 7f4392918640 1 -- 172.21.15.16:0/4207913914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f438c0feca0 msgr2=0x7f438c0ff0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:16.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.985+0000 7f4392918640 1 --2- 172.21.15.16:0/4207913914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f438c0feca0 0x7f438c0ff0a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f437c009900 tx=0x7f437c02a410 comp rx=0 tx=0).stop 2024-09-13T16:28:16.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.985+0000 7f4392918640 1 -- 172.21.15.16:0/4207913914 shutdown_connections 2024-09-13T16:28:16.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.985+0000 7f4392918640 1 --2- 172.21.15.16:0/4207913914 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f438c100850 0x7f438c1070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:16.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.985+0000 7f4392918640 1 --2- 172.21.15.16:0/4207913914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f438c0ffe90 0x7f438c100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:16.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.985+0000 7f4392918640 1 --2- 172.21.15.16:0/4207913914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f438c0feca0 0x7f438c0ff0a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:16.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.985+0000 7f4392918640 1 -- 172.21.15.16:0/4207913914 >> 172.21.15.16:0/4207913914 conn(0x7f438c0fa440 msgr2=0x7f438c0fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:16.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.985+0000 7f4392918640 1 -- 172.21.15.16:0/4207913914 shutdown_connections 2024-09-13T16:28:16.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.985+0000 7f4392918640 1 -- 172.21.15.16:0/4207913914 wait complete. 2024-09-13T16:28:16.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.985+0000 7f4392918640 1 Processor -- start 2024-09-13T16:28:16.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.985+0000 7f4392918640 1 -- start start 2024-09-13T16:28:16.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.986+0000 7f4392918640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f438c0feca0 0x7f438c068110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:16.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.986+0000 7f4392918640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f438c0ffe90 0x7f438c19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:16.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.986+0000 7f4392918640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f438c100850 0x7f438c19d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:16.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.986+0000 7f4392918640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438c19d850 con 0x7f438c0ffe90 2024-09-13T16:28:16.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.986+0000 7f4392918640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438c19d9c0 con 0x7f438c100850 2024-09-13T16:28:16.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.986+0000 7f4392918640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438c19db80 con 0x7f438c0feca0 2024-09-13T16:28:16.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.986+0000 7f4383fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f438c0ffe90 0x7f438c19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:16.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.986+0000 7f4383fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f438c0ffe90 0x7f438c19aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38226/0 (socket says 172.21.15.16:38226) 2024-09-13T16:28:16.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.986+0000 7f4383fff640 1 -- 172.21.15.16:0/1432904563 learned_addr learned my addr 172.21.15.16:0/1432904563 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:16.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.986+0000 7f438bfff640 1 --2- 172.21.15.16:0/1432904563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f438c0feca0 0x7f438c068110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:16.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.986+0000 7f4390eb5640 1 --2- 172.21.15.16:0/1432904563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f438c100850 0x7f438c19d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:16.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.986+0000 7f4383fff640 1 -- 172.21.15.16:0/1432904563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f438c0feca0 msgr2=0x7f438c068110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:16.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.986+0000 7f4383fff640 1 --2- 172.21.15.16:0/1432904563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f438c0feca0 0x7f438c068110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:16.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.986+0000 7f4383fff640 1 -- 172.21.15.16:0/1432904563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f438c100850 msgr2=0x7f438c19d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:16.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.986+0000 7f4383fff640 1 --2- 172.21.15.16:0/1432904563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f438c100850 0x7f438c19d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:16.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.986+0000 7f4383fff640 1 -- 172.21.15.16:0/1432904563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f437c009560 con 0x7f438c0ffe90 2024-09-13T16:28:16.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.986+0000 7f4383fff640 1 --2- 172.21.15.16:0/1432904563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f438c0ffe90 0x7f438c19aa50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f437400d950 tx=0x7f437400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:16.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.986+0000 7f4389ffb640 1 -- 172.21.15.16:0/1432904563 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4374009a30 con 0x7f438c0ffe90 2024-09-13T16:28:16.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.987+0000 7f4392918640 1 -- 172.21.15.16:0/1432904563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f438c19de00 con 0x7f438c0ffe90 2024-09-13T16:28:16.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.987+0000 7f4389ffb640 1 -- 172.21.15.16:0/1432904563 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f43740044b0 con 0x7f438c0ffe90 2024-09-13T16:28:16.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.987+0000 7f4389ffb640 1 -- 172.21.15.16:0/1432904563 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43740047a0 con 0x7f438c0ffe90 2024-09-13T16:28:16.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.987+0000 7f4392918640 1 -- 172.21.15.16:0/1432904563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f438c19e290 con 0x7f438c0ffe90 2024-09-13T16:28:16.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.988+0000 7f4392918640 1 -- 172.21.15.16:0/1432904563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f438c1061c0 con 0x7f438c0ffe90 2024-09-13T16:28:16.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.989+0000 7f4389ffb640 1 -- 172.21.15.16:0/1432904563 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f4374004020 con 0x7f438c0ffe90 2024-09-13T16:28:16.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.989+0000 7f4389ffb640 1 --2- 172.21.15.16:0/1432904563 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f43780a1090 0x7f43780a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:16.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.990+0000 7f438bfff640 1 --2- 172.21.15.16:0/1432904563 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f43780a1090 0x7f43780a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:16.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.990+0000 7f4389ffb640 1 -- 172.21.15.16:0/1432904563 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f43740cbe20 con 0x7f438c0ffe90 2024-09-13T16:28:16.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.990+0000 7f438bfff640 1 --2- 172.21.15.16:0/1432904563 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f43780a1090 0x7f43780a3550 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f437c0057c0 tx=0x7f437c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:16.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:16.994+0000 7f4389ffb640 1 -- 172.21.15.16:0/1432904563 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f4374095a50 con 0x7f438c0ffe90 2024-09-13T16:28:17.099 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 13 16:28:16 smithi107 podman[62858]: 2024-09-13 16:28:16.804293813 +0000 UTC m=+0.415184957 container start b50793f01f1926f2c81823a85d43b59f612f1e7749a8d4d5875584ebea278c11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-13T16:28:17.099 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 13 16:28:16 smithi107 ceph-mds[62877]: -- 172.21.15.107:0/768156589 <== mon.2 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bca3534d00 con 0x55bca3509000 2024-09-13T16:28:17.099 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 13 16:28:16 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-e[62873]: starting mds.e at 2024-09-13T16:28:17.099 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 13 16:28:16 smithi107 bash[62858]: b50793f01f1926f2c81823a85d43b59f612f1e7749a8d4d5875584ebea278c11 2024-09-13T16:28:17.099 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 13 16:28:16 smithi107 systemd[1]: Started Ceph mds.e for 0d241f6a-71ec-11ef-bceb-c7b262605968. 2024-09-13T16:28:17.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:17.202+0000 7f4392918640 1 -- 172.21.15.16:0/1432904563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f438c10a470 con 0x7f438c0ffe90 2024-09-13T16:28:17.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:17.202+0000 7f4389ffb640 1 -- 172.21.15.16:0/1432904563 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3534 (secure 0 0 0) 0x7f4374017de0 con 0x7f438c0ffe90 2024-09-13T16:28:17.203 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:28:17.203 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3292781664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3292781664},{"type":"v1","addr":"172.21.15.16:6835","nonce":3292781664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4109039027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4109039027},{"type":"v1","addr":"172.21.15.16:6837","nonce":4109039027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1984275936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1984275936},{"type":"v1","addr":"172.21.15.111:6833","nonce":1984275936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3797712690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3797712690},{"type":"v1","addr":"172.21.15.107:6833","nonce":3797712690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-09-13T16:28:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 5 2024-09-13T16:28:17.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:17.204+0000 7f4392918640 1 -- 172.21.15.16:0/1432904563 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f43780a1090 msgr2=0x7f43780a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:17.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:17.204+0000 7f4392918640 1 --2- 172.21.15.16:0/1432904563 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f43780a1090 0x7f43780a3550 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f437c0057c0 tx=0x7f437c005a20 comp rx=0 tx=0).stop 2024-09-13T16:28:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:17.204+0000 7f4392918640 1 -- 172.21.15.16:0/1432904563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f438c0ffe90 msgr2=0x7f438c19aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:17.204+0000 7f4392918640 1 --2- 172.21.15.16:0/1432904563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f438c0ffe90 0x7f438c19aa50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f437400d950 tx=0x7f437400de10 comp rx=0 tx=0).stop 2024-09-13T16:28:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:17.204+0000 7f4392918640 1 -- 172.21.15.16:0/1432904563 shutdown_connections 2024-09-13T16:28:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:17.204+0000 7f4392918640 1 --2- 172.21.15.16:0/1432904563 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f43780a1090 0x7f43780a3550 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:17.204+0000 7f4392918640 1 --2- 172.21.15.16:0/1432904563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f438c100850 0x7f438c19d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:17.204+0000 7f4392918640 1 --2- 172.21.15.16:0/1432904563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f438c0ffe90 0x7f438c19aa50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:17.204+0000 7f4392918640 1 --2- 172.21.15.16:0/1432904563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f438c0feca0 0x7f438c068110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:17.204+0000 7f4392918640 1 -- 172.21.15.16:0/1432904563 >> 172.21.15.16:0/1432904563 conn(0x7f438c0fa440 msgr2=0x7f438c103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:17.204+0000 7f4392918640 1 -- 172.21.15.16:0/1432904563 shutdown_connections 2024-09-13T16:28:17.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:17.204+0000 7f4392918640 1 -- 172.21.15.16:0/1432904563 wait complete. 2024-09-13T16:28:18.220 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:17 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:18.220 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:17 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:18.220 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:17 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:18.220 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:17 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:18.220 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:17 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:18.220 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:17 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:18.220 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:17 smithi111 ceph-mon[34600]: Deploying daemon mds.f on smithi111 2024-09-13T16:28:18.220 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:17 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1432904563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:18.220 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:17 smithi111 ceph-mon[34600]: mds.? [v2:172.21.15.107:6834/338548468,v1:172.21.15.107:6835/338548468] up:boot 2024-09-13T16:28:18.220 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:17 smithi111 ceph-mon[34600]: fsmap 5 up:standby 2024-09-13T16:28:18.221 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:17 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-13T16:28:18.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:17 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:18.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:17 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:18.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:17 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:18.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:17 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:18.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:17 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:18.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:17 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:18.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:17 smithi107 ceph-mon[34139]: Deploying daemon mds.f on smithi111 2024-09-13T16:28:18.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:17 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1432904563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:18.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:17 smithi107 ceph-mon[34139]: mds.? [v2:172.21.15.107:6834/338548468,v1:172.21.15.107:6835/338548468] up:boot 2024-09-13T16:28:18.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:17 smithi107 ceph-mon[34139]: fsmap 5 up:standby 2024-09-13T16:28:18.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:17 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-13T16:28:18.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:17 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:18.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:17 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:18.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:17 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:18.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:17 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:18.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:17 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:18.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:17 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:18.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:17 smithi016 ceph-mon[31187]: Deploying daemon mds.f on smithi111 2024-09-13T16:28:18.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:17 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1432904563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:18.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:17 smithi016 ceph-mon[31187]: mds.? [v2:172.21.15.107:6834/338548468,v1:172.21.15.107:6835/338548468] up:boot 2024-09-13T16:28:18.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:17 smithi016 ceph-mon[31187]: fsmap 5 up:standby 2024-09-13T16:28:18.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:17 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-13T16:28:19.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:18 smithi111 ceph-mon[34600]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:19.188 INFO:journalctl@ceph.mds.f.smithi111.stdout:Sep 13 16:28:19 smithi111 podman[65160]: 2024-09-13 16:28:19.107276608 +0000 UTC m=+0.131120827 container create 19749af66c83e3b0f774b86a4cb51b62d989980924771edd504b2d72f739225a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-f, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-13T16:28:19.216 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:28:19.244 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:18 smithi016 ceph-mon[31187]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:19.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.331+0000 7ff0e746b640 1 -- 172.21.15.16:0/2471390664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0e00ffec0 msgr2=0x7ff0e0100340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:19.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.331+0000 7ff0e746b640 1 --2- 172.21.15.16:0/2471390664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0e00ffec0 0x7ff0e0100340 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0009900 tx=0x7ff0d002a410 comp rx=0 tx=0).stop 2024-09-13T16:28:19.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.331+0000 7ff0e746b640 1 -- 172.21.15.16:0/2471390664 shutdown_connections 2024-09-13T16:28:19.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.331+0000 7ff0e746b640 1 --2- 172.21.15.16:0/2471390664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0e0100880 0x7ff0e0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:19.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.331+0000 7ff0e746b640 1 --2- 172.21.15.16:0/2471390664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0e00ffec0 0x7ff0e0100340 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:19.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.331+0000 7ff0e746b640 1 --2- 172.21.15.16:0/2471390664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0e00fecd0 0x7ff0e00ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:19.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.331+0000 7ff0e746b640 1 -- 172.21.15.16:0/2471390664 >> 172.21.15.16:0/2471390664 conn(0x7ff0e00fa430 msgr2=0x7ff0e00fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:19.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.331+0000 7ff0e746b640 1 -- 172.21.15.16:0/2471390664 shutdown_connections 2024-09-13T16:28:19.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.331+0000 7ff0e746b640 1 -- 172.21.15.16:0/2471390664 wait complete. 2024-09-13T16:28:19.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.332+0000 7ff0e746b640 1 Processor -- start 2024-09-13T16:28:19.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.332+0000 7ff0e746b640 1 -- start start 2024-09-13T16:28:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.332+0000 7ff0e746b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0e00fecd0 0x7ff0e0102c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.332+0000 7ff0e746b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0e00ffec0 0x7ff0e01012a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.332+0000 7ff0e746b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0e0100880 0x7ff0e01017e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.332+0000 7ff0e746b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e0101f40 con 0x7ff0e00fecd0 2024-09-13T16:28:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.332+0000 7ff0e746b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e01020b0 con 0x7ff0e00ffec0 2024-09-13T16:28:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.332+0000 7ff0e746b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e019fde0 con 0x7ff0e0100880 2024-09-13T16:28:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.332+0000 7ff0e5207640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0e00fecd0 0x7ff0e0102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.332+0000 7ff0e5207640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0e00fecd0 0x7ff0e0102c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38254/0 (socket says 172.21.15.16:38254) 2024-09-13T16:28:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.332+0000 7ff0e5207640 1 -- 172.21.15.16:0/3068546062 learned_addr learned my addr 172.21.15.16:0/3068546062 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.332+0000 7ff0e5a08640 1 --2- 172.21.15.16:0/3068546062 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0e0100880 0x7ff0e01017e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.332+0000 7ff0e4a06640 1 --2- 172.21.15.16:0/3068546062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0e00ffec0 0x7ff0e01012a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:19.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.332+0000 7ff0e5207640 1 -- 172.21.15.16:0/3068546062 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0e0100880 msgr2=0x7ff0e01017e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:19.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.332+0000 7ff0e5207640 1 --2- 172.21.15.16:0/3068546062 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0e0100880 0x7ff0e01017e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:19.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.332+0000 7ff0e5207640 1 -- 172.21.15.16:0/3068546062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0e00ffec0 msgr2=0x7ff0e01012a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:19.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.332+0000 7ff0e5207640 1 --2- 172.21.15.16:0/3068546062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0e00ffec0 0x7ff0e01012a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:19.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.332+0000 7ff0e5207640 1 -- 172.21.15.16:0/3068546062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0dc009490 con 0x7ff0e00fecd0 2024-09-13T16:28:19.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.332+0000 7ff0e5a08640 1 --2- 172.21.15.16:0/3068546062 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0e0100880 0x7ff0e01017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:28:19.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.332+0000 7ff0e4a06640 1 --2- 172.21.15.16:0/3068546062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0e00ffec0 0x7ff0e01012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:28:19.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.333+0000 7ff0e5207640 1 --2- 172.21.15.16:0/3068546062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0e00fecd0 0x7ff0e0102c60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc00e9f0 tx=0x7ff0dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:19.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.333+0000 7ff0d67fc640 1 -- 172.21.15.16:0/3068546062 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc00cde0 con 0x7ff0e00fecd0 2024-09-13T16:28:19.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.333+0000 7ff0d67fc640 1 -- 172.21.15.16:0/3068546062 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0dc0108c0 con 0x7ff0e00fecd0 2024-09-13T16:28:19.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.333+0000 7ff0e746b640 1 -- 172.21.15.16:0/3068546062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d0009560 con 0x7ff0e00fecd0 2024-09-13T16:28:19.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.333+0000 7ff0d67fc640 1 -- 172.21.15.16:0/3068546062 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc018840 con 0x7ff0e00fecd0 2024-09-13T16:28:19.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.333+0000 7ff0e746b640 1 -- 172.21.15.16:0/3068546062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e01a03e0 con 0x7ff0e00fecd0 2024-09-13T16:28:19.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.334+0000 7ff0e746b640 1 -- 172.21.15.16:0/3068546062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0e010a420 con 0x7ff0e00fecd0 2024-09-13T16:28:19.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.335+0000 7ff0d67fc640 1 -- 172.21.15.16:0/3068546062 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff0dc010430 con 0x7ff0e00fecd0 2024-09-13T16:28:19.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.336+0000 7ff0d67fc640 1 --2- 172.21.15.16:0/3068546062 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff0cc0a1090 0x7ff0cc0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:19.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.336+0000 7ff0e4a06640 1 --2- 172.21.15.16:0/3068546062 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff0cc0a1090 0x7ff0cc0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:19.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.336+0000 7ff0d67fc640 1 -- 172.21.15.16:0/3068546062 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff0dc014070 con 0x7ff0e00fecd0 2024-09-13T16:28:19.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.337+0000 7ff0e4a06640 1 --2- 172.21.15.16:0/3068546062 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff0cc0a1090 0x7ff0cc0a3550 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0000c00 tx=0x7ff0d00058b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:19.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.340+0000 7ff0d67fc640 1 -- 172.21.15.16:0/3068546062 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff0dc096770 con 0x7ff0e00fecd0 2024-09-13T16:28:19.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:18 smithi107 ceph-mon[34139]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:19.501 INFO:journalctl@ceph.mds.f.smithi111.stdout:Sep 13 16:28:19 smithi111 podman[65160]: 2024-09-13 16:28:19.376045066 +0000 UTC m=+0.399889294 container init 19749af66c83e3b0f774b86a4cb51b62d989980924771edd504b2d72f739225a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-f, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-13T16:28:19.502 INFO:journalctl@ceph.mds.f.smithi111.stdout:Sep 13 16:28:19 smithi111 podman[65160]: 2024-09-13 16:28:19.379042484 +0000 UTC m=+0.402886703 container start 19749af66c83e3b0f774b86a4cb51b62d989980924771edd504b2d72f739225a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-f, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f) 2024-09-13T16:28:19.502 INFO:journalctl@ceph.mds.f.smithi111.stdout:Sep 13 16:28:19 smithi111 ceph-mds[65179]: -- 172.21.15.111:0/819188167 <== mon.1 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5566d70f6d00 con 0x5566d70ca800 2024-09-13T16:28:19.502 INFO:journalctl@ceph.mds.f.smithi111.stdout:Sep 13 16:28:19 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-f[65175]: starting mds.f at 2024-09-13T16:28:19.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.553+0000 7ff0e746b640 1 -- 172.21.15.16:0/3068546062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0e010a5d0 con 0x7ff0e00fecd0 2024-09-13T16:28:19.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.554+0000 7ff0d67fc640 1 -- 172.21.15.16:0/3068546062 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4291 (secure 0 0 0) 0x7ff0dc099d90 con 0x7ff0e00fecd0 2024-09-13T16:28:19.555 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:28:19.555 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3292781664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3292781664},{"type":"v1","addr":"172.21.15.16:6835","nonce":3292781664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4109039027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4109039027},{"type":"v1","addr":"172.21.15.16:6837","nonce":4109039027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1984275936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1984275936},{"type":"v1","addr":"172.21.15.111:6833","nonce":1984275936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3797712690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3797712690},{"type":"v1","addr":"172.21.15.107:6833","nonce":3797712690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/338548468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":338548468},{"type":"v1","addr":"172.21.15.107:6835","nonce":338548468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2024-09-13T16:28:19.556 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 6 2024-09-13T16:28:19.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.556+0000 7ff0e746b640 1 -- 172.21.15.16:0/3068546062 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff0cc0a1090 msgr2=0x7ff0cc0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:19.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.556+0000 7ff0e746b640 1 --2- 172.21.15.16:0/3068546062 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff0cc0a1090 0x7ff0cc0a3550 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0000c00 tx=0x7ff0d00058b0 comp rx=0 tx=0).stop 2024-09-13T16:28:19.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.556+0000 7ff0e746b640 1 -- 172.21.15.16:0/3068546062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0e00fecd0 msgr2=0x7ff0e0102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:19.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.556+0000 7ff0e746b640 1 --2- 172.21.15.16:0/3068546062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0e00fecd0 0x7ff0e0102c60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc00e9f0 tx=0x7ff0dc00eeb0 comp rx=0 tx=0).stop 2024-09-13T16:28:19.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.556+0000 7ff0e746b640 1 -- 172.21.15.16:0/3068546062 shutdown_connections 2024-09-13T16:28:19.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.556+0000 7ff0e746b640 1 --2- 172.21.15.16:0/3068546062 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0e0100880 0x7ff0e01017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:19.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.556+0000 7ff0e746b640 1 --2- 172.21.15.16:0/3068546062 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff0cc0a1090 0x7ff0cc0a3550 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:19.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.556+0000 7ff0e746b640 1 --2- 172.21.15.16:0/3068546062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0e00ffec0 0x7ff0e01012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:19.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.556+0000 7ff0e746b640 1 --2- 172.21.15.16:0/3068546062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0e00fecd0 0x7ff0e0102c60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:19.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.556+0000 7ff0e746b640 1 -- 172.21.15.16:0/3068546062 >> 172.21.15.16:0/3068546062 conn(0x7ff0e00fa430 msgr2=0x7ff0e0068660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:19.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.556+0000 7ff0e746b640 1 -- 172.21.15.16:0/3068546062 shutdown_connections 2024-09-13T16:28:19.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:19.556+0000 7ff0e746b640 1 -- 172.21.15.16:0/3068546062 wait complete. 2024-09-13T16:28:19.918 INFO:journalctl@ceph.mds.f.smithi111.stdout:Sep 13 16:28:19 smithi111 bash[65160]: 19749af66c83e3b0f774b86a4cb51b62d989980924771edd504b2d72f739225a 2024-09-13T16:28:19.918 INFO:journalctl@ceph.mds.f.smithi111.stdout:Sep 13 16:28:19 smithi111 systemd[1]: Started Ceph mds.f for 0d241f6a-71ec-11ef-bceb-c7b262605968. 2024-09-13T16:28:20.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:20 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:20.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:20 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:20.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:20 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:20.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:20 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:20.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:20 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:20.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:20 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3068546062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:20.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:20 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:20.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:20 smithi107 ceph-mon[34139]: Deploying daemon mds.c on smithi111 2024-09-13T16:28:20.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:20 smithi107 ceph-mon[34139]: mds.? [v2:172.21.15.111:6834/662156111,v1:172.21.15.111:6835/662156111] up:boot 2024-09-13T16:28:20.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:20 smithi107 ceph-mon[34139]: fsmap 6 up:standby 2024-09-13T16:28:20.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:20 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-13T16:28:20.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:20 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:20.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:20 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:20.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:20 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:20.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:20 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:20.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:20 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:20.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:20 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3068546062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:20.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:20 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:20.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:20 smithi016 ceph-mon[31187]: Deploying daemon mds.c on smithi111 2024-09-13T16:28:20.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:20 smithi016 ceph-mon[31187]: mds.? [v2:172.21.15.111:6834/662156111,v1:172.21.15.111:6835/662156111] up:boot 2024-09-13T16:28:20.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:20 smithi016 ceph-mon[31187]: fsmap 6 up:standby 2024-09-13T16:28:20.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:20 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-13T16:28:20.927 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:20 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:20.927 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:20 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:20.927 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:20 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:20.927 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:20 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:20.927 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:20 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:20.927 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:20 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3068546062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:20.927 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:20 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:20.927 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:20 smithi111 ceph-mon[34600]: Deploying daemon mds.c on smithi111 2024-09-13T16:28:20.928 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:20 smithi111 ceph-mon[34600]: mds.? [v2:172.21.15.111:6834/662156111,v1:172.21.15.111:6835/662156111] up:boot 2024-09-13T16:28:20.928 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:20 smithi111 ceph-mon[34600]: fsmap 6 up:standby 2024-09-13T16:28:20.928 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:20 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-13T16:28:21.549 INFO:journalctl@ceph.mds.c.smithi111.stdout:Sep 13 16:28:21 smithi111 systemd[1]: Starting Ceph mds.c for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:28:21.569 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:28:21.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.682+0000 7fabcd890640 1 -- 172.21.15.16:0/4119549108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabc8105b20 msgr2=0x7fabc8107f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:21.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.682+0000 7fabcd890640 1 --2- 172.21.15.16:0/4119549108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabc8105b20 0x7fabc8107f10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fabc0009ca0 tx=0x7fabc002a8c0 comp rx=0 tx=0).stop 2024-09-13T16:28:21.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.682+0000 7fabcd890640 1 -- 172.21.15.16:0/4119549108 shutdown_connections 2024-09-13T16:28:21.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.682+0000 7fabcd890640 1 --2- 172.21.15.16:0/4119549108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabc8105b20 0x7fabc8107f10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:21.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.682+0000 7fabcd890640 1 --2- 172.21.15.16:0/4119549108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabc81031f0 0x7fabc81055e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:21.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.682+0000 7fabcd890640 1 --2- 172.21.15.16:0/4119549108 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabc81008c0 0x7fabc8102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:21.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.682+0000 7fabcd890640 1 -- 172.21.15.16:0/4119549108 >> 172.21.15.16:0/4119549108 conn(0x7fabc80fa440 msgr2=0x7fabc80fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:21.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.682+0000 7fabcd890640 1 -- 172.21.15.16:0/4119549108 shutdown_connections 2024-09-13T16:28:21.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.682+0000 7fabcd890640 1 -- 172.21.15.16:0/4119549108 wait complete. 2024-09-13T16:28:21.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.682+0000 7fabcd890640 1 Processor -- start 2024-09-13T16:28:21.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.683+0000 7fabcd890640 1 -- start start 2024-09-13T16:28:21.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.683+0000 7fabcd890640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabc81008c0 0x7fabc8197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:21.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.683+0000 7fabcd890640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabc81031f0 0x7fabc81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:21.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.683+0000 7fabcd890640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabc8105b20 0x7fabc819f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:21.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.683+0000 7fabcd890640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc819f990 con 0x7fabc8105b20 2024-09-13T16:28:21.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.683+0000 7fabcd890640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc819fb00 con 0x7fabc81031f0 2024-09-13T16:28:21.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.683+0000 7fabcd890640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc819fcc0 con 0x7fabc81008c0 2024-09-13T16:28:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.683+0000 7fabc77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabc8105b20 0x7fabc819f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.683+0000 7fabc77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabc8105b20 0x7fabc819f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38270/0 (socket says 172.21.15.16:38270) 2024-09-13T16:28:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.683+0000 7fabc77fe640 1 -- 172.21.15.16:0/117304640 learned_addr learned my addr 172.21.15.16:0/117304640 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.683+0000 7fabc67fc640 1 --2- 172.21.15.16:0/117304640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabc81031f0 0x7fabc81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.683+0000 7fabc6ffd640 1 --2- 172.21.15.16:0/117304640 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabc81008c0 0x7fabc8197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.683+0000 7fabc77fe640 1 -- 172.21.15.16:0/117304640 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabc81008c0 msgr2=0x7fabc8197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.683+0000 7fabc77fe640 1 --2- 172.21.15.16:0/117304640 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabc81008c0 0x7fabc8197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.683+0000 7fabc77fe640 1 -- 172.21.15.16:0/117304640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabc81031f0 msgr2=0x7fabc81983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.683+0000 7fabc77fe640 1 --2- 172.21.15.16:0/117304640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabc81031f0 0x7fabc81983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:21.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.683+0000 7fabc77fe640 1 -- 172.21.15.16:0/117304640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc0009900 con 0x7fabc8105b20 2024-09-13T16:28:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.683+0000 7fabc67fc640 1 --2- 172.21.15.16:0/117304640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabc81031f0 0x7fabc81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:28:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.683+0000 7fabc77fe640 1 --2- 172.21.15.16:0/117304640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabc8105b20 0x7fabc819f450 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fabc002aeb0 tx=0x7fabc0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.684+0000 7fabcc88e640 1 -- 172.21.15.16:0/117304640 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc00307c0 con 0x7fabc8105b20 2024-09-13T16:28:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.684+0000 7fabcc88e640 1 -- 172.21.15.16:0/117304640 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fabc0040070 con 0x7fabc8105b20 2024-09-13T16:28:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.684+0000 7fabcd890640 1 -- 172.21.15.16:0/117304640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc819ff40 con 0x7fabc8105b20 2024-09-13T16:28:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.684+0000 7fabcc88e640 1 -- 172.21.15.16:0/117304640 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc00043a0 con 0x7fabc8105b20 2024-09-13T16:28:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.684+0000 7fabcd890640 1 -- 172.21.15.16:0/117304640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc81a03c0 con 0x7fabc8105b20 2024-09-13T16:28:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.685+0000 7fabcd890640 1 -- 172.21.15.16:0/117304640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc8107610 con 0x7fabc8105b20 2024-09-13T16:28:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.686+0000 7fabcc88e640 1 -- 172.21.15.16:0/117304640 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fabc0042030 con 0x7fabc8105b20 2024-09-13T16:28:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.687+0000 7fabcc88e640 1 --2- 172.21.15.16:0/117304640 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fabb00a1090 0x7fabb00a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.687+0000 7fabc6ffd640 1 --2- 172.21.15.16:0/117304640 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fabb00a1090 0x7fabb00a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.687+0000 7fabcc88e640 1 -- 172.21.15.16:0/117304640 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fabc00ed9e0 con 0x7fabc8105b20 2024-09-13T16:28:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.687+0000 7fabc6ffd640 1 --2- 172.21.15.16:0/117304640 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fabb00a1090 0x7fabb00a3550 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fabb8009bd0 tx=0x7fabb8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:21.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.691+0000 7fabcc88e640 1 -- 172.21.15.16:0/117304640 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fabc00b7690 con 0x7fabc8105b20 2024-09-13T16:28:21.834 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:21 smithi111 ceph-mon[34600]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:21.834 INFO:journalctl@ceph.mds.c.smithi111.stdout:Sep 13 16:28:21 smithi111 podman[65466]: 2024-09-13 16:28:21.525250176 +0000 UTC m=+0.015883251 image pull 152d8bc6b19e354515e8ba109d4619445de0e09b4d4a747c2f16eb5c0edb670c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a 2024-09-13T16:28:21.834 INFO:journalctl@ceph.mds.c.smithi111.stdout:Sep 13 16:28:21 smithi111 podman[65466]: 2024-09-13 16:28:21.736909852 +0000 UTC m=+0.227542926 container create 1ae73ad6ef9689571ff8cce6fcf122db89ad545fc1007c14138642e70b9b8a93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-13T16:28:21.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:21 smithi107 ceph-mon[34139]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:21.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:21 smithi016 ceph-mon[31187]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:21.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.900+0000 7fabcd890640 1 -- 172.21.15.16:0/117304640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabc802c3e0 con 0x7fabc8105b20 2024-09-13T16:28:21.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.901+0000 7fabcc88e640 1 -- 172.21.15.16:0/117304640 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5048 (secure 0 0 0) 0x7fabc00bacb0 con 0x7fabc8105b20 2024-09-13T16:28:21.902 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:28:21.902 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3292781664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3292781664},{"type":"v1","addr":"172.21.15.16:6835","nonce":3292781664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4109039027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4109039027},{"type":"v1","addr":"172.21.15.16:6837","nonce":4109039027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1984275936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1984275936},{"type":"v1","addr":"172.21.15.111:6833","nonce":1984275936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3797712690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3797712690},{"type":"v1","addr":"172.21.15.107:6833","nonce":3797712690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/338548468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":338548468},{"type":"v1","addr":"172.21.15.107:6835","nonce":338548468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/662156111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":662156111},{"type":"v1","addr":"172.21.15.111:6835","nonce":662156111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[]} 2024-09-13T16:28:21.902 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 7 2024-09-13T16:28:21.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.903+0000 7fabcd890640 1 -- 172.21.15.16:0/117304640 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fabb00a1090 msgr2=0x7fabb00a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:21.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.903+0000 7fabcd890640 1 --2- 172.21.15.16:0/117304640 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fabb00a1090 0x7fabb00a3550 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fabb8009bd0 tx=0x7fabb8005f90 comp rx=0 tx=0).stop 2024-09-13T16:28:21.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.903+0000 7fabcd890640 1 -- 172.21.15.16:0/117304640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabc8105b20 msgr2=0x7fabc819f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:21.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.903+0000 7fabcd890640 1 --2- 172.21.15.16:0/117304640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabc8105b20 0x7fabc819f450 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fabc002aeb0 tx=0x7fabc0003c20 comp rx=0 tx=0).stop 2024-09-13T16:28:21.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.903+0000 7fabcd890640 1 -- 172.21.15.16:0/117304640 shutdown_connections 2024-09-13T16:28:21.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.903+0000 7fabcd890640 1 --2- 172.21.15.16:0/117304640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabc8105b20 0x7fabc819f450 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:21.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.903+0000 7fabcd890640 1 --2- 172.21.15.16:0/117304640 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fabb00a1090 0x7fabb00a3550 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:21.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.903+0000 7fabcd890640 1 --2- 172.21.15.16:0/117304640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabc81031f0 0x7fabc81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:21.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.903+0000 7fabcd890640 1 --2- 172.21.15.16:0/117304640 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabc81008c0 0x7fabc8197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:21.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.903+0000 7fabcd890640 1 -- 172.21.15.16:0/117304640 >> 172.21.15.16:0/117304640 conn(0x7fabc80fa440 msgr2=0x7fabc8100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:21.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.903+0000 7fabcd890640 1 -- 172.21.15.16:0/117304640 shutdown_connections 2024-09-13T16:28:21.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:21.903+0000 7fabcd890640 1 -- 172.21.15.16:0/117304640 wait complete. 2024-09-13T16:28:22.189 INFO:journalctl@ceph.mds.c.smithi111.stdout:Sep 13 16:28:21 smithi111 podman[65466]: 2024-09-13 16:28:21.948053765 +0000 UTC m=+0.438686831 container init 1ae73ad6ef9689571ff8cce6fcf122db89ad545fc1007c14138642e70b9b8a93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-13T16:28:22.189 INFO:journalctl@ceph.mds.c.smithi111.stdout:Sep 13 16:28:21 smithi111 podman[65466]: 2024-09-13 16:28:21.950881896 +0000 UTC m=+0.441514963 container start 1ae73ad6ef9689571ff8cce6fcf122db89ad545fc1007c14138642e70b9b8a93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-c, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f) 2024-09-13T16:28:22.189 INFO:journalctl@ceph.mds.c.smithi111.stdout:Sep 13 16:28:21 smithi111 ceph-mds[65485]: -- 172.21.15.111:0/746250620 <== mon.1 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5651135b8d00 con 0x56511358d000 2024-09-13T16:28:22.189 INFO:journalctl@ceph.mds.c.smithi111.stdout:Sep 13 16:28:21 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-c[65481]: starting mds.c at 2024-09-13T16:28:22.189 INFO:journalctl@ceph.mds.c.smithi111.stdout:Sep 13 16:28:22 smithi111 bash[65466]: 1ae73ad6ef9689571ff8cce6fcf122db89ad545fc1007c14138642e70b9b8a93 2024-09-13T16:28:22.189 INFO:journalctl@ceph.mds.c.smithi111.stdout:Sep 13 16:28:22 smithi111 systemd[1]: Started Ceph mds.c for 0d241f6a-71ec-11ef-bceb-c7b262605968. 2024-09-13T16:28:22.831 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:22 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/117304640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:22.831 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:22 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:22.831 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:22 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:22.831 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:22 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:22.831 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:22 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:22.831 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:22 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:22.831 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:22 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:22.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:22 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/117304640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:22.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:22 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:22.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:22 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:22.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:22 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:22.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:22 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:22.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:22 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:22.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:22 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:22.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:22 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/117304640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:22.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:22 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:22.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:22 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:22.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:22 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:22.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:22 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:22.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:22 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:22.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:22 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:23.803 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:23 smithi107 ceph-mon[34139]: Deploying daemon mds.h on smithi107 2024-09-13T16:28:23.803 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:23 smithi107 ceph-mon[34139]: mds.? [v2:172.21.15.111:6836/1877128453,v1:172.21.15.111:6837/1877128453] up:boot 2024-09-13T16:28:23.803 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:23 smithi107 ceph-mon[34139]: fsmap 7 up:standby 2024-09-13T16:28:23.803 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:23 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-13T16:28:23.803 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:23 smithi107 ceph-mon[34139]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:23.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:23 smithi016 ceph-mon[31187]: Deploying daemon mds.h on smithi107 2024-09-13T16:28:23.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:23 smithi016 ceph-mon[31187]: mds.? [v2:172.21.15.111:6836/1877128453,v1:172.21.15.111:6837/1877128453] up:boot 2024-09-13T16:28:23.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:23 smithi016 ceph-mon[31187]: fsmap 7 up:standby 2024-09-13T16:28:23.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:23 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-13T16:28:23.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:23 smithi016 ceph-mon[31187]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:23.915 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:28:23.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:23 smithi111 ceph-mon[34600]: Deploying daemon mds.h on smithi107 2024-09-13T16:28:23.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:23 smithi111 ceph-mon[34600]: mds.? [v2:172.21.15.111:6836/1877128453,v1:172.21.15.111:6837/1877128453] up:boot 2024-09-13T16:28:23.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:23 smithi111 ceph-mon[34600]: fsmap 7 up:standby 2024-09-13T16:28:23.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:23 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-13T16:28:23.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:23 smithi111 ceph-mon[34600]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:24.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.030+0000 7f9d7b8c9640 1 -- 172.21.15.16:0/818259405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d740fef80 msgr2=0x7f9d740ff3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:24.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.030+0000 7f9d7b8c9640 1 --2- 172.21.15.16:0/818259405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d740fef80 0x7f9d740ff3e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9d64009900 tx=0x7f9d6402a410 comp rx=0 tx=0).stop 2024-09-13T16:28:24.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.030+0000 7f9d7b8c9640 1 -- 172.21.15.16:0/818259405 shutdown_connections 2024-09-13T16:28:24.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.030+0000 7f9d7b8c9640 1 --2- 172.21.15.16:0/818259405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d740ff920 0x7f9d740ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:24.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.030+0000 7f9d7b8c9640 1 --2- 172.21.15.16:0/818259405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d740fef80 0x7f9d740ff3e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:24.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.030+0000 7f9d7b8c9640 1 --2- 172.21.15.16:0/818259405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d740fe640 0x7f9d740fea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:24.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.030+0000 7f9d7b8c9640 1 -- 172.21.15.16:0/818259405 >> 172.21.15.16:0/818259405 conn(0x7f9d740fa430 msgr2=0x7f9d740fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:24.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.030+0000 7f9d7b8c9640 1 -- 172.21.15.16:0/818259405 shutdown_connections 2024-09-13T16:28:24.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.030+0000 7f9d7b8c9640 1 -- 172.21.15.16:0/818259405 wait complete. 2024-09-13T16:28:24.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.031+0000 7f9d7b8c9640 1 Processor -- start 2024-09-13T16:28:24.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.031+0000 7f9d7b8c9640 1 -- start start 2024-09-13T16:28:24.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.031+0000 7f9d7b8c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d740fe640 0x7f9d74197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:24.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.031+0000 7f9d7b8c9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d740fef80 0x7f9d74198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:24.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.031+0000 7f9d7b8c9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d740ff920 0x7f9d7419f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:24.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.031+0000 7f9d7b8c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7419fa20 con 0x7f9d740fe640 2024-09-13T16:28:24.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.031+0000 7f9d7b8c9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7419fb90 con 0x7f9d740ff920 2024-09-13T16:28:24.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.031+0000 7f9d7b8c9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7419fd50 con 0x7f9d740fef80 2024-09-13T16:28:24.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.031+0000 7f9d79665640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d740fe640 0x7f9d74197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:24.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.031+0000 7f9d79665640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d740fe640 0x7f9d74197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57530/0 (socket says 172.21.15.16:57530) 2024-09-13T16:28:24.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.031+0000 7f9d79665640 1 -- 172.21.15.16:0/1056072124 learned_addr learned my addr 172.21.15.16:0/1056072124 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:24.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.031+0000 7f9d79e66640 1 --2- 172.21.15.16:0/1056072124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d740ff920 0x7f9d7419f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:24.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.032+0000 7f9d78e64640 1 --2- 172.21.15.16:0/1056072124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d740fef80 0x7f9d74198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:24.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.032+0000 7f9d79665640 1 -- 172.21.15.16:0/1056072124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d740fef80 msgr2=0x7f9d74198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:24.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.032+0000 7f9d79665640 1 --2- 172.21.15.16:0/1056072124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d740fef80 0x7f9d74198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:24.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.032+0000 7f9d79665640 1 -- 172.21.15.16:0/1056072124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d740ff920 msgr2=0x7f9d7419f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:24.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.032+0000 7f9d79665640 1 --2- 172.21.15.16:0/1056072124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d740ff920 0x7f9d7419f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:24.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.032+0000 7f9d79665640 1 -- 172.21.15.16:0/1056072124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d64009560 con 0x7f9d740fe640 2024-09-13T16:28:24.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.032+0000 7f9d78e64640 1 --2- 172.21.15.16:0/1056072124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d740fef80 0x7f9d74198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:28:24.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.032+0000 7f9d79665640 1 --2- 172.21.15.16:0/1056072124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d740fe640 0x7f9d74197f30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9d7000d770 tx=0x7f9d7000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:24.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.032+0000 7f9d6a7fc640 1 -- 172.21.15.16:0/1056072124 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d70009c30 con 0x7f9d740fe640 2024-09-13T16:28:24.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.032+0000 7f9d6a7fc640 1 -- 172.21.15.16:0/1056072124 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d7000de80 con 0x7f9d740fe640 2024-09-13T16:28:24.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.032+0000 7f9d7b8c9640 1 -- 172.21.15.16:0/1056072124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d7419ffd0 con 0x7f9d740fe640 2024-09-13T16:28:24.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.032+0000 7f9d6a7fc640 1 -- 172.21.15.16:0/1056072124 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d70004d40 con 0x7f9d740fe640 2024-09-13T16:28:24.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.032+0000 7f9d7b8c9640 1 -- 172.21.15.16:0/1056072124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d741a0450 con 0x7f9d740fe640 2024-09-13T16:28:24.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.033+0000 7f9d7b8c9640 1 -- 172.21.15.16:0/1056072124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d741096a0 con 0x7f9d740fe640 2024-09-13T16:28:24.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.034+0000 7f9d6a7fc640 1 -- 172.21.15.16:0/1056072124 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f9d70004ea0 con 0x7f9d740fe640 2024-09-13T16:28:24.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.035+0000 7f9d6a7fc640 1 --2- 172.21.15.16:0/1056072124 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f9d600a1110 0x7f9d600a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:24.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.035+0000 7f9d6a7fc640 1 -- 172.21.15.16:0/1056072124 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9d700cbe20 con 0x7f9d740fe640 2024-09-13T16:28:24.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.035+0000 7f9d78e64640 1 --2- 172.21.15.16:0/1056072124 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f9d600a1110 0x7f9d600a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:24.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.035+0000 7f9d78e64640 1 --2- 172.21.15.16:0/1056072124 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f9d600a1110 0x7f9d600a35d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9d64005ab0 tx=0x7f9d64005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:24.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.039+0000 7f9d6a7fc640 1 -- 172.21.15.16:0/1056072124 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f9d70095a50 con 0x7f9d740fe640 2024-09-13T16:28:24.099 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 13 16:28:23 smithi107 systemd[1]: Starting Ceph mds.h for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:28:24.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.250+0000 7f9d7b8c9640 1 -- 172.21.15.16:0/1056072124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d7410d740 con 0x7f9d740fe640 2024-09-13T16:28:24.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.251+0000 7f9d6a7fc640 1 -- 172.21.15.16:0/1056072124 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5808 (secure 0 0 0) 0x7f9d70099070 con 0x7f9d740fe640 2024-09-13T16:28:24.252 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:28:24.252 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3292781664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3292781664},{"type":"v1","addr":"172.21.15.16:6835","nonce":3292781664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4109039027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4109039027},{"type":"v1","addr":"172.21.15.16:6837","nonce":4109039027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1984275936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1984275936},{"type":"v1","addr":"172.21.15.111:6833","nonce":1984275936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3797712690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3797712690},{"type":"v1","addr":"172.21.15.107:6833","nonce":3797712690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/338548468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":338548468},{"type":"v1","addr":"172.21.15.107:6835","nonce":338548468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/662156111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":662156111},{"type":"v1","addr":"172.21.15.111:6835","nonce":662156111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/1877128453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1877128453},{"type":"v1","addr":"172.21.15.111:6837","nonce":1877128453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2024-09-13T16:28:24.252 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-09-13T16:28:24.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.253+0000 7f9d7b8c9640 1 -- 172.21.15.16:0/1056072124 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f9d600a1110 msgr2=0x7f9d600a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:24.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.253+0000 7f9d7b8c9640 1 --2- 172.21.15.16:0/1056072124 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f9d600a1110 0x7f9d600a35d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9d64005ab0 tx=0x7f9d64005a20 comp rx=0 tx=0).stop 2024-09-13T16:28:24.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.253+0000 7f9d7b8c9640 1 -- 172.21.15.16:0/1056072124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d740fe640 msgr2=0x7f9d74197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:24.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.253+0000 7f9d7b8c9640 1 --2- 172.21.15.16:0/1056072124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d740fe640 0x7f9d74197f30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9d7000d770 tx=0x7f9d7000dc30 comp rx=0 tx=0).stop 2024-09-13T16:28:24.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.253+0000 7f9d7b8c9640 1 -- 172.21.15.16:0/1056072124 shutdown_connections 2024-09-13T16:28:24.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.253+0000 7f9d7b8c9640 1 --2- 172.21.15.16:0/1056072124 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f9d600a1110 0x7f9d600a35d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:24.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.253+0000 7f9d7b8c9640 1 --2- 172.21.15.16:0/1056072124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d740ff920 0x7f9d7419f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:24.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.253+0000 7f9d7b8c9640 1 --2- 172.21.15.16:0/1056072124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d740fef80 0x7f9d74198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:24.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.253+0000 7f9d7b8c9640 1 --2- 172.21.15.16:0/1056072124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d740fe640 0x7f9d74197f30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:24.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.253+0000 7f9d7b8c9640 1 -- 172.21.15.16:0/1056072124 >> 172.21.15.16:0/1056072124 conn(0x7f9d740fa430 msgr2=0x7f9d740fbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:24.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.253+0000 7f9d7b8c9640 1 -- 172.21.15.16:0/1056072124 shutdown_connections 2024-09-13T16:28:24.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:24.253+0000 7f9d7b8c9640 1 -- 172.21.15.16:0/1056072124 wait complete. 2024-09-13T16:28:24.490 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 13 16:28:24 smithi107 podman[63162]: 2024-09-13 16:28:24.113412934 +0000 UTC m=+0.015950582 image pull 152d8bc6b19e354515e8ba109d4619445de0e09b4d4a747c2f16eb5c0edb670c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a 2024-09-13T16:28:24.490 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 13 16:28:24 smithi107 podman[63162]: 2024-09-13 16:28:24.236659443 +0000 UTC m=+0.139197185 container create db4523075397bd49374053864aa87e9b11d837f77c95ec8207d82f9fc537d013 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-h, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=) 2024-09-13T16:28:24.491 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 13 16:28:24 smithi107 podman[63162]: 2024-09-13 16:28:24.489527362 +0000 UTC m=+0.392064998 container init db4523075397bd49374053864aa87e9b11d837f77c95ec8207d82f9fc537d013 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-h, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T16:28:24.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:24 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1056072124' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:24.849 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 13 16:28:24 smithi107 podman[63162]: 2024-09-13 16:28:24.492474854 +0000 UTC m=+0.395012498 container start db4523075397bd49374053864aa87e9b11d837f77c95ec8207d82f9fc537d013 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T16:28:24.849 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 13 16:28:24 smithi107 ceph-mds[63181]: -- 172.21.15.107:0/167769009 <== mon.2 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559f5ff0cd00 con 0x559f5fee0800 2024-09-13T16:28:24.849 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 13 16:28:24 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-h[63177]: starting mds.h at 2024-09-13T16:28:24.850 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 13 16:28:24 smithi107 bash[63162]: db4523075397bd49374053864aa87e9b11d837f77c95ec8207d82f9fc537d013 2024-09-13T16:28:24.850 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 13 16:28:24 smithi107 systemd[1]: Started Ceph mds.h for 0d241f6a-71ec-11ef-bceb-c7b262605968. 2024-09-13T16:28:24.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:24 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1056072124' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:24.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:24 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1056072124' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:25.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:25 smithi107 ceph-mon[34139]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:25.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:25 smithi107 ceph-mon[34139]: mds.? [v2:172.21.15.107:6836/2838457611,v1:172.21.15.107:6837/2838457611] up:boot 2024-09-13T16:28:25.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:25 smithi107 ceph-mon[34139]: fsmap 8 up:standby 2024-09-13T16:28:25.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:25 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-13T16:28:25.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:25 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:25.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:25 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:25.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:25 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:25.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:25 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:25.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:25 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:25.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:25 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:25.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:25 smithi016 ceph-mon[31187]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:25.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:25 smithi016 ceph-mon[31187]: mds.? [v2:172.21.15.107:6836/2838457611,v1:172.21.15.107:6837/2838457611] up:boot 2024-09-13T16:28:25.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:25 smithi016 ceph-mon[31187]: fsmap 8 up:standby 2024-09-13T16:28:25.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:25 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-13T16:28:25.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:25 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:25.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:25 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:25.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:25 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:25.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:25 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:25.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:25 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:25.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:25 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:25.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:25 smithi111 ceph-mon[34600]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:25.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:25 smithi111 ceph-mon[34600]: mds.? [v2:172.21.15.107:6836/2838457611,v1:172.21.15.107:6837/2838457611] up:boot 2024-09-13T16:28:25.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:25 smithi111 ceph-mon[34600]: fsmap 8 up:standby 2024-09-13T16:28:25.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:25 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-13T16:28:25.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:25 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:25.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:25 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:25.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:25 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:25.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:25 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:25.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:25 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:25.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:25 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:26.268 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:28:26.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.382+0000 7f3df2d53640 1 -- 172.21.15.16:0/1118489286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dec0fecd0 msgr2=0x7f3dec0ff0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:26.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.382+0000 7f3df2d53640 1 --2- 172.21.15.16:0/1118489286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dec0fecd0 0x7f3dec0ff0d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4009900 tx=0x7f3dd402a410 comp rx=0 tx=0).stop 2024-09-13T16:28:26.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.382+0000 7f3df2d53640 1 -- 172.21.15.16:0/1118489286 shutdown_connections 2024-09-13T16:28:26.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.382+0000 7f3df2d53640 1 --2- 172.21.15.16:0/1118489286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dec100880 0x7f3dec107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:26.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.382+0000 7f3df2d53640 1 --2- 172.21.15.16:0/1118489286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3dec0ffec0 0x7f3dec100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:26.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.382+0000 7f3df2d53640 1 --2- 172.21.15.16:0/1118489286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dec0fecd0 0x7f3dec0ff0d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:26.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.382+0000 7f3df2d53640 1 -- 172.21.15.16:0/1118489286 >> 172.21.15.16:0/1118489286 conn(0x7f3dec0fa430 msgr2=0x7f3dec0fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:26.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.382+0000 7f3df2d53640 1 -- 172.21.15.16:0/1118489286 shutdown_connections 2024-09-13T16:28:26.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.382+0000 7f3df2d53640 1 -- 172.21.15.16:0/1118489286 wait complete. 2024-09-13T16:28:26.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.383+0000 7f3df2d53640 1 Processor -- start 2024-09-13T16:28:26.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.383+0000 7f3df2d53640 1 -- start start 2024-09-13T16:28:26.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.383+0000 7f3df2d53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dec0fecd0 0x7f3dec194bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:26.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.383+0000 7f3df2d53640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3dec0ffec0 0x7f3dec195130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:26.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.383+0000 7f3df2d53640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dec100880 0x7f3dec19c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:26.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.383+0000 7f3df2d53640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec19c6e0 con 0x7f3dec0fecd0 2024-09-13T16:28:26.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.383+0000 7f3df2d53640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec19c850 con 0x7f3dec100880 2024-09-13T16:28:26.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.383+0000 7f3df2d53640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec19ca10 con 0x7f3dec0ffec0 2024-09-13T16:28:26.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.383+0000 7f3df0aef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dec0fecd0 0x7f3dec194bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:26.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.383+0000 7f3df0aef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dec0fecd0 0x7f3dec194bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57556/0 (socket says 172.21.15.16:57556) 2024-09-13T16:28:26.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.383+0000 7f3df0aef640 1 -- 172.21.15.16:0/3739301948 learned_addr learned my addr 172.21.15.16:0/3739301948 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:26.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.383+0000 7f3debfff640 1 --2- 172.21.15.16:0/3739301948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3dec0ffec0 0x7f3dec195130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:26.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.383+0000 7f3df12f0640 1 --2- 172.21.15.16:0/3739301948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dec100880 0x7f3dec19c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:26.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.384+0000 7f3df0aef640 1 -- 172.21.15.16:0/3739301948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3dec0ffec0 msgr2=0x7f3dec195130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:26.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.384+0000 7f3df0aef640 1 --2- 172.21.15.16:0/3739301948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3dec0ffec0 0x7f3dec195130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:26.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.384+0000 7f3df0aef640 1 -- 172.21.15.16:0/3739301948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dec100880 msgr2=0x7f3dec19c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:26.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.384+0000 7f3df0aef640 1 --2- 172.21.15.16:0/3739301948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dec100880 0x7f3dec19c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:26.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.384+0000 7f3df0aef640 1 -- 172.21.15.16:0/3739301948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd4009560 con 0x7f3dec0fecd0 2024-09-13T16:28:26.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.384+0000 7f3df0aef640 1 --2- 172.21.15.16:0/3739301948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dec0fecd0 0x7f3dec194bf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4005ab0 tx=0x7f3dd40052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:26.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.384+0000 7f3de9ffb640 1 -- 172.21.15.16:0/3739301948 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4031d70 con 0x7f3dec0fecd0 2024-09-13T16:28:26.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.384+0000 7f3de9ffb640 1 -- 172.21.15.16:0/3739301948 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3dd40044d0 con 0x7f3dec0fecd0 2024-09-13T16:28:26.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.384+0000 7f3de9ffb640 1 -- 172.21.15.16:0/3739301948 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4003970 con 0x7f3dec0fecd0 2024-09-13T16:28:26.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.384+0000 7f3df2d53640 1 -- 172.21.15.16:0/3739301948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dec19cc90 con 0x7f3dec0fecd0 2024-09-13T16:28:26.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.384+0000 7f3df2d53640 1 -- 172.21.15.16:0/3739301948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dec19d140 con 0x7f3dec0fecd0 2024-09-13T16:28:26.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.385+0000 7f3df2d53640 1 -- 172.21.15.16:0/3739301948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dec1062a0 con 0x7f3dec0fecd0 2024-09-13T16:28:26.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.387+0000 7f3de9ffb640 1 -- 172.21.15.16:0/3739301948 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f3dd4003ad0 con 0x7f3dec0fecd0 2024-09-13T16:28:26.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.387+0000 7f3de9ffb640 1 --2- 172.21.15.16:0/3739301948 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f3dd80a1110 0x7f3dd80a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:26.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.387+0000 7f3de9ffb640 1 -- 172.21.15.16:0/3739301948 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3dd40ed5e0 con 0x7f3dec0fecd0 2024-09-13T16:28:26.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.387+0000 7f3debfff640 1 --2- 172.21.15.16:0/3739301948 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f3dd80a1110 0x7f3dd80a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:26.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.388+0000 7f3debfff640 1 --2- 172.21.15.16:0/3739301948 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f3dd80a1110 0x7f3dd80a35d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3dec196100 tx=0x7f3ddc009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:26.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.390+0000 7f3de9ffb640 1 -- 172.21.15.16:0/3739301948 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f3dd40b72c0 con 0x7f3dec0fecd0 2024-09-13T16:28:26.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.601+0000 7f3df2d53640 1 -- 172.21.15.16:0/3739301948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3dec10a420 con 0x7f3dec0fecd0 2024-09-13T16:28:26.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.601+0000 7f3de9ffb640 1 -- 172.21.15.16:0/3739301948 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6568 (secure 0 0 0) 0x7f3dd40ba8e0 con 0x7f3dec0fecd0 2024-09-13T16:28:26.603 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:28:26.603 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3292781664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3292781664},{"type":"v1","addr":"172.21.15.16:6835","nonce":3292781664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4109039027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4109039027},{"type":"v1","addr":"172.21.15.16:6837","nonce":4109039027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1984275936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1984275936},{"type":"v1","addr":"172.21.15.111:6833","nonce":1984275936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3797712690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3797712690},{"type":"v1","addr":"172.21.15.107:6833","nonce":3797712690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/338548468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":338548468},{"type":"v1","addr":"172.21.15.107:6835","nonce":338548468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/662156111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":662156111},{"type":"v1","addr":"172.21.15.111:6835","nonce":662156111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/1877128453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1877128453},{"type":"v1","addr":"172.21.15.111:6837","nonce":1877128453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2838457611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2838457611},{"type":"v1","addr":"172.21.15.107:6837","nonce":2838457611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2024-09-13T16:28:26.603 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2024-09-13T16:28:26.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.604+0000 7f3df2d53640 1 -- 172.21.15.16:0/3739301948 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f3dd80a1110 msgr2=0x7f3dd80a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:26.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.604+0000 7f3df2d53640 1 --2- 172.21.15.16:0/3739301948 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f3dd80a1110 0x7f3dd80a35d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3dec196100 tx=0x7f3ddc009500 comp rx=0 tx=0).stop 2024-09-13T16:28:26.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.604+0000 7f3df2d53640 1 -- 172.21.15.16:0/3739301948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dec0fecd0 msgr2=0x7f3dec194bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:26.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.604+0000 7f3df2d53640 1 --2- 172.21.15.16:0/3739301948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dec0fecd0 0x7f3dec194bf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4005ab0 tx=0x7f3dd40052b0 comp rx=0 tx=0).stop 2024-09-13T16:28:26.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.604+0000 7f3df2d53640 1 -- 172.21.15.16:0/3739301948 shutdown_connections 2024-09-13T16:28:26.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.604+0000 7f3df2d53640 1 --2- 172.21.15.16:0/3739301948 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f3dd80a1110 0x7f3dd80a35d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:26.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.604+0000 7f3df2d53640 1 --2- 172.21.15.16:0/3739301948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dec100880 0x7f3dec19c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:26.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.604+0000 7f3df2d53640 1 --2- 172.21.15.16:0/3739301948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3dec0ffec0 0x7f3dec195130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:26.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.604+0000 7f3df2d53640 1 --2- 172.21.15.16:0/3739301948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dec0fecd0 0x7f3dec194bf0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:26.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.604+0000 7f3df2d53640 1 -- 172.21.15.16:0/3739301948 >> 172.21.15.16:0/3739301948 conn(0x7f3dec0fa430 msgr2=0x7f3dec103470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:26.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.604+0000 7f3df2d53640 1 -- 172.21.15.16:0/3739301948 shutdown_connections 2024-09-13T16:28:26.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:26.604+0000 7f3df2d53640 1 -- 172.21.15.16:0/3739301948 wait complete. 2024-09-13T16:28:26.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:26 smithi111 ceph-mon[34600]: Deploying daemon mds.i on smithi111 2024-09-13T16:28:26.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:26 smithi107 ceph-mon[34139]: Deploying daemon mds.i on smithi111 2024-09-13T16:28:26.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:26 smithi016 ceph-mon[31187]: Deploying daemon mds.i on smithi111 2024-09-13T16:28:27.023 INFO:journalctl@ceph.mds.i.smithi111.stdout:Sep 13 16:28:27 smithi111 podman[65770]: 2024-09-13 16:28:27.019257985 +0000 UTC m=+0.362083955 container init d958eadab8860f3ca941aed13e916a331b502ce8e63041a17d80f3566d19d196 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-i, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True) 2024-09-13T16:28:27.023 INFO:journalctl@ceph.mds.i.smithi111.stdout:Sep 13 16:28:27 smithi111 podman[65770]: 2024-09-13 16:28:27.021964324 +0000 UTC m=+0.364790294 container start d958eadab8860f3ca941aed13e916a331b502ce8e63041a17d80f3566d19d196 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_BRANCH=HEAD) 2024-09-13T16:28:27.437 INFO:journalctl@ceph.mds.i.smithi111.stdout:Sep 13 16:28:27 smithi111 ceph-mds[65789]: -- 172.21.15.111:0/1289010688 <== mon.1 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559af7f36d00 con 0x559af7f0b000 2024-09-13T16:28:27.437 INFO:journalctl@ceph.mds.i.smithi111.stdout:Sep 13 16:28:27 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-i[65785]: starting mds.i at 2024-09-13T16:28:27.438 INFO:journalctl@ceph.mds.i.smithi111.stdout:Sep 13 16:28:27 smithi111 bash[65770]: d958eadab8860f3ca941aed13e916a331b502ce8e63041a17d80f3566d19d196 2024-09-13T16:28:27.438 INFO:journalctl@ceph.mds.i.smithi111.stdout:Sep 13 16:28:27 smithi111 systemd[1]: Started Ceph mds.i for 0d241f6a-71ec-11ef-bceb-c7b262605968. 2024-09-13T16:28:27.803 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:27 smithi016 ceph-mon[31187]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:27.804 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:27 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3739301948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:27.804 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:27 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:27.804 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:27 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:27.804 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:27 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:27.804 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:27 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:27.804 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:27 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:27.804 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:27 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:27.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:27 smithi107 ceph-mon[34139]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:27.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:27 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3739301948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:27.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:27 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:27.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:27 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:27.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:27 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:27.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:27 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:27.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:27 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:27.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:27 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:27.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:27 smithi111 ceph-mon[34600]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:27.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:27 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3739301948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:27.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:27 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:27.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:27 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:27.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:27 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:27.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:27 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:27.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:27 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:27.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:27 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:28.616 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:28:28.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.735+0000 7fd7ca395640 1 -- 172.21.15.16:0/1224624299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7c40ff8f0 msgr2=0x7fd7c40ffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:28.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.735+0000 7fd7ca395640 1 --2- 172.21.15.16:0/1224624299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7c40ff8f0 0x7fd7c40ffda0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc009ca0 tx=0x7fd7bc02a8c0 comp rx=0 tx=0).stop 2024-09-13T16:28:28.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.735+0000 7fd7ca395640 1 -- 172.21.15.16:0/1224624299 shutdown_connections 2024-09-13T16:28:28.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.735+0000 7fd7ca395640 1 --2- 172.21.15.16:0/1224624299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7c40ff8f0 0x7fd7c40ffda0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:28.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.735+0000 7fd7ca395640 1 --2- 172.21.15.16:0/1224624299 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7c40fef50 0x7fd7c40ff3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:28.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.735+0000 7fd7ca395640 1 --2- 172.21.15.16:0/1224624299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7c40fe610 0x7fd7c40fea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:28.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.735+0000 7fd7ca395640 1 -- 172.21.15.16:0/1224624299 >> 172.21.15.16:0/1224624299 conn(0x7fd7c40fa440 msgr2=0x7fd7c40fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:28.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.735+0000 7fd7ca395640 1 -- 172.21.15.16:0/1224624299 shutdown_connections 2024-09-13T16:28:28.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.735+0000 7fd7ca395640 1 -- 172.21.15.16:0/1224624299 wait complete. 2024-09-13T16:28:28.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.736+0000 7fd7ca395640 1 Processor -- start 2024-09-13T16:28:28.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.736+0000 7fd7ca395640 1 -- start start 2024-09-13T16:28:28.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.736+0000 7fd7ca395640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7c40fe610 0x7fd7c4197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:28.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.736+0000 7fd7ca395640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7c40fef50 0x7fd7c4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:28.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.736+0000 7fd7ca395640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7c40ff8f0 0x7fd7c419f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:28.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.736+0000 7fd7ca395640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c419f9c0 con 0x7fd7c40ff8f0 2024-09-13T16:28:28.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.736+0000 7fd7ca395640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c419fb30 con 0x7fd7c40fef50 2024-09-13T16:28:28.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.736+0000 7fd7ca395640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c419fcf0 con 0x7fd7c40fe610 2024-09-13T16:28:28.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.736+0000 7fd7c8932640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7c40ff8f0 0x7fd7c419f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:28.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.736+0000 7fd7c8932640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7c40ff8f0 0x7fd7c419f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57572/0 (socket says 172.21.15.16:57572) 2024-09-13T16:28:28.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.736+0000 7fd7c8932640 1 -- 172.21.15.16:0/3703388063 learned_addr learned my addr 172.21.15.16:0/3703388063 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:28.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.736+0000 7fd7c3fff640 1 --2- 172.21.15.16:0/3703388063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7c40fe610 0x7fd7c4197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.736+0000 7fd7bbfff640 1 --2- 172.21.15.16:0/3703388063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7c40fef50 0x7fd7c4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.737+0000 7fd7c3fff640 1 -- 172.21.15.16:0/3703388063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7c40fef50 msgr2=0x7fd7c4198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.737+0000 7fd7c3fff640 1 --2- 172.21.15.16:0/3703388063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7c40fef50 0x7fd7c4198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.737+0000 7fd7c3fff640 1 -- 172.21.15.16:0/3703388063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7c40ff8f0 msgr2=0x7fd7c419f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.737+0000 7fd7c3fff640 1 --2- 172.21.15.16:0/3703388063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7c40ff8f0 0x7fd7c419f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.737+0000 7fd7c3fff640 1 -- 172.21.15.16:0/3703388063 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7bc009900 con 0x7fd7c40fe610 2024-09-13T16:28:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.737+0000 7fd7c3fff640 1 --2- 172.21.15.16:0/3703388063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7c40fe610 0x7fd7c4197ed0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400d6d0 tx=0x7fd7b400db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.737+0000 7fd7c1ffb640 1 -- 172.21.15.16:0/3703388063 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4005220 con 0x7fd7c40fe610 2024-09-13T16:28:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.738+0000 7fd7ca395640 1 -- 172.21.15.16:0/3703388063 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c419ffd0 con 0x7fd7c40fe610 2024-09-13T16:28:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.738+0000 7fd7ca395640 1 -- 172.21.15.16:0/3703388063 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c41a0400 con 0x7fd7c40fe610 2024-09-13T16:28:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.738+0000 7fd7c1ffb640 1 -- 172.21.15.16:0/3703388063 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7b4009cb0 con 0x7fd7c40fe610 2024-09-13T16:28:28.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.738+0000 7fd7c1ffb640 1 -- 172.21.15.16:0/3703388063 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4004c60 con 0x7fd7c40fe610 2024-09-13T16:28:28.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.740+0000 7fd7ca395640 1 -- 172.21.15.16:0/3703388063 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c4109360 con 0x7fd7c40fe610 2024-09-13T16:28:28.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.749+0000 7fd7c1ffb640 1 -- 172.21.15.16:0/3703388063 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd7b4009820 con 0x7fd7c40fe610 2024-09-13T16:28:28.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.750+0000 7fd7c1ffb640 1 --2- 172.21.15.16:0/3703388063 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd7b00a1190 0x7fd7b00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:28.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.750+0000 7fd7c1ffb640 1 -- 172.21.15.16:0/3703388063 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd7b40cbeb0 con 0x7fd7c40fe610 2024-09-13T16:28:28.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.750+0000 7fd7bbfff640 1 --2- 172.21.15.16:0/3703388063 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd7b00a1190 0x7fd7b00a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:28.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.750+0000 7fd7bbfff640 1 --2- 172.21.15.16:0/3703388063 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd7b00a1190 0x7fd7b00a3650 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd7c41993e0 tx=0x7fd7ac00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:28.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.754+0000 7fd7c1ffb640 1 -- 172.21.15.16:0/3703388063 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd7b4095af0 con 0x7fd7c40fe610 2024-09-13T16:28:28.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:28 smithi107 ceph-mon[34139]: Deploying daemon mds.g on smithi016 2024-09-13T16:28:28.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:28 smithi107 ceph-mon[34139]: mds.? [v2:172.21.15.111:6838/1423842285,v1:172.21.15.111:6839/1423842285] up:boot 2024-09-13T16:28:28.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:28 smithi107 ceph-mon[34139]: fsmap 9 up:standby 2024-09-13T16:28:28.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:28 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-13T16:28:28.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:28 smithi016 ceph-mon[31187]: Deploying daemon mds.g on smithi016 2024-09-13T16:28:28.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:28 smithi016 ceph-mon[31187]: mds.? [v2:172.21.15.111:6838/1423842285,v1:172.21.15.111:6839/1423842285] up:boot 2024-09-13T16:28:28.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:28 smithi016 ceph-mon[31187]: fsmap 9 up:standby 2024-09-13T16:28:28.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:28 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-13T16:28:28.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:28 smithi111 ceph-mon[34600]: Deploying daemon mds.g on smithi016 2024-09-13T16:28:28.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:28 smithi111 ceph-mon[34600]: mds.? [v2:172.21.15.111:6838/1423842285,v1:172.21.15.111:6839/1423842285] up:boot 2024-09-13T16:28:28.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:28 smithi111 ceph-mon[34600]: fsmap 9 up:standby 2024-09-13T16:28:28.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:28 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-13T16:28:28.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.977+0000 7fd7ca395640 1 -- 172.21.15.16:0/3703388063 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7c402c410 con 0x7fd7c40fe610 2024-09-13T16:28:28.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.979+0000 7fd7c1ffb640 1 -- 172.21.15.16:0/3703388063 <== mon.1 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7330 (secure 0 0 0) 0x7fd7b4099110 con 0x7fd7c40fe610 2024-09-13T16:28:28.980 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:28:28.980 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3292781664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3292781664},{"type":"v1","addr":"172.21.15.16:6835","nonce":3292781664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4109039027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4109039027},{"type":"v1","addr":"172.21.15.16:6837","nonce":4109039027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1984275936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1984275936},{"type":"v1","addr":"172.21.15.111:6833","nonce":1984275936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3797712690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3797712690},{"type":"v1","addr":"172.21.15.107:6833","nonce":3797712690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/338548468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":338548468},{"type":"v1","addr":"172.21.15.107:6835","nonce":338548468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/662156111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":662156111},{"type":"v1","addr":"172.21.15.111:6835","nonce":662156111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/1877128453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1877128453},{"type":"v1","addr":"172.21.15.111:6837","nonce":1877128453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2838457611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2838457611},{"type":"v1","addr":"172.21.15.107:6837","nonce":2838457611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6839/1423842285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":1423842285},{"type":"v1","addr":"172.21.15.111:6839","nonce":1423842285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-09-13T16:28:28.980 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 10 2024-09-13T16:28:28.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.981+0000 7fd7ca395640 1 -- 172.21.15.16:0/3703388063 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd7b00a1190 msgr2=0x7fd7b00a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:28.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.981+0000 7fd7ca395640 1 --2- 172.21.15.16:0/3703388063 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd7b00a1190 0x7fd7b00a3650 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd7c41993e0 tx=0x7fd7ac00b390 comp rx=0 tx=0).stop 2024-09-13T16:28:28.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.981+0000 7fd7ca395640 1 -- 172.21.15.16:0/3703388063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7c40fe610 msgr2=0x7fd7c4197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:28.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.981+0000 7fd7ca395640 1 --2- 172.21.15.16:0/3703388063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7c40fe610 0x7fd7c4197ed0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400d6d0 tx=0x7fd7b400db90 comp rx=0 tx=0).stop 2024-09-13T16:28:28.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.981+0000 7fd7ca395640 1 -- 172.21.15.16:0/3703388063 shutdown_connections 2024-09-13T16:28:28.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.981+0000 7fd7ca395640 1 --2- 172.21.15.16:0/3703388063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7c40ff8f0 0x7fd7c419f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:28.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.981+0000 7fd7ca395640 1 --2- 172.21.15.16:0/3703388063 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd7b00a1190 0x7fd7b00a3650 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:28.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.981+0000 7fd7ca395640 1 --2- 172.21.15.16:0/3703388063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7c40fef50 0x7fd7c4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:28.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.981+0000 7fd7ca395640 1 --2- 172.21.15.16:0/3703388063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7c40fe610 0x7fd7c4197ed0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:28.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.981+0000 7fd7ca395640 1 -- 172.21.15.16:0/3703388063 >> 172.21.15.16:0/3703388063 conn(0x7fd7c40fa440 msgr2=0x7fd7c40fbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:28.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.981+0000 7fd7ca395640 1 -- 172.21.15.16:0/3703388063 shutdown_connections 2024-09-13T16:28:28.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:28.981+0000 7fd7ca395640 1 -- 172.21.15.16:0/3703388063 wait complete. 2024-09-13T16:28:29.315 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 13 16:28:29 smithi016 systemd[1]: Starting Ceph mds.g for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:28:29.582 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 13 16:28:29 smithi016 podman[72444]: 2024-09-13 16:28:29.217495632 +0000 UTC m=+0.023930153 image pull 152d8bc6b19e354515e8ba109d4619445de0e09b4d4a747c2f16eb5c0edb670c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a 2024-09-13T16:28:29.582 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 13 16:28:29 smithi016 podman[72444]: 2024-09-13 16:28:29.343366957 +0000 UTC m=+0.149801465 container create 8d4e0e08d9fab43c017a9e36c63899dbd7aef9c5bbf1fee8e442896c261d927d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T16:28:29.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:29 smithi107 ceph-mon[34139]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:29.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:29 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3703388063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:29.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:29 smithi016 ceph-mon[31187]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:29.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:29 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3703388063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:29.854 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 13 16:28:29 smithi016 podman[72444]: 2024-09-13 16:28:29.61307367 +0000 UTC m=+0.419508190 container init 8d4e0e08d9fab43c017a9e36c63899dbd7aef9c5bbf1fee8e442896c261d927d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-g, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T16:28:29.855 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 13 16:28:29 smithi016 podman[72444]: 2024-09-13 16:28:29.616463817 +0000 UTC m=+0.422898328 container start 8d4e0e08d9fab43c017a9e36c63899dbd7aef9c5bbf1fee8e442896c261d927d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-g, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-13T16:28:29.855 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 13 16:28:29 smithi016 ceph-mds[72471]: -- 172.21.15.16:0/837545506 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5558eb6ded00 con 0x5558eb6b2800 2024-09-13T16:28:29.855 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 13 16:28:29 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-g[72459]: starting mds.g at 2024-09-13T16:28:29.855 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 13 16:28:29 smithi016 bash[72444]: 8d4e0e08d9fab43c017a9e36c63899dbd7aef9c5bbf1fee8e442896c261d927d 2024-09-13T16:28:29.855 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 13 16:28:29 smithi016 systemd[1]: Started Ceph mds.g for 0d241f6a-71ec-11ef-bceb-c7b262605968. 2024-09-13T16:28:29.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:29 smithi111 ceph-mon[34600]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:29.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:29 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3703388063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:30.994 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:28:31.021 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:30 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:31.021 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:30 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:31.021 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:30 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:31.021 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:30 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:31.021 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:30 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:31.022 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:30 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:31.022 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:30 smithi016 ceph-mon[31187]: Deploying daemon mds.a on smithi016 2024-09-13T16:28:31.022 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:30 smithi016 ceph-mon[31187]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:31.022 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:30 smithi016 ceph-mon[31187]: mds.? [v2:172.21.15.16:6838/2391618693,v1:172.21.15.16:6839/2391618693] up:boot 2024-09-13T16:28:31.022 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:30 smithi016 ceph-mon[31187]: fsmap 10 up:standby 2024-09-13T16:28:31.022 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:30 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-13T16:28:31.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:30 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:31.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:30 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:31.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:30 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:31.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:30 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:31.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:30 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:31.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:30 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:31.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:30 smithi107 ceph-mon[34139]: Deploying daemon mds.a on smithi016 2024-09-13T16:28:31.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:30 smithi107 ceph-mon[34139]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:31.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:30 smithi107 ceph-mon[34139]: mds.? [v2:172.21.15.16:6838/2391618693,v1:172.21.15.16:6839/2391618693] up:boot 2024-09-13T16:28:31.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:30 smithi107 ceph-mon[34139]: fsmap 10 up:standby 2024-09-13T16:28:31.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:30 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-13T16:28:31.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.111+0000 7f11a3165640 1 -- 172.21.15.16:0/1096943581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f119c0ffe90 msgr2=0x7f119c100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:31.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.111+0000 7f11a3165640 1 --2- 172.21.15.16:0/1096943581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f119c0ffe90 0x7f119c100310 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1184009900 tx=0x7f118402a410 comp rx=0 tx=0).stop 2024-09-13T16:28:31.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.111+0000 7f11a3165640 1 -- 172.21.15.16:0/1096943581 shutdown_connections 2024-09-13T16:28:31.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.111+0000 7f11a3165640 1 --2- 172.21.15.16:0/1096943581 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f119c100850 0x7f119c1070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:31.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.111+0000 7f11a3165640 1 --2- 172.21.15.16:0/1096943581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f119c0ffe90 0x7f119c100310 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:31.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.111+0000 7f11a3165640 1 --2- 172.21.15.16:0/1096943581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f119c0feca0 0x7f119c0ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:31.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.111+0000 7f11a3165640 1 -- 172.21.15.16:0/1096943581 >> 172.21.15.16:0/1096943581 conn(0x7f119c0fa440 msgr2=0x7f119c0fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:31.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.111+0000 7f11a3165640 1 -- 172.21.15.16:0/1096943581 shutdown_connections 2024-09-13T16:28:31.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.111+0000 7f11a3165640 1 -- 172.21.15.16:0/1096943581 wait complete. 2024-09-13T16:28:31.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.111+0000 7f11a3165640 1 Processor -- start 2024-09-13T16:28:31.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.112+0000 7f11a3165640 1 -- start start 2024-09-13T16:28:31.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.112+0000 7f11a3165640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f119c0feca0 0x7f119c195d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:31.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.112+0000 7f11a3165640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f119c0ffe90 0x7f119c196240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:31.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.112+0000 7f11a3165640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f119c100850 0x7f119c19d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:31.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.112+0000 7f11a3165640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c19d7e0 con 0x7f119c100850 2024-09-13T16:28:31.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.112+0000 7f11a3165640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c19d950 con 0x7f119c0feca0 2024-09-13T16:28:31.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.112+0000 7f11a3165640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c19db10 con 0x7f119c0ffe90 2024-09-13T16:28:31.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.112+0000 7f11a1702640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f119c100850 0x7f119c19d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:31.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.112+0000 7f11a1702640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f119c100850 0x7f119c19d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57600/0 (socket says 172.21.15.16:57600) 2024-09-13T16:28:31.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.112+0000 7f11a1702640 1 -- 172.21.15.16:0/3617355808 learned_addr learned my addr 172.21.15.16:0/3617355808 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:31.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.112+0000 7f11a0f01640 1 --2- 172.21.15.16:0/3617355808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f119c0feca0 0x7f119c195d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:31.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.112+0000 7f119bfff640 1 --2- 172.21.15.16:0/3617355808 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f119c0ffe90 0x7f119c196240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:31.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.112+0000 7f11a1702640 1 -- 172.21.15.16:0/3617355808 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f119c0ffe90 msgr2=0x7f119c196240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:31.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.112+0000 7f11a1702640 1 --2- 172.21.15.16:0/3617355808 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f119c0ffe90 0x7f119c196240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:31.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.112+0000 7f11a1702640 1 -- 172.21.15.16:0/3617355808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f119c0feca0 msgr2=0x7f119c195d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:31.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.112+0000 7f11a1702640 1 --2- 172.21.15.16:0/3617355808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f119c0feca0 0x7f119c195d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:31.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.112+0000 7f11a1702640 1 -- 172.21.15.16:0/3617355808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1184009560 con 0x7f119c100850 2024-09-13T16:28:31.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.112+0000 7f11a1702640 1 --2- 172.21.15.16:0/3617355808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f119c100850 0x7f119c19d270 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f119400eaa0 tx=0x7f119400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:31.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.112+0000 7f1199ffb640 1 -- 172.21.15.16:0/3617355808 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119400cdb0 con 0x7f119c100850 2024-09-13T16:28:31.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.113+0000 7f1199ffb640 1 -- 172.21.15.16:0/3617355808 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1194005020 con 0x7f119c100850 2024-09-13T16:28:31.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.113+0000 7f1199ffb640 1 -- 172.21.15.16:0/3617355808 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11940116d0 con 0x7f119c100850 2024-09-13T16:28:31.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.113+0000 7f11a3165640 1 -- 172.21.15.16:0/3617355808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f119c19ddf0 con 0x7f119c100850 2024-09-13T16:28:31.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.113+0000 7f11a3165640 1 -- 172.21.15.16:0/3617355808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f119c19e160 con 0x7f119c100850 2024-09-13T16:28:31.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.114+0000 7f11a3165640 1 -- 172.21.15.16:0/3617355808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f119c1061f0 con 0x7f119c100850 2024-09-13T16:28:31.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.115+0000 7f1199ffb640 1 -- 172.21.15.16:0/3617355808 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1194005190 con 0x7f119c100850 2024-09-13T16:28:31.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.115+0000 7f1199ffb640 1 --2- 172.21.15.16:0/3617355808 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f11880a1110 0x7f11880a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:31.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.116+0000 7f1199ffb640 1 -- 172.21.15.16:0/3617355808 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f11940cc570 con 0x7f119c100850 2024-09-13T16:28:31.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.116+0000 7f11a0f01640 1 --2- 172.21.15.16:0/3617355808 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f11880a1110 0x7f11880a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:31.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.116+0000 7f11a0f01640 1 --2- 172.21.15.16:0/3617355808 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f11880a1110 0x7f11880a35d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f119c1971d0 tx=0x7f118c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:31.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.120+0000 7f1199ffb640 1 -- 172.21.15.16:0/3617355808 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f1194096120 con 0x7f119c100850 2024-09-13T16:28:31.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:30 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:31.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:30 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:31.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:30 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:31.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:30 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:31.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:30 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:31.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:30 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:31.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:30 smithi111 ceph-mon[34600]: Deploying daemon mds.a on smithi016 2024-09-13T16:28:31.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:30 smithi111 ceph-mon[34600]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:31.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:30 smithi111 ceph-mon[34600]: mds.? [v2:172.21.15.16:6838/2391618693,v1:172.21.15.16:6839/2391618693] up:boot 2024-09-13T16:28:31.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:30 smithi111 ceph-mon[34600]: fsmap 10 up:standby 2024-09-13T16:28:31.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:30 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-13T16:28:31.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.327+0000 7f11a3165640 1 -- 172.21.15.16:0/3617355808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f119c02c4f0 con 0x7f119c100850 2024-09-13T16:28:31.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.328+0000 7f1199ffb640 1 -- 172.21.15.16:0/3617355808 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8088 (secure 0 0 0) 0x7f1194098050 con 0x7f119c100850 2024-09-13T16:28:31.329 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:28:31.329 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3292781664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3292781664},{"type":"v1","addr":"172.21.15.16:6835","nonce":3292781664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4109039027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4109039027},{"type":"v1","addr":"172.21.15.16:6837","nonce":4109039027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2391618693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2391618693},{"type":"v1","addr":"172.21.15.16:6839","nonce":2391618693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1984275936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1984275936},{"type":"v1","addr":"172.21.15.111:6833","nonce":1984275936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3797712690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3797712690},{"type":"v1","addr":"172.21.15.107:6833","nonce":3797712690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/338548468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":338548468},{"type":"v1","addr":"172.21.15.107:6835","nonce":338548468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/662156111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":662156111},{"type":"v1","addr":"172.21.15.111:6835","nonce":662156111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/1877128453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1877128453},{"type":"v1","addr":"172.21.15.111:6837","nonce":1877128453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2838457611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2838457611},{"type":"v1","addr":"172.21.15.107:6837","nonce":2838457611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6839/1423842285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":1423842285},{"type":"v1","addr":"172.21.15.111:6839","nonce":1423842285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-09-13T16:28:31.329 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 11 2024-09-13T16:28:31.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.330+0000 7f11a3165640 1 -- 172.21.15.16:0/3617355808 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f11880a1110 msgr2=0x7f11880a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:31.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.330+0000 7f11a3165640 1 --2- 172.21.15.16:0/3617355808 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f11880a1110 0x7f11880a35d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f119c1971d0 tx=0x7f118c00b410 comp rx=0 tx=0).stop 2024-09-13T16:28:31.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.330+0000 7f11a3165640 1 -- 172.21.15.16:0/3617355808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f119c100850 msgr2=0x7f119c19d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:31.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.330+0000 7f11a3165640 1 --2- 172.21.15.16:0/3617355808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f119c100850 0x7f119c19d270 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f119400eaa0 tx=0x7f119400ef60 comp rx=0 tx=0).stop 2024-09-13T16:28:31.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.330+0000 7f11a3165640 1 -- 172.21.15.16:0/3617355808 shutdown_connections 2024-09-13T16:28:31.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.330+0000 7f11a3165640 1 --2- 172.21.15.16:0/3617355808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f119c100850 0x7f119c19d270 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:31.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.330+0000 7f11a3165640 1 --2- 172.21.15.16:0/3617355808 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f119c0ffe90 0x7f119c196240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:31.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.330+0000 7f11a3165640 1 --2- 172.21.15.16:0/3617355808 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f11880a1110 0x7f11880a35d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:31.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.330+0000 7f11a3165640 1 --2- 172.21.15.16:0/3617355808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f119c0feca0 0x7f119c195d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:31.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.330+0000 7f11a3165640 1 -- 172.21.15.16:0/3617355808 >> 172.21.15.16:0/3617355808 conn(0x7f119c0fa440 msgr2=0x7f119c1033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:31.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.330+0000 7f11a3165640 1 -- 172.21.15.16:0/3617355808 shutdown_connections 2024-09-13T16:28:31.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:31.331+0000 7f11a3165640 1 -- 172.21.15.16:0/3617355808 wait complete. 2024-09-13T16:28:31.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:31 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3617355808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:31.855 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 13 16:28:31 smithi016 systemd[1]: Starting Ceph mds.a for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:28:32.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:31 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3617355808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:32.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:31 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3617355808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:32.281 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 13 16:28:32 smithi016 podman[72800]: 2024-09-13 16:28:31.909765427 +0000 UTC m=+0.016571619 image pull 152d8bc6b19e354515e8ba109d4619445de0e09b4d4a747c2f16eb5c0edb670c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a 2024-09-13T16:28:32.282 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 13 16:28:32 smithi016 podman[72800]: 2024-09-13 16:28:32.009265785 +0000 UTC m=+0.116071967 container create 25504be1b45da5315437bd4db20ffc5a815f5b7f04165e5851232f373162b4af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T16:28:32.282 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 13 16:28:32 smithi016 podman[72800]: 2024-09-13 16:28:32.253861871 +0000 UTC m=+0.360668066 container init 25504be1b45da5315437bd4db20ffc5a815f5b7f04165e5851232f373162b4af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-a, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True) 2024-09-13T16:28:32.282 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 13 16:28:32 smithi016 podman[72800]: 2024-09-13 16:28:32.2569487 +0000 UTC m=+0.363754883 container start 25504be1b45da5315437bd4db20ffc5a815f5b7f04165e5851232f373162b4af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-13T16:28:32.282 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 13 16:28:32 smithi016 ceph-mds[72819]: -- 172.21.15.16:0/2454193764 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cf2f67ad00 con 0x55cf2f64f400 2024-09-13T16:28:32.604 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 13 16:28:32 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-a[72815]: starting mds.a at 2024-09-13T16:28:32.604 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 13 16:28:32 smithi016 bash[72800]: 25504be1b45da5315437bd4db20ffc5a815f5b7f04165e5851232f373162b4af 2024-09-13T16:28:32.604 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 13 16:28:32 smithi016 systemd[1]: Started Ceph mds.a for 0d241f6a-71ec-11ef-bceb-c7b262605968. 2024-09-13T16:28:33.343 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:28:33.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.463+0000 7f2f78ff4640 1 -- 172.21.15.16:0/82040234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f74103150 msgr2=0x7f2f74105540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:33.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.463+0000 7f2f78ff4640 1 --2- 172.21.15.16:0/82040234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f74103150 0x7f2f74105540 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2f68009900 tx=0x7f2f6802a410 comp rx=0 tx=0).stop 2024-09-13T16:28:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.463+0000 7f2f78ff4640 1 -- 172.21.15.16:0/82040234 shutdown_connections 2024-09-13T16:28:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.463+0000 7f2f78ff4640 1 --2- 172.21.15.16:0/82040234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f74105a80 0x7f2f74107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.463+0000 7f2f78ff4640 1 --2- 172.21.15.16:0/82040234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f74103150 0x7f2f74105540 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.463+0000 7f2f78ff4640 1 --2- 172.21.15.16:0/82040234 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f74100820 0x7f2f74102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.463+0000 7f2f78ff4640 1 -- 172.21.15.16:0/82040234 >> 172.21.15.16:0/82040234 conn(0x7f2f740fa440 msgr2=0x7f2f740fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.463+0000 7f2f78ff4640 1 -- 172.21.15.16:0/82040234 shutdown_connections 2024-09-13T16:28:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.463+0000 7f2f78ff4640 1 -- 172.21.15.16:0/82040234 wait complete. 2024-09-13T16:28:33.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.464+0000 7f2f78ff4640 1 Processor -- start 2024-09-13T16:28:33.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.464+0000 7f2f78ff4640 1 -- start start 2024-09-13T16:28:33.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.464+0000 7f2f78ff4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f74100820 0x7f2f74195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:33.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.464+0000 7f2f78ff4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f74103150 0x7f2f74196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:33.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.464+0000 7f2f78ff4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f74105a80 0x7f2f7419d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:33.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.464+0000 7f2f78ff4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7419d830 con 0x7f2f74105a80 2024-09-13T16:28:33.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.464+0000 7f2f78ff4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7419d9a0 con 0x7f2f74103150 2024-09-13T16:28:33.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.464+0000 7f2f78ff4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7419db60 con 0x7f2f74100820 2024-09-13T16:28:33.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.464+0000 7f2f72d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f74105a80 0x7f2f7419d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:33.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.464+0000 7f2f72d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f74105a80 0x7f2f7419d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33056/0 (socket says 172.21.15.16:33056) 2024-09-13T16:28:33.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.464+0000 7f2f72d9d640 1 -- 172.21.15.16:0/627195544 learned_addr learned my addr 172.21.15.16:0/627195544 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:33.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.464+0000 7f2f71d9b640 1 --2- 172.21.15.16:0/627195544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f74103150 0x7f2f74196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:33.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.464+0000 7f2f7259c640 1 --2- 172.21.15.16:0/627195544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f74100820 0x7f2f74195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:33.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.465+0000 7f2f72d9d640 1 -- 172.21.15.16:0/627195544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f74100820 msgr2=0x7f2f74195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:33.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.465+0000 7f2f72d9d640 1 --2- 172.21.15.16:0/627195544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f74100820 0x7f2f74195d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:33.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.465+0000 7f2f72d9d640 1 -- 172.21.15.16:0/627195544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f74103150 msgr2=0x7f2f74196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:33.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.465+0000 7f2f72d9d640 1 --2- 172.21.15.16:0/627195544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f74103150 0x7f2f74196290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:33.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.465+0000 7f2f72d9d640 1 -- 172.21.15.16:0/627195544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f68009560 con 0x7f2f74105a80 2024-09-13T16:28:33.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.465+0000 7f2f7259c640 1 --2- 172.21.15.16:0/627195544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f74100820 0x7f2f74195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:28:33.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.465+0000 7f2f71d9b640 1 --2- 172.21.15.16:0/627195544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f74103150 0x7f2f74196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:28:33.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.465+0000 7f2f72d9d640 1 --2- 172.21.15.16:0/627195544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f74105a80 0x7f2f7419d2c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00eb40 tx=0x7f2f6c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:33.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.465+0000 7f2f637fe640 1 -- 172.21.15.16:0/627195544 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c004b20 con 0x7f2f74105a80 2024-09-13T16:28:33.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.465+0000 7f2f78ff4640 1 -- 172.21.15.16:0/627195544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f7419de40 con 0x7f2f74105a80 2024-09-13T16:28:33.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.465+0000 7f2f637fe640 1 -- 172.21.15.16:0/627195544 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2f6c004c80 con 0x7f2f74105a80 2024-09-13T16:28:33.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.465+0000 7f2f637fe640 1 -- 172.21.15.16:0/627195544 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c010690 con 0x7f2f74105a80 2024-09-13T16:28:33.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.465+0000 7f2f78ff4640 1 -- 172.21.15.16:0/627195544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f7419e260 con 0x7f2f74105a80 2024-09-13T16:28:33.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.466+0000 7f2f78ff4640 1 -- 172.21.15.16:0/627195544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f740fbf30 con 0x7f2f74105a80 2024-09-13T16:28:33.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.467+0000 7f2f637fe640 1 -- 172.21.15.16:0/627195544 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f2f6c005020 con 0x7f2f74105a80 2024-09-13T16:28:33.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.468+0000 7f2f637fe640 1 --2- 172.21.15.16:0/627195544 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f2f5c0a1090 0x7f2f5c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:33.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.468+0000 7f2f7259c640 1 --2- 172.21.15.16:0/627195544 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f2f5c0a1090 0x7f2f5c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:33.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.468+0000 7f2f637fe640 1 -- 172.21.15.16:0/627195544 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2f6c014070 con 0x7f2f74105a80 2024-09-13T16:28:33.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.469+0000 7f2f7259c640 1 --2- 172.21.15.16:0/627195544 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f2f5c0a1090 0x7f2f5c0a3550 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2f64005fd0 tx=0x7f2f64005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:33.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.473+0000 7f2f637fe640 1 -- 172.21.15.16:0/627195544 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f2f6c096880 con 0x7f2f74105a80 2024-09-13T16:28:33.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.682+0000 7f2f78ff4640 1 -- 172.21.15.16:0/627195544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f7402c480 con 0x7f2f74105a80 2024-09-13T16:28:33.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.682+0000 7f2f637fe640 1 -- 172.21.15.16:0/627195544 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8843 (secure 0 0 0) 0x7f2f6c0987b0 con 0x7f2f74105a80 2024-09-13T16:28:33.684 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:28:33.684 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3292781664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3292781664},{"type":"v1","addr":"172.21.15.16:6835","nonce":3292781664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4109039027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4109039027},{"type":"v1","addr":"172.21.15.16:6837","nonce":4109039027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2391618693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2391618693},{"type":"v1","addr":"172.21.15.16:6839","nonce":2391618693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/521042681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":521042681},{"type":"v1","addr":"172.21.15.16:6841","nonce":521042681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1984275936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1984275936},{"type":"v1","addr":"172.21.15.111:6833","nonce":1984275936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3797712690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3797712690},{"type":"v1","addr":"172.21.15.107:6833","nonce":3797712690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/338548468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":338548468},{"type":"v1","addr":"172.21.15.107:6835","nonce":338548468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/662156111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":662156111},{"type":"v1","addr":"172.21.15.111:6835","nonce":662156111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/1877128453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1877128453},{"type":"v1","addr":"172.21.15.111:6837","nonce":1877128453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2838457611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2838457611},{"type":"v1","addr":"172.21.15.107:6837","nonce":2838457611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6839/1423842285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":1423842285},{"type":"v1","addr":"172.21.15.111:6839","nonce":1423842285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-09-13T16:28:33.684 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 12 2024-09-13T16:28:33.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.685+0000 7f2f78ff4640 1 -- 172.21.15.16:0/627195544 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f2f5c0a1090 msgr2=0x7f2f5c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:33.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.685+0000 7f2f78ff4640 1 --2- 172.21.15.16:0/627195544 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f2f5c0a1090 0x7f2f5c0a3550 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2f64005fd0 tx=0x7f2f64005850 comp rx=0 tx=0).stop 2024-09-13T16:28:33.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.685+0000 7f2f78ff4640 1 -- 172.21.15.16:0/627195544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f74105a80 msgr2=0x7f2f7419d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:33.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.685+0000 7f2f78ff4640 1 --2- 172.21.15.16:0/627195544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f74105a80 0x7f2f7419d2c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00eb40 tx=0x7f2f6c00c2d0 comp rx=0 tx=0).stop 2024-09-13T16:28:33.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.685+0000 7f2f78ff4640 1 -- 172.21.15.16:0/627195544 shutdown_connections 2024-09-13T16:28:33.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.685+0000 7f2f78ff4640 1 --2- 172.21.15.16:0/627195544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f74105a80 0x7f2f7419d2c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:33.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.685+0000 7f2f78ff4640 1 --2- 172.21.15.16:0/627195544 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f2f5c0a1090 0x7f2f5c0a3550 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:33.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.685+0000 7f2f78ff4640 1 --2- 172.21.15.16:0/627195544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f74103150 0x7f2f74196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:33.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.685+0000 7f2f78ff4640 1 --2- 172.21.15.16:0/627195544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f74100820 0x7f2f74195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:33.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.685+0000 7f2f78ff4640 1 -- 172.21.15.16:0/627195544 >> 172.21.15.16:0/627195544 conn(0x7f2f740fa440 msgr2=0x7f2f741011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:33.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.685+0000 7f2f78ff4640 1 -- 172.21.15.16:0/627195544 shutdown_connections 2024-09-13T16:28:33.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:33.685+0000 7f2f78ff4640 1 -- 172.21.15.16:0/627195544 wait complete. 2024-09-13T16:28:33.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:33 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:33.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:33 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:33.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:33 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:33.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:33 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:33.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:33 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:33.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:33 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:33.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:33 smithi111 ceph-mon[34600]: Deploying daemon mds.b on smithi107 2024-09-13T16:28:33.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:33 smithi111 ceph-mon[34600]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:33.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:33 smithi111 ceph-mon[34600]: mds.? [v2:172.21.15.16:6840/521042681,v1:172.21.15.16:6841/521042681] up:boot 2024-09-13T16:28:33.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:33 smithi111 ceph-mon[34600]: fsmap 11 up:standby 2024-09-13T16:28:33.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:33 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-13T16:28:33.695 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:33 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:33.695 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:33 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:33.695 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:33 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:33.695 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:33 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:33.696 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:33 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:33.696 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:33 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:33.696 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:33 smithi016 ceph-mon[31187]: Deploying daemon mds.b on smithi107 2024-09-13T16:28:33.696 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:33 smithi016 ceph-mon[31187]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:33.696 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:33 smithi016 ceph-mon[31187]: mds.? [v2:172.21.15.16:6840/521042681,v1:172.21.15.16:6841/521042681] up:boot 2024-09-13T16:28:33.696 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:33 smithi016 ceph-mon[31187]: fsmap 11 up:standby 2024-09-13T16:28:33.696 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:33 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-13T16:28:33.756 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:33 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:33.756 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:33 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:33.756 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:33 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:33.756 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:33 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T16:28:33.756 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:33 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T16:28:33.756 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:33 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:33.756 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:33 smithi107 ceph-mon[34139]: Deploying daemon mds.b on smithi107 2024-09-13T16:28:33.756 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:33 smithi107 ceph-mon[34139]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:33.757 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:33 smithi107 ceph-mon[34139]: mds.? [v2:172.21.15.16:6840/521042681,v1:172.21.15.16:6841/521042681] up:boot 2024-09-13T16:28:33.757 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:33 smithi107 ceph-mon[34139]: fsmap 11 up:standby 2024-09-13T16:28:33.757 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:33 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-13T16:28:34.349 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 13 16:28:34 smithi107 systemd[1]: Starting Ceph mds.b for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:28:34.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:34 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/627195544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:34.758 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:34 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/627195544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:34.758 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 13 16:28:34 smithi107 podman[63462]: 2024-09-13 16:28:34.348051807 +0000 UTC m=+0.015878164 image pull 152d8bc6b19e354515e8ba109d4619445de0e09b4d4a747c2f16eb5c0edb670c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a 2024-09-13T16:28:34.759 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 13 16:28:34 smithi107 podman[63462]: 2024-09-13 16:28:34.491264547 +0000 UTC m=+0.159090895 container create 5a0a934d5a9ee524a772f55cf809b59b520dcd87217ed4405ea7f95dd32ea159 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T16:28:34.759 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 13 16:28:34 smithi107 podman[63462]: 2024-09-13 16:28:34.758154302 +0000 UTC m=+0.425980644 container init 5a0a934d5a9ee524a772f55cf809b59b520dcd87217ed4405ea7f95dd32ea159 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-13T16:28:34.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:34 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/627195544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:35.099 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 13 16:28:34 smithi107 podman[63462]: 2024-09-13 16:28:34.761060315 +0000 UTC m=+0.428886663 container start 5a0a934d5a9ee524a772f55cf809b59b520dcd87217ed4405ea7f95dd32ea159 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-b, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f) 2024-09-13T16:28:35.099 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 13 16:28:34 smithi107 ceph-mds[63482]: -- 172.21.15.107:0/1798232796 <== mon.2 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55633bec2d00 con 0x55633be97400 2024-09-13T16:28:35.099 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 13 16:28:34 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-b[63478]: starting mds.b at 2024-09-13T16:28:35.099 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 13 16:28:34 smithi107 bash[63462]: 5a0a934d5a9ee524a772f55cf809b59b520dcd87217ed4405ea7f95dd32ea159 2024-09-13T16:28:35.100 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 13 16:28:34 smithi107 systemd[1]: Started Ceph mds.b for 0d241f6a-71ec-11ef-bceb-c7b262605968. 2024-09-13T16:28:35.678 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:35 smithi016 ceph-mon[31187]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:35.678 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:35 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:35.678 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:35 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:35.678 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:35 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:35.679 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:35 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:35.679 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:35 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:35.679 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:35 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:28:35.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:35 smithi111 ceph-mon[34600]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:35.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:35 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:35.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:35 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:35.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:35 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:35.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:35 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:35.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:35 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:35.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:35 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:28:35.698 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:28:35.804 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:35 smithi107 ceph-mon[34139]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:35.805 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:35 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:35.805 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:35 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:35.805 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:35 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:35.805 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:35 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:35.805 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:35 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:35.805 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:35 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:28:35.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.803+0000 7f8f7b648640 1 -- 172.21.15.16:0/4268464188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f740ffdc0 msgr2=0x7f8f741001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:35.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.803+0000 7f8f7b648640 1 --2- 172.21.15.16:0/4268464188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f740ffdc0 0x7f8f741001c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8f70009900 tx=0x7f8f7002a410 comp rx=0 tx=0).stop 2024-09-13T16:28:35.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.803+0000 7f8f7b648640 1 -- 172.21.15.16:0/4268464188 shutdown_connections 2024-09-13T16:28:35.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.803+0000 7f8f7b648640 1 --2- 172.21.15.16:0/4268464188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f74101970 0x7f8f74108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:35.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.803+0000 7f8f7b648640 1 --2- 172.21.15.16:0/4268464188 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f74100fb0 0x7f8f74101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:35.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.803+0000 7f8f7b648640 1 --2- 172.21.15.16:0/4268464188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f740ffdc0 0x7f8f741001c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:35.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.803+0000 7f8f7b648640 1 -- 172.21.15.16:0/4268464188 >> 172.21.15.16:0/4268464188 conn(0x7f8f740fb540 msgr2=0x7f8f740fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:35.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.803+0000 7f8f7b648640 1 -- 172.21.15.16:0/4268464188 shutdown_connections 2024-09-13T16:28:35.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.803+0000 7f8f7b648640 1 -- 172.21.15.16:0/4268464188 wait complete. 2024-09-13T16:28:35.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.804+0000 7f8f7b648640 1 Processor -- start 2024-09-13T16:28:35.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.804+0000 7f8f7b648640 1 -- start start 2024-09-13T16:28:35.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.804+0000 7f8f7b648640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f740ffdc0 0x7f8f74195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:35.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.804+0000 7f8f7b648640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f74100fb0 0x7f8f74196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:35.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.804+0000 7f8f7b648640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f74101970 0x7f8f7419d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:35.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.804+0000 7f8f7b648640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7419d730 con 0x7f8f74101970 2024-09-13T16:28:35.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.804+0000 7f8f7b648640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7419d8a0 con 0x7f8f740ffdc0 2024-09-13T16:28:35.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.804+0000 7f8f7b648640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7419da60 con 0x7f8f74100fb0 2024-09-13T16:28:35.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.804+0000 7f8f79be5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f74101970 0x7f8f7419d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:35.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.804+0000 7f8f79be5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f74101970 0x7f8f7419d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33068/0 (socket says 172.21.15.16:33068) 2024-09-13T16:28:35.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.804+0000 7f8f79be5640 1 -- 172.21.15.16:0/4213396696 learned_addr learned my addr 172.21.15.16:0/4213396696 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:35.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.804+0000 7f8f793e4640 1 --2- 172.21.15.16:0/4213396696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f740ffdc0 0x7f8f74195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:35.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.804+0000 7f8f78be3640 1 --2- 172.21.15.16:0/4213396696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f74100fb0 0x7f8f74196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:35.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.804+0000 7f8f79be5640 1 -- 172.21.15.16:0/4213396696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f74100fb0 msgr2=0x7f8f74196190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:35.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.804+0000 7f8f79be5640 1 --2- 172.21.15.16:0/4213396696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f74100fb0 0x7f8f74196190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:35.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.804+0000 7f8f79be5640 1 -- 172.21.15.16:0/4213396696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f740ffdc0 msgr2=0x7f8f74195c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:35.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.804+0000 7f8f79be5640 1 --2- 172.21.15.16:0/4213396696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f740ffdc0 0x7f8f74195c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:35.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.804+0000 7f8f79be5640 1 -- 172.21.15.16:0/4213396696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f70009560 con 0x7f8f74101970 2024-09-13T16:28:35.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.804+0000 7f8f78be3640 1 --2- 172.21.15.16:0/4213396696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f74100fb0 0x7f8f74196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:28:35.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.805+0000 7f8f79be5640 1 --2- 172.21.15.16:0/4213396696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f74101970 0x7f8f7419d1c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c00cce0 tx=0x7f8f6c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:35.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.805+0000 7f8f6a7fc640 1 -- 172.21.15.16:0/4213396696 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c00f450 con 0x7f8f74101970 2024-09-13T16:28:35.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.805+0000 7f8f6a7fc640 1 -- 172.21.15.16:0/4213396696 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8f6c00fa40 con 0x7f8f74101970 2024-09-13T16:28:35.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.805+0000 7f8f6a7fc640 1 -- 172.21.15.16:0/4213396696 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c0189a0 con 0x7f8f74101970 2024-09-13T16:28:35.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.805+0000 7f8f7b648640 1 -- 172.21.15.16:0/4213396696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f7419dd40 con 0x7f8f74101970 2024-09-13T16:28:35.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.805+0000 7f8f7b648640 1 -- 172.21.15.16:0/4213396696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f7419e1d0 con 0x7f8f74101970 2024-09-13T16:28:35.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.806+0000 7f8f7b648640 1 -- 172.21.15.16:0/4213396696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f74107250 con 0x7f8f74101970 2024-09-13T16:28:35.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.807+0000 7f8f6a7fc640 1 -- 172.21.15.16:0/4213396696 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8f6c00fbb0 con 0x7f8f74101970 2024-09-13T16:28:35.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.808+0000 7f8f6a7fc640 1 --2- 172.21.15.16:0/4213396696 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8f600a1110 0x7f8f600a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:35.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.808+0000 7f8f793e4640 1 --2- 172.21.15.16:0/4213396696 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8f600a1110 0x7f8f600a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:35.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.808+0000 7f8f6a7fc640 1 -- 172.21.15.16:0/4213396696 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8f6c0cc8a0 con 0x7f8f74101970 2024-09-13T16:28:35.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.808+0000 7f8f793e4640 1 --2- 172.21.15.16:0/4213396696 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8f600a1110 0x7f8f600a35d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8f700057c0 tx=0x7f8f70005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:35.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:35.812+0000 7f8f6a7fc640 1 -- 172.21.15.16:0/4213396696 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f8f6c096440 con 0x7f8f74101970 2024-09-13T16:28:36.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.020+0000 7f8f7b648640 1 -- 172.21.15.16:0/4213396696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f7410b510 con 0x7f8f74101970 2024-09-13T16:28:36.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.021+0000 7f8f6a7fc640 1 -- 172.21.15.16:0/4213396696 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9604 (secure 0 0 0) 0x7f8f7410b510 con 0x7f8f74101970 2024-09-13T16:28:36.023 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:28:36.023 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3292781664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3292781664},{"type":"v1","addr":"172.21.15.16:6835","nonce":3292781664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4109039027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4109039027},{"type":"v1","addr":"172.21.15.16:6837","nonce":4109039027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2391618693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2391618693},{"type":"v1","addr":"172.21.15.16:6839","nonce":2391618693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/521042681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":521042681},{"type":"v1","addr":"172.21.15.16:6841","nonce":521042681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1984275936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1984275936},{"type":"v1","addr":"172.21.15.111:6833","nonce":1984275936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3797712690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3797712690},{"type":"v1","addr":"172.21.15.107:6833","nonce":3797712690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/338548468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":338548468},{"type":"v1","addr":"172.21.15.107:6835","nonce":338548468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/662156111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":662156111},{"type":"v1","addr":"172.21.15.111:6835","nonce":662156111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/1877128453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1877128453},{"type":"v1","addr":"172.21.15.111:6837","nonce":1877128453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2838457611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2838457611},{"type":"v1","addr":"172.21.15.107:6837","nonce":2838457611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6839/1423842285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":1423842285},{"type":"v1","addr":"172.21.15.111:6839","nonce":1423842285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24518,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/1914689806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1914689806},{"type":"v1","addr":"172.21.15.107:6839","nonce":1914689806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2024-09-13T16:28:36.023 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 13 2024-09-13T16:28:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.023+0000 7f8f7b648640 1 -- 172.21.15.16:0/4213396696 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8f600a1110 msgr2=0x7f8f600a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.023+0000 7f8f7b648640 1 --2- 172.21.15.16:0/4213396696 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8f600a1110 0x7f8f600a35d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8f700057c0 tx=0x7f8f70005a20 comp rx=0 tx=0).stop 2024-09-13T16:28:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.023+0000 7f8f7b648640 1 -- 172.21.15.16:0/4213396696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f74101970 msgr2=0x7f8f7419d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.023+0000 7f8f7b648640 1 --2- 172.21.15.16:0/4213396696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f74101970 0x7f8f7419d1c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c00cce0 tx=0x7f8f6c00a620 comp rx=0 tx=0).stop 2024-09-13T16:28:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.023+0000 7f8f7b648640 1 -- 172.21.15.16:0/4213396696 shutdown_connections 2024-09-13T16:28:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.023+0000 7f8f7b648640 1 --2- 172.21.15.16:0/4213396696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f74101970 0x7f8f7419d1c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.023+0000 7f8f7b648640 1 --2- 172.21.15.16:0/4213396696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f74100fb0 0x7f8f74196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.023+0000 7f8f7b648640 1 --2- 172.21.15.16:0/4213396696 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8f600a1110 0x7f8f600a35d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.023+0000 7f8f7b648640 1 --2- 172.21.15.16:0/4213396696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f740ffdc0 0x7f8f74195c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.023+0000 7f8f7b648640 1 -- 172.21.15.16:0/4213396696 >> 172.21.15.16:0/4213396696 conn(0x7f8f740fb540 msgr2=0x7f8f74104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.023+0000 7f8f7b648640 1 -- 172.21.15.16:0/4213396696 shutdown_connections 2024-09-13T16:28:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.023+0000 7f8f7b648640 1 -- 172.21.15.16:0/4213396696 wait complete. 2024-09-13T16:28:36.036 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:28:36.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.137+0000 7fe1046a3640 1 -- 172.21.15.16:0/2266518833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0fc0ffe90 msgr2=0x7fe0fc100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:36.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.137+0000 7fe1046a3640 1 --2- 172.21.15.16:0/2266518833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0fc0ffe90 0x7fe0fc100310 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec009900 tx=0x7fe0ec02a410 comp rx=0 tx=0).stop 2024-09-13T16:28:36.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.137+0000 7fe1046a3640 1 -- 172.21.15.16:0/2266518833 shutdown_connections 2024-09-13T16:28:36.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.137+0000 7fe1046a3640 1 --2- 172.21.15.16:0/2266518833 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0fc100850 0x7fe0fc1070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:36.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.137+0000 7fe1046a3640 1 --2- 172.21.15.16:0/2266518833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0fc0ffe90 0x7fe0fc100310 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:36.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.137+0000 7fe1046a3640 1 --2- 172.21.15.16:0/2266518833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0fc0feca0 0x7fe0fc0ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:36.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.137+0000 7fe1046a3640 1 -- 172.21.15.16:0/2266518833 >> 172.21.15.16:0/2266518833 conn(0x7fe0fc0fa440 msgr2=0x7fe0fc0fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:36.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.137+0000 7fe1046a3640 1 -- 172.21.15.16:0/2266518833 shutdown_connections 2024-09-13T16:28:36.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.137+0000 7fe1046a3640 1 -- 172.21.15.16:0/2266518833 wait complete. 2024-09-13T16:28:36.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.138+0000 7fe1046a3640 1 Processor -- start 2024-09-13T16:28:36.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.138+0000 7fe1046a3640 1 -- start start 2024-09-13T16:28:36.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.138+0000 7fe1046a3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0fc0feca0 0x7fe0fc195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:36.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.138+0000 7fe1046a3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0fc0ffe90 0x7fe0fc196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:36.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.138+0000 7fe1046a3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0fc100850 0x7fe0fc19d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:36.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.138+0000 7fe1046a3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc19d830 con 0x7fe0fc0ffe90 2024-09-13T16:28:36.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.138+0000 7fe1046a3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc19d9a0 con 0x7fe0fc0feca0 2024-09-13T16:28:36.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.138+0000 7fe1046a3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc19db60 con 0x7fe0fc100850 2024-09-13T16:28:36.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.138+0000 7fe10243f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0fc0feca0 0x7fe0fc195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:36.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.138+0000 7fe101c3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0fc0ffe90 0x7fe0fc196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:36.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.138+0000 7fe10243f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0fc0feca0 0x7fe0fc195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.16:54900/0 (socket says 172.21.15.16:54900) 2024-09-13T16:28:36.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.138+0000 7fe10243f640 1 -- 172.21.15.16:0/447868802 learned_addr learned my addr 172.21.15.16:0/447868802 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:36.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.139+0000 7fe102c40640 1 --2- 172.21.15.16:0/447868802 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0fc100850 0x7fe0fc19d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:36.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.139+0000 7fe10243f640 1 -- 172.21.15.16:0/447868802 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0fc100850 msgr2=0x7fe0fc19d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:36.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.139+0000 7fe10243f640 1 --2- 172.21.15.16:0/447868802 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0fc100850 0x7fe0fc19d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:36.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.139+0000 7fe10243f640 1 -- 172.21.15.16:0/447868802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0fc0ffe90 msgr2=0x7fe0fc196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:36.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.139+0000 7fe10243f640 1 --2- 172.21.15.16:0/447868802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0fc0ffe90 0x7fe0fc196290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:36.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.139+0000 7fe10243f640 1 -- 172.21.15.16:0/447868802 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ec009560 con 0x7fe0fc0feca0 2024-09-13T16:28:36.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.139+0000 7fe10243f640 1 --2- 172.21.15.16:0/447868802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0fc0feca0 0x7fe0fc195d50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe0f800d950 tx=0x7fe0f800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:36.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.139+0000 7fe0f37fe640 1 -- 172.21.15.16:0/447868802 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8009c30 con 0x7fe0fc0feca0 2024-09-13T16:28:36.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.139+0000 7fe1046a3640 1 -- 172.21.15.16:0/447868802 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0fc19de40 con 0x7fe0fc0feca0 2024-09-13T16:28:36.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.139+0000 7fe0f37fe640 1 -- 172.21.15.16:0/447868802 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe0f8009d90 con 0x7fe0fc0feca0 2024-09-13T16:28:36.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.139+0000 7fe0f37fe640 1 -- 172.21.15.16:0/447868802 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f800f6b0 con 0x7fe0fc0feca0 2024-09-13T16:28:36.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.139+0000 7fe1046a3640 1 -- 172.21.15.16:0/447868802 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0fc19e260 con 0x7fe0fc0feca0 2024-09-13T16:28:36.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.140+0000 7fe1046a3640 1 -- 172.21.15.16:0/447868802 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0fc10a3f0 con 0x7fe0fc0feca0 2024-09-13T16:28:36.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.141+0000 7fe0f37fe640 1 -- 172.21.15.16:0/447868802 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe0f8010460 con 0x7fe0fc0feca0 2024-09-13T16:28:36.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.142+0000 7fe0f37fe640 1 --2- 172.21.15.16:0/447868802 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe0e80a1090 0x7fe0e80a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:36.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.142+0000 7fe0f37fe640 1 -- 172.21.15.16:0/447868802 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe0f80cc6b0 con 0x7fe0fc0feca0 2024-09-13T16:28:36.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.142+0000 7fe101c3e640 1 --2- 172.21.15.16:0/447868802 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe0e80a1090 0x7fe0e80a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:36.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.143+0000 7fe101c3e640 1 --2- 172.21.15.16:0/447868802 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe0e80a1090 0x7fe0e80a3550 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec02aa00 tx=0x7fe0ec005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:36.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.145+0000 7fe0f37fe640 1 -- 172.21.15.16:0/447868802 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fe0f80961b0 con 0x7fe0fc0feca0 2024-09-13T16:28:36.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.342+0000 7fe1046a3640 1 -- 172.21.15.16:0/447868802 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0fc19e710 con 0x7fe0fc0feca0 2024-09-13T16:28:36.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.344+0000 7fe0f37fe640 1 -- 172.21.15.16:0/447868802 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18433 (secure 0 0 0) 0x7fe0f80980e0 con 0x7fe0fc0feca0 2024-09-13T16:28:36.345 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:28:36.345 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"fsid":"0d241f6a-71ec-11ef-bceb-c7b262605968","created":"2024-09-13T16:21:17.971738+0000","modified":"2024-09-13T16:27:55.677869+0000","last_up_change":"2024-09-13T16:27:54.673413+0000","last_in_change":"2024-09-13T16:27:35.718390+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:24:22.981019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0e11ef8f-08d6-4e74-afd6-f49f4bd40032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6803","nonce":2079394186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6805","nonce":2079394186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6809","nonce":2079394186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6807","nonce":2079394186}]},"public_addr":"172.21.15.16:6803/2079394186","cluster_addr":"172.21.15.16:6805/2079394186","heartbeat_back_addr":"172.21.15.16:6809/2079394186","heartbeat_front_addr":"172.21.15.16:6807/2079394186","state":["exists","up"]},{"osd":1,"uuid":"f2cd3694-9c9e-4da7-976e-f80a8df120f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6801","nonce":4021360958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6803","nonce":4021360958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6807","nonce":4021360958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6805","nonce":4021360958}]},"public_addr":"172.21.15.107:6801/4021360958","cluster_addr":"172.21.15.107:6803/4021360958","heartbeat_back_addr":"172.21.15.107:6807/4021360958","heartbeat_front_addr":"172.21.15.107:6805/4021360958","state":["exists","up"]},{"osd":2,"uuid":"a5bbf2eb-6599-44d5-ad96-68bb89ee19f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6801","nonce":3227179825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6803","nonce":3227179825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6807","nonce":3227179825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6805","nonce":3227179825}]},"public_addr":"172.21.15.111:6801/3227179825","cluster_addr":"172.21.15.111:6803/3227179825","heartbeat_back_addr":"172.21.15.111:6807/3227179825","heartbeat_front_addr":"172.21.15.111:6805/3227179825","state":["exists","up"]},{"osd":3,"uuid":"d6295ffe-11bc-4964-a250-9fecca870b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6811","nonce":2390375937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6813","nonce":2390375937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6817","nonce":2390375937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6815","nonce":2390375937}]},"public_addr":"172.21.15.16:6811/2390375937","cluster_addr":"172.21.15.16:6813/2390375937","heartbeat_back_addr":"172.21.15.16:6817/2390375937","heartbeat_front_addr":"172.21.15.16:6815/2390375937","state":["exists","up"]},{"osd":4,"uuid":"0a8446e0-fc86-4646-beb6-a7b9d433093d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6809","nonce":1936582048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6811","nonce":1936582048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6815","nonce":1936582048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6813","nonce":1936582048}]},"public_addr":"172.21.15.107:6809/1936582048","cluster_addr":"172.21.15.107:6811/1936582048","heartbeat_back_addr":"172.21.15.107:6815/1936582048","heartbeat_front_addr":"172.21.15.107:6813/1936582048","state":["exists","up"]},{"osd":5,"uuid":"2c97a446-357e-4975-9e51-9c95fe84a663","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6809","nonce":483858448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6811","nonce":483858448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6815","nonce":483858448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6813","nonce":483858448}]},"public_addr":"172.21.15.111:6809/483858448","cluster_addr":"172.21.15.111:6811/483858448","heartbeat_back_addr":"172.21.15.111:6815/483858448","heartbeat_front_addr":"172.21.15.111:6813/483858448","state":["exists","up"]},{"osd":6,"uuid":"890239e6-e6fb-4f9c-b973-05cc66d199bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6819","nonce":1330214141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6821","nonce":1330214141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6825","nonce":1330214141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6823","nonce":1330214141}]},"public_addr":"172.21.15.16:6819/1330214141","cluster_addr":"172.21.15.16:6821/1330214141","heartbeat_back_addr":"172.21.15.16:6825/1330214141","heartbeat_front_addr":"172.21.15.16:6823/1330214141","state":["exists","up"]},{"osd":7,"uuid":"ee826a9c-f441-4903-9b1b-66b627f7b712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6817","nonce":1133876754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6819","nonce":1133876754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6823","nonce":1133876754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6821","nonce":1133876754}]},"public_addr":"172.21.15.107:6817/1133876754","cluster_addr":"172.21.15.107:6819/1133876754","heartbeat_back_addr":"172.21.15.107:6823/1133876754","heartbeat_front_addr":"172.21.15.107:6821/1133876754","state":["exists","up"]},{"osd":8,"uuid":"e19773f2-dec8-44e0-9d33-0252e07d6415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6817","nonce":837548190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6819","nonce":837548190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6823","nonce":837548190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6821","nonce":837548190}]},"public_addr":"172.21.15.111:6817/837548190","cluster_addr":"172.21.15.111:6819/837548190","heartbeat_back_addr":"172.21.15.111:6823/837548190","heartbeat_front_addr":"172.21.15.111:6821/837548190","state":["exists","up"]},{"osd":9,"uuid":"bae4d6db-8b90-4d02-8e83-7d8f2a6f8fdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6827","nonce":4175383686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6829","nonce":4175383686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6833","nonce":4175383686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6831","nonce":4175383686}]},"public_addr":"172.21.15.16:6827/4175383686","cluster_addr":"172.21.15.16:6829/4175383686","heartbeat_back_addr":"172.21.15.16:6833/4175383686","heartbeat_front_addr":"172.21.15.16:6831/4175383686","state":["exists","up"]},{"osd":10,"uuid":"fd0e7740-76c8-4c99-a349-7077b4905143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6825","nonce":2659550426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6827","nonce":2659550426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6831","nonce":2659550426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6829","nonce":2659550426}]},"public_addr":"172.21.15.107:6825/2659550426","cluster_addr":"172.21.15.107:6827/2659550426","heartbeat_back_addr":"172.21.15.107:6831/2659550426","heartbeat_front_addr":"172.21.15.107:6829/2659550426","state":["exists","up"]},{"osd":11,"uuid":"94a1751d-b00c-4e23-815d-3a8bc3dea146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6825","nonce":1633548683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6827","nonce":1633548683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6831","nonce":1633548683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6829","nonce":1633548683}]},"public_addr":"172.21.15.111:6825/1633548683","cluster_addr":"172.21.15.111:6827/1633548683","heartbeat_back_addr":"172.21.15.111:6831/1633548683","heartbeat_front_addr":"172.21.15.111:6829/1633548683","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:36.359852+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:58.014202+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:19.753672+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:43.404817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:06.001677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:29.361587+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:52.554847+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:15.957553+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:39.581093+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:04.911389+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:28.971280+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:53.473561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/2098378667":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/3536229016":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6801/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/2684687547":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6800/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/368410612":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/12158160":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6800/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6801/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/3754423379":"2024-09-14T16:21:40.126459+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:28:36.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.347+0000 7fe1046a3640 1 -- 172.21.15.16:0/447868802 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe0e80a1090 msgr2=0x7fe0e80a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:36.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.347+0000 7fe1046a3640 1 --2- 172.21.15.16:0/447868802 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe0e80a1090 0x7fe0e80a3550 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec02aa00 tx=0x7fe0ec005a20 comp rx=0 tx=0).stop 2024-09-13T16:28:36.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.347+0000 7fe1046a3640 1 -- 172.21.15.16:0/447868802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0fc0feca0 msgr2=0x7fe0fc195d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:36.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.347+0000 7fe1046a3640 1 --2- 172.21.15.16:0/447868802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0fc0feca0 0x7fe0fc195d50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe0f800d950 tx=0x7fe0f800de10 comp rx=0 tx=0).stop 2024-09-13T16:28:36.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.347+0000 7fe1046a3640 1 -- 172.21.15.16:0/447868802 shutdown_connections 2024-09-13T16:28:36.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.347+0000 7fe1046a3640 1 --2- 172.21.15.16:0/447868802 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0fc100850 0x7fe0fc19d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:36.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.347+0000 7fe1046a3640 1 --2- 172.21.15.16:0/447868802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0fc0ffe90 0x7fe0fc196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:36.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.347+0000 7fe1046a3640 1 --2- 172.21.15.16:0/447868802 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe0e80a1090 0x7fe0e80a3550 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:36.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.347+0000 7fe1046a3640 1 --2- 172.21.15.16:0/447868802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0fc0feca0 0x7fe0fc195d50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:36.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.347+0000 7fe1046a3640 1 -- 172.21.15.16:0/447868802 >> 172.21.15.16:0/447868802 conn(0x7fe0fc0fa440 msgr2=0x7fe0fc067690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:36.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.347+0000 7fe1046a3640 1 -- 172.21.15.16:0/447868802 shutdown_connections 2024-09-13T16:28:36.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.347+0000 7fe1046a3640 1 -- 172.21.15.16:0/447868802 wait complete. 2024-09-13T16:28:36.363 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:24:22.981019+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T16:28:36.363 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:28:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.476+0000 7f2c6fc53640 1 -- 172.21.15.16:0/2843223124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c68100880 msgr2=0x7f2c68102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.476+0000 7f2c6fc53640 1 --2- 172.21.15.16:0/2843223124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c68100880 0x7f2c68102c70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2c64009900 tx=0x7f2c6402a410 comp rx=0 tx=0).stop 2024-09-13T16:28:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.476+0000 7f2c6fc53640 1 -- 172.21.15.16:0/2843223124 shutdown_connections 2024-09-13T16:28:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.476+0000 7f2c6fc53640 1 --2- 172.21.15.16:0/2843223124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c68105ae0 0x7f2c68107ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.476+0000 7f2c6fc53640 1 --2- 172.21.15.16:0/2843223124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c681031b0 0x7f2c681055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.476+0000 7f2c6fc53640 1 --2- 172.21.15.16:0/2843223124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c68100880 0x7f2c68102c70 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:36.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.476+0000 7f2c6fc53640 1 -- 172.21.15.16:0/2843223124 >> 172.21.15.16:0/2843223124 conn(0x7f2c680fa3e0 msgr2=0x7f2c680fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.476+0000 7f2c6fc53640 1 -- 172.21.15.16:0/2843223124 shutdown_connections 2024-09-13T16:28:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.476+0000 7f2c6fc53640 1 -- 172.21.15.16:0/2843223124 wait complete. 2024-09-13T16:28:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.477+0000 7f2c6fc53640 1 Processor -- start 2024-09-13T16:28:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.477+0000 7f2c6fc53640 1 -- start start 2024-09-13T16:28:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.477+0000 7f2c6fc53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c68100880 0x7f2c68197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.477+0000 7f2c6fc53640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c681031b0 0x7f2c68198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.477+0000 7f2c6fc53640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c68105ae0 0x7f2c6819f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.477+0000 7f2c6fc53640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6819fa20 con 0x7f2c68100880 2024-09-13T16:28:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.477+0000 7f2c6fc53640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6819fb90 con 0x7f2c68105ae0 2024-09-13T16:28:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.477+0000 7f2c6fc53640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6819fd50 con 0x7f2c681031b0 2024-09-13T16:28:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.477+0000 7f2c6d9ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c68100880 0x7f2c68197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.477+0000 7f2c6d9ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c68100880 0x7f2c68197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33130/0 (socket says 172.21.15.16:33130) 2024-09-13T16:28:36.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.477+0000 7f2c6d9ef640 1 -- 172.21.15.16:0/1417269678 learned_addr learned my addr 172.21.15.16:0/1417269678 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.477+0000 7f2c6d1ee640 1 --2- 172.21.15.16:0/1417269678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c681031b0 0x7f2c68198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.477+0000 7f2c6e1f0640 1 --2- 172.21.15.16:0/1417269678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c68105ae0 0x7f2c6819f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.477+0000 7f2c6d9ef640 1 -- 172.21.15.16:0/1417269678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c681031b0 msgr2=0x7f2c68198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.477+0000 7f2c6d9ef640 1 --2- 172.21.15.16:0/1417269678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c681031b0 0x7f2c68198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.477+0000 7f2c6d9ef640 1 -- 172.21.15.16:0/1417269678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c68105ae0 msgr2=0x7f2c6819f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.477+0000 7f2c6d9ef640 1 --2- 172.21.15.16:0/1417269678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c68105ae0 0x7f2c6819f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.477+0000 7f2c6d9ef640 1 -- 172.21.15.16:0/1417269678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c64009560 con 0x7f2c68100880 2024-09-13T16:28:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.477+0000 7f2c6d1ee640 1 --2- 172.21.15.16:0/1417269678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c681031b0 0x7f2c68198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:28:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.477+0000 7f2c6d9ef640 1 --2- 172.21.15.16:0/1417269678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c68100880 0x7f2c68197f30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2c640057c0 tx=0x7f2c640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.478+0000 7f2c5effd640 1 -- 172.21.15.16:0/1417269678 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c640054b0 con 0x7f2c68100880 2024-09-13T16:28:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.478+0000 7f2c5effd640 1 -- 172.21.15.16:0/1417269678 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c640044e0 con 0x7f2c68100880 2024-09-13T16:28:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.478+0000 7f2c6fc53640 1 -- 172.21.15.16:0/1417269678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c6819ffd0 con 0x7f2c68100880 2024-09-13T16:28:36.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.478+0000 7f2c5effd640 1 -- 172.21.15.16:0/1417269678 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c64003a30 con 0x7f2c68100880 2024-09-13T16:28:36.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.478+0000 7f2c6fc53640 1 -- 172.21.15.16:0/1417269678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c681a03a0 con 0x7f2c68100880 2024-09-13T16:28:36.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.479+0000 7f2c6fc53640 1 -- 172.21.15.16:0/1417269678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c681075d0 con 0x7f2c68100880 2024-09-13T16:28:36.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.480+0000 7f2c5effd640 1 -- 172.21.15.16:0/1417269678 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f2c64003b90 con 0x7f2c68100880 2024-09-13T16:28:36.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.481+0000 7f2c5effd640 1 --2- 172.21.15.16:0/1417269678 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f2c540a1090 0x7f2c540a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:36.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.481+0000 7f2c5effd640 1 -- 172.21.15.16:0/1417269678 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2c640ed850 con 0x7f2c68100880 2024-09-13T16:28:36.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.481+0000 7f2c6d1ee640 1 --2- 172.21.15.16:0/1417269678 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f2c540a1090 0x7f2c540a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:36.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.481+0000 7f2c6d1ee640 1 --2- 172.21.15.16:0/1417269678 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f2c540a1090 0x7f2c540a3550 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2c68199440 tx=0x7f2c58005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:36.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.484+0000 7f2c5effd640 1 -- 172.21.15.16:0/1417269678 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f2c640b7530 con 0x7f2c68100880 2024-09-13T16:28:36.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.670+0000 7f2c6fc53640 1 -- 172.21.15.16:0/1417269678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2c6810b760 con 0x7f2c68100880 2024-09-13T16:28:36.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.670+0000 7f2c5effd640 1 -- 172.21.15.16:0/1417269678 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2c640bab50 con 0x7f2c68100880 2024-09-13T16:28:36.672 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-13T16:28:36.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.672+0000 7f2c6fc53640 1 -- 172.21.15.16:0/1417269678 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f2c540a1090 msgr2=0x7f2c540a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:36.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.672+0000 7f2c6fc53640 1 --2- 172.21.15.16:0/1417269678 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f2c540a1090 0x7f2c540a3550 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2c68199440 tx=0x7f2c58005f90 comp rx=0 tx=0).stop 2024-09-13T16:28:36.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.672+0000 7f2c6fc53640 1 -- 172.21.15.16:0/1417269678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c68100880 msgr2=0x7f2c68197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:36.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.672+0000 7f2c6fc53640 1 --2- 172.21.15.16:0/1417269678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c68100880 0x7f2c68197f30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2c640057c0 tx=0x7f2c640042f0 comp rx=0 tx=0).stop 2024-09-13T16:28:36.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.672+0000 7f2c6fc53640 1 -- 172.21.15.16:0/1417269678 shutdown_connections 2024-09-13T16:28:36.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.672+0000 7f2c6fc53640 1 --2- 172.21.15.16:0/1417269678 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f2c540a1090 0x7f2c540a3550 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:36.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.672+0000 7f2c6fc53640 1 --2- 172.21.15.16:0/1417269678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c68105ae0 0x7f2c6819f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:36.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.672+0000 7f2c6fc53640 1 --2- 172.21.15.16:0/1417269678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c681031b0 0x7f2c68198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:36.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.672+0000 7f2c6fc53640 1 --2- 172.21.15.16:0/1417269678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c68100880 0x7f2c68197f30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:36.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.672+0000 7f2c6fc53640 1 -- 172.21.15.16:0/1417269678 >> 172.21.15.16:0/1417269678 conn(0x7f2c680fa3e0 msgr2=0x7f2c680feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:36.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.672+0000 7f2c6fc53640 1 -- 172.21.15.16:0/1417269678 shutdown_connections 2024-09-13T16:28:36.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.672+0000 7f2c6fc53640 1 -- 172.21.15.16:0/1417269678 wait complete. 2024-09-13T16:28:36.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:36 smithi016 ceph-mon[31187]: mds.? [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] up:boot 2024-09-13T16:28:36.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:36 smithi016 ceph-mon[31187]: fsmap 12 up:standby 2024-09-13T16:28:36.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:36 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-13T16:28:36.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:36 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/4213396696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:36.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:36 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/447868802' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:28:36.685 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T16:28:36.686 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:36 smithi111 ceph-mon[34600]: mds.? [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] up:boot 2024-09-13T16:28:36.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:36 smithi111 ceph-mon[34600]: fsmap 12 up:standby 2024-09-13T16:28:36.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:36 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-13T16:28:36.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:36 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/4213396696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:36.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:36 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/447868802' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:28:36.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.785+0000 7fa994d25640 1 -- 172.21.15.16:0/542192648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9900ffec0 msgr2=0x7fa990100340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.785+0000 7fa994d25640 1 --2- 172.21.15.16:0/542192648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9900ffec0 0x7fa990100340 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa984009900 tx=0x7fa98402a410 comp rx=0 tx=0).stop 2024-09-13T16:28:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.785+0000 7fa994d25640 1 -- 172.21.15.16:0/542192648 shutdown_connections 2024-09-13T16:28:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.785+0000 7fa994d25640 1 --2- 172.21.15.16:0/542192648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa990100880 0x7fa990107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.785+0000 7fa994d25640 1 --2- 172.21.15.16:0/542192648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9900ffec0 0x7fa990100340 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.785+0000 7fa994d25640 1 --2- 172.21.15.16:0/542192648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9900fecd0 0x7fa9900ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.785+0000 7fa994d25640 1 -- 172.21.15.16:0/542192648 >> 172.21.15.16:0/542192648 conn(0x7fa9900fa430 msgr2=0x7fa9900fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.785+0000 7fa994d25640 1 -- 172.21.15.16:0/542192648 shutdown_connections 2024-09-13T16:28:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.785+0000 7fa994d25640 1 -- 172.21.15.16:0/542192648 wait complete. 2024-09-13T16:28:36.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.786+0000 7fa994d25640 1 Processor -- start 2024-09-13T16:28:36.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.786+0000 7fa994d25640 1 -- start start 2024-09-13T16:28:36.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.786+0000 7fa994d25640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9900fecd0 0x7fa990102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:36.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.786+0000 7fa994d25640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9900ffec0 0x7fa9901015c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:36.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.786+0000 7fa994d25640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa990100880 0x7fa990101b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:36.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.786+0000 7fa994d25640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa990102260 con 0x7fa9900fecd0 2024-09-13T16:28:36.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.786+0000 7fa994d25640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9901023d0 con 0x7fa9900ffec0 2024-09-13T16:28:36.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.786+0000 7fa994d25640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99019fdc0 con 0x7fa990100880 2024-09-13T16:28:36.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.786+0000 7fa98e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9900fecd0 0x7fa990102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:36.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.786+0000 7fa98e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9900fecd0 0x7fa990102f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33152/0 (socket says 172.21.15.16:33152) 2024-09-13T16:28:36.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.786+0000 7fa98e59c640 1 -- 172.21.15.16:0/4187993205 learned_addr learned my addr 172.21.15.16:0/4187993205 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:36.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.786+0000 7fa98ed9d640 1 --2- 172.21.15.16:0/4187993205 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa990100880 0x7fa990101b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:36.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.786+0000 7fa98dd9b640 1 --2- 172.21.15.16:0/4187993205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9900ffec0 0x7fa9901015c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:36.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.786+0000 7fa98e59c640 1 -- 172.21.15.16:0/4187993205 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa990100880 msgr2=0x7fa990101b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:36.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.786+0000 7fa98e59c640 1 --2- 172.21.15.16:0/4187993205 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa990100880 0x7fa990101b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:36.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.786+0000 7fa98e59c640 1 -- 172.21.15.16:0/4187993205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9900ffec0 msgr2=0x7fa9901015c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:36.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.786+0000 7fa98e59c640 1 --2- 172.21.15.16:0/4187993205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9900ffec0 0x7fa9901015c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:36.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.786+0000 7fa98e59c640 1 -- 172.21.15.16:0/4187993205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa984009560 con 0x7fa9900fecd0 2024-09-13T16:28:36.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.787+0000 7fa98e59c640 1 --2- 172.21.15.16:0/4187993205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9900fecd0 0x7fa990102f80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa98000e9f0 tx=0x7fa98000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:36.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.787+0000 7fa97f7fe640 1 -- 172.21.15.16:0/4187993205 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa980009700 con 0x7fa9900fecd0 2024-09-13T16:28:36.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.787+0000 7fa97f7fe640 1 -- 172.21.15.16:0/4187993205 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa9800044b0 con 0x7fa9900fecd0 2024-09-13T16:28:36.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.787+0000 7fa994d25640 1 -- 172.21.15.16:0/4187993205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9901a0070 con 0x7fa9900fecd0 2024-09-13T16:28:36.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.787+0000 7fa97f7fe640 1 -- 172.21.15.16:0/4187993205 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa980010430 con 0x7fa9900fecd0 2024-09-13T16:28:36.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.787+0000 7fa994d25640 1 -- 172.21.15.16:0/4187993205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9901a05a0 con 0x7fa9900fecd0 2024-09-13T16:28:36.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.788+0000 7fa994d25640 1 -- 172.21.15.16:0/4187993205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9900ff0d0 con 0x7fa9900fecd0 2024-09-13T16:28:36.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.789+0000 7fa97f7fe640 1 -- 172.21.15.16:0/4187993205 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa98000cd80 con 0x7fa9900fecd0 2024-09-13T16:28:36.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.790+0000 7fa97f7fe640 1 --2- 172.21.15.16:0/4187993205 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa9780a1110 0x7fa9780a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:36.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.790+0000 7fa98dd9b640 1 --2- 172.21.15.16:0/4187993205 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa9780a1110 0x7fa9780a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:36.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.790+0000 7fa97f7fe640 1 -- 172.21.15.16:0/4187993205 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa980014070 con 0x7fa9900fecd0 2024-09-13T16:28:36.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.790+0000 7fa98dd9b640 1 --2- 172.21.15.16:0/4187993205 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa9780a1110 0x7fa9780a35d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa98402aa00 tx=0x7fa984005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:36.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.794+0000 7fa97f7fe640 1 -- 172.21.15.16:0/4187993205 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa980096350 con 0x7fa9900fecd0 2024-09-13T16:28:36.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:36 smithi107 ceph-mon[34139]: mds.? [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] up:boot 2024-09-13T16:28:36.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:36 smithi107 ceph-mon[34139]: fsmap 12 up:standby 2024-09-13T16:28:36.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:36 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-13T16:28:36.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:36 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/4213396696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:36.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:36 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/447868802' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:28:36.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.997+0000 7fa994d25640 1 -- 172.21.15.16:0/4187993205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa9901a08c0 con 0x7fa9900fecd0 2024-09-13T16:28:36.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.997+0000 7fa97f7fe640 1 -- 172.21.15.16:0/4187993205 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa980099970 con 0x7fa9900fecd0 2024-09-13T16:28:36.999 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:28:36.999 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-13T16:28:36.999 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:28:37.000 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-13T16:28:37.000 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-13T16:28:37.000 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:28:37.000 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-13T16:28:37.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.999+0000 7fa994d25640 1 -- 172.21.15.16:0/4187993205 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa9780a1110 msgr2=0x7fa9780a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:37.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.999+0000 7fa994d25640 1 --2- 172.21.15.16:0/4187993205 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa9780a1110 0x7fa9780a35d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa98402aa00 tx=0x7fa984005a20 comp rx=0 tx=0).stop 2024-09-13T16:28:37.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.999+0000 7fa994d25640 1 -- 172.21.15.16:0/4187993205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9900fecd0 msgr2=0x7fa990102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:37.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.999+0000 7fa994d25640 1 --2- 172.21.15.16:0/4187993205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9900fecd0 0x7fa990102f80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa98000e9f0 tx=0x7fa98000eeb0 comp rx=0 tx=0).stop 2024-09-13T16:28:37.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.999+0000 7fa994d25640 1 -- 172.21.15.16:0/4187993205 shutdown_connections 2024-09-13T16:28:37.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.999+0000 7fa994d25640 1 --2- 172.21.15.16:0/4187993205 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa990100880 0x7fa990101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:37.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.999+0000 7fa994d25640 1 --2- 172.21.15.16:0/4187993205 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa9780a1110 0x7fa9780a35d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:37.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.999+0000 7fa994d25640 1 --2- 172.21.15.16:0/4187993205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9900ffec0 0x7fa9901015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:37.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.999+0000 7fa994d25640 1 --2- 172.21.15.16:0/4187993205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9900fecd0 0x7fa990102f80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:37.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.999+0000 7fa994d25640 1 -- 172.21.15.16:0/4187993205 >> 172.21.15.16:0/4187993205 conn(0x7fa9900fa430 msgr2=0x7fa9900685f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:37.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.999+0000 7fa994d25640 1 -- 172.21.15.16:0/4187993205 shutdown_connections 2024-09-13T16:28:37.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:36.999+0000 7fa994d25640 1 -- 172.21.15.16:0/4187993205 wait complete. 2024-09-13T16:28:37.013 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-13T16:28:37.013 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-13T16:28:37.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.117+0000 7f8e26441640 1 -- 172.21.15.16:0/4244829045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e20100810 msgr2=0x7f8e201070a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:37.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.117+0000 7f8e26441640 1 --2- 172.21.15.16:0/4244829045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e20100810 0x7f8e201070a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8e18009ca0 tx=0x7f8e1802a8c0 comp rx=0 tx=0).stop 2024-09-13T16:28:37.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.117+0000 7f8e26441640 1 -- 172.21.15.16:0/4244829045 shutdown_connections 2024-09-13T16:28:37.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.117+0000 7f8e26441640 1 --2- 172.21.15.16:0/4244829045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e20100810 0x7f8e201070a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:37.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.117+0000 7f8e26441640 1 --2- 172.21.15.16:0/4244829045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e200ffe50 0x7f8e201002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:37.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.117+0000 7f8e26441640 1 --2- 172.21.15.16:0/4244829045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e200fec60 0x7f8e200ff060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:37.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.117+0000 7f8e26441640 1 -- 172.21.15.16:0/4244829045 >> 172.21.15.16:0/4244829045 conn(0x7f8e200fa3e0 msgr2=0x7f8e200fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:37.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.118+0000 7f8e26441640 1 -- 172.21.15.16:0/4244829045 shutdown_connections 2024-09-13T16:28:37.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.118+0000 7f8e26441640 1 -- 172.21.15.16:0/4244829045 wait complete. 2024-09-13T16:28:37.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.118+0000 7f8e26441640 1 Processor -- start 2024-09-13T16:28:37.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.118+0000 7f8e26441640 1 -- start start 2024-09-13T16:28:37.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.118+0000 7f8e26441640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e200fec60 0x7f8e20195c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:37.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.118+0000 7f8e26441640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e200ffe50 0x7f8e20196160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:37.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.118+0000 7f8e26441640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e20100810 0x7f8e2019d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:37.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.118+0000 7f8e26441640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2019d700 con 0x7f8e200ffe50 2024-09-13T16:28:37.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.119+0000 7f8e26441640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2019d870 con 0x7f8e200fec60 2024-09-13T16:28:37.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.119+0000 7f8e26441640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2019da30 con 0x7f8e20100810 2024-09-13T16:28:37.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.119+0000 7f8e1f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e200ffe50 0x7f8e20196160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:37.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.119+0000 7f8e1ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e200fec60 0x7f8e20195c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:37.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.119+0000 7f8e1ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e200fec60 0x7f8e20195c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.16:54956/0 (socket says 172.21.15.16:54956) 2024-09-13T16:28:37.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.119+0000 7f8e1ffff640 1 -- 172.21.15.16:0/306652787 learned_addr learned my addr 172.21.15.16:0/306652787 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:37.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.119+0000 7f8e249de640 1 --2- 172.21.15.16:0/306652787 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e20100810 0x7f8e2019d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:37.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.119+0000 7f8e249de640 1 -- 172.21.15.16:0/306652787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e200fec60 msgr2=0x7f8e20195c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:37.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.119+0000 7f8e249de640 1 --2- 172.21.15.16:0/306652787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e200fec60 0x7f8e20195c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:37.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.119+0000 7f8e249de640 1 -- 172.21.15.16:0/306652787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e200ffe50 msgr2=0x7f8e20196160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:37.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.119+0000 7f8e249de640 1 --2- 172.21.15.16:0/306652787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e200ffe50 0x7f8e20196160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:37.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.119+0000 7f8e249de640 1 -- 172.21.15.16:0/306652787 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e18009900 con 0x7f8e20100810 2024-09-13T16:28:37.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.119+0000 7f8e1ffff640 1 --2- 172.21.15.16:0/306652787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e200fec60 0x7f8e20195c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:28:37.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.119+0000 7f8e1f7fe640 1 --2- 172.21.15.16:0/306652787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e200ffe50 0x7f8e20196160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:28:37.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.119+0000 7f8e249de640 1 --2- 172.21.15.16:0/306652787 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e20100810 0x7f8e2019d190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8e1802aeb0 tx=0x7f8e18003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:37.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.120+0000 7f8e1d7fa640 1 -- 172.21.15.16:0/306652787 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e180307c0 con 0x7f8e20100810 2024-09-13T16:28:37.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.120+0000 7f8e1d7fa640 1 -- 172.21.15.16:0/306652787 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8e18040070 con 0x7f8e20100810 2024-09-13T16:28:37.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.120+0000 7f8e1d7fa640 1 -- 172.21.15.16:0/306652787 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e180043c0 con 0x7f8e20100810 2024-09-13T16:28:37.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.120+0000 7f8e26441640 1 -- 172.21.15.16:0/306652787 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e2019dcb0 con 0x7f8e20100810 2024-09-13T16:28:37.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.120+0000 7f8e26441640 1 -- 172.21.15.16:0/306652787 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e2019e1c0 con 0x7f8e20100810 2024-09-13T16:28:37.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.121+0000 7f8e26441640 1 -- 172.21.15.16:0/306652787 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e20106190 con 0x7f8e20100810 2024-09-13T16:28:37.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.122+0000 7f8e1d7fa640 1 -- 172.21.15.16:0/306652787 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8e18042030 con 0x7f8e20100810 2024-09-13T16:28:37.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.123+0000 7f8e1d7fa640 1 --2- 172.21.15.16:0/306652787 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8e080a1110 0x7f8e080a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:37.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.123+0000 7f8e1ffff640 1 --2- 172.21.15.16:0/306652787 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8e080a1110 0x7f8e080a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:37.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.123+0000 7f8e1d7fa640 1 -- 172.21.15.16:0/306652787 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8e180edb50 con 0x7f8e20100810 2024-09-13T16:28:37.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.123+0000 7f8e1ffff640 1 --2- 172.21.15.16:0/306652787 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8e080a1110 0x7f8e080a35d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8e201970f0 tx=0x7f8e1000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:37.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.126+0000 7f8e1d7fa640 1 -- 172.21.15.16:0/306652787 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f8e180b7650 con 0x7f8e20100810 2024-09-13T16:28:37.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.310+0000 7f8e26441640 1 -- 172.21.15.16:0/306652787 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8e200681f0 con 0x7f8e20100810 2024-09-13T16:28:37.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.828+0000 7f8e1d7fa640 1 -- 172.21.15.16:0/306652787 <== mon.1 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8e180b9580 con 0x7f8e20100810 2024-09-13T16:28:37.832 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-13T16:28:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.831+0000 7f8e26441640 1 -- 172.21.15.16:0/306652787 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8e080a1110 msgr2=0x7f8e080a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.831+0000 7f8e26441640 1 --2- 172.21.15.16:0/306652787 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8e080a1110 0x7f8e080a35d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8e201970f0 tx=0x7f8e1000b410 comp rx=0 tx=0).stop 2024-09-13T16:28:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.831+0000 7f8e26441640 1 -- 172.21.15.16:0/306652787 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e20100810 msgr2=0x7f8e2019d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.831+0000 7f8e26441640 1 --2- 172.21.15.16:0/306652787 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e20100810 0x7f8e2019d190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8e1802aeb0 tx=0x7f8e18003c20 comp rx=0 tx=0).stop 2024-09-13T16:28:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.831+0000 7f8e26441640 1 -- 172.21.15.16:0/306652787 shutdown_connections 2024-09-13T16:28:37.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.831+0000 7f8e26441640 1 --2- 172.21.15.16:0/306652787 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e20100810 0x7f8e2019d190 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:37.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.831+0000 7f8e26441640 1 --2- 172.21.15.16:0/306652787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e200ffe50 0x7f8e20196160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:37.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.831+0000 7f8e26441640 1 --2- 172.21.15.16:0/306652787 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8e080a1110 0x7f8e080a35d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:37.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.831+0000 7f8e26441640 1 --2- 172.21.15.16:0/306652787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e200fec60 0x7f8e20195c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:37.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.831+0000 7f8e26441640 1 -- 172.21.15.16:0/306652787 >> 172.21.15.16:0/306652787 conn(0x7f8e200fa3e0 msgr2=0x7f8e20103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:37.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.831+0000 7f8e26441640 1 -- 172.21.15.16:0/306652787 shutdown_connections 2024-09-13T16:28:37.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.831+0000 7f8e26441640 1 -- 172.21.15.16:0/306652787 wait complete. 2024-09-13T16:28:37.844 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:37 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:37.844 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:37 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:37.844 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:37 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:37.844 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:37 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:37.844 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:37 smithi016 ceph-mon[31187]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:37.845 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:37 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:37.845 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:37 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:37.845 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:37 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1417269678' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:28:37.845 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:37 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:37.845 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:37 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:28:37.845 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:37 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:37.845 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:37 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:28:37.846 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:37 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/4187993205' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-13T16:28:37.846 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:37 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/306652787' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-13T16:28:37.846 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:37 smithi016 ceph-mon[31187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-13T16:28:37.847 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-13T16:28:37.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:37 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:37.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:37 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:37.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:37 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:37.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:37 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:37.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:37 smithi111 ceph-mon[34600]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:37.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:37 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:37.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:37 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:37.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:37 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1417269678' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:28:37.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:37 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:37.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:37 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:28:37.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:37 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:37.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:37 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:28:37.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:37 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/4187993205' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-13T16:28:37.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:37 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/306652787' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-13T16:28:37.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:37 smithi111 ceph-mon[34600]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-13T16:28:37.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.955+0000 7fc11bd45640 1 -- 172.21.15.16:0/531002878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1140ff600 msgr2=0x7fc1140ffa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:37.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.955+0000 7fc11bd45640 1 --2- 172.21.15.16:0/531002878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1140ff600 0x7fc1140ffa80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc104009900 tx=0x7fc10402a410 comp rx=0 tx=0).stop 2024-09-13T16:28:37.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.955+0000 7fc11bd45640 1 -- 172.21.15.16:0/531002878 shutdown_connections 2024-09-13T16:28:37.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.955+0000 7fc11bd45640 1 --2- 172.21.15.16:0/531002878 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1140fffc0 0x7fc114106850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:37.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.955+0000 7fc11bd45640 1 --2- 172.21.15.16:0/531002878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1140ff600 0x7fc1140ffa80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:37.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.955+0000 7fc11bd45640 1 --2- 172.21.15.16:0/531002878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1140fe410 0x7fc1140fe810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:37.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.955+0000 7fc11bd45640 1 -- 172.21.15.16:0/531002878 >> 172.21.15.16:0/531002878 conn(0x7fc1140f9b70 msgr2=0x7fc1140fbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:37.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.955+0000 7fc11bd45640 1 -- 172.21.15.16:0/531002878 shutdown_connections 2024-09-13T16:28:37.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.955+0000 7fc11bd45640 1 -- 172.21.15.16:0/531002878 wait complete. 2024-09-13T16:28:37.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.955+0000 7fc11bd45640 1 Processor -- start 2024-09-13T16:28:37.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.955+0000 7fc11bd45640 1 -- start start 2024-09-13T16:28:37.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.956+0000 7fc11bd45640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1140fe410 0x7fc114195c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:37.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.956+0000 7fc11bd45640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1140ff600 0x7fc114196180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:37.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.956+0000 7fc11bd45640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1140fffc0 0x7fc11419d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:37.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.956+0000 7fc11bd45640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11419d720 con 0x7fc1140fe410 2024-09-13T16:28:37.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.956+0000 7fc11bd45640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11419d890 con 0x7fc1140ff600 2024-09-13T16:28:37.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.956+0000 7fc11bd45640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11419da50 con 0x7fc1140fffc0 2024-09-13T16:28:37.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.956+0000 7fc119ae1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1140fe410 0x7fc114195c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:37.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.956+0000 7fc119ae1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1140fe410 0x7fc114195c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33186/0 (socket says 172.21.15.16:33186) 2024-09-13T16:28:37.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.956+0000 7fc119ae1640 1 -- 172.21.15.16:0/2590873044 learned_addr learned my addr 172.21.15.16:0/2590873044 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:37.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.956+0000 7fc1192e0640 1 --2- 172.21.15.16:0/2590873044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1140ff600 0x7fc114196180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:37.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.956+0000 7fc11a2e2640 1 --2- 172.21.15.16:0/2590873044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1140fffc0 0x7fc11419d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:37.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.956+0000 7fc119ae1640 1 -- 172.21.15.16:0/2590873044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1140fffc0 msgr2=0x7fc11419d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:37.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.956+0000 7fc119ae1640 1 --2- 172.21.15.16:0/2590873044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1140fffc0 0x7fc11419d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:37.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.956+0000 7fc119ae1640 1 -- 172.21.15.16:0/2590873044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1140ff600 msgr2=0x7fc114196180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:37.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.956+0000 7fc119ae1640 1 --2- 172.21.15.16:0/2590873044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1140ff600 0x7fc114196180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:37.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.956+0000 7fc119ae1640 1 -- 172.21.15.16:0/2590873044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc104009560 con 0x7fc1140fe410 2024-09-13T16:28:37.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.956+0000 7fc119ae1640 1 --2- 172.21.15.16:0/2590873044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1140fe410 0x7fc114195c40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc11000c970 tx=0x7fc11000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:37.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.956+0000 7fc10affd640 1 -- 172.21.15.16:0/2590873044 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc110007c10 con 0x7fc1140fe410 2024-09-13T16:28:37.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.956+0000 7fc10affd640 1 -- 172.21.15.16:0/2590873044 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc110007d70 con 0x7fc1140fe410 2024-09-13T16:28:37.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.956+0000 7fc11bd45640 1 -- 172.21.15.16:0/2590873044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11419dd30 con 0x7fc1140fe410 2024-09-13T16:28:37.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.956+0000 7fc10affd640 1 -- 172.21.15.16:0/2590873044 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11000f720 con 0x7fc1140fe410 2024-09-13T16:28:37.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.957+0000 7fc11bd45640 1 -- 172.21.15.16:0/2590873044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11419e190 con 0x7fc1140fe410 2024-09-13T16:28:37.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.957+0000 7fc11bd45640 1 -- 172.21.15.16:0/2590873044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1141058a0 con 0x7fc1140fe410 2024-09-13T16:28:37.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.959+0000 7fc10affd640 1 -- 172.21.15.16:0/2590873044 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fc110018480 con 0x7fc1140fe410 2024-09-13T16:28:37.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.959+0000 7fc10affd640 1 --2- 172.21.15.16:0/2590873044 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc1000a1090 0x7fc1000a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:37.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.959+0000 7fc1192e0640 1 --2- 172.21.15.16:0/2590873044 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc1000a1090 0x7fc1000a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:37.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.959+0000 7fc10affd640 1 -- 172.21.15.16:0/2590873044 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fc110013070 con 0x7fc1140fe410 2024-09-13T16:28:37.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.960+0000 7fc1192e0640 1 --2- 172.21.15.16:0/2590873044 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc1000a1090 0x7fc1000a3550 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc10402aa00 tx=0x7fc104005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:37.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:37.963+0000 7fc10affd640 1 -- 172.21.15.16:0/2590873044 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc1100966d0 con 0x7fc1140fe410 2024-09-13T16:28:38.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:37 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:38.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:37 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:38.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:37 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:38.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:37 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:38.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:37 smithi107 ceph-mon[34139]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:38.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:37 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:38.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:37 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:38.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:37 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1417269678' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:28:38.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:37 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:38.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:37 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:28:38.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:37 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:38.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:37 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:28:38.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:37 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/4187993205' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-13T16:28:38.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:37 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/306652787' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-13T16:28:38.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:37 smithi107 ceph-mon[34139]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-13T16:28:38.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.160+0000 7fc11bd45640 1 -- 172.21.15.16:0/2590873044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fc11410b580 con 0x7fc1140fe410 2024-09-13T16:28:38.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.836+0000 7fc10affd640 1 -- 172.21.15.16:0/2590873044 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v83) v1 ==== 171+0+0 (secure 0 0 0) 0x7fc110099cf0 con 0x7fc1140fe410 2024-09-13T16:28:38.841 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-13T16:28:38.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.840+0000 7fc11bd45640 1 -- 172.21.15.16:0/2590873044 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc1000a1090 msgr2=0x7fc1000a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:38.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.840+0000 7fc11bd45640 1 --2- 172.21.15.16:0/2590873044 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc1000a1090 0x7fc1000a3550 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc10402aa00 tx=0x7fc104005a20 comp rx=0 tx=0).stop 2024-09-13T16:28:38.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.840+0000 7fc11bd45640 1 -- 172.21.15.16:0/2590873044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1140fe410 msgr2=0x7fc114195c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:38.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.840+0000 7fc11bd45640 1 --2- 172.21.15.16:0/2590873044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1140fe410 0x7fc114195c40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc11000c970 tx=0x7fc11000ce30 comp rx=0 tx=0).stop 2024-09-13T16:28:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.841+0000 7fc11bd45640 1 -- 172.21.15.16:0/2590873044 shutdown_connections 2024-09-13T16:28:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.841+0000 7fc11bd45640 1 --2- 172.21.15.16:0/2590873044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1140fffc0 0x7fc11419d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.841+0000 7fc11bd45640 1 --2- 172.21.15.16:0/2590873044 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc1000a1090 0x7fc1000a3550 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.841+0000 7fc11bd45640 1 --2- 172.21.15.16:0/2590873044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1140ff600 0x7fc114196180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.841+0000 7fc11bd45640 1 --2- 172.21.15.16:0/2590873044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1140fe410 0x7fc114195c40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.841+0000 7fc11bd45640 1 -- 172.21.15.16:0/2590873044 >> 172.21.15.16:0/2590873044 conn(0x7fc1140f9b70 msgr2=0x7fc114102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.841+0000 7fc11bd45640 1 -- 172.21.15.16:0/2590873044 shutdown_connections 2024-09-13T16:28:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.842+0000 7fc11bd45640 1 -- 172.21.15.16:0/2590873044 wait complete. 2024-09-13T16:28:38.859 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-13T16:28:38.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.962+0000 7fa4643db640 1 -- 172.21.15.16:0/676008807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa45c103140 msgr2=0x7fa45c105530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:38.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.962+0000 7fa4643db640 1 --2- 172.21.15.16:0/676008807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa45c103140 0x7fa45c105530 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa44c009900 tx=0x7fa44c02a410 comp rx=0 tx=0).stop 2024-09-13T16:28:38.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.962+0000 7fa4643db640 1 -- 172.21.15.16:0/676008807 shutdown_connections 2024-09-13T16:28:38.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.962+0000 7fa4643db640 1 --2- 172.21.15.16:0/676008807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa45c105a70 0x7fa45c107ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:38.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.962+0000 7fa4643db640 1 --2- 172.21.15.16:0/676008807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa45c103140 0x7fa45c105530 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:38.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.962+0000 7fa4643db640 1 --2- 172.21.15.16:0/676008807 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa45c100810 0x7fa45c102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:38.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.962+0000 7fa4643db640 1 -- 172.21.15.16:0/676008807 >> 172.21.15.16:0/676008807 conn(0x7fa45c0fa3f0 msgr2=0x7fa45c0fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:38.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.962+0000 7fa4643db640 1 -- 172.21.15.16:0/676008807 shutdown_connections 2024-09-13T16:28:38.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.962+0000 7fa4643db640 1 -- 172.21.15.16:0/676008807 wait complete. 2024-09-13T16:28:38.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.963+0000 7fa4643db640 1 Processor -- start 2024-09-13T16:28:38.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.963+0000 7fa4643db640 1 -- start start 2024-09-13T16:28:38.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.963+0000 7fa4643db640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa45c100810 0x7fa45c100180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:38.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.963+0000 7fa4643db640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa45c103140 0x7fa45c0fe7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:38.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.963+0000 7fa4643db640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa45c105a70 0x7fa45c0fed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:38.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.963+0000 7fa4643db640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c0ff460 con 0x7fa45c100810 2024-09-13T16:28:38.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.963+0000 7fa4643db640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c0ff5d0 con 0x7fa45c103140 2024-09-13T16:28:38.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.963+0000 7fa4643db640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c19fd70 con 0x7fa45c105a70 2024-09-13T16:28:38.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.963+0000 7fa462177640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa45c100810 0x7fa45c100180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:38.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.963+0000 7fa462177640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa45c100810 0x7fa45c100180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33206/0 (socket says 172.21.15.16:33206) 2024-09-13T16:28:38.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.963+0000 7fa462177640 1 -- 172.21.15.16:0/1951427821 learned_addr learned my addr 172.21.15.16:0/1951427821 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:38.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.963+0000 7fa461976640 1 --2- 172.21.15.16:0/1951427821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa45c103140 0x7fa45c0fe7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:38.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.963+0000 7fa462978640 1 --2- 172.21.15.16:0/1951427821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa45c105a70 0x7fa45c0fed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:38.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.963+0000 7fa462177640 1 -- 172.21.15.16:0/1951427821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa45c105a70 msgr2=0x7fa45c0fed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:38.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.963+0000 7fa462177640 1 --2- 172.21.15.16:0/1951427821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa45c105a70 0x7fa45c0fed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:38.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.963+0000 7fa462177640 1 -- 172.21.15.16:0/1951427821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa45c103140 msgr2=0x7fa45c0fe7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:38.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.963+0000 7fa462177640 1 --2- 172.21.15.16:0/1951427821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa45c103140 0x7fa45c0fe7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:38.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.963+0000 7fa462177640 1 -- 172.21.15.16:0/1951427821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44c009560 con 0x7fa45c100810 2024-09-13T16:28:38.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.964+0000 7fa462177640 1 --2- 172.21.15.16:0/1951427821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa45c100810 0x7fa45c100180 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa45800ea20 tx=0x7fa45800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:38.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.964+0000 7fa4537fe640 1 -- 172.21.15.16:0/1951427821 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa458009700 con 0x7fa45c100810 2024-09-13T16:28:38.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.964+0000 7fa4537fe640 1 -- 172.21.15.16:0/1951427821 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa45800cd80 con 0x7fa45c100810 2024-09-13T16:28:38.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.964+0000 7fa4643db640 1 -- 172.21.15.16:0/1951427821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa45c19ffc0 con 0x7fa45c100810 2024-09-13T16:28:38.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.964+0000 7fa4537fe640 1 -- 172.21.15.16:0/1951427821 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa458018660 con 0x7fa45c100810 2024-09-13T16:28:38.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.964+0000 7fa4643db640 1 -- 172.21.15.16:0/1951427821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa45c1a0440 con 0x7fa45c100810 2024-09-13T16:28:38.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.965+0000 7fa4643db640 1 -- 172.21.15.16:0/1951427821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa45c0fbee0 con 0x7fa45c100810 2024-09-13T16:28:38.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.966+0000 7fa4537fe640 1 -- 172.21.15.16:0/1951427821 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa458010430 con 0x7fa45c100810 2024-09-13T16:28:38.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.967+0000 7fa4537fe640 1 --2- 172.21.15.16:0/1951427821 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa4480a1090 0x7fa4480a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:38.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.967+0000 7fa461976640 1 --2- 172.21.15.16:0/1951427821 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa4480a1090 0x7fa4480a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:38.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.967+0000 7fa4537fe640 1 -- 172.21.15.16:0/1951427821 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7879+0+0 (secure 0 0 0) 0x7fa458014070 con 0x7fa45c100810 2024-09-13T16:28:38.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.967+0000 7fa461976640 1 --2- 172.21.15.16:0/1951427821 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa4480a1090 0x7fa4480a3550 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa44c0057c0 tx=0x7fa44c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:38.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:38.972+0000 7fa4537fe640 1 -- 172.21.15.16:0/1951427821 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa458052e20 con 0x7fa45c100810 2024-09-13T16:28:39.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:38 smithi107 ceph-mon[34139]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-13T16:28:39.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:38 smithi107 ceph-mon[34139]: osdmap e82: 12 total, 12 up, 12 in 2024-09-13T16:28:39.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:38 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/2590873044' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-13T16:28:39.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:38 smithi107 ceph-mon[34139]: pgmap v269: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:39.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:38 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:39.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:38 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:39.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:38 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:39.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:38 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:39.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:38 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:39.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:38 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:39.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:38 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:39.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:38 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:28:39.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:38 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:39.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:38 smithi016 ceph-mon[31187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-13T16:28:39.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:38 smithi016 ceph-mon[31187]: osdmap e82: 12 total, 12 up, 12 in 2024-09-13T16:28:39.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:38 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2590873044' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-13T16:28:39.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:38 smithi016 ceph-mon[31187]: pgmap v269: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:39.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:38 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:39.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:38 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:39.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:38 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:39.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:38 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:39.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:38 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:39.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:38 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:39.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:38 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:39.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:38 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:28:39.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:38 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:39.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.182+0000 7fa4643db640 1 -- 172.21.15.16:0/1951427821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa45c10b510 con 0x7fa45c100810 2024-09-13T16:28:39.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:38 smithi111 ceph-mon[34600]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-13T16:28:39.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:38 smithi111 ceph-mon[34600]: osdmap e82: 12 total, 12 up, 12 in 2024-09-13T16:28:39.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:38 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/2590873044' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-13T16:28:39.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:38 smithi111 ceph-mon[34600]: pgmap v269: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T16:28:39.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:38 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:39.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:38 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:39.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:38 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:39.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:38 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:39.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:38 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:39.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:38 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:39.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:38 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:28:39.189 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:38 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:28:39.189 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:38 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:39.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.188+0000 7fa4537fe640 1 -- 172.21.15.16:0/1951427821 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fa458096440 con 0x7fa45c100810 2024-09-13T16:28:39.192 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 2 and data pool 3 2024-09-13T16:28:39.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.195+0000 7fa4643db640 1 -- 172.21.15.16:0/1951427821 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa4480a1090 msgr2=0x7fa4480a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:39.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.195+0000 7fa4643db640 1 --2- 172.21.15.16:0/1951427821 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa4480a1090 0x7fa4480a3550 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa44c0057c0 tx=0x7fa44c005a20 comp rx=0 tx=0).stop 2024-09-13T16:28:39.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.195+0000 7fa4643db640 1 -- 172.21.15.16:0/1951427821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa45c100810 msgr2=0x7fa45c100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:39.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.195+0000 7fa4643db640 1 --2- 172.21.15.16:0/1951427821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa45c100810 0x7fa45c100180 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa45800ea20 tx=0x7fa45800eee0 comp rx=0 tx=0).stop 2024-09-13T16:28:39.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.195+0000 7fa4643db640 1 -- 172.21.15.16:0/1951427821 shutdown_connections 2024-09-13T16:28:39.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.195+0000 7fa4643db640 1 --2- 172.21.15.16:0/1951427821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa45c105a70 0x7fa45c0fed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:39.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.195+0000 7fa4643db640 1 --2- 172.21.15.16:0/1951427821 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa4480a1090 0x7fa4480a3550 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:39.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.196+0000 7fa4643db640 1 --2- 172.21.15.16:0/1951427821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa45c103140 0x7fa45c0fe7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:39.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.196+0000 7fa4643db640 1 --2- 172.21.15.16:0/1951427821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa45c100810 0x7fa45c100180 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:39.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.196+0000 7fa4643db640 1 -- 172.21.15.16:0/1951427821 >> 172.21.15.16:0/1951427821 conn(0x7fa45c0fa3f0 msgr2=0x7fa45c101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:39.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.196+0000 7fa4643db640 1 -- 172.21.15.16:0/1951427821 shutdown_connections 2024-09-13T16:28:39.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.196+0000 7fa4643db640 1 -- 172.21.15.16:0/1951427821 wait complete. 2024-09-13T16:28:39.217 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-13T16:28:39.218 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-13T16:28:39.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.323+0000 7f0d44553640 1 -- 172.21.15.16:0/3759683263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d3c100840 msgr2=0x7f0d3c1070d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:39.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.323+0000 7f0d44553640 1 --2- 172.21.15.16:0/3759683263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d3c100840 0x7f0d3c1070d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0d34009ca0 tx=0x7f0d3402a8c0 comp rx=0 tx=0).stop 2024-09-13T16:28:39.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.323+0000 7f0d44553640 1 -- 172.21.15.16:0/3759683263 shutdown_connections 2024-09-13T16:28:39.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.323+0000 7f0d44553640 1 --2- 172.21.15.16:0/3759683263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d3c100840 0x7f0d3c1070d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:39.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.323+0000 7f0d44553640 1 --2- 172.21.15.16:0/3759683263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d3c0ffe80 0x7f0d3c100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:39.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.323+0000 7f0d44553640 1 --2- 172.21.15.16:0/3759683263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d3c0fec90 0x7f0d3c0ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:39.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.323+0000 7f0d44553640 1 -- 172.21.15.16:0/3759683263 >> 172.21.15.16:0/3759683263 conn(0x7f0d3c0fa3f0 msgr2=0x7f0d3c0fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:39.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.323+0000 7f0d44553640 1 -- 172.21.15.16:0/3759683263 shutdown_connections 2024-09-13T16:28:39.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.323+0000 7f0d44553640 1 -- 172.21.15.16:0/3759683263 wait complete. 2024-09-13T16:28:39.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.323+0000 7f0d44553640 1 Processor -- start 2024-09-13T16:28:39.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.324+0000 7f0d44553640 1 -- start start 2024-09-13T16:28:39.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.324+0000 7f0d44553640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d3c0fec90 0x7f0d3c195c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:39.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.324+0000 7f0d44553640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d3c0ffe80 0x7f0d3c1961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:39.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.324+0000 7f0d44553640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d3c100840 0x7f0d3c19d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:39.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.324+0000 7f0d44553640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3c19d750 con 0x7f0d3c100840 2024-09-13T16:28:39.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.324+0000 7f0d44553640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3c19d8c0 con 0x7f0d3c0fec90 2024-09-13T16:28:39.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.324+0000 7f0d44553640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3c19da80 con 0x7f0d3c0ffe80 2024-09-13T16:28:39.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.324+0000 7f0d42af0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d3c100840 0x7f0d3c19d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:39.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.324+0000 7f0d42af0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d3c100840 0x7f0d3c19d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33220/0 (socket says 172.21.15.16:33220) 2024-09-13T16:28:39.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.324+0000 7f0d42af0640 1 -- 172.21.15.16:0/2149956675 learned_addr learned my addr 172.21.15.16:0/2149956675 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:39.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.324+0000 7f0d41aee640 1 --2- 172.21.15.16:0/2149956675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d3c0ffe80 0x7f0d3c1961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:39.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.324+0000 7f0d422ef640 1 --2- 172.21.15.16:0/2149956675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d3c0fec90 0x7f0d3c195c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:39.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.324+0000 7f0d42af0640 1 -- 172.21.15.16:0/2149956675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d3c0ffe80 msgr2=0x7f0d3c1961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:39.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.324+0000 7f0d42af0640 1 --2- 172.21.15.16:0/2149956675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d3c0ffe80 0x7f0d3c1961b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:39.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.324+0000 7f0d42af0640 1 -- 172.21.15.16:0/2149956675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d3c0fec90 msgr2=0x7f0d3c195c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:39.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.324+0000 7f0d42af0640 1 --2- 172.21.15.16:0/2149956675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d3c0fec90 0x7f0d3c195c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:39.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.324+0000 7f0d42af0640 1 -- 172.21.15.16:0/2149956675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d34009900 con 0x7f0d3c100840 2024-09-13T16:28:39.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.324+0000 7f0d42af0640 1 --2- 172.21.15.16:0/2149956675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d3c100840 0x7f0d3c19d1e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0d3402aeb0 tx=0x7f0d34003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:39.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.325+0000 7f0d337fe640 1 -- 172.21.15.16:0/2149956675 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d340307c0 con 0x7f0d3c100840 2024-09-13T16:28:39.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.325+0000 7f0d337fe640 1 -- 172.21.15.16:0/2149956675 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0d34040070 con 0x7f0d3c100840 2024-09-13T16:28:39.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.325+0000 7f0d44553640 1 -- 172.21.15.16:0/2149956675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d3c19dd00 con 0x7f0d3c100840 2024-09-13T16:28:39.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.325+0000 7f0d337fe640 1 -- 172.21.15.16:0/2149956675 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d340044e0 con 0x7f0d3c100840 2024-09-13T16:28:39.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.325+0000 7f0d44553640 1 -- 172.21.15.16:0/2149956675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d3c19e1c0 con 0x7f0d3c100840 2024-09-13T16:28:39.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.326+0000 7f0d44553640 1 -- 172.21.15.16:0/2149956675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d3c10a3e0 con 0x7f0d3c100840 2024-09-13T16:28:39.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.327+0000 7f0d337fe640 1 -- 172.21.15.16:0/2149956675 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f0d34042030 con 0x7f0d3c100840 2024-09-13T16:28:39.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.327+0000 7f0d337fe640 1 --2- 172.21.15.16:0/2149956675 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f0d280a1090 0x7f0d280a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:39.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.327+0000 7f0d422ef640 1 --2- 172.21.15.16:0/2149956675 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f0d280a1090 0x7f0d280a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:39.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.327+0000 7f0d337fe640 1 -- 172.21.15.16:0/2149956675 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0d340edf30 con 0x7f0d3c100840 2024-09-13T16:28:39.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.328+0000 7f0d422ef640 1 --2- 172.21.15.16:0/2149956675 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f0d280a1090 0x7f0d280a3550 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c197140 tx=0x7f0d3800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:39.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.332+0000 7f0d337fe640 1 -- 172.21.15.16:0/2149956675 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0d340b7780 con 0x7f0d3c100840 2024-09-13T16:28:39.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:39.511+0000 7f0d44553640 1 -- 172.21.15.16:0/2149956675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f0d3c10a5c0 con 0x7f0d3c100840 2024-09-13T16:28:40.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:39 smithi107 ceph-mon[34139]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T16:28:40.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:39 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/2590873044' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-13T16:28:40.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:39 smithi107 ceph-mon[34139]: osdmap e83: 12 total, 12 up, 12 in 2024-09-13T16:28:40.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:39 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1951427821' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-13T16:28:40.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:39 smithi107 ceph-mon[34139]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-13T16:28:40.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:39 smithi107 ceph-mon[34139]: osdmap e84: 12 total, 12 up, 12 in 2024-09-13T16:28:40.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:39 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1951427821' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-13T16:28:40.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:39 smithi107 ceph-mon[34139]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-13T16:28:40.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:39 smithi107 ceph-mon[34139]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-13T16:28:40.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:39 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/2149956675' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-13T16:28:40.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:39 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:40.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:39 smithi016 ceph-mon[31187]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T16:28:40.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:39 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2590873044' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-13T16:28:40.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:39 smithi016 ceph-mon[31187]: osdmap e83: 12 total, 12 up, 12 in 2024-09-13T16:28:40.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:39 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1951427821' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-13T16:28:40.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:39 smithi016 ceph-mon[31187]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-13T16:28:40.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:39 smithi016 ceph-mon[31187]: osdmap e84: 12 total, 12 up, 12 in 2024-09-13T16:28:40.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:39 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1951427821' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-13T16:28:40.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:39 smithi016 ceph-mon[31187]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-13T16:28:40.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:39 smithi016 ceph-mon[31187]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-13T16:28:40.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:39 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2149956675' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-13T16:28:40.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:39 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:40.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:39 smithi111 ceph-mon[34600]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T16:28:40.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:39 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/2590873044' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-13T16:28:40.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:39 smithi111 ceph-mon[34600]: osdmap e83: 12 total, 12 up, 12 in 2024-09-13T16:28:40.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:39 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1951427821' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-13T16:28:40.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:39 smithi111 ceph-mon[34600]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-13T16:28:40.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:39 smithi111 ceph-mon[34600]: osdmap e84: 12 total, 12 up, 12 in 2024-09-13T16:28:40.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:39 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1951427821' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-13T16:28:40.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:39 smithi111 ceph-mon[34600]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-13T16:28:40.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:39 smithi111 ceph-mon[34600]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-13T16:28:40.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:39 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/2149956675' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-13T16:28:40.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:39 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:28:40.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.194+0000 7f0d337fe640 1 -- 172.21.15.16:0/2149956675 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v85) v1 ==== 157+0+0 (secure 0 0 0) 0x7f0d340b96b0 con 0x7f0d3c100840 2024-09-13T16:28:40.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.197+0000 7f0d44553640 1 -- 172.21.15.16:0/2149956675 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f0d280a1090 msgr2=0x7f0d280a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:40.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.197+0000 7f0d44553640 1 --2- 172.21.15.16:0/2149956675 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f0d280a1090 0x7f0d280a3550 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c197140 tx=0x7f0d3800b410 comp rx=0 tx=0).stop 2024-09-13T16:28:40.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.197+0000 7f0d44553640 1 -- 172.21.15.16:0/2149956675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d3c100840 msgr2=0x7f0d3c19d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:40.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.197+0000 7f0d44553640 1 --2- 172.21.15.16:0/2149956675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d3c100840 0x7f0d3c19d1e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0d3402aeb0 tx=0x7f0d34003c20 comp rx=0 tx=0).stop 2024-09-13T16:28:40.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.197+0000 7f0d44553640 1 -- 172.21.15.16:0/2149956675 shutdown_connections 2024-09-13T16:28:40.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.197+0000 7f0d44553640 1 --2- 172.21.15.16:0/2149956675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d3c100840 0x7f0d3c19d1e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:40.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.197+0000 7f0d44553640 1 --2- 172.21.15.16:0/2149956675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d3c0ffe80 0x7f0d3c1961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:40.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.197+0000 7f0d44553640 1 --2- 172.21.15.16:0/2149956675 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f0d280a1090 0x7f0d280a3550 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:40.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.197+0000 7f0d44553640 1 --2- 172.21.15.16:0/2149956675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d3c0fec90 0x7f0d3c195c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:40.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.197+0000 7f0d44553640 1 -- 172.21.15.16:0/2149956675 >> 172.21.15.16:0/2149956675 conn(0x7f0d3c0fa3f0 msgr2=0x7f0d3c0685c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:40.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.197+0000 7f0d44553640 1 -- 172.21.15.16:0/2149956675 shutdown_connections 2024-09-13T16:28:40.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.197+0000 7f0d44553640 1 -- 172.21.15.16:0/2149956675 wait complete. 2024-09-13T16:28:40.212 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T16:28:40.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.327+0000 7f3dbefc7640 1 -- 172.21.15.16:0/2871646247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db80ffdc0 msgr2=0x7f3db81001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:40.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.327+0000 7f3dbefc7640 1 --2- 172.21.15.16:0/2871646247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db80ffdc0 0x7f3db81001c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3dac009900 tx=0x7f3dac02a410 comp rx=0 tx=0).stop 2024-09-13T16:28:40.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.328+0000 7f3dbefc7640 1 -- 172.21.15.16:0/2871646247 shutdown_connections 2024-09-13T16:28:40.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.328+0000 7f3dbefc7640 1 --2- 172.21.15.16:0/2871646247 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3db8101970 0x7f3db8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:40.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.328+0000 7f3dbefc7640 1 --2- 172.21.15.16:0/2871646247 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3db8100fb0 0x7f3db8101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:40.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.328+0000 7f3dbefc7640 1 --2- 172.21.15.16:0/2871646247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db80ffdc0 0x7f3db81001c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:40.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.328+0000 7f3dbefc7640 1 -- 172.21.15.16:0/2871646247 >> 172.21.15.16:0/2871646247 conn(0x7f3db80fb520 msgr2=0x7f3db80fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:40.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.328+0000 7f3dbefc7640 1 -- 172.21.15.16:0/2871646247 shutdown_connections 2024-09-13T16:28:40.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.328+0000 7f3dbefc7640 1 -- 172.21.15.16:0/2871646247 wait complete. 2024-09-13T16:28:40.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.328+0000 7f3dbefc7640 1 Processor -- start 2024-09-13T16:28:40.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.328+0000 7f3dbefc7640 1 -- start start 2024-09-13T16:28:40.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.328+0000 7f3dbefc7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db80ffdc0 0x7f3db8195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:40.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.328+0000 7f3dbefc7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3db8100fb0 0x7f3db8196230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:40.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.328+0000 7f3dbefc7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3db8101970 0x7f3db819d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:40.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.328+0000 7f3dbefc7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db819d7d0 con 0x7f3db80ffdc0 2024-09-13T16:28:40.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.328+0000 7f3dbefc7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db819d940 con 0x7f3db8100fb0 2024-09-13T16:28:40.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.328+0000 7f3dbefc7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db819db00 con 0x7f3db8101970 2024-09-13T16:28:40.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.328+0000 7f3dbcd63640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db80ffdc0 0x7f3db8195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:40.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.328+0000 7f3dbcd63640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db80ffdc0 0x7f3db8195cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33232/0 (socket says 172.21.15.16:33232) 2024-09-13T16:28:40.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.328+0000 7f3dbcd63640 1 -- 172.21.15.16:0/676324072 learned_addr learned my addr 172.21.15.16:0/676324072 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:40.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.329+0000 7f3dbd564640 1 --2- 172.21.15.16:0/676324072 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3db8101970 0x7f3db819d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:40.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.329+0000 7f3db7fff640 1 --2- 172.21.15.16:0/676324072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3db8100fb0 0x7f3db8196230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:40.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.329+0000 7f3dbcd63640 1 -- 172.21.15.16:0/676324072 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3db8101970 msgr2=0x7f3db819d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:40.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.329+0000 7f3dbcd63640 1 --2- 172.21.15.16:0/676324072 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3db8101970 0x7f3db819d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:40.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.329+0000 7f3dbcd63640 1 -- 172.21.15.16:0/676324072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3db8100fb0 msgr2=0x7f3db8196230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:40.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.329+0000 7f3dbcd63640 1 --2- 172.21.15.16:0/676324072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3db8100fb0 0x7f3db8196230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:40.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.329+0000 7f3dbcd63640 1 -- 172.21.15.16:0/676324072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dac009560 con 0x7f3db80ffdc0 2024-09-13T16:28:40.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.329+0000 7f3dbd564640 1 --2- 172.21.15.16:0/676324072 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3db8101970 0x7f3db819d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:28:40.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.329+0000 7f3db7fff640 1 --2- 172.21.15.16:0/676324072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3db8100fb0 0x7f3db8196230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:28:40.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.329+0000 7f3dbcd63640 1 --2- 172.21.15.16:0/676324072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db80ffdc0 0x7f3db8195cf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3dac009230 tx=0x7f3dac004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:40.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.329+0000 7f3db5ffb640 1 -- 172.21.15.16:0/676324072 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dac031d70 con 0x7f3db80ffdc0 2024-09-13T16:28:40.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.329+0000 7f3db5ffb640 1 -- 172.21.15.16:0/676324072 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3dac004b50 con 0x7f3db80ffdc0 2024-09-13T16:28:40.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.329+0000 7f3dbefc7640 1 -- 172.21.15.16:0/676324072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db819dd80 con 0x7f3db80ffdc0 2024-09-13T16:28:40.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.329+0000 7f3db5ffb640 1 -- 172.21.15.16:0/676324072 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dac003c30 con 0x7f3db80ffdc0 2024-09-13T16:28:40.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.329+0000 7f3dbefc7640 1 -- 172.21.15.16:0/676324072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db819e180 con 0x7f3db80ffdc0 2024-09-13T16:28:40.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.330+0000 7f3dbefc7640 1 -- 172.21.15.16:0/676324072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db81072a0 con 0x7f3db80ffdc0 2024-09-13T16:28:40.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.332+0000 7f3db5ffb640 1 -- 172.21.15.16:0/676324072 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f3dac0046c0 con 0x7f3db80ffdc0 2024-09-13T16:28:40.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.332+0000 7f3db5ffb640 1 --2- 172.21.15.16:0/676324072 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f3da80a1090 0x7f3da80a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:40.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.333+0000 7f3db5ffb640 1 -- 172.21.15.16:0/676324072 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8205+0+0 (secure 0 0 0) 0x7f3dac0edf20 con 0x7f3db80ffdc0 2024-09-13T16:28:40.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.333+0000 7f3db7fff640 1 --2- 172.21.15.16:0/676324072 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f3da80a1090 0x7f3da80a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:40.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.333+0000 7f3db7fff640 1 --2- 172.21.15.16:0/676324072 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f3da80a1090 0x7f3da80a3550 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3da4006fd0 tx=0x7f3da4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:40.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.335+0000 7f3db5ffb640 1 -- 172.21.15.16:0/676324072 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f3dac0b7640 con 0x7f3db80ffdc0 2024-09-13T16:28:40.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:40.509+0000 7f3dbefc7640 1 -- 172.21.15.16:0/676324072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3db810b590 con 0x7f3db80ffdc0 2024-09-13T16:28:40.849 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 13 16:28:40 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-b[63478]: 2024-09-13T16:28:40.432+0000 7f87d005d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T16:28:41.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:41 smithi016 ceph-mon[31187]: mds.? [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] up:active 2024-09-13T16:28:41.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:41 smithi016 ceph-mon[31187]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-13T16:28:41.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:41 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2149956675' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-13T16:28:41.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:41 smithi016 ceph-mon[31187]: osdmap e85: 12 total, 12 up, 12 in 2024-09-13T16:28:41.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:41 smithi016 ceph-mon[31187]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-13T16:28:41.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:41 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/676324072' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-13T16:28:41.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:41 smithi016 ceph-mon[31187]: pgmap v273: 97 pgs: 11 creating+peering, 66 unknown, 20 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-13T16:28:41.601 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:41 smithi107 ceph-mon[34139]: mds.? [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] up:active 2024-09-13T16:28:41.601 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:41 smithi107 ceph-mon[34139]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-13T16:28:41.601 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:41 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/2149956675' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-13T16:28:41.602 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:41 smithi107 ceph-mon[34139]: osdmap e85: 12 total, 12 up, 12 in 2024-09-13T16:28:41.602 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:41 smithi107 ceph-mon[34139]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-13T16:28:41.602 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:41 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/676324072' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-13T16:28:41.602 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:41 smithi107 ceph-mon[34139]: pgmap v273: 97 pgs: 11 creating+peering, 66 unknown, 20 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-13T16:28:41.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:41 smithi111 ceph-mon[34600]: mds.? [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] up:active 2024-09-13T16:28:41.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:41 smithi111 ceph-mon[34600]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-13T16:28:41.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:41 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/2149956675' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-13T16:28:41.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:41 smithi111 ceph-mon[34600]: osdmap e85: 12 total, 12 up, 12 in 2024-09-13T16:28:41.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:41 smithi111 ceph-mon[34600]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-13T16:28:41.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:41 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/676324072' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-13T16:28:41.688 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:41 smithi111 ceph-mon[34600]: pgmap v273: 97 pgs: 11 creating+peering, 66 unknown, 20 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-13T16:28:42.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.202+0000 7f3db5ffb640 1 -- 172.21.15.16:0/676324072 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v87) v1 ==== 228+0+0 (secure 0 0 0) 0x7f3dac0b9570 con 0x7f3db80ffdc0 2024-09-13T16:28:42.205 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data_ec' created 2024-09-13T16:28:42.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.207+0000 7f3dbefc7640 1 -- 172.21.15.16:0/676324072 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f3da80a1090 msgr2=0x7f3da80a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:42.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.207+0000 7f3dbefc7640 1 --2- 172.21.15.16:0/676324072 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f3da80a1090 0x7f3da80a3550 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3da4006fd0 tx=0x7f3da4008040 comp rx=0 tx=0).stop 2024-09-13T16:28:42.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.207+0000 7f3dbefc7640 1 -- 172.21.15.16:0/676324072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db80ffdc0 msgr2=0x7f3db8195cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:42.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.207+0000 7f3dbefc7640 1 --2- 172.21.15.16:0/676324072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db80ffdc0 0x7f3db8195cf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3dac009230 tx=0x7f3dac004360 comp rx=0 tx=0).stop 2024-09-13T16:28:42.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.208+0000 7f3dbefc7640 1 -- 172.21.15.16:0/676324072 shutdown_connections 2024-09-13T16:28:42.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.208+0000 7f3dbefc7640 1 --2- 172.21.15.16:0/676324072 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3db8101970 0x7f3db819d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:42.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.208+0000 7f3dbefc7640 1 --2- 172.21.15.16:0/676324072 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f3da80a1090 0x7f3da80a3550 secure :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3da4006fd0 tx=0x7f3da4008040 comp rx=0 tx=0).stop 2024-09-13T16:28:42.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.208+0000 7f3dbefc7640 1 --2- 172.21.15.16:0/676324072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3db8100fb0 0x7f3db8196230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:42.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.208+0000 7f3dbefc7640 1 --2- 172.21.15.16:0/676324072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db80ffdc0 0x7f3db8195cf0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:42.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.208+0000 7f3dbefc7640 1 -- 172.21.15.16:0/676324072 >> 172.21.15.16:0/676324072 conn(0x7f3db80fb520 msgr2=0x7f3db8104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:42.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.208+0000 7f3dbefc7640 1 -- 172.21.15.16:0/676324072 shutdown_connections 2024-09-13T16:28:42.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.208+0000 7f3dbefc7640 1 -- 172.21.15.16:0/676324072 wait complete. 2024-09-13T16:28:42.225 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-13T16:28:42.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.337+0000 7f0378308640 1 -- 172.21.15.16:0/64961360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03700ffe80 msgr2=0x7f0370100300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:42.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.337+0000 7f0378308640 1 --2- 172.21.15.16:0/64961360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03700ffe80 0x7f0370100300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0360009900 tx=0x7f036002a410 comp rx=0 tx=0).stop 2024-09-13T16:28:42.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.337+0000 7f0378308640 1 -- 172.21.15.16:0/64961360 shutdown_connections 2024-09-13T16:28:42.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.337+0000 7f0378308640 1 --2- 172.21.15.16:0/64961360 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0370100840 0x7f03701070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:42.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.337+0000 7f0378308640 1 --2- 172.21.15.16:0/64961360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03700ffe80 0x7f0370100300 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:42.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.337+0000 7f0378308640 1 --2- 172.21.15.16:0/64961360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03700fec90 0x7f03700ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:42.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.337+0000 7f0378308640 1 -- 172.21.15.16:0/64961360 >> 172.21.15.16:0/64961360 conn(0x7f03700fa3f0 msgr2=0x7f03700fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:42.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.337+0000 7f0378308640 1 -- 172.21.15.16:0/64961360 shutdown_connections 2024-09-13T16:28:42.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.337+0000 7f0378308640 1 -- 172.21.15.16:0/64961360 wait complete. 2024-09-13T16:28:42.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.338+0000 7f0378308640 1 Processor -- start 2024-09-13T16:28:42.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.338+0000 7f0378308640 1 -- start start 2024-09-13T16:28:42.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.338+0000 7f0378308640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03700fec90 0x7f03700690e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:42.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.338+0000 7f0378308640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03700ffe80 0x7f037019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:42.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.338+0000 7f0378308640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0370100840 0x7f037019cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:42.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.338+0000 7f0378308640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037019d740 con 0x7f03700ffe80 2024-09-13T16:28:42.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.338+0000 7f0378308640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037019d8b0 con 0x7f03700fec90 2024-09-13T16:28:42.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.338+0000 7f0378308640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037019da70 con 0x7f0370100840 2024-09-13T16:28:42.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.338+0000 7f03758a3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03700ffe80 0x7f037019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:42.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.338+0000 7f03758a3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03700ffe80 0x7f037019a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33238/0 (socket says 172.21.15.16:33238) 2024-09-13T16:28:42.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.338+0000 7f03758a3640 1 -- 172.21.15.16:0/3334139402 learned_addr learned my addr 172.21.15.16:0/3334139402 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:42.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.338+0000 7f03760a4640 1 --2- 172.21.15.16:0/3334139402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03700fec90 0x7f03700690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:42.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.338+0000 7f03768a5640 1 --2- 172.21.15.16:0/3334139402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0370100840 0x7f037019cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:42.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.338+0000 7f03758a3640 1 -- 172.21.15.16:0/3334139402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0370100840 msgr2=0x7f037019cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:42.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.338+0000 7f03758a3640 1 --2- 172.21.15.16:0/3334139402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0370100840 0x7f037019cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:42.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.338+0000 7f03758a3640 1 -- 172.21.15.16:0/3334139402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03700fec90 msgr2=0x7f03700690e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:42.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.338+0000 7f03758a3640 1 --2- 172.21.15.16:0/3334139402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03700fec90 0x7f03700690e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:42.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.338+0000 7f03758a3640 1 -- 172.21.15.16:0/3334139402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0360009560 con 0x7f03700ffe80 2024-09-13T16:28:42.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.338+0000 7f03768a5640 1 --2- 172.21.15.16:0/3334139402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0370100840 0x7f037019cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:28:42.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.339+0000 7f03758a3640 1 --2- 172.21.15.16:0/3334139402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03700ffe80 0x7f037019a940 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f03600057c0 tx=0x7f03600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:42.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.339+0000 7f03677fe640 1 -- 172.21.15.16:0/3334139402 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03600054b0 con 0x7f03700ffe80 2024-09-13T16:28:42.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.339+0000 7f03677fe640 1 -- 172.21.15.16:0/3334139402 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f03600044e0 con 0x7f03700ffe80 2024-09-13T16:28:42.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.339+0000 7f0378308640 1 -- 172.21.15.16:0/3334139402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f037019dc90 con 0x7f03700ffe80 2024-09-13T16:28:42.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.339+0000 7f03677fe640 1 -- 172.21.15.16:0/3334139402 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0360003a30 con 0x7f03700ffe80 2024-09-13T16:28:42.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.339+0000 7f0378308640 1 -- 172.21.15.16:0/3334139402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f037019e140 con 0x7f03700ffe80 2024-09-13T16:28:42.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.340+0000 7f0378308640 1 -- 172.21.15.16:0/3334139402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0370106170 con 0x7f03700ffe80 2024-09-13T16:28:42.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.341+0000 7f03677fe640 1 -- 172.21.15.16:0/3334139402 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f0360003b90 con 0x7f03700ffe80 2024-09-13T16:28:42.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.342+0000 7f03677fe640 1 --2- 172.21.15.16:0/3334139402 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f035c0a1090 0x7f035c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:42.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.342+0000 7f03677fe640 1 -- 172.21.15.16:0/3334139402 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f03600edfa0 con 0x7f03700ffe80 2024-09-13T16:28:42.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.342+0000 7f03760a4640 1 --2- 172.21.15.16:0/3334139402 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f035c0a1090 0x7f035c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:42.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.342+0000 7f03760a4640 1 --2- 172.21.15.16:0/3334139402 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f035c0a1090 0x7f035c0a3550 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f03700ffce0 tx=0x7f036c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:42.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.345+0000 7f03677fe640 1 -- 172.21.15.16:0/3334139402 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f03600b7560 con 0x7f03700ffe80 2024-09-13T16:28:42.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:42.516+0000 7f0378308640 1 -- 172.21.15.16:0/3334139402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f037010a400 con 0x7f03700ffe80 2024-09-13T16:28:42.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:42 smithi107 ceph-mon[34139]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-13T16:28:42.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:42 smithi107 ceph-mon[34139]: Cluster is now healthy 2024-09-13T16:28:42.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:42 smithi107 ceph-mon[34139]: osdmap e86: 12 total, 12 up, 12 in 2024-09-13T16:28:42.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:42 smithi016 ceph-mon[31187]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-13T16:28:42.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:42 smithi016 ceph-mon[31187]: Cluster is now healthy 2024-09-13T16:28:42.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:42 smithi016 ceph-mon[31187]: osdmap e86: 12 total, 12 up, 12 in 2024-09-13T16:28:42.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:42 smithi111 ceph-mon[34600]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-13T16:28:42.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:42 smithi111 ceph-mon[34600]: Cluster is now healthy 2024-09-13T16:28:42.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:42 smithi111 ceph-mon[34600]: osdmap e86: 12 total, 12 up, 12 in 2024-09-13T16:28:43.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.210+0000 7f03677fe640 1 -- 172.21.15.16:0/3334139402 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v88) v1 ==== 169+0+0 (secure 0 0 0) 0x7f03600bab80 con 0x7f03700ffe80 2024-09-13T16:28:43.213 INFO:teuthology.orchestra.run.smithi016.stderr:set pool 4 allow_ec_overwrites to true 2024-09-13T16:28:43.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.215+0000 7f0378308640 1 -- 172.21.15.16:0/3334139402 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f035c0a1090 msgr2=0x7f035c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:43.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.215+0000 7f0378308640 1 --2- 172.21.15.16:0/3334139402 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f035c0a1090 0x7f035c0a3550 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f03700ffce0 tx=0x7f036c005f90 comp rx=0 tx=0).stop 2024-09-13T16:28:43.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.215+0000 7f0378308640 1 -- 172.21.15.16:0/3334139402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03700ffe80 msgr2=0x7f037019a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:43.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.215+0000 7f0378308640 1 --2- 172.21.15.16:0/3334139402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03700ffe80 0x7f037019a940 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f03600057c0 tx=0x7f03600042f0 comp rx=0 tx=0).stop 2024-09-13T16:28:43.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.215+0000 7f0378308640 1 -- 172.21.15.16:0/3334139402 shutdown_connections 2024-09-13T16:28:43.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.215+0000 7f0378308640 1 --2- 172.21.15.16:0/3334139402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0370100840 0x7f037019cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:43.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.215+0000 7f0378308640 1 --2- 172.21.15.16:0/3334139402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03700ffe80 0x7f037019a940 secure :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f03600057c0 tx=0x7f03600042f0 comp rx=0 tx=0).stop 2024-09-13T16:28:43.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.215+0000 7f0378308640 1 --2- 172.21.15.16:0/3334139402 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f035c0a1090 0x7f035c0a3550 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:43.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.215+0000 7f0378308640 1 --2- 172.21.15.16:0/3334139402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03700fec90 0x7f03700690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:43.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.215+0000 7f0378308640 1 -- 172.21.15.16:0/3334139402 >> 172.21.15.16:0/3334139402 conn(0x7f03700fa3f0 msgr2=0x7f0370103340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:43.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.215+0000 7f0378308640 1 -- 172.21.15.16:0/3334139402 shutdown_connections 2024-09-13T16:28:43.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.216+0000 7f0378308640 1 -- 172.21.15.16:0/3334139402 wait complete. 2024-09-13T16:28:43.242 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-13T16:28:43.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.352+0000 7faa32820640 1 -- 172.21.15.16:0/3678566988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa2c0fec60 msgr2=0x7faa2c0ff060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:43.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.352+0000 7faa32820640 1 --2- 172.21.15.16:0/3678566988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa2c0fec60 0x7faa2c0ff060 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7faa1c009900 tx=0x7faa1c02a410 comp rx=0 tx=0).stop 2024-09-13T16:28:43.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.352+0000 7faa32820640 1 -- 172.21.15.16:0/3678566988 shutdown_connections 2024-09-13T16:28:43.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.352+0000 7faa32820640 1 --2- 172.21.15.16:0/3678566988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa2c100810 0x7faa2c1070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:43.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.352+0000 7faa32820640 1 --2- 172.21.15.16:0/3678566988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa2c0ffe50 0x7faa2c1002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:43.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.352+0000 7faa32820640 1 --2- 172.21.15.16:0/3678566988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa2c0fec60 0x7faa2c0ff060 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:43.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.352+0000 7faa32820640 1 -- 172.21.15.16:0/3678566988 >> 172.21.15.16:0/3678566988 conn(0x7faa2c0fa3e0 msgr2=0x7faa2c0fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:43.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.352+0000 7faa32820640 1 -- 172.21.15.16:0/3678566988 shutdown_connections 2024-09-13T16:28:43.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.352+0000 7faa32820640 1 -- 172.21.15.16:0/3678566988 wait complete. 2024-09-13T16:28:43.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.353+0000 7faa32820640 1 Processor -- start 2024-09-13T16:28:43.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.353+0000 7faa32820640 1 -- start start 2024-09-13T16:28:43.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.353+0000 7faa32820640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa2c0fec60 0x7faa2c102bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:43.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.353+0000 7faa32820640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa2c0ffe50 0x7faa2c101230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:43.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.353+0000 7faa32820640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa2c100810 0x7faa2c101770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:43.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.353+0000 7faa32820640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c101ed0 con 0x7faa2c0fec60 2024-09-13T16:28:43.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.353+0000 7faa32820640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c102040 con 0x7faa2c100810 2024-09-13T16:28:43.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.353+0000 7faa32820640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c19fcc0 con 0x7faa2c0ffe50 2024-09-13T16:28:43.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.353+0000 7faa2bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa2c0fec60 0x7faa2c102bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:43.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.353+0000 7faa2bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa2c0fec60 0x7faa2c102bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33258/0 (socket says 172.21.15.16:33258) 2024-09-13T16:28:43.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.353+0000 7faa2bfff640 1 -- 172.21.15.16:0/298307710 learned_addr learned my addr 172.21.15.16:0/298307710 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:43.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.354+0000 7faa2bfff640 1 -- 172.21.15.16:0/298307710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa2c0ffe50 msgr2=0x7faa2c101230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:43.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.354+0000 7faa2bfff640 1 --2- 172.21.15.16:0/298307710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa2c0ffe50 0x7faa2c101230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:43.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.354+0000 7faa2bfff640 1 -- 172.21.15.16:0/298307710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa2c100810 msgr2=0x7faa2c101770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:43.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.354+0000 7faa2bfff640 1 --2- 172.21.15.16:0/298307710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa2c100810 0x7faa2c101770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:43.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.354+0000 7faa2bfff640 1 -- 172.21.15.16:0/298307710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1c009560 con 0x7faa2c0fec60 2024-09-13T16:28:43.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.354+0000 7faa2bfff640 1 --2- 172.21.15.16:0/298307710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa2c0fec60 0x7faa2c102bf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7faa1c006010 tx=0x7faa1c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:43.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.354+0000 7faa297fa640 1 -- 172.21.15.16:0/298307710 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1c031d70 con 0x7faa2c0fec60 2024-09-13T16:28:43.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.354+0000 7faa32820640 1 -- 172.21.15.16:0/298307710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa2c19ff10 con 0x7faa2c0fec60 2024-09-13T16:28:43.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.354+0000 7faa297fa640 1 -- 172.21.15.16:0/298307710 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faa1c004b50 con 0x7faa2c0fec60 2024-09-13T16:28:43.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.354+0000 7faa297fa640 1 -- 172.21.15.16:0/298307710 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1c003cd0 con 0x7faa2c0fec60 2024-09-13T16:28:43.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.354+0000 7faa32820640 1 -- 172.21.15.16:0/298307710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa2c1a0340 con 0x7faa2c0fec60 2024-09-13T16:28:43.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.355+0000 7faa32820640 1 -- 172.21.15.16:0/298307710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa2c10a3d0 con 0x7faa2c0fec60 2024-09-13T16:28:43.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.356+0000 7faa297fa640 1 -- 172.21.15.16:0/298307710 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7faa1c0046c0 con 0x7faa2c0fec60 2024-09-13T16:28:43.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.357+0000 7faa297fa640 1 --2- 172.21.15.16:0/298307710 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7faa140a1090 0x7faa140a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:43.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.357+0000 7faa2b7fe640 1 --2- 172.21.15.16:0/298307710 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7faa140a1090 0x7faa140a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:43.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.357+0000 7faa297fa640 1 -- 172.21.15.16:0/298307710 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8698+0+0 (secure 0 0 0) 0x7faa1c0edd50 con 0x7faa2c0fec60 2024-09-13T16:28:43.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.357+0000 7faa2b7fe640 1 --2- 172.21.15.16:0/298307710 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7faa140a1090 0x7faa140a3550 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faa2c102960 tx=0x7faa20005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:43.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.361+0000 7faa297fa640 1 -- 172.21.15.16:0/298307710 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7faa1c0b72b0 con 0x7faa2c0fec60 2024-09-13T16:28:43.500 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:43 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/676324072' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-13T16:28:43.501 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:43 smithi107 ceph-mon[34139]: osdmap e87: 12 total, 12 up, 12 in 2024-09-13T16:28:43.501 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:43 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3334139402' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-13T16:28:43.501 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:43 smithi107 ceph-mon[34139]: pgmap v276: 129 pgs: 16 creating+peering, 47 unknown, 66 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 10 op/s 2024-09-13T16:28:43.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.572+0000 7faa32820640 1 -- 172.21.15.16:0/298307710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7faa2c10a5e0 con 0x7faa2c0fec60 2024-09-13T16:28:43.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.577+0000 7faa297fa640 1 -- 172.21.15.16:0/298307710 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7faa1c0ba8d0 con 0x7faa2c0fec60 2024-09-13T16:28:43.579 INFO:teuthology.orchestra.run.smithi016.stderr:added data pool 4 to fsmap 2024-09-13T16:28:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.579+0000 7faa32820640 1 -- 172.21.15.16:0/298307710 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7faa140a1090 msgr2=0x7faa140a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.579+0000 7faa32820640 1 --2- 172.21.15.16:0/298307710 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7faa140a1090 0x7faa140a3550 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faa2c102960 tx=0x7faa20005c70 comp rx=0 tx=0).stop 2024-09-13T16:28:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.579+0000 7faa32820640 1 -- 172.21.15.16:0/298307710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa2c0fec60 msgr2=0x7faa2c102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.579+0000 7faa32820640 1 --2- 172.21.15.16:0/298307710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa2c0fec60 0x7faa2c102bf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7faa1c006010 tx=0x7faa1c004360 comp rx=0 tx=0).stop 2024-09-13T16:28:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.579+0000 7faa32820640 1 -- 172.21.15.16:0/298307710 shutdown_connections 2024-09-13T16:28:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.579+0000 7faa32820640 1 --2- 172.21.15.16:0/298307710 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7faa140a1090 0x7faa140a3550 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.579+0000 7faa32820640 1 --2- 172.21.15.16:0/298307710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa2c100810 0x7faa2c101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.579+0000 7faa32820640 1 --2- 172.21.15.16:0/298307710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa2c0ffe50 0x7faa2c101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.579+0000 7faa32820640 1 --2- 172.21.15.16:0/298307710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa2c0fec60 0x7faa2c102bf0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.579+0000 7faa32820640 1 -- 172.21.15.16:0/298307710 >> 172.21.15.16:0/298307710 conn(0x7faa2c0fa3e0 msgr2=0x7faa2c067610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.579+0000 7faa32820640 1 -- 172.21.15.16:0/298307710 shutdown_connections 2024-09-13T16:28:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.579+0000 7faa32820640 1 -- 172.21.15.16:0/298307710 wait complete. 2024-09-13T16:28:43.591 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:43 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/676324072' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-13T16:28:43.591 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:43 smithi016 ceph-mon[31187]: osdmap e87: 12 total, 12 up, 12 in 2024-09-13T16:28:43.591 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:43 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3334139402' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-13T16:28:43.591 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:43 smithi016 ceph-mon[31187]: pgmap v276: 129 pgs: 16 creating+peering, 47 unknown, 66 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 10 op/s 2024-09-13T16:28:43.592 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:28:43.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:43 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/676324072' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-13T16:28:43.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:43 smithi111 ceph-mon[34600]: osdmap e87: 12 total, 12 up, 12 in 2024-09-13T16:28:43.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:43 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3334139402' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-13T16:28:43.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:43 smithi111 ceph-mon[34600]: pgmap v276: 129 pgs: 16 creating+peering, 47 unknown, 66 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 10 op/s 2024-09-13T16:28:43.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.696+0000 7fc31d391640 1 -- 172.21.15.16:0/1661432716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc318100880 msgr2=0x7fc318107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:43.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.696+0000 7fc31d391640 1 --2- 172.21.15.16:0/1661432716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc318100880 0x7fc318107110 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc310009ca0 tx=0x7fc31002a8c0 comp rx=0 tx=0).stop 2024-09-13T16:28:43.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.696+0000 7fc31d391640 1 -- 172.21.15.16:0/1661432716 shutdown_connections 2024-09-13T16:28:43.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.696+0000 7fc31d391640 1 --2- 172.21.15.16:0/1661432716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc318100880 0x7fc318107110 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:43.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.696+0000 7fc31d391640 1 --2- 172.21.15.16:0/1661432716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3180ffec0 0x7fc318100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:43.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.696+0000 7fc31d391640 1 --2- 172.21.15.16:0/1661432716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3180fecd0 0x7fc3180ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:43.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.696+0000 7fc31d391640 1 -- 172.21.15.16:0/1661432716 >> 172.21.15.16:0/1661432716 conn(0x7fc3180fa430 msgr2=0x7fc3180fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:43.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.696+0000 7fc31d391640 1 -- 172.21.15.16:0/1661432716 shutdown_connections 2024-09-13T16:28:43.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.696+0000 7fc31d391640 1 -- 172.21.15.16:0/1661432716 wait complete. 2024-09-13T16:28:43.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.697+0000 7fc31d391640 1 Processor -- start 2024-09-13T16:28:43.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.697+0000 7fc31d391640 1 -- start start 2024-09-13T16:28:43.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.697+0000 7fc31d391640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3180fecd0 0x7fc318194b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:43.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.697+0000 7fc31d391640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3180ffec0 0x7fc3181950c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.697+0000 7fc31d391640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc318100880 0x7fc31819c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.697+0000 7fc31d391640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31819c670 con 0x7fc3180fecd0 2024-09-13T16:28:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.697+0000 7fc31d391640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31819c7e0 con 0x7fc318100880 2024-09-13T16:28:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.697+0000 7fc31d391640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31819c9a0 con 0x7fc3180ffec0 2024-09-13T16:28:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.697+0000 7fc316ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3180fecd0 0x7fc318194b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.697+0000 7fc316ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3180fecd0 0x7fc318194b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44612/0 (socket says 172.21.15.16:44612) 2024-09-13T16:28:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.697+0000 7fc316ffd640 1 -- 172.21.15.16:0/739742202 learned_addr learned my addr 172.21.15.16:0/739742202 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.697+0000 7fc3177fe640 1 --2- 172.21.15.16:0/739742202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc318100880 0x7fc31819c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.698+0000 7fc316ffd640 1 -- 172.21.15.16:0/739742202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3180ffec0 msgr2=0x7fc3181950c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.698+0000 7fc316ffd640 1 --2- 172.21.15.16:0/739742202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3180ffec0 0x7fc3181950c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.698+0000 7fc316ffd640 1 -- 172.21.15.16:0/739742202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc318100880 msgr2=0x7fc31819c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.698+0000 7fc316ffd640 1 --2- 172.21.15.16:0/739742202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc318100880 0x7fc31819c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.698+0000 7fc316ffd640 1 -- 172.21.15.16:0/739742202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc310009900 con 0x7fc3180fecd0 2024-09-13T16:28:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.698+0000 7fc316ffd640 1 --2- 172.21.15.16:0/739742202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3180fecd0 0x7fc318194b80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc30800c970 tx=0x7fc30800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.698+0000 7fc2fffff640 1 -- 172.21.15.16:0/739742202 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc308004220 con 0x7fc3180fecd0 2024-09-13T16:28:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.698+0000 7fc31d391640 1 -- 172.21.15.16:0/739742202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc31819cc80 con 0x7fc3180fecd0 2024-09-13T16:28:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.698+0000 7fc2fffff640 1 -- 172.21.15.16:0/739742202 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc308007e40 con 0x7fc3180fecd0 2024-09-13T16:28:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.698+0000 7fc2fffff640 1 -- 172.21.15.16:0/739742202 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc308003e30 con 0x7fc3180fecd0 2024-09-13T16:28:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.698+0000 7fc31d391640 1 -- 172.21.15.16:0/739742202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc31819d190 con 0x7fc3180fecd0 2024-09-13T16:28:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.699+0000 7fc31d391640 1 -- 172.21.15.16:0/739742202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3181061d0 con 0x7fc3180fecd0 2024-09-13T16:28:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.700+0000 7fc2fffff640 1 -- 172.21.15.16:0/739742202 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fc3080079b0 con 0x7fc3180fecd0 2024-09-13T16:28:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.701+0000 7fc2fffff640 1 --2- 172.21.15.16:0/739742202 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc3000a1090 0x7fc3000a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.701+0000 7fc3167fc640 1 --2- 172.21.15.16:0/739742202 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc3000a1090 0x7fc3000a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.701+0000 7fc2fffff640 1 -- 172.21.15.16:0/739742202 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc3080cc800 con 0x7fc3180fecd0 2024-09-13T16:28:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.702+0000 7fc3167fc640 1 --2- 172.21.15.16:0/739742202 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc3000a1090 0x7fc3000a3550 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc30c0058c0 tx=0x7fc30c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:43.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.705+0000 7fc2fffff640 1 -- 172.21.15.16:0/739742202 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc308095d40 con 0x7fc3180fecd0 2024-09-13T16:28:43.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.916+0000 7fc31d391640 1 -- 172.21.15.16:0/739742202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc31810a420 con 0x7fc3180fecd0 2024-09-13T16:28:43.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.917+0000 7fc2fffff640 1 -- 172.21.15.16:0/739742202 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10813 (secure 0 0 0) 0x7fc308099360 con 0x7fc3180fecd0 2024-09-13T16:28:43.919 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:28:43.919 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3292781664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3292781664},{"type":"v1","addr":"172.21.15.16:6835","nonce":3292781664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4109039027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4109039027},{"type":"v1","addr":"172.21.15.16:6837","nonce":4109039027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2391618693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2391618693},{"type":"v1","addr":"172.21.15.16:6839","nonce":2391618693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/521042681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":521042681},{"type":"v1","addr":"172.21.15.16:6841","nonce":521042681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1984275936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1984275936},{"type":"v1","addr":"172.21.15.111:6833","nonce":1984275936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3797712690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3797712690},{"type":"v1","addr":"172.21.15.107:6833","nonce":3797712690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/338548468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":338548468},{"type":"v1","addr":"172.21.15.107:6835","nonce":338548468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/662156111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":662156111},{"type":"v1","addr":"172.21.15.111:6835","nonce":662156111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/1877128453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1877128453},{"type":"v1","addr":"172.21.15.111:6837","nonce":1877128453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2838457611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2838457611},{"type":"v1","addr":"172.21.15.107:6837","nonce":2838457611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6839/1423842285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":1423842285},{"type":"v1","addr":"172.21.15.111:6839","nonce":1423842285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:28:39.184734+0000","modified":"2024-09-13T16:28:43.574336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6839/1914689806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1914689806},{"type":"v1","addr":"172.21.15.107:6839","nonce":1914689806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-09-13T16:28:43.919 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 17 2024-09-13T16:28:43.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.919+0000 7fc31d391640 1 -- 172.21.15.16:0/739742202 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc3000a1090 msgr2=0x7fc3000a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:43.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.919+0000 7fc31d391640 1 --2- 172.21.15.16:0/739742202 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc3000a1090 0x7fc3000a3550 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc30c0058c0 tx=0x7fc30c005850 comp rx=0 tx=0).stop 2024-09-13T16:28:43.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.919+0000 7fc31d391640 1 -- 172.21.15.16:0/739742202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3180fecd0 msgr2=0x7fc318194b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:43.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.919+0000 7fc31d391640 1 --2- 172.21.15.16:0/739742202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3180fecd0 0x7fc318194b80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc30800c970 tx=0x7fc30800ce30 comp rx=0 tx=0).stop 2024-09-13T16:28:43.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.919+0000 7fc31d391640 1 -- 172.21.15.16:0/739742202 shutdown_connections 2024-09-13T16:28:43.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.919+0000 7fc31d391640 1 --2- 172.21.15.16:0/739742202 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc3000a1090 0x7fc3000a3550 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:43.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.919+0000 7fc31d391640 1 --2- 172.21.15.16:0/739742202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc318100880 0x7fc31819c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:43.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.919+0000 7fc31d391640 1 --2- 172.21.15.16:0/739742202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3180ffec0 0x7fc3181950c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:43.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.919+0000 7fc31d391640 1 --2- 172.21.15.16:0/739742202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3180fecd0 0x7fc318194b80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:43.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.919+0000 7fc31d391640 1 -- 172.21.15.16:0/739742202 >> 172.21.15.16:0/739742202 conn(0x7fc3180fa430 msgr2=0x7fc3181033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:43.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.919+0000 7fc31d391640 1 -- 172.21.15.16:0/739742202 shutdown_connections 2024-09-13T16:28:43.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:43.919+0000 7fc31d391640 1 -- 172.21.15.16:0/739742202 wait complete. 2024-09-13T16:28:43.932 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:28:44.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.032+0000 7f1f5699c640 1 -- 172.21.15.16:0/3119073411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f50100880 msgr2=0x7f1f50107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:44.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.032+0000 7f1f5699c640 1 --2- 172.21.15.16:0/3119073411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f50100880 0x7f1f50107110 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1f48009ca0 tx=0x7f1f4802a8c0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.032+0000 7f1f5699c640 1 -- 172.21.15.16:0/3119073411 shutdown_connections 2024-09-13T16:28:44.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.032+0000 7f1f5699c640 1 --2- 172.21.15.16:0/3119073411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f50100880 0x7f1f50107110 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.032+0000 7f1f5699c640 1 --2- 172.21.15.16:0/3119073411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f500ffec0 0x7f1f50100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.032+0000 7f1f5699c640 1 --2- 172.21.15.16:0/3119073411 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f500fecd0 0x7f1f500ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.032+0000 7f1f5699c640 1 -- 172.21.15.16:0/3119073411 >> 172.21.15.16:0/3119073411 conn(0x7f1f500fa430 msgr2=0x7f1f500fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:44.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.032+0000 7f1f5699c640 1 -- 172.21.15.16:0/3119073411 shutdown_connections 2024-09-13T16:28:44.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.032+0000 7f1f5699c640 1 -- 172.21.15.16:0/3119073411 wait complete. 2024-09-13T16:28:44.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.032+0000 7f1f5699c640 1 Processor -- start 2024-09-13T16:28:44.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.032+0000 7f1f5699c640 1 -- start start 2024-09-13T16:28:44.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.033+0000 7f1f5699c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f500fecd0 0x7f1f50194ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:44.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.033+0000 7f1f5699c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f500ffec0 0x7f1f501950e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:44.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.033+0000 7f1f5699c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f50100880 0x7f1f5019c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:44.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.033+0000 7f1f5699c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5019c690 con 0x7f1f500fecd0 2024-09-13T16:28:44.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.033+0000 7f1f5699c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5019c800 con 0x7f1f50100880 2024-09-13T16:28:44.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.033+0000 7f1f5699c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5019c9c0 con 0x7f1f500ffec0 2024-09-13T16:28:44.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.033+0000 7f1f4ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f500fecd0 0x7f1f50194ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:44.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.033+0000 7f1f4ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f500fecd0 0x7f1f50194ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44634/0 (socket says 172.21.15.16:44634) 2024-09-13T16:28:44.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.033+0000 7f1f4ffff640 1 -- 172.21.15.16:0/814710286 learned_addr learned my addr 172.21.15.16:0/814710286 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:44.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.033+0000 7f1f4f7fe640 1 --2- 172.21.15.16:0/814710286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f500ffec0 0x7f1f501950e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:44.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.033+0000 7f1f54f39640 1 --2- 172.21.15.16:0/814710286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f50100880 0x7f1f5019c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:44.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.033+0000 7f1f4ffff640 1 -- 172.21.15.16:0/814710286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f500ffec0 msgr2=0x7f1f501950e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:44.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.033+0000 7f1f4ffff640 1 --2- 172.21.15.16:0/814710286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f500ffec0 0x7f1f501950e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.033+0000 7f1f4ffff640 1 -- 172.21.15.16:0/814710286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f50100880 msgr2=0x7f1f5019c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:44.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.033+0000 7f1f4ffff640 1 --2- 172.21.15.16:0/814710286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f50100880 0x7f1f5019c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.033+0000 7f1f4ffff640 1 -- 172.21.15.16:0/814710286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f48009900 con 0x7f1f500fecd0 2024-09-13T16:28:44.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.033+0000 7f1f4f7fe640 1 --2- 172.21.15.16:0/814710286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f500ffec0 0x7f1f501950e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:28:44.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.033+0000 7f1f4ffff640 1 --2- 172.21.15.16:0/814710286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f500fecd0 0x7f1f50194ba0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1f4000e9c0 tx=0x7f1f4000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:44.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.034+0000 7f1f4d7fa640 1 -- 172.21.15.16:0/814710286 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f400096e0 con 0x7f1f500fecd0 2024-09-13T16:28:44.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.034+0000 7f1f5699c640 1 -- 172.21.15.16:0/814710286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f5019cca0 con 0x7f1f500fecd0 2024-09-13T16:28:44.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.034+0000 7f1f4d7fa640 1 -- 172.21.15.16:0/814710286 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f400044c0 con 0x7f1f500fecd0 2024-09-13T16:28:44.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.034+0000 7f1f4d7fa640 1 -- 172.21.15.16:0/814710286 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f40003e30 con 0x7f1f500fecd0 2024-09-13T16:28:44.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.034+0000 7f1f5699c640 1 -- 172.21.15.16:0/814710286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f5019d1b0 con 0x7f1f500fecd0 2024-09-13T16:28:44.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.035+0000 7f1f5699c640 1 -- 172.21.15.16:0/814710286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f501061d0 con 0x7f1f500fecd0 2024-09-13T16:28:44.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.036+0000 7f1f4d7fa640 1 -- 172.21.15.16:0/814710286 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1f400035f0 con 0x7f1f500fecd0 2024-09-13T16:28:44.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.036+0000 7f1f4d7fa640 1 --2- 172.21.15.16:0/814710286 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f1f3c0a1110 0x7f1f3c0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:44.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.037+0000 7f1f4f7fe640 1 --2- 172.21.15.16:0/814710286 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f1f3c0a1110 0x7f1f3c0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:44.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.037+0000 7f1f4d7fa640 1 -- 172.21.15.16:0/814710286 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1f40014070 con 0x7f1f500fecd0 2024-09-13T16:28:44.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.037+0000 7f1f4f7fe640 1 --2- 172.21.15.16:0/814710286 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f1f3c0a1110 0x7f1f3c0a35d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1f501960b0 tx=0x7f1f38005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:44.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.041+0000 7f1f4d7fa640 1 -- 172.21.15.16:0/814710286 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f1f400964b0 con 0x7f1f500fecd0 2024-09-13T16:28:44.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.214+0000 7f1f5699c640 1 -- 172.21.15.16:0/814710286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f5002c410 con 0x7f1f500fecd0 2024-09-13T16:28:44.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.215+0000 7f1f4d7fa640 1 -- 172.21.15.16:0/814710286 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23971 (secure 0 0 0) 0x7f1f40099ad0 con 0x7f1f500fecd0 2024-09-13T16:28:44.218 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:28:44.218 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":89,"fsid":"0d241f6a-71ec-11ef-bceb-c7b262605968","created":"2024-09-13T16:21:17.971738+0000","modified":"2024-09-13T16:28:43.574106+0000","last_up_change":"2024-09-13T16:27:54.673413+0000","last_in_change":"2024-09-13T16:27:35.718390+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:24:22.981019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:28:37.746027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:28:38.602731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:28:41.633322+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e11ef8f-08d6-4e74-afd6-f49f4bd40032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6803","nonce":2079394186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6805","nonce":2079394186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6809","nonce":2079394186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6807","nonce":2079394186}]},"public_addr":"172.21.15.16:6803/2079394186","cluster_addr":"172.21.15.16:6805/2079394186","heartbeat_back_addr":"172.21.15.16:6809/2079394186","heartbeat_front_addr":"172.21.15.16:6807/2079394186","state":["exists","up"]},{"osd":1,"uuid":"f2cd3694-9c9e-4da7-976e-f80a8df120f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6801","nonce":4021360958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6803","nonce":4021360958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6807","nonce":4021360958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6805","nonce":4021360958}]},"public_addr":"172.21.15.107:6801/4021360958","cluster_addr":"172.21.15.107:6803/4021360958","heartbeat_back_addr":"172.21.15.107:6807/4021360958","heartbeat_front_addr":"172.21.15.107:6805/4021360958","state":["exists","up"]},{"osd":2,"uuid":"a5bbf2eb-6599-44d5-ad96-68bb89ee19f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6801","nonce":3227179825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6803","nonce":3227179825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6807","nonce":3227179825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6805","nonce":3227179825}]},"public_addr":"172.21.15.111:6801/3227179825","cluster_addr":"172.21.15.111:6803/3227179825","heartbeat_back_addr":"172.21.15.111:6807/3227179825","heartbeat_front_addr":"172.21.15.111:6805/3227179825","state":["exists","up"]},{"osd":3,"uuid":"d6295ffe-11bc-4964-a250-9fecca870b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6811","nonce":2390375937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6813","nonce":2390375937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6817","nonce":2390375937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6815","nonce":2390375937}]},"public_addr":"172.21.15.16:6811/2390375937","cluster_addr":"172.21.15.16:6813/2390375937","heartbeat_back_addr":"172.21.15.16:6817/2390375937","heartbeat_front_addr":"172.21.15.16:6815/2390375937","state":["exists","up"]},{"osd":4,"uuid":"0a8446e0-fc86-4646-beb6-a7b9d433093d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6809","nonce":1936582048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6811","nonce":1936582048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6815","nonce":1936582048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6813","nonce":1936582048}]},"public_addr":"172.21.15.107:6809/1936582048","cluster_addr":"172.21.15.107:6811/1936582048","heartbeat_back_addr":"172.21.15.107:6815/1936582048","heartbeat_front_addr":"172.21.15.107:6813/1936582048","state":["exists","up"]},{"osd":5,"uuid":"2c97a446-357e-4975-9e51-9c95fe84a663","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6809","nonce":483858448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6811","nonce":483858448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6815","nonce":483858448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6813","nonce":483858448}]},"public_addr":"172.21.15.111:6809/483858448","cluster_addr":"172.21.15.111:6811/483858448","heartbeat_back_addr":"172.21.15.111:6815/483858448","heartbeat_front_addr":"172.21.15.111:6813/483858448","state":["exists","up"]},{"osd":6,"uuid":"890239e6-e6fb-4f9c-b973-05cc66d199bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6819","nonce":1330214141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6821","nonce":1330214141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6825","nonce":1330214141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6823","nonce":1330214141}]},"public_addr":"172.21.15.16:6819/1330214141","cluster_addr":"172.21.15.16:6821/1330214141","heartbeat_back_addr":"172.21.15.16:6825/1330214141","heartbeat_front_addr":"172.21.15.16:6823/1330214141","state":["exists","up"]},{"osd":7,"uuid":"ee826a9c-f441-4903-9b1b-66b627f7b712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6817","nonce":1133876754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6819","nonce":1133876754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6823","nonce":1133876754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6821","nonce":1133876754}]},"public_addr":"172.21.15.107:6817/1133876754","cluster_addr":"172.21.15.107:6819/1133876754","heartbeat_back_addr":"172.21.15.107:6823/1133876754","heartbeat_front_addr":"172.21.15.107:6821/1133876754","state":["exists","up"]},{"osd":8,"uuid":"e19773f2-dec8-44e0-9d33-0252e07d6415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6817","nonce":837548190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6819","nonce":837548190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6823","nonce":837548190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6821","nonce":837548190}]},"public_addr":"172.21.15.111:6817/837548190","cluster_addr":"172.21.15.111:6819/837548190","heartbeat_back_addr":"172.21.15.111:6823/837548190","heartbeat_front_addr":"172.21.15.111:6821/837548190","state":["exists","up"]},{"osd":9,"uuid":"bae4d6db-8b90-4d02-8e83-7d8f2a6f8fdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6827","nonce":4175383686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6829","nonce":4175383686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6833","nonce":4175383686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6831","nonce":4175383686}]},"public_addr":"172.21.15.16:6827/4175383686","cluster_addr":"172.21.15.16:6829/4175383686","heartbeat_back_addr":"172.21.15.16:6833/4175383686","heartbeat_front_addr":"172.21.15.16:6831/4175383686","state":["exists","up"]},{"osd":10,"uuid":"fd0e7740-76c8-4c99-a349-7077b4905143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6825","nonce":2659550426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6827","nonce":2659550426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6831","nonce":2659550426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6829","nonce":2659550426}]},"public_addr":"172.21.15.107:6825/2659550426","cluster_addr":"172.21.15.107:6827/2659550426","heartbeat_back_addr":"172.21.15.107:6831/2659550426","heartbeat_front_addr":"172.21.15.107:6829/2659550426","state":["exists","up"]},{"osd":11,"uuid":"94a1751d-b00c-4e23-815d-3a8bc3dea146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6825","nonce":1633548683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6827","nonce":1633548683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6831","nonce":1633548683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6829","nonce":1633548683}]},"public_addr":"172.21.15.111:6825/1633548683","cluster_addr":"172.21.15.111:6827/1633548683","heartbeat_back_addr":"172.21.15.111:6831/1633548683","heartbeat_front_addr":"172.21.15.111:6829/1633548683","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:36.359852+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:58.014202+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:19.753672+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:43.404817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:06.001677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:29.361587+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:52.554847+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:15.957553+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:39.581093+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:04.911389+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:28.971280+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:53.473561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/2098378667":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/3536229016":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6801/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/2684687547":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6800/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/368410612":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/12158160":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6800/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6801/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/3754423379":"2024-09-14T16:21:40.126459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:28:44.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.217+0000 7f1f5699c640 1 -- 172.21.15.16:0/814710286 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f1f3c0a1110 msgr2=0x7f1f3c0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:44.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.217+0000 7f1f5699c640 1 --2- 172.21.15.16:0/814710286 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f1f3c0a1110 0x7f1f3c0a35d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1f501960b0 tx=0x7f1f38005f90 comp rx=0 tx=0).stop 2024-09-13T16:28:44.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.217+0000 7f1f5699c640 1 -- 172.21.15.16:0/814710286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f500fecd0 msgr2=0x7f1f50194ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:44.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.217+0000 7f1f5699c640 1 --2- 172.21.15.16:0/814710286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f500fecd0 0x7f1f50194ba0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1f4000e9c0 tx=0x7f1f4000ee80 comp rx=0 tx=0).stop 2024-09-13T16:28:44.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.218+0000 7f1f5699c640 1 -- 172.21.15.16:0/814710286 shutdown_connections 2024-09-13T16:28:44.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.218+0000 7f1f5699c640 1 --2- 172.21.15.16:0/814710286 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f1f3c0a1110 0x7f1f3c0a35d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.218+0000 7f1f5699c640 1 --2- 172.21.15.16:0/814710286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f50100880 0x7f1f5019c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.218+0000 7f1f5699c640 1 --2- 172.21.15.16:0/814710286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f500ffec0 0x7f1f501950e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.218+0000 7f1f5699c640 1 --2- 172.21.15.16:0/814710286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f500fecd0 0x7f1f50194ba0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.218+0000 7f1f5699c640 1 -- 172.21.15.16:0/814710286 >> 172.21.15.16:0/814710286 conn(0x7f1f500fa430 msgr2=0x7f1f501033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:44.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.218+0000 7f1f5699c640 1 -- 172.21.15.16:0/814710286 shutdown_connections 2024-09-13T16:28:44.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.218+0000 7f1f5699c640 1 -- 172.21.15.16:0/814710286 wait complete. 2024-09-13T16:28:44.231 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:28:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.332+0000 7f9eddf1f640 1 -- 172.21.15.16:0/3206787193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ed80ff8e0 msgr2=0x7f9ed80ffd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.332+0000 7f9eddf1f640 1 --2- 172.21.15.16:0/3206787193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ed80ff8e0 0x7f9ed80ffd90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0009ca0 tx=0x7f9ed002a8c0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.332+0000 7f9eddf1f640 1 -- 172.21.15.16:0/3206787193 shutdown_connections 2024-09-13T16:28:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.332+0000 7f9eddf1f640 1 --2- 172.21.15.16:0/3206787193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ed80ff8e0 0x7f9ed80ffd90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.332+0000 7f9eddf1f640 1 --2- 172.21.15.16:0/3206787193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ed80fef40 0x7f9ed80ff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.332+0000 7f9eddf1f640 1 --2- 172.21.15.16:0/3206787193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ed80fe600 0x7f9ed80fea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.332+0000 7f9eddf1f640 1 -- 172.21.15.16:0/3206787193 >> 172.21.15.16:0/3206787193 conn(0x7f9ed80fa3f0 msgr2=0x7f9ed80fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.332+0000 7f9eddf1f640 1 -- 172.21.15.16:0/3206787193 shutdown_connections 2024-09-13T16:28:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.332+0000 7f9eddf1f640 1 -- 172.21.15.16:0/3206787193 wait complete. 2024-09-13T16:28:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.333+0000 7f9eddf1f640 1 Processor -- start 2024-09-13T16:28:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.333+0000 7f9eddf1f640 1 -- start start 2024-09-13T16:28:44.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.333+0000 7f9eddf1f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ed80fe600 0x7f9ed8197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:44.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.333+0000 7f9eddf1f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ed80fef40 0x7f9ed8198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:44.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.333+0000 7f9eddf1f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ed80ff8e0 0x7f9ed819f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.333+0000 7f9eddf1f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed819f930 con 0x7f9ed80ff8e0 2024-09-13T16:28:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.333+0000 7f9eddf1f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed819faa0 con 0x7f9ed80fe600 2024-09-13T16:28:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.333+0000 7f9eddf1f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed819fc60 con 0x7f9ed80fef40 2024-09-13T16:28:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.333+0000 7f9ed7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ed80ff8e0 0x7f9ed819f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.333+0000 7f9ed7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ed80ff8e0 0x7f9ed819f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44650/0 (socket says 172.21.15.16:44650) 2024-09-13T16:28:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.333+0000 7f9ed7fff640 1 -- 172.21.15.16:0/3410280895 learned_addr learned my addr 172.21.15.16:0/3410280895 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.333+0000 7f9ed77fe640 1 --2- 172.21.15.16:0/3410280895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ed80fe600 0x7f9ed8197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.333+0000 7f9ed6ffd640 1 --2- 172.21.15.16:0/3410280895 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ed80fef40 0x7f9ed8198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.333+0000 7f9ed7fff640 1 -- 172.21.15.16:0/3410280895 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ed80fef40 msgr2=0x7f9ed8198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.333+0000 7f9ed7fff640 1 --2- 172.21.15.16:0/3410280895 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ed80fef40 0x7f9ed8198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.333+0000 7f9ed7fff640 1 -- 172.21.15.16:0/3410280895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ed80fe600 msgr2=0x7f9ed8197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.333+0000 7f9ed7fff640 1 --2- 172.21.15.16:0/3410280895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ed80fe600 0x7f9ed8197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.333+0000 7f9ed7fff640 1 -- 172.21.15.16:0/3410280895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed0009900 con 0x7f9ed80ff8e0 2024-09-13T16:28:44.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.334+0000 7f9ed77fe640 1 --2- 172.21.15.16:0/3410280895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ed80fe600 0x7f9ed8197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:28:44.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.334+0000 7f9ed7fff640 1 --2- 172.21.15.16:0/3410280895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ed80ff8e0 0x7f9ed819f3f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9ed002aeb0 tx=0x7f9ed0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:44.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.334+0000 7f9ed4ff9640 1 -- 172.21.15.16:0/3410280895 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed00307c0 con 0x7f9ed80ff8e0 2024-09-13T16:28:44.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.334+0000 7f9ed4ff9640 1 -- 172.21.15.16:0/3410280895 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9ed0040070 con 0x7f9ed80ff8e0 2024-09-13T16:28:44.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.334+0000 7f9eddf1f640 1 -- 172.21.15.16:0/3410280895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed819fee0 con 0x7f9ed80ff8e0 2024-09-13T16:28:44.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.334+0000 7f9ed4ff9640 1 -- 172.21.15.16:0/3410280895 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed00043c0 con 0x7f9ed80ff8e0 2024-09-13T16:28:44.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.334+0000 7f9eddf1f640 1 -- 172.21.15.16:0/3410280895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed81a0360 con 0x7f9ed80ff8e0 2024-09-13T16:28:44.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.335+0000 7f9eddf1f640 1 -- 172.21.15.16:0/3410280895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed81095a0 con 0x7f9ed80ff8e0 2024-09-13T16:28:44.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.336+0000 7f9ed4ff9640 1 -- 172.21.15.16:0/3410280895 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f9ed0042030 con 0x7f9ed80ff8e0 2024-09-13T16:28:44.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.337+0000 7f9ed4ff9640 1 --2- 172.21.15.16:0/3410280895 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f9ec00a1090 0x7f9ec00a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:44.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.337+0000 7f9ed77fe640 1 --2- 172.21.15.16:0/3410280895 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f9ec00a1090 0x7f9ec00a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:44.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.337+0000 7f9ed4ff9640 1 -- 172.21.15.16:0/3410280895 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9ed00ee180 con 0x7f9ed80ff8e0 2024-09-13T16:28:44.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.338+0000 7f9ed77fe640 1 --2- 172.21.15.16:0/3410280895 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f9ec00a1090 0x7f9ec00a3550 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8009fc0 tx=0x7f9ec8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:44.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.342+0000 7f9ed4ff9640 1 -- 172.21.15.16:0/3410280895 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f9ed00b7720 con 0x7f9ed80ff8e0 2024-09-13T16:28:44.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.512+0000 7f9eddf1f640 1 -- 172.21.15.16:0/3410280895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ed802c410 con 0x7f9ed80ff8e0 2024-09-13T16:28:44.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.514+0000 7f9ed4ff9640 1 -- 172.21.15.16:0/3410280895 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23971 (secure 0 0 0) 0x7f9ed00bad40 con 0x7f9ed80ff8e0 2024-09-13T16:28:44.516 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:28:44.516 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":89,"fsid":"0d241f6a-71ec-11ef-bceb-c7b262605968","created":"2024-09-13T16:21:17.971738+0000","modified":"2024-09-13T16:28:43.574106+0000","last_up_change":"2024-09-13T16:27:54.673413+0000","last_in_change":"2024-09-13T16:27:35.718390+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:24:22.981019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:28:37.746027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:28:38.602731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:28:41.633322+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e11ef8f-08d6-4e74-afd6-f49f4bd40032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6803","nonce":2079394186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6805","nonce":2079394186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6809","nonce":2079394186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6807","nonce":2079394186}]},"public_addr":"172.21.15.16:6803/2079394186","cluster_addr":"172.21.15.16:6805/2079394186","heartbeat_back_addr":"172.21.15.16:6809/2079394186","heartbeat_front_addr":"172.21.15.16:6807/2079394186","state":["exists","up"]},{"osd":1,"uuid":"f2cd3694-9c9e-4da7-976e-f80a8df120f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6801","nonce":4021360958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6803","nonce":4021360958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6807","nonce":4021360958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6805","nonce":4021360958}]},"public_addr":"172.21.15.107:6801/4021360958","cluster_addr":"172.21.15.107:6803/4021360958","heartbeat_back_addr":"172.21.15.107:6807/4021360958","heartbeat_front_addr":"172.21.15.107:6805/4021360958","state":["exists","up"]},{"osd":2,"uuid":"a5bbf2eb-6599-44d5-ad96-68bb89ee19f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6801","nonce":3227179825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6803","nonce":3227179825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6807","nonce":3227179825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6805","nonce":3227179825}]},"public_addr":"172.21.15.111:6801/3227179825","cluster_addr":"172.21.15.111:6803/3227179825","heartbeat_back_addr":"172.21.15.111:6807/3227179825","heartbeat_front_addr":"172.21.15.111:6805/3227179825","state":["exists","up"]},{"osd":3,"uuid":"d6295ffe-11bc-4964-a250-9fecca870b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6811","nonce":2390375937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6813","nonce":2390375937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6817","nonce":2390375937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6815","nonce":2390375937}]},"public_addr":"172.21.15.16:6811/2390375937","cluster_addr":"172.21.15.16:6813/2390375937","heartbeat_back_addr":"172.21.15.16:6817/2390375937","heartbeat_front_addr":"172.21.15.16:6815/2390375937","state":["exists","up"]},{"osd":4,"uuid":"0a8446e0-fc86-4646-beb6-a7b9d433093d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6809","nonce":1936582048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6811","nonce":1936582048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6815","nonce":1936582048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6813","nonce":1936582048}]},"public_addr":"172.21.15.107:6809/1936582048","cluster_addr":"172.21.15.107:6811/1936582048","heartbeat_back_addr":"172.21.15.107:6815/1936582048","heartbeat_front_addr":"172.21.15.107:6813/1936582048","state":["exists","up"]},{"osd":5,"uuid":"2c97a446-357e-4975-9e51-9c95fe84a663","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6809","nonce":483858448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6811","nonce":483858448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6815","nonce":483858448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6813","nonce":483858448}]},"public_addr":"172.21.15.111:6809/483858448","cluster_addr":"172.21.15.111:6811/483858448","heartbeat_back_addr":"172.21.15.111:6815/483858448","heartbeat_front_addr":"172.21.15.111:6813/483858448","state":["exists","up"]},{"osd":6,"uuid":"890239e6-e6fb-4f9c-b973-05cc66d199bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6819","nonce":1330214141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6821","nonce":1330214141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6825","nonce":1330214141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6823","nonce":1330214141}]},"public_addr":"172.21.15.16:6819/1330214141","cluster_addr":"172.21.15.16:6821/1330214141","heartbeat_back_addr":"172.21.15.16:6825/1330214141","heartbeat_front_addr":"172.21.15.16:6823/1330214141","state":["exists","up"]},{"osd":7,"uuid":"ee826a9c-f441-4903-9b1b-66b627f7b712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6817","nonce":1133876754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6819","nonce":1133876754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6823","nonce":1133876754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6821","nonce":1133876754}]},"public_addr":"172.21.15.107:6817/1133876754","cluster_addr":"172.21.15.107:6819/1133876754","heartbeat_back_addr":"172.21.15.107:6823/1133876754","heartbeat_front_addr":"172.21.15.107:6821/1133876754","state":["exists","up"]},{"osd":8,"uuid":"e19773f2-dec8-44e0-9d33-0252e07d6415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6817","nonce":837548190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6819","nonce":837548190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6823","nonce":837548190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6821","nonce":837548190}]},"public_addr":"172.21.15.111:6817/837548190","cluster_addr":"172.21.15.111:6819/837548190","heartbeat_back_addr":"172.21.15.111:6823/837548190","heartbeat_front_addr":"172.21.15.111:6821/837548190","state":["exists","up"]},{"osd":9,"uuid":"bae4d6db-8b90-4d02-8e83-7d8f2a6f8fdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6827","nonce":4175383686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6829","nonce":4175383686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6833","nonce":4175383686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6831","nonce":4175383686}]},"public_addr":"172.21.15.16:6827/4175383686","cluster_addr":"172.21.15.16:6829/4175383686","heartbeat_back_addr":"172.21.15.16:6833/4175383686","heartbeat_front_addr":"172.21.15.16:6831/4175383686","state":["exists","up"]},{"osd":10,"uuid":"fd0e7740-76c8-4c99-a349-7077b4905143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6825","nonce":2659550426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6827","nonce":2659550426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6831","nonce":2659550426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6829","nonce":2659550426}]},"public_addr":"172.21.15.107:6825/2659550426","cluster_addr":"172.21.15.107:6827/2659550426","heartbeat_back_addr":"172.21.15.107:6831/2659550426","heartbeat_front_addr":"172.21.15.107:6829/2659550426","state":["exists","up"]},{"osd":11,"uuid":"94a1751d-b00c-4e23-815d-3a8bc3dea146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6825","nonce":1633548683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6827","nonce":1633548683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6831","nonce":1633548683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6829","nonce":1633548683}]},"public_addr":"172.21.15.111:6825/1633548683","cluster_addr":"172.21.15.111:6827/1633548683","heartbeat_back_addr":"172.21.15.111:6831/1633548683","heartbeat_front_addr":"172.21.15.111:6829/1633548683","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:36.359852+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:58.014202+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:19.753672+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:43.404817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:06.001677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:29.361587+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:52.554847+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:15.957553+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:39.581093+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:04.911389+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:28.971280+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:53.473561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/2098378667":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/3536229016":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6801/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/2684687547":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6800/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/368410612":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/12158160":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6800/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6801/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/3754423379":"2024-09-14T16:21:40.126459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:28:44.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.516+0000 7f9eddf1f640 1 -- 172.21.15.16:0/3410280895 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f9ec00a1090 msgr2=0x7f9ec00a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:44.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.516+0000 7f9eddf1f640 1 --2- 172.21.15.16:0/3410280895 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f9ec00a1090 0x7f9ec00a3550 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8009fc0 tx=0x7f9ec8005f90 comp rx=0 tx=0).stop 2024-09-13T16:28:44.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.516+0000 7f9eddf1f640 1 -- 172.21.15.16:0/3410280895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ed80ff8e0 msgr2=0x7f9ed819f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:44.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.516+0000 7f9eddf1f640 1 --2- 172.21.15.16:0/3410280895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ed80ff8e0 0x7f9ed819f3f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9ed002aeb0 tx=0x7f9ed0003c20 comp rx=0 tx=0).stop 2024-09-13T16:28:44.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.516+0000 7f9eddf1f640 1 -- 172.21.15.16:0/3410280895 shutdown_connections 2024-09-13T16:28:44.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.516+0000 7f9eddf1f640 1 --2- 172.21.15.16:0/3410280895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ed80ff8e0 0x7f9ed819f3f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.516+0000 7f9eddf1f640 1 --2- 172.21.15.16:0/3410280895 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ed80fef40 0x7f9ed8198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.516+0000 7f9eddf1f640 1 --2- 172.21.15.16:0/3410280895 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f9ec00a1090 0x7f9ec00a3550 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.516+0000 7f9eddf1f640 1 --2- 172.21.15.16:0/3410280895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ed80fe600 0x7f9ed8197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.516+0000 7f9eddf1f640 1 -- 172.21.15.16:0/3410280895 >> 172.21.15.16:0/3410280895 conn(0x7f9ed80fa3f0 msgr2=0x7f9ed80fbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:44.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.516+0000 7f9eddf1f640 1 -- 172.21.15.16:0/3410280895 shutdown_connections 2024-09-13T16:28:44.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.516+0000 7f9eddf1f640 1 -- 172.21.15.16:0/3410280895 wait complete. 2024-09-13T16:28:44.528 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:44 smithi016 ceph-mon[31187]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T16:28:44.529 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:44 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3334139402' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-13T16:28:44.529 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:44 smithi016 ceph-mon[31187]: osdmap e88: 12 total, 12 up, 12 in 2024-09-13T16:28:44.529 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:44 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/298307710' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-13T16:28:44.529 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:44 smithi016 ceph-mon[31187]: osdmap e89: 12 total, 12 up, 12 in 2024-09-13T16:28:44.529 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:44 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/298307710' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-13T16:28:44.529 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:44 smithi016 ceph-mon[31187]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-13T16:28:44.529 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:44 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/739742202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:44.550 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:28:44.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:44 smithi107 ceph-mon[34139]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T16:28:44.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:44 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3334139402' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-13T16:28:44.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:44 smithi107 ceph-mon[34139]: osdmap e88: 12 total, 12 up, 12 in 2024-09-13T16:28:44.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:44 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/298307710' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-13T16:28:44.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:44 smithi107 ceph-mon[34139]: osdmap e89: 12 total, 12 up, 12 in 2024-09-13T16:28:44.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:44 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/298307710' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-13T16:28:44.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:44 smithi107 ceph-mon[34139]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-13T16:28:44.600 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:44 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/739742202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:44.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.650+0000 7fab07be2640 1 -- 172.21.15.16:0/3468722120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab00100850 msgr2=0x7fab001070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:44.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.650+0000 7fab07be2640 1 --2- 172.21.15.16:0/3468722120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab00100850 0x7fab001070e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7faaf8009ca0 tx=0x7faaf802a8c0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.650+0000 7fab07be2640 1 -- 172.21.15.16:0/3468722120 shutdown_connections 2024-09-13T16:28:44.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.650+0000 7fab07be2640 1 --2- 172.21.15.16:0/3468722120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab00100850 0x7fab001070e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.651+0000 7fab07be2640 1 --2- 172.21.15.16:0/3468722120 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab000ffe90 0x7fab00100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.651+0000 7fab07be2640 1 --2- 172.21.15.16:0/3468722120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab000feca0 0x7fab000ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.651+0000 7fab07be2640 1 -- 172.21.15.16:0/3468722120 >> 172.21.15.16:0/3468722120 conn(0x7fab000fa440 msgr2=0x7fab000fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.651+0000 7fab07be2640 1 -- 172.21.15.16:0/3468722120 shutdown_connections 2024-09-13T16:28:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.651+0000 7fab07be2640 1 -- 172.21.15.16:0/3468722120 wait complete. 2024-09-13T16:28:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.651+0000 7fab07be2640 1 Processor -- start 2024-09-13T16:28:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.651+0000 7fab07be2640 1 -- start start 2024-09-13T16:28:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.651+0000 7fab07be2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab000feca0 0x7fab00195c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.651+0000 7fab07be2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab000ffe90 0x7fab001961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.651+0000 7fab07be2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab00100850 0x7fab0019d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.651+0000 7fab07be2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0019d740 con 0x7fab000feca0 2024-09-13T16:28:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.651+0000 7fab07be2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0019d8b0 con 0x7fab00100850 2024-09-13T16:28:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.651+0000 7fab07be2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0019da70 con 0x7fab000ffe90 2024-09-13T16:28:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.652+0000 7fab0597e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab000feca0 0x7fab00195c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.652+0000 7fab0597e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab000feca0 0x7fab00195c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44676/0 (socket says 172.21.15.16:44676) 2024-09-13T16:28:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.652+0000 7fab0597e640 1 -- 172.21.15.16:0/1237869844 learned_addr learned my addr 172.21.15.16:0/1237869844 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.652+0000 7fab0617f640 1 --2- 172.21.15.16:0/1237869844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab00100850 0x7fab0019d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.652+0000 7fab0517d640 1 --2- 172.21.15.16:0/1237869844 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab000ffe90 0x7fab001961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.652+0000 7fab0597e640 1 -- 172.21.15.16:0/1237869844 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab000ffe90 msgr2=0x7fab001961a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.652+0000 7fab0597e640 1 --2- 172.21.15.16:0/1237869844 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab000ffe90 0x7fab001961a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.652+0000 7fab0597e640 1 -- 172.21.15.16:0/1237869844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab00100850 msgr2=0x7fab0019d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.652+0000 7fab0597e640 1 --2- 172.21.15.16:0/1237869844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab00100850 0x7fab0019d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.652+0000 7fab0597e640 1 -- 172.21.15.16:0/1237869844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf8009900 con 0x7fab000feca0 2024-09-13T16:28:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.652+0000 7fab0617f640 1 --2- 172.21.15.16:0/1237869844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab00100850 0x7fab0019d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:28:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.652+0000 7fab0517d640 1 --2- 172.21.15.16:0/1237869844 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab000ffe90 0x7fab001961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:28:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.652+0000 7fab0597e640 1 --2- 172.21.15.16:0/1237869844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab000feca0 0x7fab00195c60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faafc00e9c0 tx=0x7faafc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.652+0000 7faaf6ffd640 1 -- 172.21.15.16:0/1237869844 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc009700 con 0x7fab000feca0 2024-09-13T16:28:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.652+0000 7faaf6ffd640 1 -- 172.21.15.16:0/1237869844 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faafc0044b0 con 0x7fab000feca0 2024-09-13T16:28:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.652+0000 7fab07be2640 1 -- 172.21.15.16:0/1237869844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab0019dd50 con 0x7fab000feca0 2024-09-13T16:28:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.652+0000 7faaf6ffd640 1 -- 172.21.15.16:0/1237869844 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc003e40 con 0x7fab000feca0 2024-09-13T16:28:44.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.652+0000 7fab07be2640 1 -- 172.21.15.16:0/1237869844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab0019e1f0 con 0x7fab000feca0 2024-09-13T16:28:44.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.653+0000 7fab07be2640 1 -- 172.21.15.16:0/1237869844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab0010a3f0 con 0x7fab000feca0 2024-09-13T16:28:44.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.655+0000 7faaf6ffd640 1 -- 172.21.15.16:0/1237869844 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7faafc003fa0 con 0x7fab000feca0 2024-09-13T16:28:44.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.655+0000 7faaf6ffd640 1 --2- 172.21.15.16:0/1237869844 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7faaec0a1110 0x7faaec0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:44.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.655+0000 7faaf6ffd640 1 -- 172.21.15.16:0/1237869844 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7faafc014070 con 0x7fab000feca0 2024-09-13T16:28:44.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.655+0000 7fab0517d640 1 --2- 172.21.15.16:0/1237869844 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7faaec0a1110 0x7faaec0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:44.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.656+0000 7fab0517d640 1 --2- 172.21.15.16:0/1237869844 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7faaec0a1110 0x7faaec0a35d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7faaf0009c20 tx=0x7faaf0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:44.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.658+0000 7faaf6ffd640 1 -- 172.21.15.16:0/1237869844 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7faafc096640 con 0x7fab000feca0 2024-09-13T16:28:44.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:44 smithi111 ceph-mon[34600]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T16:28:44.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:44 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3334139402' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-13T16:28:44.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:44 smithi111 ceph-mon[34600]: osdmap e88: 12 total, 12 up, 12 in 2024-09-13T16:28:44.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:44 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/298307710' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-13T16:28:44.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:44 smithi111 ceph-mon[34600]: osdmap e89: 12 total, 12 up, 12 in 2024-09-13T16:28:44.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:44 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/298307710' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-13T16:28:44.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:44 smithi111 ceph-mon[34600]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-13T16:28:44.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:44 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/739742202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:44.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.866+0000 7fab07be2640 1 -- 172.21.15.16:0/1237869844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab0010a5d0 con 0x7fab000feca0 2024-09-13T16:28:44.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.867+0000 7faaf6ffd640 1 -- 172.21.15.16:0/1237869844 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10813 (secure 0 0 0) 0x7faafc099c60 con 0x7fab000feca0 2024-09-13T16:28:44.869 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:28:44.869 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3292781664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3292781664},{"type":"v1","addr":"172.21.15.16:6835","nonce":3292781664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4109039027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4109039027},{"type":"v1","addr":"172.21.15.16:6837","nonce":4109039027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2391618693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2391618693},{"type":"v1","addr":"172.21.15.16:6839","nonce":2391618693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/521042681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":521042681},{"type":"v1","addr":"172.21.15.16:6841","nonce":521042681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1984275936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1984275936},{"type":"v1","addr":"172.21.15.111:6833","nonce":1984275936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/3797712690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3797712690},{"type":"v1","addr":"172.21.15.107:6833","nonce":3797712690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/338548468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":338548468},{"type":"v1","addr":"172.21.15.107:6835","nonce":338548468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/662156111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":662156111},{"type":"v1","addr":"172.21.15.111:6835","nonce":662156111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/1877128453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1877128453},{"type":"v1","addr":"172.21.15.111:6837","nonce":1877128453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2838457611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2838457611},{"type":"v1","addr":"172.21.15.107:6837","nonce":2838457611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6839/1423842285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":1423842285},{"type":"v1","addr":"172.21.15.111:6839","nonce":1423842285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:28:39.184734+0000","modified":"2024-09-13T16:28:43.574336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6839/1914689806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1914689806},{"type":"v1","addr":"172.21.15.107:6839","nonce":1914689806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-09-13T16:28:44.869 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 17 2024-09-13T16:28:44.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.868+0000 7fab07be2640 1 -- 172.21.15.16:0/1237869844 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7faaec0a1110 msgr2=0x7faaec0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:44.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.868+0000 7fab07be2640 1 --2- 172.21.15.16:0/1237869844 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7faaec0a1110 0x7faaec0a35d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7faaf0009c20 tx=0x7faaf0005f90 comp rx=0 tx=0).stop 2024-09-13T16:28:44.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.868+0000 7fab07be2640 1 -- 172.21.15.16:0/1237869844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab000feca0 msgr2=0x7fab00195c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:44.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.868+0000 7fab07be2640 1 --2- 172.21.15.16:0/1237869844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab000feca0 0x7fab00195c60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faafc00e9c0 tx=0x7faafc00ee80 comp rx=0 tx=0).stop 2024-09-13T16:28:44.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.868+0000 7fab07be2640 1 -- 172.21.15.16:0/1237869844 shutdown_connections 2024-09-13T16:28:44.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.868+0000 7fab07be2640 1 --2- 172.21.15.16:0/1237869844 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7faaec0a1110 0x7faaec0a35d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.868+0000 7fab07be2640 1 --2- 172.21.15.16:0/1237869844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab00100850 0x7fab0019d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.868+0000 7fab07be2640 1 --2- 172.21.15.16:0/1237869844 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab000ffe90 0x7fab001961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.868+0000 7fab07be2640 1 --2- 172.21.15.16:0/1237869844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab000feca0 0x7fab00195c60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.868+0000 7fab07be2640 1 -- 172.21.15.16:0/1237869844 >> 172.21.15.16:0/1237869844 conn(0x7fab000fa440 msgr2=0x7fab00067690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:44.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.869+0000 7fab07be2640 1 -- 172.21.15.16:0/1237869844 shutdown_connections 2024-09-13T16:28:44.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.869+0000 7fab07be2640 1 -- 172.21.15.16:0/1237869844 wait complete. 2024-09-13T16:28:44.883 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:28:44.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.983+0000 7fa8e2c96640 1 -- 172.21.15.16:0/2210054001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8dc105a80 msgr2=0x7fa8dc107ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:44.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.983+0000 7fa8e2c96640 1 --2- 172.21.15.16:0/2210054001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8dc105a80 0x7fa8dc107ef0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4009900 tx=0x7fa8d402a410 comp rx=0 tx=0).stop 2024-09-13T16:28:44.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.984+0000 7fa8e2c96640 1 -- 172.21.15.16:0/2210054001 shutdown_connections 2024-09-13T16:28:44.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.984+0000 7fa8e2c96640 1 --2- 172.21.15.16:0/2210054001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8dc105a80 0x7fa8dc107ef0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.984+0000 7fa8e2c96640 1 --2- 172.21.15.16:0/2210054001 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8dc103150 0x7fa8dc105540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.984+0000 7fa8e2c96640 1 --2- 172.21.15.16:0/2210054001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8dc100820 0x7fa8dc102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.984+0000 7fa8e2c96640 1 -- 172.21.15.16:0/2210054001 >> 172.21.15.16:0/2210054001 conn(0x7fa8dc0fa440 msgr2=0x7fa8dc0fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:44.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.984+0000 7fa8e2c96640 1 -- 172.21.15.16:0/2210054001 shutdown_connections 2024-09-13T16:28:44.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.984+0000 7fa8e2c96640 1 -- 172.21.15.16:0/2210054001 wait complete. 2024-09-13T16:28:44.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.984+0000 7fa8e2c96640 1 Processor -- start 2024-09-13T16:28:44.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.984+0000 7fa8e2c96640 1 -- start start 2024-09-13T16:28:44.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.984+0000 7fa8e2c96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8dc100820 0x7fa8dc195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:44.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.984+0000 7fa8e2c96640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8dc103150 0x7fa8dc1961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.984+0000 7fa8e2c96640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8dc105a80 0x7fa8dc19d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.984+0000 7fa8e2c96640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc19d760 con 0x7fa8dc100820 2024-09-13T16:28:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.984+0000 7fa8e2c96640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc19d8d0 con 0x7fa8dc105a80 2024-09-13T16:28:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.984+0000 7fa8e2c96640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc19da90 con 0x7fa8dc103150 2024-09-13T16:28:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.984+0000 7fa8e0a32640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8dc100820 0x7fa8dc195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.984+0000 7fa8e0a32640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8dc100820 0x7fa8dc195c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44706/0 (socket says 172.21.15.16:44706) 2024-09-13T16:28:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.984+0000 7fa8e0a32640 1 -- 172.21.15.16:0/4161767618 learned_addr learned my addr 172.21.15.16:0/4161767618 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.985+0000 7fa8e1233640 1 --2- 172.21.15.16:0/4161767618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8dc105a80 0x7fa8dc19d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.985+0000 7fa8e0a32640 1 -- 172.21.15.16:0/4161767618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8dc103150 msgr2=0x7fa8dc1961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.985+0000 7fa8e0a32640 1 --2- 172.21.15.16:0/4161767618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8dc103150 0x7fa8dc1961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.985+0000 7fa8e0a32640 1 -- 172.21.15.16:0/4161767618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8dc105a80 msgr2=0x7fa8dc19d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.985+0000 7fa8e0a32640 1 --2- 172.21.15.16:0/4161767618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8dc105a80 0x7fa8dc19d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.985+0000 7fa8e0a32640 1 -- 172.21.15.16:0/4161767618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d4009560 con 0x7fa8dc100820 2024-09-13T16:28:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.985+0000 7fa8e0a32640 1 --2- 172.21.15.16:0/4161767618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8dc100820 0x7fa8dc195c80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc00e9f0 tx=0x7fa8cc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:44.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.985+0000 7fa8d9ffb640 1 -- 172.21.15.16:0/4161767618 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc009700 con 0x7fa8dc100820 2024-09-13T16:28:44.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.985+0000 7fa8d9ffb640 1 -- 172.21.15.16:0/4161767618 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa8cc00cd80 con 0x7fa8dc100820 2024-09-13T16:28:44.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.985+0000 7fa8e2c96640 1 -- 172.21.15.16:0/4161767618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8dc19dd70 con 0x7fa8dc100820 2024-09-13T16:28:44.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.985+0000 7fa8d9ffb640 1 -- 172.21.15.16:0/4161767618 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc0187e0 con 0x7fa8dc100820 2024-09-13T16:28:44.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.985+0000 7fa8e2c96640 1 -- 172.21.15.16:0/4161767618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8dc19e210 con 0x7fa8dc100820 2024-09-13T16:28:44.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.986+0000 7fa8e2c96640 1 -- 172.21.15.16:0/4161767618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8dc0fbf30 con 0x7fa8dc100820 2024-09-13T16:28:44.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.988+0000 7fa8d9ffb640 1 -- 172.21.15.16:0/4161767618 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa8cc010430 con 0x7fa8dc100820 2024-09-13T16:28:44.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.988+0000 7fa8d9ffb640 1 --2- 172.21.15.16:0/4161767618 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa8c40a1110 0x7fa8c40a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:44.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.988+0000 7fa8dbfff640 1 --2- 172.21.15.16:0/4161767618 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa8c40a1110 0x7fa8c40a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:44.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.988+0000 7fa8d9ffb640 1 -- 172.21.15.16:0/4161767618 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa8cc014070 con 0x7fa8dc100820 2024-09-13T16:28:44.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.989+0000 7fa8dbfff640 1 --2- 172.21.15.16:0/4161767618 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa8c40a1110 0x7fa8c40a35d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0005fd0 tx=0x7fa8d00071c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:44.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:44.992+0000 7fa8d9ffb640 1 -- 172.21.15.16:0/4161767618 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa8cc052e20 con 0x7fa8dc100820 2024-09-13T16:28:45.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.165+0000 7fa8e2c96640 1 -- 172.21.15.16:0/4161767618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8dc02c4b0 con 0x7fa8dc100820 2024-09-13T16:28:45.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.167+0000 7fa8d9ffb640 1 -- 172.21.15.16:0/4161767618 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23971 (secure 0 0 0) 0x7fa8cc096480 con 0x7fa8dc100820 2024-09-13T16:28:45.170 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:28:45.170 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":89,"fsid":"0d241f6a-71ec-11ef-bceb-c7b262605968","created":"2024-09-13T16:21:17.971738+0000","modified":"2024-09-13T16:28:43.574106+0000","last_up_change":"2024-09-13T16:27:54.673413+0000","last_in_change":"2024-09-13T16:27:35.718390+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:24:22.981019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:28:37.746027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:28:38.602731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:28:41.633322+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e11ef8f-08d6-4e74-afd6-f49f4bd40032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6803","nonce":2079394186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6805","nonce":2079394186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6809","nonce":2079394186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6807","nonce":2079394186}]},"public_addr":"172.21.15.16:6803/2079394186","cluster_addr":"172.21.15.16:6805/2079394186","heartbeat_back_addr":"172.21.15.16:6809/2079394186","heartbeat_front_addr":"172.21.15.16:6807/2079394186","state":["exists","up"]},{"osd":1,"uuid":"f2cd3694-9c9e-4da7-976e-f80a8df120f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6801","nonce":4021360958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6803","nonce":4021360958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6807","nonce":4021360958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6805","nonce":4021360958}]},"public_addr":"172.21.15.107:6801/4021360958","cluster_addr":"172.21.15.107:6803/4021360958","heartbeat_back_addr":"172.21.15.107:6807/4021360958","heartbeat_front_addr":"172.21.15.107:6805/4021360958","state":["exists","up"]},{"osd":2,"uuid":"a5bbf2eb-6599-44d5-ad96-68bb89ee19f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6801","nonce":3227179825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6803","nonce":3227179825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6807","nonce":3227179825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6805","nonce":3227179825}]},"public_addr":"172.21.15.111:6801/3227179825","cluster_addr":"172.21.15.111:6803/3227179825","heartbeat_back_addr":"172.21.15.111:6807/3227179825","heartbeat_front_addr":"172.21.15.111:6805/3227179825","state":["exists","up"]},{"osd":3,"uuid":"d6295ffe-11bc-4964-a250-9fecca870b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6811","nonce":2390375937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6813","nonce":2390375937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6817","nonce":2390375937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6815","nonce":2390375937}]},"public_addr":"172.21.15.16:6811/2390375937","cluster_addr":"172.21.15.16:6813/2390375937","heartbeat_back_addr":"172.21.15.16:6817/2390375937","heartbeat_front_addr":"172.21.15.16:6815/2390375937","state":["exists","up"]},{"osd":4,"uuid":"0a8446e0-fc86-4646-beb6-a7b9d433093d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6809","nonce":1936582048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6811","nonce":1936582048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6815","nonce":1936582048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6813","nonce":1936582048}]},"public_addr":"172.21.15.107:6809/1936582048","cluster_addr":"172.21.15.107:6811/1936582048","heartbeat_back_addr":"172.21.15.107:6815/1936582048","heartbeat_front_addr":"172.21.15.107:6813/1936582048","state":["exists","up"]},{"osd":5,"uuid":"2c97a446-357e-4975-9e51-9c95fe84a663","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6809","nonce":483858448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6811","nonce":483858448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6815","nonce":483858448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6813","nonce":483858448}]},"public_addr":"172.21.15.111:6809/483858448","cluster_addr":"172.21.15.111:6811/483858448","heartbeat_back_addr":"172.21.15.111:6815/483858448","heartbeat_front_addr":"172.21.15.111:6813/483858448","state":["exists","up"]},{"osd":6,"uuid":"890239e6-e6fb-4f9c-b973-05cc66d199bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6819","nonce":1330214141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6821","nonce":1330214141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6825","nonce":1330214141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6823","nonce":1330214141}]},"public_addr":"172.21.15.16:6819/1330214141","cluster_addr":"172.21.15.16:6821/1330214141","heartbeat_back_addr":"172.21.15.16:6825/1330214141","heartbeat_front_addr":"172.21.15.16:6823/1330214141","state":["exists","up"]},{"osd":7,"uuid":"ee826a9c-f441-4903-9b1b-66b627f7b712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6817","nonce":1133876754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6819","nonce":1133876754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6823","nonce":1133876754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6821","nonce":1133876754}]},"public_addr":"172.21.15.107:6817/1133876754","cluster_addr":"172.21.15.107:6819/1133876754","heartbeat_back_addr":"172.21.15.107:6823/1133876754","heartbeat_front_addr":"172.21.15.107:6821/1133876754","state":["exists","up"]},{"osd":8,"uuid":"e19773f2-dec8-44e0-9d33-0252e07d6415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6817","nonce":837548190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6819","nonce":837548190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6823","nonce":837548190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6821","nonce":837548190}]},"public_addr":"172.21.15.111:6817/837548190","cluster_addr":"172.21.15.111:6819/837548190","heartbeat_back_addr":"172.21.15.111:6823/837548190","heartbeat_front_addr":"172.21.15.111:6821/837548190","state":["exists","up"]},{"osd":9,"uuid":"bae4d6db-8b90-4d02-8e83-7d8f2a6f8fdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6827","nonce":4175383686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6829","nonce":4175383686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6833","nonce":4175383686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6831","nonce":4175383686}]},"public_addr":"172.21.15.16:6827/4175383686","cluster_addr":"172.21.15.16:6829/4175383686","heartbeat_back_addr":"172.21.15.16:6833/4175383686","heartbeat_front_addr":"172.21.15.16:6831/4175383686","state":["exists","up"]},{"osd":10,"uuid":"fd0e7740-76c8-4c99-a349-7077b4905143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6825","nonce":2659550426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6827","nonce":2659550426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6831","nonce":2659550426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6829","nonce":2659550426}]},"public_addr":"172.21.15.107:6825/2659550426","cluster_addr":"172.21.15.107:6827/2659550426","heartbeat_back_addr":"172.21.15.107:6831/2659550426","heartbeat_front_addr":"172.21.15.107:6829/2659550426","state":["exists","up"]},{"osd":11,"uuid":"94a1751d-b00c-4e23-815d-3a8bc3dea146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6825","nonce":1633548683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6827","nonce":1633548683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6831","nonce":1633548683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6829","nonce":1633548683}]},"public_addr":"172.21.15.111:6825/1633548683","cluster_addr":"172.21.15.111:6827/1633548683","heartbeat_back_addr":"172.21.15.111:6831/1633548683","heartbeat_front_addr":"172.21.15.111:6829/1633548683","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:36.359852+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:58.014202+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:19.753672+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:43.404817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:06.001677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:29.361587+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:52.554847+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:15.957553+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:39.581093+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:04.911389+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:28.971280+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:53.473561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/2098378667":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/3536229016":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6801/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/2684687547":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6800/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/368410612":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/12158160":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6800/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6801/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/3754423379":"2024-09-14T16:21:40.126459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:28:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.169+0000 7fa8e2c96640 1 -- 172.21.15.16:0/4161767618 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa8c40a1110 msgr2=0x7fa8c40a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:45.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.169+0000 7fa8e2c96640 1 --2- 172.21.15.16:0/4161767618 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa8c40a1110 0x7fa8c40a35d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0005fd0 tx=0x7fa8d00071c0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.169+0000 7fa8e2c96640 1 -- 172.21.15.16:0/4161767618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8dc100820 msgr2=0x7fa8dc195c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:45.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.169+0000 7fa8e2c96640 1 --2- 172.21.15.16:0/4161767618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8dc100820 0x7fa8dc195c80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc00e9f0 tx=0x7fa8cc00eeb0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.169+0000 7fa8e2c96640 1 -- 172.21.15.16:0/4161767618 shutdown_connections 2024-09-13T16:28:45.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.169+0000 7fa8e2c96640 1 --2- 172.21.15.16:0/4161767618 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa8c40a1110 0x7fa8c40a35d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.169+0000 7fa8e2c96640 1 --2- 172.21.15.16:0/4161767618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8dc105a80 0x7fa8dc19d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.169+0000 7fa8e2c96640 1 --2- 172.21.15.16:0/4161767618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8dc103150 0x7fa8dc1961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.169+0000 7fa8e2c96640 1 --2- 172.21.15.16:0/4161767618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8dc100820 0x7fa8dc195c80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.169+0000 7fa8e2c96640 1 -- 172.21.15.16:0/4161767618 >> 172.21.15.16:0/4161767618 conn(0x7fa8dc0fa440 msgr2=0x7fa8dc101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:45.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.169+0000 7fa8e2c96640 1 -- 172.21.15.16:0/4161767618 shutdown_connections 2024-09-13T16:28:45.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.169+0000 7fa8e2c96640 1 -- 172.21.15.16:0/4161767618 wait complete. 2024-09-13T16:28:45.182 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T16:28:39.184734+0000', 'modified': '2024-09-13T16:28:43.574336+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24518}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24518': {'gid': 24518, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.107:6839/1914689806', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6838', 'nonce': 1914689806}, {'type': 'v1', 'addr': '172.21.15.107:6839', 'nonce': 1914689806}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-09-13T16:28:45.183 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T16:28:45.183 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-13T16:28:45.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.285+0000 7f3080a96640 1 -- 172.21.15.16:0/3845323054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f307c0ffe60 msgr2=0x7f307c1002e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:45.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.285+0000 7f3080a96640 1 --2- 172.21.15.16:0/3845323054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f307c0ffe60 0x7f307c1002e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3070009900 tx=0x7f307002a410 comp rx=0 tx=0).stop 2024-09-13T16:28:45.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.285+0000 7f3080a96640 1 -- 172.21.15.16:0/3845323054 shutdown_connections 2024-09-13T16:28:45.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.285+0000 7f3080a96640 1 --2- 172.21.15.16:0/3845323054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f307c100820 0x7f307c1070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.285+0000 7f3080a96640 1 --2- 172.21.15.16:0/3845323054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f307c0ffe60 0x7f307c1002e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.285+0000 7f3080a96640 1 --2- 172.21.15.16:0/3845323054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f307c0fec70 0x7f307c0ff070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.285+0000 7f3080a96640 1 -- 172.21.15.16:0/3845323054 >> 172.21.15.16:0/3845323054 conn(0x7f307c0fa410 msgr2=0x7f307c0fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:45.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.285+0000 7f3080a96640 1 -- 172.21.15.16:0/3845323054 shutdown_connections 2024-09-13T16:28:45.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.285+0000 7f3080a96640 1 -- 172.21.15.16:0/3845323054 wait complete. 2024-09-13T16:28:45.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.286+0000 7f3080a96640 1 Processor -- start 2024-09-13T16:28:45.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.286+0000 7f3080a96640 1 -- start start 2024-09-13T16:28:45.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.286+0000 7f3080a96640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f307c0fec70 0x7f307c195bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:45.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.286+0000 7f3080a96640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f307c0ffe60 0x7f307c196110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:45.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.286+0000 7f3080a96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f307c100820 0x7f307c19d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:45.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.286+0000 7f3080a96640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c19d6c0 con 0x7f307c100820 2024-09-13T16:28:45.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.286+0000 7f3080a96640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c19d830 con 0x7f307c0ffe60 2024-09-13T16:28:45.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.286+0000 7f3080a96640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c19d9f0 con 0x7f307c0fec70 2024-09-13T16:28:45.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.286+0000 7f307ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f307c100820 0x7f307c19d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:45.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.286+0000 7f307ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f307c100820 0x7f307c19d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44730/0 (socket says 172.21.15.16:44730) 2024-09-13T16:28:45.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.286+0000 7f307ad9d640 1 -- 172.21.15.16:0/1921661111 learned_addr learned my addr 172.21.15.16:0/1921661111 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:45.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.286+0000 7f307a59c640 1 --2- 172.21.15.16:0/1921661111 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f307c0fec70 0x7f307c195bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:45.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.286+0000 7f3079d9b640 1 --2- 172.21.15.16:0/1921661111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f307c0ffe60 0x7f307c196110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:45.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.286+0000 7f307ad9d640 1 -- 172.21.15.16:0/1921661111 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f307c0fec70 msgr2=0x7f307c195bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:45.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.286+0000 7f307ad9d640 1 --2- 172.21.15.16:0/1921661111 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f307c0fec70 0x7f307c195bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.286+0000 7f307ad9d640 1 -- 172.21.15.16:0/1921661111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f307c0ffe60 msgr2=0x7f307c196110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:45.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.286+0000 7f307ad9d640 1 --2- 172.21.15.16:0/1921661111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f307c0ffe60 0x7f307c196110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.286+0000 7f307ad9d640 1 -- 172.21.15.16:0/1921661111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3070009560 con 0x7f307c100820 2024-09-13T16:28:45.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.286+0000 7f307ad9d640 1 --2- 172.21.15.16:0/1921661111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f307c100820 0x7f307c19d180 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f307400cce0 tx=0x7f307400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:45.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.286+0000 7f306b7fe640 1 -- 172.21.15.16:0/1921661111 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307400f450 con 0x7f307c100820 2024-09-13T16:28:45.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.286+0000 7f306b7fe640 1 -- 172.21.15.16:0/1921661111 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f307400fa40 con 0x7f307c100820 2024-09-13T16:28:45.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.286+0000 7f3080a96640 1 -- 172.21.15.16:0/1921661111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f307c19dcd0 con 0x7f307c100820 2024-09-13T16:28:45.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.287+0000 7f306b7fe640 1 -- 172.21.15.16:0/1921661111 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30740189a0 con 0x7f307c100820 2024-09-13T16:28:45.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.287+0000 7f3080a96640 1 -- 172.21.15.16:0/1921661111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f307c19e190 con 0x7f307c100820 2024-09-13T16:28:45.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.289+0000 7f306b7fe640 1 -- 172.21.15.16:0/1921661111 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f307400fbb0 con 0x7f307c100820 2024-09-13T16:28:45.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.290+0000 7f308388f740 1 Processor -- start 2024-09-13T16:28:45.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.290+0000 7f308388f740 1 -- start start 2024-09-13T16:28:45.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.290+0000 7f308388f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d13f51d500 0x55d13f51d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:45.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.290+0000 7f308388f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d13f51de40 0x55d13f522900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:45.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.290+0000 7f308388f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d13f522ed0 0x55d13f525390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:45.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.290+0000 7f308388f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d13f4787c0 con 0x55d13f51de40 2024-09-13T16:28:45.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.290+0000 7f308388f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d13f487b70 con 0x55d13f522ed0 2024-09-13T16:28:45.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.290+0000 7f308388f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d13f485300 con 0x55d13f51d500 2024-09-13T16:28:45.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.290+0000 7f307a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d13f51de40 0x55d13f522900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:45.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.290+0000 7f307a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d13f51de40 0x55d13f522900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44734/0 (socket says 172.21.15.16:44734) 2024-09-13T16:28:45.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.290+0000 7f307a59c640 1 -- 172.21.15.16:0/3036302944 learned_addr learned my addr 172.21.15.16:0/3036302944 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:45.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.290+0000 7f306b7fe640 1 --2- 172.21.15.16:0/1921661111 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f30640a1110 0x7f30640a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:45.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.290+0000 7f3079d9b640 1 --2- 172.21.15.16:0/3036302944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d13f51d500 0x55d13f51d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:45.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.290+0000 7f307a59c640 1 -- 172.21.15.16:0/3036302944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d13f51d500 msgr2=0x55d13f51d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:28:45.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.290+0000 7f307a59c640 1 --2- 172.21.15.16:0/3036302944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d13f51d500 0x55d13f51d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.290+0000 7f3079d9b640 1 --2- 172.21.15.16:0/3036302944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d13f522ed0 0x55d13f525390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:45.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.290+0000 7f307a59c640 1 -- 172.21.15.16:0/3036302944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d13f522ed0 msgr2=0x55d13f525390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:28:45.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.290+0000 7f307ad9d640 1 --2- 172.21.15.16:0/1921661111 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f30640a1110 0x7f30640a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:45.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.290+0000 7f307a59c640 1 --2- 172.21.15.16:0/3036302944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d13f522ed0 0x55d13f525390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.290+0000 7f307a59c640 1 -- 172.21.15.16:0/3036302944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d13f40d4b0 con 0x55d13f51de40 2024-09-13T16:28:45.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.290+0000 7f3079d9b640 1 --2- 172.21.15.16:0/3036302944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d13f522ed0 0x55d13f525390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:28:45.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.291+0000 7f306b7fe640 1 -- 172.21.15.16:0/1921661111 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f30740ccf60 con 0x7f307c100820 2024-09-13T16:28:45.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.291+0000 7f307a59c640 1 --2- 172.21.15.16:0/3036302944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d13f51de40 0x55d13f522900 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f306c009c20 tx=0x7f306c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6ead13469272af server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:45.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.291+0000 7f3068ff9640 1 -- 172.21.15.16:0/3036302944 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c009720 con 0x55d13f51de40 2024-09-13T16:28:45.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.291+0000 7f307ad9d640 1 --2- 172.21.15.16:0/1921661111 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f30640a1110 0x7f30640a35d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f30740966c0 tx=0x7f307400af30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:45.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.291+0000 7f3068ff9640 1 -- 172.21.15.16:0/3036302944 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f306c0128e0 con 0x55d13f51de40 2024-09-13T16:28:45.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.291+0000 7f3068ff9640 1 -- 172.21.15.16:0/3036302944 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c00f8d0 con 0x55d13f51de40 2024-09-13T16:28:45.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.292+0000 7f308388f740 1 -- 172.21.15.16:0/3036302944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d13f51de40 msgr2=0x55d13f522900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:45.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.292+0000 7f308388f740 1 --2- 172.21.15.16:0/3036302944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d13f51de40 0x55d13f522900 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f306c009c20 tx=0x7f306c0117b0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.292+0000 7f308388f740 1 -- 172.21.15.16:0/3036302944 shutdown_connections 2024-09-13T16:28:45.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.292+0000 7f308388f740 1 --2- 172.21.15.16:0/3036302944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d13f522ed0 0x55d13f525390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.292+0000 7f308388f740 1 --2- 172.21.15.16:0/3036302944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d13f51de40 0x55d13f522900 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.292+0000 7f308388f740 1 --2- 172.21.15.16:0/3036302944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d13f51d500 0x55d13f51d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.292+0000 7f308388f740 1 -- 172.21.15.16:0/3036302944 >> 172.21.15.16:0/3036302944 conn(0x55d13f518b70 msgr2=0x55d13f518f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:45.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.292+0000 7f308388f740 1 -- 172.21.15.16:0/3036302944 shutdown_connections 2024-09-13T16:28:45.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.292+0000 7f308388f740 1 -- 172.21.15.16:0/3036302944 wait complete. 2024-09-13T16:28:45.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.292+0000 7f308388f740 1 -- start start 2024-09-13T16:28:45.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.292+0000 7f308388f740 1 Processor -- start 2024-09-13T16:28:45.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.292+0000 7f308388f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d13f51d500 0x55d13f5270e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:45.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.292+0000 7f308388f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d13f51de40 0x55d13f527620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:45.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.292+0000 7f308388f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d13f522ed0 0x55d13f52e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:45.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.292+0000 7f308388f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d13f480bf0 con 0x55d13f522ed0 2024-09-13T16:28:45.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.292+0000 7f308388f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d13f43f250 con 0x55d13f51d500 2024-09-13T16:28:45.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.292+0000 7f308388f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d13f43eef0 con 0x55d13f51de40 2024-09-13T16:28:45.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.292+0000 7f308388f740 10 client.0 _ll_register_callbacks cb 0x55d13f471740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:28:45.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.292+0000 7f3079d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d13f522ed0 0x55d13f52e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:45.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.292+0000 7f3079d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d13f522ed0 0x55d13f52e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44750/0 (socket says 172.21.15.16:44750) 2024-09-13T16:28:45.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.293+0000 7f3079d9b640 1 -- 172.21.15.16:0/2706745607 learned_addr learned my addr 172.21.15.16:0/2706745607 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:45.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.293+0000 7f3079d9b640 1 --2- 172.21.15.16:0/2706745607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d13f51d500 0x55d13f5270e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:45.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.293+0000 7f307a59c640 1 --2- 172.21.15.16:0/2706745607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d13f51de40 0x55d13f527620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:45.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.293+0000 7f3079d9b640 1 -- 172.21.15.16:0/2706745607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d13f51de40 msgr2=0x55d13f527620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:28:45.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.293+0000 7f3079d9b640 1 --2- 172.21.15.16:0/2706745607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d13f51de40 0x55d13f527620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.293+0000 7f3079d9b640 1 -- 172.21.15.16:0/2706745607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d13f51d500 msgr2=0x55d13f5270e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:28:45.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.293+0000 7f3079d9b640 1 --2- 172.21.15.16:0/2706745607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d13f51d500 0x55d13f5270e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.293+0000 7f3079d9b640 1 -- 172.21.15.16:0/2706745607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f306c009880 con 0x55d13f522ed0 2024-09-13T16:28:45.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.293+0000 7f3079d9b640 1 --2- 172.21.15.16:0/2706745607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d13f522ed0 0x55d13f52e660 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3070005ec0 tx=0x7f3070009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7bcea0c286284b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:45.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.293+0000 7f3061ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:28:45.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.293+0000 7f3061ffb640 1 -- 172.21.15.16:0/2706745607 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3070032c20 con 0x55d13f522ed0 2024-09-13T16:28:45.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.293+0000 7f3061ffb640 1 -- 172.21.15.16:0/2706745607 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3070032d80 con 0x55d13f522ed0 2024-09-13T16:28:45.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.293+0000 7f308388f740 1 -- 172.21.15.16:0/2706745607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d13f1743c0 con 0x55d13f522ed0 2024-09-13T16:28:45.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.293+0000 7f3061ffb640 1 -- 172.21.15.16:0/2706745607 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30700035f0 con 0x55d13f522ed0 2024-09-13T16:28:45.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.293+0000 7f308388f740 10 client.14718 fetch_fsmap learned FSMap version 17 2024-09-13T16:28:45.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.293+0000 7f308388f740 1 -- 172.21.15.16:0/2706745607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55d13f47f270 con 0x55d13f522ed0 2024-09-13T16:28:45.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.293+0000 7f3061ffb640 1 -- 172.21.15.16:0/2706745607 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3070055080 con 0x55d13f522ed0 2024-09-13T16:28:45.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.294+0000 7f3061ffb640 1 -- 172.21.15.16:0/2706745607 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f30700515d0 con 0x55d13f522ed0 2024-09-13T16:28:45.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.294+0000 7f308388f740 10 client.14718 fetch_fsmap finished waiting for FSMap version 17 2024-09-13T16:28:45.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.294+0000 7f308388f740 10 client.14718 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-13T16:28:45.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.294+0000 7f308388f740 20 client.14718 populate_metadata read hostname 'smithi016' 2024-09-13T16:28:45.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.294+0000 7f308388f740 1 --2- 172.21.15.16:0/2706745607 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55d13f532f10 0x55d13f537680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:45.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.294+0000 7f308388f740 4 client.14718 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T16:28:45.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.294+0000 7f308388f740 1 -- 172.21.15.16:0/2706745607 --> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d13f1856c0 con 0x55d13f532f10 2024-09-13T16:28:45.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.294+0000 7f307a59c640 1 --2- 172.21.15.16:0/2706745607 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55d13f532f10 0x55d13f537680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:45.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.294+0000 7f3061ffb640 10 client.14718 ms_handle_connect on v2:172.21.15.107:6838/1914689806 2024-09-13T16:28:45.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.294+0000 7f307a59c640 1 --2- 172.21.15.16:0/2706745607 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55d13f532f10 0x55d13f537680 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15ad22c9dcd9c447 server_cookie=f07c6b2aaed18299 in_seq=0 out_seq=0 2024-09-13T16:28:45.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.295+0000 7f3061ffb640 1 -- 172.21.15.16:0/2706745607 <== mds.0 v2:172.21.15.107:6838/1914689806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d13f1856c0 con 0x55d13f532f10 2024-09-13T16:28:45.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.295+0000 7f3061ffb640 10 client.14718 handle_command_reply: tid=0 2024-09-13T16:28:45.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.295+0000 7f308388f740 1 client.14718 shutdown 2024-09-13T16:28:45.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.295+0000 7f308388f740 1 -- 172.21.15.16:0/2706745607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d13f522ed0 msgr2=0x55d13f52e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:45.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.295+0000 7f308388f740 1 --2- 172.21.15.16:0/2706745607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d13f522ed0 0x55d13f52e660 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3070005ec0 tx=0x7f3070009260 comp rx=0 tx=0).stop 2024-09-13T16:28:45.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.295+0000 7f308388f740 1 -- 172.21.15.16:0/2706745607 shutdown_connections 2024-09-13T16:28:45.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.295+0000 7f308388f740 1 --2- 172.21.15.16:0/2706745607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d13f522ed0 0x55d13f52e660 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.295+0000 7f308388f740 1 --2- 172.21.15.16:0/2706745607 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55d13f532f10 0x55d13f537680 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.295+0000 7f308388f740 1 --2- 172.21.15.16:0/2706745607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d13f51de40 0x55d13f527620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.295+0000 7f308388f740 1 --2- 172.21.15.16:0/2706745607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d13f51d500 0x55d13f5270e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.295+0000 7f3061ffb640 0 client.14718 ms_handle_reset on v2:172.21.15.107:6838/1914689806 2024-09-13T16:28:45.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.295+0000 7f308388f740 1 -- 172.21.15.16:0/2706745607 >> 172.21.15.16:0/2706745607 conn(0x55d13f518b70 msgr2=0x55d13f51a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:45.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.295+0000 7f308388f740 1 -- 172.21.15.16:0/2706745607 shutdown_connections 2024-09-13T16:28:45.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.295+0000 7f308388f740 1 -- 172.21.15.16:0/2706745607 wait complete. 2024-09-13T16:28:45.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.295+0000 7f308388f740 20 client.14718 trim_cache size 0 max 16384 2024-09-13T16:28:45.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.310+0000 7f308388f740 1 Processor -- start 2024-09-13T16:28:45.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.310+0000 7f308388f740 1 -- start start 2024-09-13T16:28:45.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.310+0000 7f308388f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d13f518b70 0x55d13f526e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:45.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.310+0000 7f308388f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d13f51d500 0x55d13f5273d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:45.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.310+0000 7f308388f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d13f51de40 0x55d13f53b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:45.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.310+0000 7f308388f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d13f404460 con 0x55d13f51d500 2024-09-13T16:28:45.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.310+0000 7f308388f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d13f514600 con 0x55d13f51de40 2024-09-13T16:28:45.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.310+0000 7f308388f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d13f1856c0 con 0x55d13f518b70 2024-09-13T16:28:45.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.310+0000 7f307a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d13f51d500 0x55d13f5273d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:45.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.310+0000 7f307a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d13f51d500 0x55d13f5273d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44760/0 (socket says 172.21.15.16:44760) 2024-09-13T16:28:45.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.310+0000 7f307a59c640 1 -- 172.21.15.16:0/403702307 learned_addr learned my addr 172.21.15.16:0/403702307 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:45.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.310+0000 7f307a59c640 1 -- 172.21.15.16:0/403702307 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d13f518b70 msgr2=0x55d13f526e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:28:45.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.310+0000 7f307a59c640 1 --2- 172.21.15.16:0/403702307 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d13f518b70 0x55d13f526e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.310+0000 7f307a59c640 1 -- 172.21.15.16:0/403702307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d13f51de40 msgr2=0x55d13f53b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:28:45.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.310+0000 7f307a59c640 1 --2- 172.21.15.16:0/403702307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d13f51de40 0x55d13f53b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.311+0000 7f307a59c640 1 -- 172.21.15.16:0/403702307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d13f4215f0 con 0x55d13f51d500 2024-09-13T16:28:45.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.311+0000 7f307a59c640 1 --2- 172.21.15.16:0/403702307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d13f51d500 0x55d13f5273d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f306c012fd0 tx=0x7f306c012f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a61a7f9992fd8fb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:45.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.311+0000 7f30697fa640 1 -- 172.21.15.16:0/403702307 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c012450 con 0x55d13f51d500 2024-09-13T16:28:45.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.311+0000 7f30697fa640 1 -- 172.21.15.16:0/403702307 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f306c009700 con 0x55d13f51d500 2024-09-13T16:28:45.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.311+0000 7f30697fa640 1 -- 172.21.15.16:0/403702307 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c0054c0 con 0x55d13f51d500 2024-09-13T16:28:45.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.311+0000 7f308388f740 1 -- 172.21.15.16:0/403702307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d13f51d500 msgr2=0x55d13f5273d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:45.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.311+0000 7f308388f740 1 --2- 172.21.15.16:0/403702307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d13f51d500 0x55d13f5273d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f306c012fd0 tx=0x7f306c012f60 comp rx=0 tx=0).stop 2024-09-13T16:28:45.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.311+0000 7f308388f740 1 -- 172.21.15.16:0/403702307 shutdown_connections 2024-09-13T16:28:45.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.311+0000 7f308388f740 1 --2- 172.21.15.16:0/403702307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d13f51de40 0x55d13f53b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.311+0000 7f308388f740 1 --2- 172.21.15.16:0/403702307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d13f51d500 0x55d13f5273d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.311+0000 7f308388f740 1 --2- 172.21.15.16:0/403702307 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d13f518b70 0x55d13f526e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.311+0000 7f308388f740 1 -- 172.21.15.16:0/403702307 >> 172.21.15.16:0/403702307 conn(0x55d13f525630 msgr2=0x55d13f5222b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:45.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.311+0000 7f308388f740 1 -- 172.21.15.16:0/403702307 shutdown_connections 2024-09-13T16:28:45.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.311+0000 7f308388f740 1 -- 172.21.15.16:0/403702307 wait complete. 2024-09-13T16:28:45.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.312+0000 7f308388f740 1 -- start start 2024-09-13T16:28:45.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.312+0000 7f308388f740 1 Processor -- start 2024-09-13T16:28:45.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.312+0000 7f308388f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d13f518b70 0x55d13f5865b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:45.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.312+0000 7f308388f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d13f51d500 0x55d13f586af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:45.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.312+0000 7f308388f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d13f51de40 0x55d13f58db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:45.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.312+0000 7f308388f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d13f58e070 con 0x55d13f518b70 2024-09-13T16:28:45.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.312+0000 7f308388f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d13f58e1b0 con 0x55d13f51de40 2024-09-13T16:28:45.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.312+0000 7f308388f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d13f58e340 con 0x55d13f51d500 2024-09-13T16:28:45.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.312+0000 7f308388f740 10 client.0 _ll_register_callbacks cb 0x7f306c009650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:28:45.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.312+0000 7f3079d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d13f518b70 0x55d13f5865b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:45.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.312+0000 7f3079d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d13f518b70 0x55d13f5865b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44774/0 (socket says 172.21.15.16:44774) 2024-09-13T16:28:45.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.312+0000 7f3079d9b640 1 -- 172.21.15.16:0/2519296195 learned_addr learned my addr 172.21.15.16:0/2519296195 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:45.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.312+0000 7f3079d9b640 1 --2- 172.21.15.16:0/2519296195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d13f51de40 0x55d13f58db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:45.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.312+0000 7f307a59c640 1 --2- 172.21.15.16:0/2519296195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d13f51d500 0x55d13f586af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:45.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.312+0000 7f3079d9b640 1 -- 172.21.15.16:0/2519296195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d13f51d500 msgr2=0x55d13f586af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:28:45.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.312+0000 7f3079d9b640 1 --2- 172.21.15.16:0/2519296195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d13f51d500 0x55d13f586af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.312+0000 7f3079d9b640 1 -- 172.21.15.16:0/2519296195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d13f51de40 msgr2=0x55d13f58db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:28:45.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.312+0000 7f3079d9b640 1 --2- 172.21.15.16:0/2519296195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d13f51de40 0x55d13f58db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.312+0000 7f3079d9b640 1 -- 172.21.15.16:0/2519296195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d13f40d4b0 con 0x55d13f518b70 2024-09-13T16:28:45.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.312+0000 7f307a59c640 1 --2- 172.21.15.16:0/2519296195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d13f51d500 0x55d13f586af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:28:45.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.312+0000 7f3079d9b640 1 --2- 172.21.15.16:0/2519296195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d13f518b70 0x55d13f5865b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3070051b50 tx=0x7f3070051ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d9de3619b02336 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:45.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.312+0000 7f3061ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:28:45.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.313+0000 7f3061ffb640 1 -- 172.21.15.16:0/2519296195 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3070051b80 con 0x55d13f518b70 2024-09-13T16:28:45.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.313+0000 7f3061ffb640 1 -- 172.21.15.16:0/2519296195 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3070000e40 con 0x55d13f518b70 2024-09-13T16:28:45.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.313+0000 7f308388f740 1 -- 172.21.15.16:0/2519296195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d13f1d8d10 con 0x55d13f518b70 2024-09-13T16:28:45.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.313+0000 7f3061ffb640 1 -- 172.21.15.16:0/2519296195 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3070032d90 con 0x55d13f518b70 2024-09-13T16:28:45.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.313+0000 7f308388f740 10 client.14724 fetch_fsmap learned FSMap version 17 2024-09-13T16:28:45.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.313+0000 7f308388f740 1 -- 172.21.15.16:0/2519296195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55d13f446d00 con 0x55d13f518b70 2024-09-13T16:28:45.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.313+0000 7f3061ffb640 1 -- 172.21.15.16:0/2519296195 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d13f1d8d10 con 0x55d13f518b70 2024-09-13T16:28:45.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.313+0000 7f3061ffb640 1 -- 172.21.15.16:0/2519296195 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f307005b030 con 0x55d13f518b70 2024-09-13T16:28:45.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.313+0000 7f308388f740 10 client.14724 fetch_fsmap finished waiting for FSMap version 17 2024-09-13T16:28:45.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.313+0000 7f308388f740 10 client.14724 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-13T16:28:45.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.313+0000 7f308388f740 20 client.14724 populate_metadata read hostname 'smithi016' 2024-09-13T16:28:45.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.313+0000 7f308388f740 1 --2- 172.21.15.16:0/2519296195 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55d13f522ed0 0x55d13f58f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:45.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.313+0000 7f308388f740 4 client.14724 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T16:28:45.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.313+0000 7f308388f740 1 -- 172.21.15.16:0/2519296195 --> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d13f533910 con 0x55d13f522ed0 2024-09-13T16:28:45.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.313+0000 7f307a59c640 1 --2- 172.21.15.16:0/2519296195 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55d13f522ed0 0x55d13f58f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:45.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.314+0000 7f3061ffb640 10 client.14724 ms_handle_connect on v2:172.21.15.107:6838/1914689806 2024-09-13T16:28:45.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.314+0000 7f307a59c640 1 --2- 172.21.15.16:0/2519296195 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55d13f522ed0 0x55d13f58f7d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51a44a8e2ba59163 server_cookie=468182b32752afa9 in_seq=0 out_seq=0 2024-09-13T16:28:45.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.314+0000 7f3061ffb640 1 -- 172.21.15.16:0/2519296195 <== mds.0 v2:172.21.15.107:6838/1914689806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x7f306c009700 con 0x55d13f522ed0 2024-09-13T16:28:45.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.314+0000 7f3061ffb640 10 client.14724 handle_command_reply: tid=0 2024-09-13T16:28:45.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.314+0000 7f308388f740 1 client.14724 shutdown 2024-09-13T16:28:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.314+0000 7f308388f740 1 -- 172.21.15.16:0/2519296195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d13f518b70 msgr2=0x55d13f5865b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.314+0000 7f308388f740 1 --2- 172.21.15.16:0/2519296195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d13f518b70 0x55d13f5865b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3070051b50 tx=0x7f3070051ae0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.314+0000 7f308388f740 1 -- 172.21.15.16:0/2519296195 shutdown_connections 2024-09-13T16:28:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.314+0000 7f308388f740 1 --2- 172.21.15.16:0/2519296195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d13f51de40 0x55d13f58db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.314+0000 7f308388f740 1 --2- 172.21.15.16:0/2519296195 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55d13f522ed0 0x55d13f58f7d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.314+0000 7f308388f740 1 --2- 172.21.15.16:0/2519296195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d13f51d500 0x55d13f586af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.314+0000 7f308388f740 1 --2- 172.21.15.16:0/2519296195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d13f518b70 0x55d13f5865b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.314+0000 7f3061ffb640 0 client.14724 ms_handle_reset on v2:172.21.15.107:6838/1914689806 2024-09-13T16:28:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.314+0000 7f308388f740 1 -- 172.21.15.16:0/2519296195 >> 172.21.15.16:0/2519296195 conn(0x55d13f525630 msgr2=0x55d13f53b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.314+0000 7f308388f740 1 -- 172.21.15.16:0/2519296195 shutdown_connections 2024-09-13T16:28:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.314+0000 7f308388f740 1 -- 172.21.15.16:0/2519296195 wait complete. 2024-09-13T16:28:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.314+0000 7f308388f740 20 client.14724 trim_cache size 0 max 16384 2024-09-13T16:28:45.324 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-13T16:28:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "0d241f6a-71ec-11ef-bceb-c7b262605968", 2024-09-13T16:28:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-13T16:28:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24518, 2024-09-13T16:28:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-13T16:28:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-13T16:28:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-13T16:28:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 6.1283045009999997, 2024-09-13T16:28:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 17, 2024-09-13T16:28:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 89, 2024-09-13T16:28:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 88, 2024-09-13T16:28:45.324 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 10.529086963999999 2024-09-13T16:28:45.325 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-13T16:28:45.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.315+0000 7f3080a96640 1 -- 172.21.15.16:0/1921661111 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f30640a1110 msgr2=0x7f30640a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:45.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.315+0000 7f3080a96640 1 --2- 172.21.15.16:0/1921661111 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f30640a1110 0x7f30640a35d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f30740966c0 tx=0x7f307400af30 comp rx=0 tx=0).stop 2024-09-13T16:28:45.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.315+0000 7f3080a96640 1 -- 172.21.15.16:0/1921661111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f307c100820 msgr2=0x7f307c19d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:45.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.315+0000 7f3080a96640 1 --2- 172.21.15.16:0/1921661111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f307c100820 0x7f307c19d180 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f307400cce0 tx=0x7f307400a620 comp rx=0 tx=0).stop 2024-09-13T16:28:45.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.315+0000 7f3080a96640 1 -- 172.21.15.16:0/1921661111 shutdown_connections 2024-09-13T16:28:45.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.315+0000 7f3080a96640 1 --2- 172.21.15.16:0/1921661111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f307c100820 0x7f307c19d180 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.315+0000 7f3080a96640 1 --2- 172.21.15.16:0/1921661111 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f30640a1110 0x7f30640a35d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.315+0000 7f3080a96640 1 --2- 172.21.15.16:0/1921661111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f307c0ffe60 0x7f307c196110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.315+0000 7f3080a96640 1 --2- 172.21.15.16:0/1921661111 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f307c0fec70 0x7f307c195bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:45.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.315+0000 7f3080a96640 1 -- 172.21.15.16:0/1921661111 >> 172.21.15.16:0/1921661111 conn(0x7f307c0fa410 msgr2=0x7f307c103270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:45.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.315+0000 7f3080a96640 1 -- 172.21.15.16:0/1921661111 shutdown_connections 2024-09-13T16:28:45.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:45.315+0000 7f3080a96640 1 -- 172.21.15.16:0/1921661111 wait complete. 2024-09-13T16:28:45.327 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T16:28:45.327 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-13T16:28:45.328 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-13T16:28:45.529 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:28:45.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:45 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/814710286' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:28:45.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:45 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3410280895' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:28:45.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:45 smithi107 ceph-mon[34139]: pgmap v279: 129 pgs: 10 creating+peering, 13 unknown, 106 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-09-13T16:28:45.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:45 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1237869844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:45.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:45 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/4161767618' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:28:45.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:45 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/814710286' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:28:45.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:45 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3410280895' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:28:45.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:45 smithi016 ceph-mon[31187]: pgmap v279: 129 pgs: 10 creating+peering, 13 unknown, 106 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-09-13T16:28:45.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:45 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1237869844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:45.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:45 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/4161767618' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:28:45.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:45 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/814710286' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:28:45.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:45 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3410280895' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:28:45.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:45 smithi111 ceph-mon[34600]: pgmap v279: 129 pgs: 10 creating+peering, 13 unknown, 106 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-09-13T16:28:45.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:45 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1237869844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:28:45.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:45 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/4161767618' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:28:46.588 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:46 smithi111 ceph-mon[34600]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-13T16:28:46.588 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:46 smithi111 ceph-mon[34600]: Cluster is now healthy 2024-09-13T16:28:46.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:46 smithi107 ceph-mon[34139]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-13T16:28:46.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:46 smithi107 ceph-mon[34139]: Cluster is now healthy 2024-09-13T16:28:46.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:46 smithi016 ceph-mon[31187]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-13T16:28:46.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:46 smithi016 ceph-mon[31187]: Cluster is now healthy 2024-09-13T16:28:47.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.396+0000 7f6754d54640 1 -- 172.21.15.16:0/1754646793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67500ff880 msgr2=0x7f67501038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:47.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.396+0000 7f6754d54640 1 --2- 172.21.15.16:0/1754646793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67500ff880 0x7f67501038d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6738009930 tx=0x7f673802a420 comp rx=0 tx=0).stop 2024-09-13T16:28:47.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.396+0000 7f6754d54640 1 -- 172.21.15.16:0/1754646793 shutdown_connections 2024-09-13T16:28:47.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.396+0000 7f6754d54640 1 --2- 172.21.15.16:0/1754646793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6750104000 0x7f675010bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:47.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.396+0000 7f6754d54640 1 --2- 172.21.15.16:0/1754646793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67500ff880 0x7f67501038d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:47.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.396+0000 7f6754d54640 1 --2- 172.21.15.16:0/1754646793 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67500feed0 0x7f67500ff2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:47.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.396+0000 7f6754d54640 1 -- 172.21.15.16:0/1754646793 >> 172.21.15.16:0/1754646793 conn(0x7f67500fa9b0 msgr2=0x7f67500fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:47.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.396+0000 7f6754d54640 1 -- 172.21.15.16:0/1754646793 shutdown_connections 2024-09-13T16:28:47.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.396+0000 7f6754d54640 1 -- 172.21.15.16:0/1754646793 wait complete. 2024-09-13T16:28:47.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.397+0000 7f6754d54640 1 Processor -- start 2024-09-13T16:28:47.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.397+0000 7f6754d54640 1 -- start start 2024-09-13T16:28:47.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.397+0000 7f6754d54640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67500feed0 0x7f67501970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:47.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.397+0000 7f6754d54640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67500ff880 0x7f67501975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:47.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.397+0000 7f6754d54640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6750104000 0x7f67501a2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:47.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.397+0000 7f6754d54640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6750197e80 con 0x7f67500ff880 2024-09-13T16:28:47.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.397+0000 7f6754d54640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67501a2660 con 0x7f67500feed0 2024-09-13T16:28:47.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.397+0000 7f6754d54640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67501a27a0 con 0x7f6750104000 2024-09-13T16:28:47.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.397+0000 7f674dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67500ff880 0x7f67501975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:47.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.397+0000 7f674dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67500ff880 0x7f67501975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44794/0 (socket says 172.21.15.16:44794) 2024-09-13T16:28:47.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.398+0000 7f674ed9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6750104000 0x7f67501a2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:47.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.398+0000 7f674dd9b640 1 -- 172.21.15.16:0/1829733666 learned_addr learned my addr 172.21.15.16:0/1829733666 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:47.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.398+0000 7f674e59c640 1 --2- 172.21.15.16:0/1829733666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67500feed0 0x7f67501970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:47.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.398+0000 7f674ed9d640 1 -- 172.21.15.16:0/1829733666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67500feed0 msgr2=0x7f67501970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:47.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.398+0000 7f674ed9d640 1 --2- 172.21.15.16:0/1829733666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67500feed0 0x7f67501970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:47.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.398+0000 7f674ed9d640 1 -- 172.21.15.16:0/1829733666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67500ff880 msgr2=0x7f67501975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:47.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.398+0000 7f674ed9d640 1 --2- 172.21.15.16:0/1829733666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67500ff880 0x7f67501975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:47.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.398+0000 7f674ed9d640 1 -- 172.21.15.16:0/1829733666 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6738009560 con 0x7f6750104000 2024-09-13T16:28:47.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.398+0000 7f674ed9d640 1 --2- 172.21.15.16:0/1829733666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6750104000 0x7f67501a2120 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f674800e940 tx=0x7f674800ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:47.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.398+0000 7f67477fe640 1 -- 172.21.15.16:0/1829733666 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6748004d20 con 0x7f6750104000 2024-09-13T16:28:47.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.398+0000 7f67477fe640 1 -- 172.21.15.16:0/1829733666 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6748004e80 con 0x7f6750104000 2024-09-13T16:28:47.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.398+0000 7f6754d54640 1 -- 172.21.15.16:0/1829733666 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67501a2a50 con 0x7f6750104000 2024-09-13T16:28:47.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.398+0000 7f67477fe640 1 -- 172.21.15.16:0/1829733666 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6748011470 con 0x7f6750104000 2024-09-13T16:28:47.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.398+0000 7f6754d54640 1 -- 172.21.15.16:0/1829733666 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67501a2f00 con 0x7f6750104000 2024-09-13T16:28:47.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.399+0000 7f6754d54640 1 -- 172.21.15.16:0/1829733666 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67501003b0 con 0x7f6750104000 2024-09-13T16:28:47.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.400+0000 7f67477fe640 1 -- 172.21.15.16:0/1829733666 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f674800cb20 con 0x7f6750104000 2024-09-13T16:28:47.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.401+0000 7f67477fe640 1 --2- 172.21.15.16:0/1829733666 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f673c0a1080 0x7f673c0a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:47.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.401+0000 7f674e59c640 1 --2- 172.21.15.16:0/1829733666 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f673c0a1080 0x7f673c0a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:47.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.401+0000 7f67477fe640 1 -- 172.21.15.16:0/1829733666 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f67480ccb20 con 0x7f6750104000 2024-09-13T16:28:47.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.402+0000 7f674e59c640 1 --2- 172.21.15.16:0/1829733666 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f673c0a1080 0x7f673c0a3540 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6740009bd0 tx=0x7f6740005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:47.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.404+0000 7f67477fe640 1 -- 172.21.15.16:0/1829733666 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f6748096110 con 0x7f6750104000 2024-09-13T16:28:47.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:47 smithi107 ceph-mon[34139]: pgmap v280: 129 pgs: 10 creating+peering, 4 unknown, 115 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 7 op/s 2024-09-13T16:28:47.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:47 smithi016 ceph-mon[31187]: pgmap v280: 129 pgs: 10 creating+peering, 4 unknown, 115 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 7 op/s 2024-09-13T16:28:47.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.610+0000 7f6754d54640 1 -- 172.21.15.16:0/1829733666 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f67500ff2b0 con 0x7f6750104000 2024-09-13T16:28:47.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.611+0000 7f67477fe640 1 -- 172.21.15.16:0/1829733666 <== mon.1 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7f674801f020 con 0x7f6750104000 2024-09-13T16:28:47.613 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-13T16:28:47.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.613+0000 7f6754d54640 1 -- 172.21.15.16:0/1829733666 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f673c0a1080 msgr2=0x7f673c0a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:47.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.613+0000 7f6754d54640 1 --2- 172.21.15.16:0/1829733666 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f673c0a1080 0x7f673c0a3540 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6740009bd0 tx=0x7f6740005f90 comp rx=0 tx=0).stop 2024-09-13T16:28:47.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.613+0000 7f6754d54640 1 -- 172.21.15.16:0/1829733666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6750104000 msgr2=0x7f67501a2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:47.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.613+0000 7f6754d54640 1 --2- 172.21.15.16:0/1829733666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6750104000 0x7f67501a2120 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f674800e940 tx=0x7f674800ee00 comp rx=0 tx=0).stop 2024-09-13T16:28:47.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.613+0000 7f6754d54640 1 -- 172.21.15.16:0/1829733666 shutdown_connections 2024-09-13T16:28:47.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.613+0000 7f6754d54640 1 --2- 172.21.15.16:0/1829733666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6750104000 0x7f67501a2120 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:47.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.613+0000 7f6754d54640 1 --2- 172.21.15.16:0/1829733666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67500ff880 0x7f67501975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:47.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.613+0000 7f6754d54640 1 --2- 172.21.15.16:0/1829733666 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f673c0a1080 0x7f673c0a3540 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:47.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.613+0000 7f6754d54640 1 --2- 172.21.15.16:0/1829733666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67500feed0 0x7f67501970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:47.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.613+0000 7f6754d54640 1 -- 172.21.15.16:0/1829733666 >> 172.21.15.16:0/1829733666 conn(0x7f67500fa9b0 msgr2=0x7f67500fbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:47.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.613+0000 7f6754d54640 1 -- 172.21.15.16:0/1829733666 shutdown_connections 2024-09-13T16:28:47.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:47.613+0000 7f6754d54640 1 -- 172.21.15.16:0/1829733666 wait complete. 2024-09-13T16:28:47.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:47 smithi111 ceph-mon[34600]: pgmap v280: 129 pgs: 10 creating+peering, 4 unknown, 115 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 7 op/s 2024-09-13T16:28:48.281 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-13T16:28:48.281 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T16:28:48.281 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-13T16:28:48.281 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-13T16:28:48.281 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-13T16:28:48.281 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-13T16:28:48.281 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T16:28:48.281 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T16:28:48.281 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-13T16:28:48.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:28:48.282 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-13T16:28:48.303 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T16:28:48.303 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T16:28:48.303 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T16:28:48.304 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:28:48.304 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-13T16:28:48.304 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:28:48.304 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T16:28:48.304 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-13T16:28:48.304 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-13T16:28:48.304 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T16:28:48.304 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-13T16:28:48.304 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-13T16:28:48.305 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T16:28:48.305 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-13T16:28:48.305 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-13T16:28:48.305 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-13T16:28:48.305 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42322sec preferred_lft 42322sec 2024-09-13T16:28:48.305 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-13T16:28:48.305 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:28:48.305 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T16:28:48.305 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-13T16:28:48.305 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-13T16:28:48.306 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-13T16:28:48.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:28:48.306 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-09-13T16:28:48.306 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-13T16:28:48.306 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2024-09-13T16:28:48.306 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2024-09-13T16:28:48.307 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2024-09-13T16:28:48.307 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-13T16:28:48.307 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-13T16:28:48.467 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:48 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1829733666' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-13T16:28:48.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:28:48.470 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T16:28:48.495 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-13T16:28:48.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:28:48.497 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-13T16:28:48.568 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-13T16:28:48.568 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T16:28:48.568 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-13T16:28:48.568 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-13T16:28:48.568 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T16:28:48.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:28:48.569 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-09-13T16:28:48.569 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-13T16:28:48.569 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-13T16:28:48.569 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-13T16:28:48.569 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-13T16:28:48.569 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-13T16:28:48.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:48 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1829733666' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-13T16:28:48.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:48 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1829733666' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-13T16:28:48.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:28:48.811 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-13T16:28:48.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:28:48.827 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2024-09-13T16:28:48.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:28:48.886 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-09-13T16:28:48.886 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-13T16:28:48.886 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-13T16:28:48.886 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-13T16:28:48.886 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-13T16:28:48.967 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-13T16:28:48.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:28:48.967 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-09-13T16:28:48.967 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-13T16:28:48.967 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-13T16:28:48.967 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-13T16:28:48.968 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-13T16:28:48.968 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-13T16:28:48.968 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-13T16:28:48.968 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-13T16:28:49.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:28:49.124 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-09-13T16:28:49.124 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-13T16:28:49.124 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-13T16:28:49.124 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2024-09-13T16:28:49.124 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-13T16:28:49.165 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-13T16:28:49.165 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T16:28:49.165 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-13T16:28:49.223 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-13T16:28:49.224 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T16:28:49.224 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-13T16:28:49.281 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-13T16:28:49.344 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:49 smithi016 ceph-mon[31187]: pgmap v281: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-09-13T16:28:49.349 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T16:28:49.349 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T16:28:49.415 INFO:teuthology.orchestra.run.smithi016.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T16:28:49.417 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T16:28:49.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:28:49.417 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-13T16:28:49.475 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-13T16:28:49.476 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-13T16:28:49.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-13T16:28:49.545 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T16:28:49.545 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T16:28:49.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:49 smithi107 ceph-mon[34139]: pgmap v281: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-09-13T16:28:49.611 INFO:teuthology.orchestra.run.smithi016.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T16:28:49.612 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T16:28:49.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:28:49.612 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-13T16:28:49.621 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:2024-09-13T16:28:49.619+0000 7f09297e6480 -1 init, newargv = 0x55899fde5490 newargc=15 2024-09-13T16:28:49.621 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[74912]: starting ceph client 2024-09-13T16:28:49.627 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[74912]: starting fuse 2024-09-13T16:28:49.637 INFO:teuthology.orchestra.run.smithi016.stdout:120 2024-09-13T16:28:49.638 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-13T16:28:49.638 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T16:28:49.638 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-13T16:28:49.638 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-13T16:28:49.638 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-13T16:28:49.638 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-13T16:28:49.638 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-13T16:28:49.638 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-13T16:28:49.638 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-13T16:28:49.639 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-13T16:28:49.639 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-13T16:28:49.639 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-13T16:28:49.639 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-13T16:28:49.639 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-13T16:28:49.639 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-13T16:28:49.639 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-13T16:28:49.639 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-13T16:28:49.639 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-13T16:28:49.639 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T16:28:49.639 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T16:28:49.639 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T16:28:49.639 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-13T16:28:49.639 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-13T16:28:49.639 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-13T16:28:49.640 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T16:28:49.640 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-13T16:28:49.640 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.admin")) 2024-09-13T16:28:49.640 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-13T16:28:49.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:49 smithi111 ceph-mon[34600]: pgmap v281: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-09-13T16:28:49.755 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.admin.74912.asok 2024-09-13T16:28:49.942 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.74912.asok 2024-09-13T16:28:49.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:28:49.942 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.74912.asok status 2024-09-13T16:28:50.080 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-13T16:28:50.080 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-13T16:28:50.080 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "50bbe7f395997dcd871813949c49addbbf630361", 2024-09-13T16:28:50.080 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 18.2.4-850-g50bbe7f3 (50bbe7f395997dcd871813949c49addbbf630361) reef (stable)", 2024-09-13T16:28:50.081 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "admin", 2024-09-13T16:28:50.081 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-13T16:28:50.081 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-13T16:28:50.081 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "74912", 2024-09-13T16:28:50.081 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2024-09-13T16:28:50.081 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:28:50.081 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-09-13T16:28:50.081 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-09-13T16:28:50.081 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 14736, 2024-09-13T16:28:50.081 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-13T16:28:50.081 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-13T16:28:50.081 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-13T16:28:50.081 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 14736 2024-09-13T16:28:50.082 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:28:50.082 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-13T16:28:50.082 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-13T16:28:50.082 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-13T16:28:50.082 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3176376122 2024-09-13T16:28:50.082 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:28:50.082 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:28:50.082 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-13T16:28:50.082 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-13T16:28:50.082 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-13T16:28:50.082 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3176376122 2024-09-13T16:28:50.082 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:28:50.082 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.14736 192.168.0.1:0/3176376122", 2024-09-13T16:28:50.082 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/3176376122", 2024-09-13T16:28:50.082 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2024-09-13T16:28:50.082 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 17, 2024-09-13T16:28:50.083 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 89, 2024-09-13T16:28:50.083 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-09-13T16:28:50.083 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-13T16:28:50.083 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-13T16:28:50.083 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-13T16:28:50.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:28:50.089 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-09-13T16:28:50.089 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-13T16:28:50.106 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-09-13T16:28:50.106 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-13T16:28:50.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:28:50.106 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-13T16:28:50.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:28:50.173 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-13T16:28:50.247 INFO:teuthology.orchestra.run.smithi016.stdout:# file: . 2024-09-13T16:28:50.248 INFO:teuthology.orchestra.run.smithi016.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-13T16:28:50.248 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:28:50.248 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-13T16:28:50.248 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:28:50.305 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:28:50.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.445+0000 7f88a39db640 1 -- 172.21.15.16:0/727607103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f889c0feca0 msgr2=0x7f889c0ff0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.445+0000 7f88a39db640 1 --2- 172.21.15.16:0/727607103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f889c0feca0 0x7f889c0ff0a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8898009900 tx=0x7f889802a410 comp rx=0 tx=0).stop 2024-09-13T16:28:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.445+0000 7f88a39db640 1 -- 172.21.15.16:0/727607103 shutdown_connections 2024-09-13T16:28:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.445+0000 7f88a39db640 1 --2- 172.21.15.16:0/727607103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f889c100850 0x7f889c1070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.445+0000 7f88a39db640 1 --2- 172.21.15.16:0/727607103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f889c0ffe90 0x7f889c100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.445+0000 7f88a39db640 1 --2- 172.21.15.16:0/727607103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f889c0feca0 0x7f889c0ff0a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.445+0000 7f88a39db640 1 -- 172.21.15.16:0/727607103 >> 172.21.15.16:0/727607103 conn(0x7f889c0fa420 msgr2=0x7f889c0fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.445+0000 7f88a39db640 1 -- 172.21.15.16:0/727607103 shutdown_connections 2024-09-13T16:28:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.445+0000 7f88a39db640 1 -- 172.21.15.16:0/727607103 wait complete. 2024-09-13T16:28:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.446+0000 7f88a39db640 1 Processor -- start 2024-09-13T16:28:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.446+0000 7f88a39db640 1 -- start start 2024-09-13T16:28:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.446+0000 7f88a39db640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f889c0feca0 0x7f889c195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.446+0000 7f88a39db640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f889c0ffe90 0x7f889c1962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.446+0000 7f88a39db640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f889c100850 0x7f889c19d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.446+0000 7f88a39db640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889c19d880 con 0x7f889c0feca0 2024-09-13T16:28:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.446+0000 7f88a39db640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889c19d9f0 con 0x7f889c100850 2024-09-13T16:28:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.446+0000 7f88a39db640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889c19dbb0 con 0x7f889c0ffe90 2024-09-13T16:28:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.446+0000 7f88a1777640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f889c0feca0 0x7f889c195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.446+0000 7f88a1777640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f889c0feca0 0x7f889c195da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44802/0 (socket says 172.21.15.16:44802) 2024-09-13T16:28:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.446+0000 7f88a1777640 1 -- 172.21.15.16:0/4161085663 learned_addr learned my addr 172.21.15.16:0/4161085663 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.446+0000 7f88a1f78640 1 --2- 172.21.15.16:0/4161085663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f889c100850 0x7f889c19d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.446+0000 7f88a1777640 1 -- 172.21.15.16:0/4161085663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f889c0ffe90 msgr2=0x7f889c1962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.446+0000 7f88a0f76640 1 --2- 172.21.15.16:0/4161085663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f889c0ffe90 0x7f889c1962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.446+0000 7f88a1777640 1 --2- 172.21.15.16:0/4161085663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f889c0ffe90 0x7f889c1962e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.446+0000 7f88a1777640 1 -- 172.21.15.16:0/4161085663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f889c100850 msgr2=0x7f889c19d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.446+0000 7f88a1777640 1 --2- 172.21.15.16:0/4161085663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f889c100850 0x7f889c19d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.446+0000 7f88a1777640 1 -- 172.21.15.16:0/4161085663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8898009560 con 0x7f889c0feca0 2024-09-13T16:28:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.446+0000 7f88a1f78640 1 --2- 172.21.15.16:0/4161085663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f889c100850 0x7f889c19d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:28:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.447+0000 7f88a1777640 1 --2- 172.21.15.16:0/4161085663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f889c0feca0 0x7f889c195da0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8898009230 tx=0x7f8898005320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.447+0000 7f88927fc640 1 -- 172.21.15.16:0/4161085663 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8898031d70 con 0x7f889c0feca0 2024-09-13T16:28:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.447+0000 7f88a39db640 1 -- 172.21.15.16:0/4161085663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f889c19de30 con 0x7f889c0feca0 2024-09-13T16:28:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.447+0000 7f88927fc640 1 -- 172.21.15.16:0/4161085663 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8898004b50 con 0x7f889c0feca0 2024-09-13T16:28:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.447+0000 7f88927fc640 1 -- 172.21.15.16:0/4161085663 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8898003c30 con 0x7f889c0feca0 2024-09-13T16:28:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.447+0000 7f88a39db640 1 -- 172.21.15.16:0/4161085663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f889c19e200 con 0x7f889c0feca0 2024-09-13T16:28:50.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.448+0000 7f88a39db640 1 -- 172.21.15.16:0/4161085663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f889c10a410 con 0x7f889c0feca0 2024-09-13T16:28:50.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.449+0000 7f88927fc640 1 -- 172.21.15.16:0/4161085663 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f88980046c0 con 0x7f889c0feca0 2024-09-13T16:28:50.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.450+0000 7f88927fc640 1 --2- 172.21.15.16:0/4161085663 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f88880a1090 0x7f88880a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:50.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.450+0000 7f88a0f76640 1 --2- 172.21.15.16:0/4161085663 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f88880a1090 0x7f88880a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:50.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.450+0000 7f88927fc640 1 -- 172.21.15.16:0/4161085663 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f88980ee100 con 0x7f889c0feca0 2024-09-13T16:28:50.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.450+0000 7f88a0f76640 1 --2- 172.21.15.16:0/4161085663 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f88880a1090 0x7f88880a3550 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f888c0058c0 tx=0x7f888c005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:50.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.453+0000 7f88927fc640 1 -- 172.21.15.16:0/4161085663 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f88980b75b0 con 0x7f889c0feca0 2024-09-13T16:28:50.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.626+0000 7f88a39db640 1 -- 172.21.15.16:0/4161085663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f889c10a5f0 con 0x7f889c0feca0 2024-09-13T16:28:50.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.628+0000 7f88927fc640 1 -- 172.21.15.16:0/4161085663 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23971 (secure 0 0 0) 0x7f88980b94e0 con 0x7f889c0feca0 2024-09-13T16:28:50.631 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:28:50.631 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":89,"fsid":"0d241f6a-71ec-11ef-bceb-c7b262605968","created":"2024-09-13T16:21:17.971738+0000","modified":"2024-09-13T16:28:43.574106+0000","last_up_change":"2024-09-13T16:27:54.673413+0000","last_in_change":"2024-09-13T16:27:35.718390+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:24:22.981019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:28:37.746027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:28:38.602731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:28:41.633322+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e11ef8f-08d6-4e74-afd6-f49f4bd40032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6803","nonce":2079394186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6805","nonce":2079394186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6809","nonce":2079394186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6807","nonce":2079394186}]},"public_addr":"172.21.15.16:6803/2079394186","cluster_addr":"172.21.15.16:6805/2079394186","heartbeat_back_addr":"172.21.15.16:6809/2079394186","heartbeat_front_addr":"172.21.15.16:6807/2079394186","state":["exists","up"]},{"osd":1,"uuid":"f2cd3694-9c9e-4da7-976e-f80a8df120f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6801","nonce":4021360958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6803","nonce":4021360958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6807","nonce":4021360958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6805","nonce":4021360958}]},"public_addr":"172.21.15.107:6801/4021360958","cluster_addr":"172.21.15.107:6803/4021360958","heartbeat_back_addr":"172.21.15.107:6807/4021360958","heartbeat_front_addr":"172.21.15.107:6805/4021360958","state":["exists","up"]},{"osd":2,"uuid":"a5bbf2eb-6599-44d5-ad96-68bb89ee19f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6801","nonce":3227179825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6803","nonce":3227179825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6807","nonce":3227179825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6805","nonce":3227179825}]},"public_addr":"172.21.15.111:6801/3227179825","cluster_addr":"172.21.15.111:6803/3227179825","heartbeat_back_addr":"172.21.15.111:6807/3227179825","heartbeat_front_addr":"172.21.15.111:6805/3227179825","state":["exists","up"]},{"osd":3,"uuid":"d6295ffe-11bc-4964-a250-9fecca870b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6811","nonce":2390375937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6813","nonce":2390375937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6817","nonce":2390375937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6815","nonce":2390375937}]},"public_addr":"172.21.15.16:6811/2390375937","cluster_addr":"172.21.15.16:6813/2390375937","heartbeat_back_addr":"172.21.15.16:6817/2390375937","heartbeat_front_addr":"172.21.15.16:6815/2390375937","state":["exists","up"]},{"osd":4,"uuid":"0a8446e0-fc86-4646-beb6-a7b9d433093d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6809","nonce":1936582048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6811","nonce":1936582048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6815","nonce":1936582048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6813","nonce":1936582048}]},"public_addr":"172.21.15.107:6809/1936582048","cluster_addr":"172.21.15.107:6811/1936582048","heartbeat_back_addr":"172.21.15.107:6815/1936582048","heartbeat_front_addr":"172.21.15.107:6813/1936582048","state":["exists","up"]},{"osd":5,"uuid":"2c97a446-357e-4975-9e51-9c95fe84a663","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6809","nonce":483858448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6811","nonce":483858448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6815","nonce":483858448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6813","nonce":483858448}]},"public_addr":"172.21.15.111:6809/483858448","cluster_addr":"172.21.15.111:6811/483858448","heartbeat_back_addr":"172.21.15.111:6815/483858448","heartbeat_front_addr":"172.21.15.111:6813/483858448","state":["exists","up"]},{"osd":6,"uuid":"890239e6-e6fb-4f9c-b973-05cc66d199bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6819","nonce":1330214141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6821","nonce":1330214141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6825","nonce":1330214141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6823","nonce":1330214141}]},"public_addr":"172.21.15.16:6819/1330214141","cluster_addr":"172.21.15.16:6821/1330214141","heartbeat_back_addr":"172.21.15.16:6825/1330214141","heartbeat_front_addr":"172.21.15.16:6823/1330214141","state":["exists","up"]},{"osd":7,"uuid":"ee826a9c-f441-4903-9b1b-66b627f7b712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6817","nonce":1133876754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6819","nonce":1133876754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6823","nonce":1133876754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6821","nonce":1133876754}]},"public_addr":"172.21.15.107:6817/1133876754","cluster_addr":"172.21.15.107:6819/1133876754","heartbeat_back_addr":"172.21.15.107:6823/1133876754","heartbeat_front_addr":"172.21.15.107:6821/1133876754","state":["exists","up"]},{"osd":8,"uuid":"e19773f2-dec8-44e0-9d33-0252e07d6415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6817","nonce":837548190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6819","nonce":837548190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6823","nonce":837548190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6821","nonce":837548190}]},"public_addr":"172.21.15.111:6817/837548190","cluster_addr":"172.21.15.111:6819/837548190","heartbeat_back_addr":"172.21.15.111:6823/837548190","heartbeat_front_addr":"172.21.15.111:6821/837548190","state":["exists","up"]},{"osd":9,"uuid":"bae4d6db-8b90-4d02-8e83-7d8f2a6f8fdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6827","nonce":4175383686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6829","nonce":4175383686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6833","nonce":4175383686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6831","nonce":4175383686}]},"public_addr":"172.21.15.16:6827/4175383686","cluster_addr":"172.21.15.16:6829/4175383686","heartbeat_back_addr":"172.21.15.16:6833/4175383686","heartbeat_front_addr":"172.21.15.16:6831/4175383686","state":["exists","up"]},{"osd":10,"uuid":"fd0e7740-76c8-4c99-a349-7077b4905143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6825","nonce":2659550426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6827","nonce":2659550426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6831","nonce":2659550426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6829","nonce":2659550426}]},"public_addr":"172.21.15.107:6825/2659550426","cluster_addr":"172.21.15.107:6827/2659550426","heartbeat_back_addr":"172.21.15.107:6831/2659550426","heartbeat_front_addr":"172.21.15.107:6829/2659550426","state":["exists","up"]},{"osd":11,"uuid":"94a1751d-b00c-4e23-815d-3a8bc3dea146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6825","nonce":1633548683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6827","nonce":1633548683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6831","nonce":1633548683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6829","nonce":1633548683}]},"public_addr":"172.21.15.111:6825/1633548683","cluster_addr":"172.21.15.111:6827/1633548683","heartbeat_back_addr":"172.21.15.111:6831/1633548683","heartbeat_front_addr":"172.21.15.111:6829/1633548683","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:36.359852+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:58.014202+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:19.753672+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:43.404817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:06.001677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:29.361587+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:52.554847+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:15.957553+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:39.581093+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:04.911389+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:28.971280+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:53.473561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/2098378667":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/3536229016":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6801/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/2684687547":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6800/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/368410612":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/12158160":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6800/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6801/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/3754423379":"2024-09-14T16:21:40.126459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:28:50.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.630+0000 7f88a39db640 1 -- 172.21.15.16:0/4161085663 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f88880a1090 msgr2=0x7f88880a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:50.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.630+0000 7f88a39db640 1 --2- 172.21.15.16:0/4161085663 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f88880a1090 0x7f88880a3550 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f888c0058c0 tx=0x7f888c005c60 comp rx=0 tx=0).stop 2024-09-13T16:28:50.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.630+0000 7f88a39db640 1 -- 172.21.15.16:0/4161085663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f889c0feca0 msgr2=0x7f889c195da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:50.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.630+0000 7f88a39db640 1 --2- 172.21.15.16:0/4161085663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f889c0feca0 0x7f889c195da0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8898009230 tx=0x7f8898005320 comp rx=0 tx=0).stop 2024-09-13T16:28:50.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.630+0000 7f88a39db640 1 -- 172.21.15.16:0/4161085663 shutdown_connections 2024-09-13T16:28:50.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.630+0000 7f88a39db640 1 --2- 172.21.15.16:0/4161085663 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f88880a1090 0x7f88880a3550 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:50.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.630+0000 7f88a39db640 1 --2- 172.21.15.16:0/4161085663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f889c100850 0x7f889c19d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:50.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.630+0000 7f88a39db640 1 --2- 172.21.15.16:0/4161085663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f889c0ffe90 0x7f889c1962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:50.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.630+0000 7f88a39db640 1 --2- 172.21.15.16:0/4161085663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f889c0feca0 0x7f889c195da0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:50.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.630+0000 7f88a39db640 1 -- 172.21.15.16:0/4161085663 >> 172.21.15.16:0/4161085663 conn(0x7f889c0fa420 msgr2=0x7f889c0686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:50.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.630+0000 7f88a39db640 1 -- 172.21.15.16:0/4161085663 shutdown_connections 2024-09-13T16:28:50.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.630+0000 7f88a39db640 1 -- 172.21.15.16:0/4161085663 wait complete. 2024-09-13T16:28:50.643 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:24:22.981019+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T16:28:37.746027+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T16:28:38.602731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T16:28:41.633322+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T16:28:50.644 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:28:50.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.754+0000 7fdad8a53640 1 -- 172.21.15.16:0/3395523552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdad4105a40 msgr2=0x7fdad4107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:50.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.754+0000 7fdad8a53640 1 --2- 172.21.15.16:0/3395523552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdad4105a40 0x7fdad4107eb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009ca0 tx=0x7fdacc02a8c0 comp rx=0 tx=0).stop 2024-09-13T16:28:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.754+0000 7fdad8a53640 1 -- 172.21.15.16:0/3395523552 shutdown_connections 2024-09-13T16:28:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.754+0000 7fdad8a53640 1 --2- 172.21.15.16:0/3395523552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdad4105a40 0x7fdad4107eb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.754+0000 7fdad8a53640 1 --2- 172.21.15.16:0/3395523552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdad4103110 0x7fdad4105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.754+0000 7fdad8a53640 1 --2- 172.21.15.16:0/3395523552 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdad41007e0 0x7fdad4102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.754+0000 7fdad8a53640 1 -- 172.21.15.16:0/3395523552 >> 172.21.15.16:0/3395523552 conn(0x7fdad40fa3e0 msgr2=0x7fdad40fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.754+0000 7fdad8a53640 1 -- 172.21.15.16:0/3395523552 shutdown_connections 2024-09-13T16:28:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.754+0000 7fdad8a53640 1 -- 172.21.15.16:0/3395523552 wait complete. 2024-09-13T16:28:50.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.755+0000 7fdad8a53640 1 Processor -- start 2024-09-13T16:28:50.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.755+0000 7fdad8a53640 1 -- start start 2024-09-13T16:28:50.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.755+0000 7fdad8a53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdad41007e0 0x7fdad4197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:50.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.755+0000 7fdad8a53640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdad4103110 0x7fdad4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:50.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.755+0000 7fdad8a53640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdad4105a40 0x7fdad419f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:50.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.755+0000 7fdad8a53640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad419f9c0 con 0x7fdad41007e0 2024-09-13T16:28:50.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.755+0000 7fdad8a53640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad419fb30 con 0x7fdad4105a40 2024-09-13T16:28:50.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.755+0000 7fdad8a53640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad419fcf0 con 0x7fdad4103110 2024-09-13T16:28:50.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.755+0000 7fdad259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdad41007e0 0x7fdad4197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:50.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.755+0000 7fdad259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdad41007e0 0x7fdad4197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44806/0 (socket says 172.21.15.16:44806) 2024-09-13T16:28:50.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.755+0000 7fdad259c640 1 -- 172.21.15.16:0/4238553851 learned_addr learned my addr 172.21.15.16:0/4238553851 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:50.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.755+0000 7fdad2d9d640 1 --2- 172.21.15.16:0/4238553851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdad4105a40 0x7fdad419f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.755+0000 7fdad259c640 1 -- 172.21.15.16:0/4238553851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdad4103110 msgr2=0x7fdad4198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.755+0000 7fdad259c640 1 --2- 172.21.15.16:0/4238553851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdad4103110 0x7fdad4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.755+0000 7fdad259c640 1 -- 172.21.15.16:0/4238553851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdad4105a40 msgr2=0x7fdad419f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.755+0000 7fdad259c640 1 --2- 172.21.15.16:0/4238553851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdad4105a40 0x7fdad419f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.756+0000 7fdad259c640 1 -- 172.21.15.16:0/4238553851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdacc009900 con 0x7fdad41007e0 2024-09-13T16:28:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.756+0000 7fdad2d9d640 1 --2- 172.21.15.16:0/4238553851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdad4105a40 0x7fdad419f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:28:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.756+0000 7fdad259c640 1 --2- 172.21.15.16:0/4238553851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdad41007e0 0x7fdad4197ed0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdac400e9f0 tx=0x7fdac400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.756+0000 7fdac37fe640 1 -- 172.21.15.16:0/4238553851 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac4009720 con 0x7fdad41007e0 2024-09-13T16:28:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.756+0000 7fdad8a53640 1 -- 172.21.15.16:0/4238553851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad419ffd0 con 0x7fdad41007e0 2024-09-13T16:28:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.756+0000 7fdac37fe640 1 -- 172.21.15.16:0/4238553851 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdac400cd80 con 0x7fdad41007e0 2024-09-13T16:28:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.756+0000 7fdac37fe640 1 -- 172.21.15.16:0/4238553851 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac4010430 con 0x7fdad41007e0 2024-09-13T16:28:50.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.756+0000 7fdad8a53640 1 -- 172.21.15.16:0/4238553851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad41a0400 con 0x7fdad41007e0 2024-09-13T16:28:50.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.757+0000 7fdad8a53640 1 -- 172.21.15.16:0/4238553851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdad4107530 con 0x7fdad41007e0 2024-09-13T16:28:50.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.758+0000 7fdac37fe640 1 -- 172.21.15.16:0/4238553851 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fdac4004fb0 con 0x7fdad41007e0 2024-09-13T16:28:50.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.759+0000 7fdac37fe640 1 --2- 172.21.15.16:0/4238553851 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdabc0a1090 0x7fdabc0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:50.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.759+0000 7fdad1d9b640 1 --2- 172.21.15.16:0/4238553851 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdabc0a1090 0x7fdabc0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:50.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.759+0000 7fdac37fe640 1 -- 172.21.15.16:0/4238553851 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fdac4014070 con 0x7fdad41007e0 2024-09-13T16:28:50.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.759+0000 7fdad1d9b640 1 --2- 172.21.15.16:0/4238553851 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdabc0a1090 0x7fdabc0a3550 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdac8005fd0 tx=0x7fdac8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:50.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.763+0000 7fdac37fe640 1 -- 172.21.15.16:0/4238553851 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fdac4096540 con 0x7fdad41007e0 2024-09-13T16:28:50.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.932+0000 7fdad8a53640 1 -- 172.21.15.16:0/4238553851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdad402c480 con 0x7fdad41007e0 2024-09-13T16:28:50.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.933+0000 7fdac37fe640 1 -- 172.21.15.16:0/4238553851 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdac4099b60 con 0x7fdad41007e0 2024-09-13T16:28:50.935 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-13T16:28:50.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.934+0000 7fdad8a53640 1 -- 172.21.15.16:0/4238553851 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdabc0a1090 msgr2=0x7fdabc0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:50.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.934+0000 7fdad8a53640 1 --2- 172.21.15.16:0/4238553851 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdabc0a1090 0x7fdabc0a3550 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdac8005fd0 tx=0x7fdac8005850 comp rx=0 tx=0).stop 2024-09-13T16:28:50.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.934+0000 7fdad8a53640 1 -- 172.21.15.16:0/4238553851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdad41007e0 msgr2=0x7fdad4197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:50.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.934+0000 7fdad8a53640 1 --2- 172.21.15.16:0/4238553851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdad41007e0 0x7fdad4197ed0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdac400e9f0 tx=0x7fdac400edf0 comp rx=0 tx=0).stop 2024-09-13T16:28:50.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.934+0000 7fdad8a53640 1 -- 172.21.15.16:0/4238553851 shutdown_connections 2024-09-13T16:28:50.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.934+0000 7fdad8a53640 1 --2- 172.21.15.16:0/4238553851 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdabc0a1090 0x7fdabc0a3550 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:50.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.934+0000 7fdad8a53640 1 --2- 172.21.15.16:0/4238553851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdad4105a40 0x7fdad419f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:50.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.934+0000 7fdad8a53640 1 --2- 172.21.15.16:0/4238553851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdad4103110 0x7fdad4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:50.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.934+0000 7fdad8a53640 1 --2- 172.21.15.16:0/4238553851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdad41007e0 0x7fdad4197ed0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:50.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.934+0000 7fdad8a53640 1 -- 172.21.15.16:0/4238553851 >> 172.21.15.16:0/4238553851 conn(0x7fdad40fa3e0 msgr2=0x7fdad40fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:50.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.935+0000 7fdad8a53640 1 -- 172.21.15.16:0/4238553851 shutdown_connections 2024-09-13T16:28:50.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:50.935+0000 7fdad8a53640 1 -- 172.21.15.16:0/4238553851 wait complete. 2024-09-13T16:28:50.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:50 smithi111 ceph-mon[34600]: pgmap v282: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s wr, 0 op/s 2024-09-13T16:28:50.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:50 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/4161085663' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:28:50.946 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:50 smithi016 ceph-mon[31187]: pgmap v282: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s wr, 0 op/s 2024-09-13T16:28:50.946 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:50 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/4161085663' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:28:50.947 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T16:28:51.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.051+0000 7f06f00d7640 1 -- 172.21.15.16:0/3231776962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06e8101930 msgr2=0x7f06e81081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:51.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.051+0000 7f06f00d7640 1 --2- 172.21.15.16:0/3231776962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06e8101930 0x7f06e81081c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f06e0009900 tx=0x7f06e002a410 comp rx=0 tx=0).stop 2024-09-13T16:28:51.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.052+0000 7f06f00d7640 1 -- 172.21.15.16:0/3231776962 shutdown_connections 2024-09-13T16:28:51.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.052+0000 7f06f00d7640 1 --2- 172.21.15.16:0/3231776962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06e8101930 0x7f06e81081c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.052+0000 7f06f00d7640 1 --2- 172.21.15.16:0/3231776962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06e8100f70 0x7f06e81013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.052+0000 7f06f00d7640 1 --2- 172.21.15.16:0/3231776962 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06e80ffd80 0x7f06e8100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.052+0000 7f06f00d7640 1 -- 172.21.15.16:0/3231776962 >> 172.21.15.16:0/3231776962 conn(0x7f06e80fb4e0 msgr2=0x7f06e80fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.052+0000 7f06f00d7640 1 -- 172.21.15.16:0/3231776962 shutdown_connections 2024-09-13T16:28:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.052+0000 7f06f00d7640 1 -- 172.21.15.16:0/3231776962 wait complete. 2024-09-13T16:28:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.052+0000 7f06f00d7640 1 Processor -- start 2024-09-13T16:28:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.052+0000 7f06f00d7640 1 -- start start 2024-09-13T16:28:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.052+0000 7f06f00d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06e80ffd80 0x7f06e8195ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.052+0000 7f06f00d7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06e8100f70 0x7f06e81960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.052+0000 7f06f00d7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06e8101930 0x7f06e819d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.052+0000 7f06f00d7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e819d690 con 0x7f06e80ffd80 2024-09-13T16:28:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.052+0000 7f06f00d7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e819d800 con 0x7f06e8100f70 2024-09-13T16:28:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.052+0000 7f06f00d7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e819d9c0 con 0x7f06e8101930 2024-09-13T16:28:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.053+0000 7f06ede73640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06e80ffd80 0x7f06e8195ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.053+0000 7f06ede73640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06e80ffd80 0x7f06e8195ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44826/0 (socket says 172.21.15.16:44826) 2024-09-13T16:28:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.053+0000 7f06ede73640 1 -- 172.21.15.16:0/2267788140 learned_addr learned my addr 172.21.15.16:0/2267788140 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.053+0000 7f06ee674640 1 --2- 172.21.15.16:0/2267788140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06e8101930 0x7f06e819d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.053+0000 7f06ed672640 1 --2- 172.21.15.16:0/2267788140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06e8100f70 0x7f06e81960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.053+0000 7f06ede73640 1 -- 172.21.15.16:0/2267788140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06e8101930 msgr2=0x7f06e819d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.053+0000 7f06ede73640 1 --2- 172.21.15.16:0/2267788140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06e8101930 0x7f06e819d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.053+0000 7f06ede73640 1 -- 172.21.15.16:0/2267788140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06e8100f70 msgr2=0x7f06e81960e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.053+0000 7f06ede73640 1 --2- 172.21.15.16:0/2267788140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06e8100f70 0x7f06e81960e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.053+0000 7f06ede73640 1 -- 172.21.15.16:0/2267788140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e0009560 con 0x7f06e80ffd80 2024-09-13T16:28:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.053+0000 7f06ede73640 1 --2- 172.21.15.16:0/2267788140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06e80ffd80 0x7f06e8195ba0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f06e400ebc0 tx=0x7f06e400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.053+0000 7f06deffd640 1 -- 172.21.15.16:0/2267788140 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e4003b70 con 0x7f06e80ffd80 2024-09-13T16:28:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.053+0000 7f06deffd640 1 -- 172.21.15.16:0/2267788140 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f06e400eec0 con 0x7f06e80ffd80 2024-09-13T16:28:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.053+0000 7f06f00d7640 1 -- 172.21.15.16:0/2267788140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06e819dca0 con 0x7f06e80ffd80 2024-09-13T16:28:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.053+0000 7f06deffd640 1 -- 172.21.15.16:0/2267788140 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e40196e0 con 0x7f06e80ffd80 2024-09-13T16:28:51.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.053+0000 7f06f00d7640 1 -- 172.21.15.16:0/2267788140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06e819e1b0 con 0x7f06e80ffd80 2024-09-13T16:28:51.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.054+0000 7f06f00d7640 1 -- 172.21.15.16:0/2267788140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06e81071c0 con 0x7f06e80ffd80 2024-09-13T16:28:51.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.055+0000 7f06deffd640 1 -- 172.21.15.16:0/2267788140 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f06e4004030 con 0x7f06e80ffd80 2024-09-13T16:28:51.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.056+0000 7f06deffd640 1 --2- 172.21.15.16:0/2267788140 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f06d40a1110 0x7f06d40a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:51.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.056+0000 7f06deffd640 1 -- 172.21.15.16:0/2267788140 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f06e4014070 con 0x7f06e80ffd80 2024-09-13T16:28:51.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.056+0000 7f06ed672640 1 --2- 172.21.15.16:0/2267788140 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f06d40a1110 0x7f06d40a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:51.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.056+0000 7f06ed672640 1 --2- 172.21.15.16:0/2267788140 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f06d40a1110 0x7f06d40a35d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f06d8005e10 tx=0x7f06d8005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:51.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.059+0000 7f06deffd640 1 -- 172.21.15.16:0/2267788140 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f06e4096b10 con 0x7f06e80ffd80 2024-09-13T16:28:51.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:50 smithi107 ceph-mon[34139]: pgmap v282: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s wr, 0 op/s 2024-09-13T16:28:51.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:50 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/4161085663' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:28:51.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.232+0000 7f06f00d7640 1 -- 172.21.15.16:0/2267788140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f06e810b4d0 con 0x7f06e80ffd80 2024-09-13T16:28:51.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.232+0000 7f06deffd640 1 -- 172.21.15.16:0/2267788140 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f06e409a130 con 0x7f06e80ffd80 2024-09-13T16:28:51.235 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-13T16:28:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.234+0000 7f06f00d7640 1 -- 172.21.15.16:0/2267788140 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f06d40a1110 msgr2=0x7f06d40a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.234+0000 7f06f00d7640 1 --2- 172.21.15.16:0/2267788140 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f06d40a1110 0x7f06d40a35d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f06d8005e10 tx=0x7f06d8005da0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.234+0000 7f06f00d7640 1 -- 172.21.15.16:0/2267788140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06e80ffd80 msgr2=0x7f06e8195ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.234+0000 7f06f00d7640 1 --2- 172.21.15.16:0/2267788140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06e80ffd80 0x7f06e8195ba0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f06e400ebc0 tx=0x7f06e400c6a0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.234+0000 7f06f00d7640 1 -- 172.21.15.16:0/2267788140 shutdown_connections 2024-09-13T16:28:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.234+0000 7f06f00d7640 1 --2- 172.21.15.16:0/2267788140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06e8101930 0x7f06e819d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.234+0000 7f06f00d7640 1 --2- 172.21.15.16:0/2267788140 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f06d40a1110 0x7f06d40a35d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.234+0000 7f06f00d7640 1 --2- 172.21.15.16:0/2267788140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06e8100f70 0x7f06e81960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.234+0000 7f06f00d7640 1 --2- 172.21.15.16:0/2267788140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06e80ffd80 0x7f06e8195ba0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.234+0000 7f06f00d7640 1 -- 172.21.15.16:0/2267788140 >> 172.21.15.16:0/2267788140 conn(0x7f06e80fb4e0 msgr2=0x7f06e8104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.234+0000 7f06f00d7640 1 -- 172.21.15.16:0/2267788140 shutdown_connections 2024-09-13T16:28:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.234+0000 7f06f00d7640 1 -- 172.21.15.16:0/2267788140 wait complete. 2024-09-13T16:28:51.247 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T16:28:51.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.351+0000 7facd245a640 1 -- 172.21.15.16:0/2638129089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faccc0ffd70 msgr2=0x7faccc100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:51.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.351+0000 7facd245a640 1 --2- 172.21.15.16:0/2638129089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faccc0ffd70 0x7faccc100170 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7facbc009900 tx=0x7facbc02a410 comp rx=0 tx=0).stop 2024-09-13T16:28:51.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.352+0000 7facd245a640 1 -- 172.21.15.16:0/2638129089 shutdown_connections 2024-09-13T16:28:51.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.352+0000 7facd245a640 1 --2- 172.21.15.16:0/2638129089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faccc101920 0x7faccc1081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.352+0000 7facd245a640 1 --2- 172.21.15.16:0/2638129089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faccc100f60 0x7faccc1013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.352+0000 7facd245a640 1 --2- 172.21.15.16:0/2638129089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faccc0ffd70 0x7faccc100170 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.352+0000 7facd245a640 1 -- 172.21.15.16:0/2638129089 >> 172.21.15.16:0/2638129089 conn(0x7faccc0fb4d0 msgr2=0x7faccc0fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:51.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.352+0000 7facd245a640 1 -- 172.21.15.16:0/2638129089 shutdown_connections 2024-09-13T16:28:51.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.352+0000 7facd245a640 1 -- 172.21.15.16:0/2638129089 wait complete. 2024-09-13T16:28:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.352+0000 7facd245a640 1 Processor -- start 2024-09-13T16:28:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.352+0000 7facd245a640 1 -- start start 2024-09-13T16:28:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.352+0000 7facd245a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faccc0ffd70 0x7faccc074260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.352+0000 7facd245a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faccc100f60 0x7faccc0728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.353+0000 7facd245a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faccc101920 0x7faccc072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.353+0000 7facd245a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc073540 con 0x7faccc0ffd70 2024-09-13T16:28:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.353+0000 7facd245a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc0736b0 con 0x7faccc101920 2024-09-13T16:28:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.353+0000 7facd245a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc19fd50 con 0x7faccc100f60 2024-09-13T16:28:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.353+0000 7faccbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faccc0ffd70 0x7faccc074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.353+0000 7faccbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faccc0ffd70 0x7faccc074260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44836/0 (socket says 172.21.15.16:44836) 2024-09-13T16:28:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.353+0000 7faccbfff640 1 -- 172.21.15.16:0/3811190444 learned_addr learned my addr 172.21.15.16:0/3811190444 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.353+0000 7faccb7fe640 1 --2- 172.21.15.16:0/3811190444 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faccc100f60 0x7faccc0728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:51.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.353+0000 7faccbfff640 1 -- 172.21.15.16:0/3811190444 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faccc100f60 msgr2=0x7faccc0728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:51.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.353+0000 7faccbfff640 1 --2- 172.21.15.16:0/3811190444 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faccc100f60 0x7faccc0728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.353+0000 7faccbfff640 1 -- 172.21.15.16:0/3811190444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faccc101920 msgr2=0x7faccc072de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:51.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.353+0000 7facd09f7640 1 --2- 172.21.15.16:0/3811190444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faccc101920 0x7faccc072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:51.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.353+0000 7faccbfff640 1 --2- 172.21.15.16:0/3811190444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faccc101920 0x7faccc072de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.353+0000 7faccbfff640 1 -- 172.21.15.16:0/3811190444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facbc009560 con 0x7faccc0ffd70 2024-09-13T16:28:51.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.353+0000 7facd09f7640 1 --2- 172.21.15.16:0/3811190444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faccc101920 0x7faccc072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:28:51.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.353+0000 7faccbfff640 1 --2- 172.21.15.16:0/3811190444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faccc0ffd70 0x7faccc074260 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7facbc0048f0 tx=0x7facbc004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:51.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.353+0000 7facc97fa640 1 -- 172.21.15.16:0/3811190444 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc0044d0 con 0x7faccc0ffd70 2024-09-13T16:28:51.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.353+0000 7facc97fa640 1 -- 172.21.15.16:0/3811190444 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7facbc003a80 con 0x7faccc0ffd70 2024-09-13T16:28:51.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.353+0000 7facd245a640 1 -- 172.21.15.16:0/3811190444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faccc19ffa0 con 0x7faccc0ffd70 2024-09-13T16:28:51.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.353+0000 7facc97fa640 1 -- 172.21.15.16:0/3811190444 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc03cae0 con 0x7faccc0ffd70 2024-09-13T16:28:51.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.353+0000 7facd245a640 1 -- 172.21.15.16:0/3811190444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faccc1a03c0 con 0x7faccc0ffd70 2024-09-13T16:28:51.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.354+0000 7facd245a640 1 -- 172.21.15.16:0/3811190444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faccc107290 con 0x7faccc0ffd70 2024-09-13T16:28:51.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.356+0000 7facc97fa640 1 -- 172.21.15.16:0/3811190444 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7facbc003bf0 con 0x7faccc0ffd70 2024-09-13T16:28:51.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.356+0000 7facc97fa640 1 --2- 172.21.15.16:0/3811190444 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7facb40a1110 0x7facb40a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:51.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.357+0000 7faccb7fe640 1 --2- 172.21.15.16:0/3811190444 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7facb40a1110 0x7facb40a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:51.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.357+0000 7facc97fa640 1 -- 172.21.15.16:0/3811190444 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7facbc0edf70 con 0x7faccc0ffd70 2024-09-13T16:28:51.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.357+0000 7faccb7fe640 1 --2- 172.21.15.16:0/3811190444 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7facb40a1110 0x7facb40a35d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7facc0005d00 tx=0x7facc0005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:51.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.361+0000 7facc97fa640 1 -- 172.21.15.16:0/3811190444 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7facbc0b74b0 con 0x7faccc0ffd70 2024-09-13T16:28:51.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.534+0000 7facd245a640 1 -- 172.21.15.16:0/3811190444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faccc02c3e0 con 0x7faccc0ffd70 2024-09-13T16:28:51.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.535+0000 7facc97fa640 1 -- 172.21.15.16:0/3811190444 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7facbc035020 con 0x7faccc0ffd70 2024-09-13T16:28:51.538 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-09-13T16:28:51.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.537+0000 7facd245a640 1 -- 172.21.15.16:0/3811190444 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7facb40a1110 msgr2=0x7facb40a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:51.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.537+0000 7facd245a640 1 --2- 172.21.15.16:0/3811190444 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7facb40a1110 0x7facb40a35d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7facc0005d00 tx=0x7facc0005c50 comp rx=0 tx=0).stop 2024-09-13T16:28:51.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.537+0000 7facd245a640 1 -- 172.21.15.16:0/3811190444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faccc0ffd70 msgr2=0x7faccc074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:51.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.537+0000 7facd245a640 1 --2- 172.21.15.16:0/3811190444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faccc0ffd70 0x7faccc074260 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7facbc0048f0 tx=0x7facbc004920 comp rx=0 tx=0).stop 2024-09-13T16:28:51.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.537+0000 7facd245a640 1 -- 172.21.15.16:0/3811190444 shutdown_connections 2024-09-13T16:28:51.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.537+0000 7facd245a640 1 --2- 172.21.15.16:0/3811190444 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7facb40a1110 0x7facb40a35d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.537+0000 7facd245a640 1 --2- 172.21.15.16:0/3811190444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faccc101920 0x7faccc072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.537+0000 7facd245a640 1 --2- 172.21.15.16:0/3811190444 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faccc100f60 0x7faccc0728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.537+0000 7facd245a640 1 --2- 172.21.15.16:0/3811190444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faccc0ffd70 0x7faccc074260 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.537+0000 7facd245a640 1 -- 172.21.15.16:0/3811190444 >> 172.21.15.16:0/3811190444 conn(0x7faccc0fb4d0 msgr2=0x7faccc104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:51.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.537+0000 7facd245a640 1 -- 172.21.15.16:0/3811190444 shutdown_connections 2024-09-13T16:28:51.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.537+0000 7facd245a640 1 -- 172.21.15.16:0/3811190444 wait complete. 2024-09-13T16:28:51.550 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T16:28:51.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.655+0000 7f090c5c7640 1 -- 172.21.15.16:0/3503266679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0904105a90 msgr2=0x7f0904107ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:51.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.655+0000 7f090c5c7640 1 --2- 172.21.15.16:0/3503266679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0904105a90 0x7f0904107ec0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009ca0 tx=0x7f08fc02a8c0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.655+0000 7f090c5c7640 1 -- 172.21.15.16:0/3503266679 shutdown_connections 2024-09-13T16:28:51.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.655+0000 7f090c5c7640 1 --2- 172.21.15.16:0/3503266679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0904105a90 0x7f0904107ec0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.655+0000 7f090c5c7640 1 --2- 172.21.15.16:0/3503266679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0904103160 0x7f0904105550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.655+0000 7f090c5c7640 1 --2- 172.21.15.16:0/3503266679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0904100830 0x7f0904102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.655+0000 7f090c5c7640 1 -- 172.21.15.16:0/3503266679 >> 172.21.15.16:0/3503266679 conn(0x7f09040fa3d0 msgr2=0x7f09040fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:51.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.655+0000 7f090c5c7640 1 -- 172.21.15.16:0/3503266679 shutdown_connections 2024-09-13T16:28:51.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.655+0000 7f090c5c7640 1 -- 172.21.15.16:0/3503266679 wait complete. 2024-09-13T16:28:51.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.656+0000 7f090c5c7640 1 Processor -- start 2024-09-13T16:28:51.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.656+0000 7f090c5c7640 1 -- start start 2024-09-13T16:28:51.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.656+0000 7f090c5c7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0904100830 0x7f0904195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:51.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.656+0000 7f090c5c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0904103160 0x7f0904196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:51.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.656+0000 7f090c5c7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0904105a90 0x7f090419d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:51.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.656+0000 7f090c5c7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090419d8b0 con 0x7f0904103160 2024-09-13T16:28:51.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.656+0000 7f090c5c7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090419da20 con 0x7f0904105a90 2024-09-13T16:28:51.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.656+0000 7f090c5c7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090419dbe0 con 0x7f0904100830 2024-09-13T16:28:51.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.656+0000 7f0909b62640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0904103160 0x7f0904196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:51.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.656+0000 7f0909b62640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0904103160 0x7f0904196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44858/0 (socket says 172.21.15.16:44858) 2024-09-13T16:28:51.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.656+0000 7f0909b62640 1 -- 172.21.15.16:0/4060103254 learned_addr learned my addr 172.21.15.16:0/4060103254 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:51.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.656+0000 7f090a363640 1 --2- 172.21.15.16:0/4060103254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0904100830 0x7f0904195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:51.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.656+0000 7f090ab64640 1 --2- 172.21.15.16:0/4060103254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0904105a90 0x7f090419d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:51.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.656+0000 7f0909b62640 1 -- 172.21.15.16:0/4060103254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0904100830 msgr2=0x7f0904195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:51.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.656+0000 7f0909b62640 1 --2- 172.21.15.16:0/4060103254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0904100830 0x7f0904195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.656+0000 7f0909b62640 1 -- 172.21.15.16:0/4060103254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0904105a90 msgr2=0x7f090419d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:51.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.656+0000 7f0909b62640 1 --2- 172.21.15.16:0/4060103254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0904105a90 0x7f090419d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.657+0000 7f0909b62640 1 -- 172.21.15.16:0/4060103254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08fc009900 con 0x7f0904103160 2024-09-13T16:28:51.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.657+0000 7f0909b62640 1 --2- 172.21.15.16:0/4060103254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0904103160 0x7f0904196300 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f08f400e9f0 tx=0x7f08f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:51.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.657+0000 7f08fb7fe640 1 -- 172.21.15.16:0/4060103254 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f40096c0 con 0x7f0904103160 2024-09-13T16:28:51.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.657+0000 7f090c5c7640 1 -- 172.21.15.16:0/4060103254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f090419dec0 con 0x7f0904103160 2024-09-13T16:28:51.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.657+0000 7f08fb7fe640 1 -- 172.21.15.16:0/4060103254 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f08f400ccc0 con 0x7f0904103160 2024-09-13T16:28:51.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.657+0000 7f08fb7fe640 1 -- 172.21.15.16:0/4060103254 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f4004e60 con 0x7f0904103160 2024-09-13T16:28:51.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.657+0000 7f090c5c7640 1 -- 172.21.15.16:0/4060103254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f090419e2f0 con 0x7f0904103160 2024-09-13T16:28:51.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.658+0000 7f090c5c7640 1 -- 172.21.15.16:0/4060103254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09040fbec0 con 0x7f0904103160 2024-09-13T16:28:51.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.659+0000 7f08fb7fe640 1 -- 172.21.15.16:0/4060103254 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f08f4005020 con 0x7f0904103160 2024-09-13T16:28:51.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.660+0000 7f08fb7fe640 1 --2- 172.21.15.16:0/4060103254 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f08f00a1090 0x7f08f00a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:51.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.660+0000 7f090a363640 1 --2- 172.21.15.16:0/4060103254 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f08f00a1090 0x7f08f00a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:51.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.660+0000 7f08fb7fe640 1 -- 172.21.15.16:0/4060103254 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f08f4014070 con 0x7f0904103160 2024-09-13T16:28:51.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.660+0000 7f090a363640 1 --2- 172.21.15.16:0/4060103254 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f08f00a1090 0x7f08f00a3550 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0900009bd0 tx=0x7f0900005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:51.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.664+0000 7f08fb7fe640 1 -- 172.21.15.16:0/4060103254 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f08f4096400 con 0x7f0904103160 2024-09-13T16:28:51.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.835+0000 7f090c5c7640 1 -- 172.21.15.16:0/4060103254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f090402c410 con 0x7f0904103160 2024-09-13T16:28:51.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.836+0000 7f08fb7fe640 1 -- 172.21.15.16:0/4060103254 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7f08f4099a20 con 0x7f0904103160 2024-09-13T16:28:51.838 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-13T16:28:51.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.837+0000 7f090c5c7640 1 -- 172.21.15.16:0/4060103254 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f08f00a1090 msgr2=0x7f08f00a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:51.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.837+0000 7f090c5c7640 1 --2- 172.21.15.16:0/4060103254 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f08f00a1090 0x7f08f00a3550 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0900009bd0 tx=0x7f0900005f90 comp rx=0 tx=0).stop 2024-09-13T16:28:51.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.838+0000 7f090c5c7640 1 -- 172.21.15.16:0/4060103254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0904103160 msgr2=0x7f0904196300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:51.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.838+0000 7f090c5c7640 1 --2- 172.21.15.16:0/4060103254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0904103160 0x7f0904196300 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f08f400e9f0 tx=0x7f08f400eeb0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.838+0000 7f090c5c7640 1 -- 172.21.15.16:0/4060103254 shutdown_connections 2024-09-13T16:28:51.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.838+0000 7f090c5c7640 1 --2- 172.21.15.16:0/4060103254 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f08f00a1090 0x7f08f00a3550 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.838+0000 7f090c5c7640 1 --2- 172.21.15.16:0/4060103254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0904105a90 0x7f090419d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.838+0000 7f090c5c7640 1 --2- 172.21.15.16:0/4060103254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0904103160 0x7f0904196300 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.838+0000 7f090c5c7640 1 --2- 172.21.15.16:0/4060103254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0904100830 0x7f0904195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.838+0000 7f090c5c7640 1 -- 172.21.15.16:0/4060103254 >> 172.21.15.16:0/4060103254 conn(0x7f09040fa3d0 msgr2=0x7f0904102630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:51.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.838+0000 7f090c5c7640 1 -- 172.21.15.16:0/4060103254 shutdown_connections 2024-09-13T16:28:51.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.838+0000 7f090c5c7640 1 -- 172.21.15.16:0/4060103254 wait complete. 2024-09-13T16:28:51.849 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:51 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/4238553851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:28:51.849 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:51 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2267788140' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T16:28:51.849 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:51 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3811190444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T16:28:51.850 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T16:28:51.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:51 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/4238553851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:28:51.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:51 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/2267788140' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T16:28:51.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:51 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3811190444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T16:28:51.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.952+0000 7f065d98a640 1 -- 172.21.15.16:0/3222328505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0658105a90 msgr2=0x7f0658107f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:51.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.952+0000 7f065d98a640 1 --2- 172.21.15.16:0/3222328505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0658105a90 0x7f0658107f00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0650009ca0 tx=0x7f065002a8c0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.952+0000 7f065d98a640 1 -- 172.21.15.16:0/3222328505 shutdown_connections 2024-09-13T16:28:51.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.952+0000 7f065d98a640 1 --2- 172.21.15.16:0/3222328505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0658105a90 0x7f0658107f00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.952+0000 7f065d98a640 1 --2- 172.21.15.16:0/3222328505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0658103160 0x7f0658105550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.952+0000 7f065d98a640 1 --2- 172.21.15.16:0/3222328505 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0658100830 0x7f0658102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.952+0000 7f065d98a640 1 -- 172.21.15.16:0/3222328505 >> 172.21.15.16:0/3222328505 conn(0x7f06580fa430 msgr2=0x7f06580fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:51.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.952+0000 7f065d98a640 1 -- 172.21.15.16:0/3222328505 shutdown_connections 2024-09-13T16:28:51.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.952+0000 7f065d98a640 1 -- 172.21.15.16:0/3222328505 wait complete. 2024-09-13T16:28:51.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.952+0000 7f065d98a640 1 Processor -- start 2024-09-13T16:28:51.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.953+0000 7f065d98a640 1 -- start start 2024-09-13T16:28:51.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.953+0000 7f065d98a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0658100830 0x7f06581001a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:51.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.953+0000 7f065d98a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0658103160 0x7f06580fe7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:51.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.953+0000 7f065d98a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0658105a90 0x7f06580fed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:51.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.953+0000 7f065d98a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06580ff480 con 0x7f0658105a90 2024-09-13T16:28:51.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.953+0000 7f065d98a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06580ff5f0 con 0x7f0658100830 2024-09-13T16:28:51.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.953+0000 7f065d98a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065819fd10 con 0x7f0658103160 2024-09-13T16:28:51.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.953+0000 7f06577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0658105a90 0x7f06580fed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:51.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.953+0000 7f06577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0658105a90 0x7f06580fed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44866/0 (socket says 172.21.15.16:44866) 2024-09-13T16:28:51.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.953+0000 7f06577fe640 1 -- 172.21.15.16:0/4186578390 learned_addr learned my addr 172.21.15.16:0/4186578390 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:51.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.953+0000 7f0656ffd640 1 --2- 172.21.15.16:0/4186578390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0658100830 0x7f06581001a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:51.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.953+0000 7f06567fc640 1 --2- 172.21.15.16:0/4186578390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0658103160 0x7f06580fe7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:51.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.953+0000 7f06577fe640 1 -- 172.21.15.16:0/4186578390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0658103160 msgr2=0x7f06580fe7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:51.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.953+0000 7f06577fe640 1 --2- 172.21.15.16:0/4186578390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0658103160 0x7f06580fe7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.953+0000 7f06577fe640 1 -- 172.21.15.16:0/4186578390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0658100830 msgr2=0x7f06581001a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:51.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.953+0000 7f06577fe640 1 --2- 172.21.15.16:0/4186578390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0658100830 0x7f06581001a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:51.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.953+0000 7f06577fe640 1 -- 172.21.15.16:0/4186578390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0650009900 con 0x7f0658105a90 2024-09-13T16:28:51.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.953+0000 7f06567fc640 1 --2- 172.21.15.16:0/4186578390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0658103160 0x7f06580fe7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:28:51.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.953+0000 7f06577fe640 1 --2- 172.21.15.16:0/4186578390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0658105a90 0x7f06580fed20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0650033020 tx=0x7f0650030f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:51.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.954+0000 7f065c988640 1 -- 172.21.15.16:0/4186578390 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0650003b70 con 0x7f0658105a90 2024-09-13T16:28:51.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.954+0000 7f065c988640 1 -- 172.21.15.16:0/4186578390 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0650040070 con 0x7f0658105a90 2024-09-13T16:28:51.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.954+0000 7f065c988640 1 -- 172.21.15.16:0/4186578390 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0650004380 con 0x7f0658105a90 2024-09-13T16:28:51.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.954+0000 7f065d98a640 1 -- 172.21.15.16:0/4186578390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f065819ff60 con 0x7f0658105a90 2024-09-13T16:28:51.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.954+0000 7f065d98a640 1 -- 172.21.15.16:0/4186578390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06581a03e0 con 0x7f0658105a90 2024-09-13T16:28:51.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.955+0000 7f065d98a640 1 -- 172.21.15.16:0/4186578390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06580fbf20 con 0x7f0658105a90 2024-09-13T16:28:51.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.956+0000 7f065c988640 1 -- 172.21.15.16:0/4186578390 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f0650042030 con 0x7f0658105a90 2024-09-13T16:28:51.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.957+0000 7f065c988640 1 --2- 172.21.15.16:0/4186578390 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f06440a1090 0x7f06440a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:51.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.957+0000 7f0656ffd640 1 --2- 172.21.15.16:0/4186578390 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f06440a1090 0x7f06440a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:51.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.957+0000 7f065c988640 1 -- 172.21.15.16:0/4186578390 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f06500ee190 con 0x7f0658105a90 2024-09-13T16:28:51.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.957+0000 7f0656ffd640 1 --2- 172.21.15.16:0/4186578390 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f06440a1090 0x7f06440a3550 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0648009c50 tx=0x7f0648005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:51.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:51.960+0000 7f065c988640 1 -- 172.21.15.16:0/4186578390 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f06500b76d0 con 0x7f0658105a90 2024-09-13T16:28:52.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:51 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/4238553851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:28:52.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:51 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/2267788140' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T16:28:52.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:51 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3811190444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T16:28:52.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.143+0000 7f065d98a640 1 -- 172.21.15.16:0/4186578390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f065810b560 con 0x7f0658105a90 2024-09-13T16:28:52.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.143+0000 7f065c988640 1 -- 172.21.15.16:0/4186578390 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) v1 ==== 81+0+589 (secure 0 0 0) 0x7f06500bacf0 con 0x7f0658105a90 2024-09-13T16:28:52.146 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2098378667 2024-09-14T16:22:04.485277+0000 2024-09-13T16:28:52.146 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3536229016 2024-09-14T16:22:04.485277+0000 2024-09-13T16:28:52.146 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/1756940569 2024-09-14T16:22:04.485277+0000 2024-09-13T16:28:52.146 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2684687547 2024-09-14T16:22:04.485277+0000 2024-09-13T16:28:52.146 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/1756940569 2024-09-14T16:22:04.485277+0000 2024-09-13T16:28:52.146 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/368410612 2024-09-14T16:21:40.126459+0000 2024-09-13T16:28:52.147 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/12158160 2024-09-14T16:21:40.126459+0000 2024-09-13T16:28:52.147 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/1147765163 2024-09-14T16:21:40.126459+0000 2024-09-13T16:28:52.147 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/1147765163 2024-09-14T16:21:40.126459+0000 2024-09-13T16:28:52.147 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3754423379 2024-09-14T16:21:40.126459+0000 2024-09-13T16:28:52.149 INFO:teuthology.orchestra.run.smithi016.stderr:listed 10 entries 2024-09-13T16:28:52.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.145+0000 7f065d98a640 1 -- 172.21.15.16:0/4186578390 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f06440a1090 msgr2=0x7f06440a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:52.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.145+0000 7f065d98a640 1 --2- 172.21.15.16:0/4186578390 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f06440a1090 0x7f06440a3550 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0648009c50 tx=0x7f0648005f90 comp rx=0 tx=0).stop 2024-09-13T16:28:52.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.145+0000 7f065d98a640 1 -- 172.21.15.16:0/4186578390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0658105a90 msgr2=0x7f06580fed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:52.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.145+0000 7f065d98a640 1 --2- 172.21.15.16:0/4186578390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0658105a90 0x7f06580fed20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0650033020 tx=0x7f0650030f20 comp rx=0 tx=0).stop 2024-09-13T16:28:52.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.145+0000 7f065d98a640 1 -- 172.21.15.16:0/4186578390 shutdown_connections 2024-09-13T16:28:52.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.145+0000 7f065d98a640 1 --2- 172.21.15.16:0/4186578390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0658105a90 0x7f06580fed20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:52.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.145+0000 7f065d98a640 1 --2- 172.21.15.16:0/4186578390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0658103160 0x7f06580fe7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:52.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.145+0000 7f065d98a640 1 --2- 172.21.15.16:0/4186578390 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f06440a1090 0x7f06440a3550 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:52.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.145+0000 7f065d98a640 1 --2- 172.21.15.16:0/4186578390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0658100830 0x7f06581001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:52.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.145+0000 7f065d98a640 1 -- 172.21.15.16:0/4186578390 >> 172.21.15.16:0/4186578390 conn(0x7f06580fa430 msgr2=0x7f0658101090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:52.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.145+0000 7f065d98a640 1 -- 172.21.15.16:0/4186578390 shutdown_connections 2024-09-13T16:28:52.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.145+0000 7f065d98a640 1 -- 172.21.15.16:0/4186578390 wait complete. 2024-09-13T16:28:52.158 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-13T16:28:52.158 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:28:52.176 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:28:52.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.316+0000 7ff107f53640 1 -- 172.21.15.16:0/1923540448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff100100f90 msgr2=0x7ff100101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:52.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.316+0000 7ff107f53640 1 --2- 172.21.15.16:0/1923540448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff100100f90 0x7ff100101410 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0009900 tx=0x7ff0f002a410 comp rx=0 tx=0).stop 2024-09-13T16:28:52.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.316+0000 7ff107f53640 1 -- 172.21.15.16:0/1923540448 shutdown_connections 2024-09-13T16:28:52.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.316+0000 7ff107f53640 1 --2- 172.21.15.16:0/1923540448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff100101950 0x7ff1001081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:52.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.316+0000 7ff107f53640 1 --2- 172.21.15.16:0/1923540448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff100100f90 0x7ff100101410 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:52.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.316+0000 7ff107f53640 1 --2- 172.21.15.16:0/1923540448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1000ffda0 0x7ff1001001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:52.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.316+0000 7ff107f53640 1 -- 172.21.15.16:0/1923540448 >> 172.21.15.16:0/1923540448 conn(0x7ff1000fb520 msgr2=0x7ff1000fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:52.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.317+0000 7ff107f53640 1 -- 172.21.15.16:0/1923540448 shutdown_connections 2024-09-13T16:28:52.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.317+0000 7ff107f53640 1 -- 172.21.15.16:0/1923540448 wait complete. 2024-09-13T16:28:52.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.317+0000 7ff107f53640 1 Processor -- start 2024-09-13T16:28:52.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.317+0000 7ff107f53640 1 -- start start 2024-09-13T16:28:52.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.317+0000 7ff107f53640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1000ffda0 0x7ff100195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:52.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.317+0000 7ff107f53640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff100100f90 0x7ff100196230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:52.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.317+0000 7ff107f53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff100101950 0x7ff10019d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:52.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.317+0000 7ff107f53640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10019d7b0 con 0x7ff100101950 2024-09-13T16:28:52.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.317+0000 7ff107f53640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10019d920 con 0x7ff100100f90 2024-09-13T16:28:52.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.317+0000 7ff107f53640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10019dae0 con 0x7ff1000ffda0 2024-09-13T16:28:52.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.317+0000 7ff1064f0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff100101950 0x7ff10019d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:52.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.317+0000 7ff1064f0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff100101950 0x7ff10019d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44880/0 (socket says 172.21.15.16:44880) 2024-09-13T16:28:52.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.317+0000 7ff1064f0640 1 -- 172.21.15.16:0/3601289937 learned_addr learned my addr 172.21.15.16:0/3601289937 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:52.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.318+0000 7ff1054ee640 1 --2- 172.21.15.16:0/3601289937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff100100f90 0x7ff100196230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:52.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.318+0000 7ff105cef640 1 --2- 172.21.15.16:0/3601289937 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1000ffda0 0x7ff100195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:52.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.318+0000 7ff1064f0640 1 -- 172.21.15.16:0/3601289937 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1000ffda0 msgr2=0x7ff100195cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:52.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.318+0000 7ff1064f0640 1 --2- 172.21.15.16:0/3601289937 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1000ffda0 0x7ff100195cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:52.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.318+0000 7ff1064f0640 1 -- 172.21.15.16:0/3601289937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff100100f90 msgr2=0x7ff100196230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:52.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.318+0000 7ff1064f0640 1 --2- 172.21.15.16:0/3601289937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff100100f90 0x7ff100196230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:52.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.318+0000 7ff1064f0640 1 -- 172.21.15.16:0/3601289937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f0009560 con 0x7ff100101950 2024-09-13T16:28:52.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.318+0000 7ff1054ee640 1 --2- 172.21.15.16:0/3601289937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff100100f90 0x7ff100196230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:28:52.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.318+0000 7ff1064f0640 1 --2- 172.21.15.16:0/3601289937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff100101950 0x7ff10019d240 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800eaa0 tx=0x7ff0f800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:52.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.318+0000 7ff0f6ffd640 1 -- 172.21.15.16:0/3601289937 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f800cdb0 con 0x7ff100101950 2024-09-13T16:28:52.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.318+0000 7ff0f6ffd640 1 -- 172.21.15.16:0/3601289937 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0f8005020 con 0x7ff100101950 2024-09-13T16:28:52.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.318+0000 7ff0f6ffd640 1 -- 172.21.15.16:0/3601289937 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f80116d0 con 0x7ff100101950 2024-09-13T16:28:52.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.318+0000 7ff107f53640 1 -- 172.21.15.16:0/3601289937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff10019ddc0 con 0x7ff100101950 2024-09-13T16:28:52.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.318+0000 7ff107f53640 1 -- 172.21.15.16:0/3601289937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff10019e1f0 con 0x7ff100101950 2024-09-13T16:28:52.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.319+0000 7ff107f53640 1 -- 172.21.15.16:0/3601289937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1001072d0 con 0x7ff100101950 2024-09-13T16:28:52.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.320+0000 7ff0f6ffd640 1 -- 172.21.15.16:0/3601289937 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff0f8005190 con 0x7ff100101950 2024-09-13T16:28:52.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.321+0000 7ff0f6ffd640 1 --2- 172.21.15.16:0/3601289937 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff0ec0a1110 0x7ff0ec0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:52.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.321+0000 7ff105cef640 1 --2- 172.21.15.16:0/3601289937 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff0ec0a1110 0x7ff0ec0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:52.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.321+0000 7ff0f6ffd640 1 -- 172.21.15.16:0/3601289937 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff0f80ccd80 con 0x7ff100101950 2024-09-13T16:28:52.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.322+0000 7ff105cef640 1 --2- 172.21.15.16:0/3601289937 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff0ec0a1110 0x7ff0ec0a35d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff1000fec10 tx=0x7ff0fc005c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:52.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.325+0000 7ff0f6ffd640 1 -- 172.21.15.16:0/3601289937 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff0f80962c0 con 0x7ff100101950 2024-09-13T16:28:52.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.496+0000 7ff107f53640 1 -- 172.21.15.16:0/3601289937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff10002c480 con 0x7ff100101950 2024-09-13T16:28:52.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.497+0000 7ff0f6ffd640 1 -- 172.21.15.16:0/3601289937 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23971 (secure 0 0 0) 0x7ff0f801f020 con 0x7ff100101950 2024-09-13T16:28:52.500 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:28:52.500 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":89,"fsid":"0d241f6a-71ec-11ef-bceb-c7b262605968","created":"2024-09-13T16:21:17.971738+0000","modified":"2024-09-13T16:28:43.574106+0000","last_up_change":"2024-09-13T16:27:54.673413+0000","last_in_change":"2024-09-13T16:27:35.718390+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:24:22.981019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:28:37.746027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:28:38.602731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:28:41.633322+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e11ef8f-08d6-4e74-afd6-f49f4bd40032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6803","nonce":2079394186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6805","nonce":2079394186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6809","nonce":2079394186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6807","nonce":2079394186}]},"public_addr":"172.21.15.16:6803/2079394186","cluster_addr":"172.21.15.16:6805/2079394186","heartbeat_back_addr":"172.21.15.16:6809/2079394186","heartbeat_front_addr":"172.21.15.16:6807/2079394186","state":["exists","up"]},{"osd":1,"uuid":"f2cd3694-9c9e-4da7-976e-f80a8df120f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6801","nonce":4021360958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6803","nonce":4021360958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6807","nonce":4021360958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6805","nonce":4021360958}]},"public_addr":"172.21.15.107:6801/4021360958","cluster_addr":"172.21.15.107:6803/4021360958","heartbeat_back_addr":"172.21.15.107:6807/4021360958","heartbeat_front_addr":"172.21.15.107:6805/4021360958","state":["exists","up"]},{"osd":2,"uuid":"a5bbf2eb-6599-44d5-ad96-68bb89ee19f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6801","nonce":3227179825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6803","nonce":3227179825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6807","nonce":3227179825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6805","nonce":3227179825}]},"public_addr":"172.21.15.111:6801/3227179825","cluster_addr":"172.21.15.111:6803/3227179825","heartbeat_back_addr":"172.21.15.111:6807/3227179825","heartbeat_front_addr":"172.21.15.111:6805/3227179825","state":["exists","up"]},{"osd":3,"uuid":"d6295ffe-11bc-4964-a250-9fecca870b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6811","nonce":2390375937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6813","nonce":2390375937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6817","nonce":2390375937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6815","nonce":2390375937}]},"public_addr":"172.21.15.16:6811/2390375937","cluster_addr":"172.21.15.16:6813/2390375937","heartbeat_back_addr":"172.21.15.16:6817/2390375937","heartbeat_front_addr":"172.21.15.16:6815/2390375937","state":["exists","up"]},{"osd":4,"uuid":"0a8446e0-fc86-4646-beb6-a7b9d433093d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6809","nonce":1936582048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6811","nonce":1936582048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6815","nonce":1936582048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6813","nonce":1936582048}]},"public_addr":"172.21.15.107:6809/1936582048","cluster_addr":"172.21.15.107:6811/1936582048","heartbeat_back_addr":"172.21.15.107:6815/1936582048","heartbeat_front_addr":"172.21.15.107:6813/1936582048","state":["exists","up"]},{"osd":5,"uuid":"2c97a446-357e-4975-9e51-9c95fe84a663","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6809","nonce":483858448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6811","nonce":483858448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6815","nonce":483858448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6813","nonce":483858448}]},"public_addr":"172.21.15.111:6809/483858448","cluster_addr":"172.21.15.111:6811/483858448","heartbeat_back_addr":"172.21.15.111:6815/483858448","heartbeat_front_addr":"172.21.15.111:6813/483858448","state":["exists","up"]},{"osd":6,"uuid":"890239e6-e6fb-4f9c-b973-05cc66d199bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6819","nonce":1330214141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6821","nonce":1330214141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6825","nonce":1330214141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6823","nonce":1330214141}]},"public_addr":"172.21.15.16:6819/1330214141","cluster_addr":"172.21.15.16:6821/1330214141","heartbeat_back_addr":"172.21.15.16:6825/1330214141","heartbeat_front_addr":"172.21.15.16:6823/1330214141","state":["exists","up"]},{"osd":7,"uuid":"ee826a9c-f441-4903-9b1b-66b627f7b712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6817","nonce":1133876754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6819","nonce":1133876754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6823","nonce":1133876754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6821","nonce":1133876754}]},"public_addr":"172.21.15.107:6817/1133876754","cluster_addr":"172.21.15.107:6819/1133876754","heartbeat_back_addr":"172.21.15.107:6823/1133876754","heartbeat_front_addr":"172.21.15.107:6821/1133876754","state":["exists","up"]},{"osd":8,"uuid":"e19773f2-dec8-44e0-9d33-0252e07d6415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6817","nonce":837548190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6819","nonce":837548190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6823","nonce":837548190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6821","nonce":837548190}]},"public_addr":"172.21.15.111:6817/837548190","cluster_addr":"172.21.15.111:6819/837548190","heartbeat_back_addr":"172.21.15.111:6823/837548190","heartbeat_front_addr":"172.21.15.111:6821/837548190","state":["exists","up"]},{"osd":9,"uuid":"bae4d6db-8b90-4d02-8e83-7d8f2a6f8fdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6827","nonce":4175383686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6829","nonce":4175383686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6833","nonce":4175383686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6831","nonce":4175383686}]},"public_addr":"172.21.15.16:6827/4175383686","cluster_addr":"172.21.15.16:6829/4175383686","heartbeat_back_addr":"172.21.15.16:6833/4175383686","heartbeat_front_addr":"172.21.15.16:6831/4175383686","state":["exists","up"]},{"osd":10,"uuid":"fd0e7740-76c8-4c99-a349-7077b4905143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6825","nonce":2659550426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6827","nonce":2659550426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6831","nonce":2659550426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6829","nonce":2659550426}]},"public_addr":"172.21.15.107:6825/2659550426","cluster_addr":"172.21.15.107:6827/2659550426","heartbeat_back_addr":"172.21.15.107:6831/2659550426","heartbeat_front_addr":"172.21.15.107:6829/2659550426","state":["exists","up"]},{"osd":11,"uuid":"94a1751d-b00c-4e23-815d-3a8bc3dea146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6825","nonce":1633548683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6827","nonce":1633548683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6831","nonce":1633548683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6829","nonce":1633548683}]},"public_addr":"172.21.15.111:6825/1633548683","cluster_addr":"172.21.15.111:6827/1633548683","heartbeat_back_addr":"172.21.15.111:6831/1633548683","heartbeat_front_addr":"172.21.15.111:6829/1633548683","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:36.359852+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:58.014202+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:19.753672+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:43.404817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:06.001677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:29.361587+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:52.554847+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:15.957553+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:39.581093+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:04.911389+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:28.971280+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:53.473561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/2098378667":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/3536229016":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6801/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/2684687547":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6800/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/368410612":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/12158160":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6800/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6801/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/3754423379":"2024-09-14T16:21:40.126459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:28:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.499+0000 7ff107f53640 1 -- 172.21.15.16:0/3601289937 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff0ec0a1110 msgr2=0x7ff0ec0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.499+0000 7ff107f53640 1 --2- 172.21.15.16:0/3601289937 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff0ec0a1110 0x7ff0ec0a35d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff1000fec10 tx=0x7ff0fc005c80 comp rx=0 tx=0).stop 2024-09-13T16:28:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.499+0000 7ff107f53640 1 -- 172.21.15.16:0/3601289937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff100101950 msgr2=0x7ff10019d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.499+0000 7ff107f53640 1 --2- 172.21.15.16:0/3601289937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff100101950 0x7ff10019d240 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800eaa0 tx=0x7ff0f800ef60 comp rx=0 tx=0).stop 2024-09-13T16:28:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.499+0000 7ff107f53640 1 -- 172.21.15.16:0/3601289937 shutdown_connections 2024-09-13T16:28:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.499+0000 7ff107f53640 1 --2- 172.21.15.16:0/3601289937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff100101950 0x7ff10019d240 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.499+0000 7ff107f53640 1 --2- 172.21.15.16:0/3601289937 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff0ec0a1110 0x7ff0ec0a35d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.499+0000 7ff107f53640 1 --2- 172.21.15.16:0/3601289937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff100100f90 0x7ff100196230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.499+0000 7ff107f53640 1 --2- 172.21.15.16:0/3601289937 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1000ffda0 0x7ff100195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.499+0000 7ff107f53640 1 -- 172.21.15.16:0/3601289937 >> 172.21.15.16:0/3601289937 conn(0x7ff1000fb520 msgr2=0x7ff1001044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.500+0000 7ff107f53640 1 -- 172.21.15.16:0/3601289937 shutdown_connections 2024-09-13T16:28:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.500+0000 7ff107f53640 1 -- 172.21.15.16:0/3601289937 wait complete. 2024-09-13T16:28:52.512 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:24:22.981019+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T16:28:37.746027+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T16:28:38.602731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T16:28:41.633322+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T16:28:52.513 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:28:52.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.617+0000 7fcc11496640 1 -- 172.21.15.16:0/410607245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc0c101920 msgr2=0x7fcc0c1081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:52.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.617+0000 7fcc11496640 1 --2- 172.21.15.16:0/410607245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc0c101920 0x7fcc0c1081b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcc04009ca0 tx=0x7fcc0402a8c0 comp rx=0 tx=0).stop 2024-09-13T16:28:52.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.617+0000 7fcc11496640 1 -- 172.21.15.16:0/410607245 shutdown_connections 2024-09-13T16:28:52.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.617+0000 7fcc11496640 1 --2- 172.21.15.16:0/410607245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc0c101920 0x7fcc0c1081b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:52.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.617+0000 7fcc11496640 1 --2- 172.21.15.16:0/410607245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc0c100f60 0x7fcc0c1013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:52.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.617+0000 7fcc11496640 1 --2- 172.21.15.16:0/410607245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc0c0ffd70 0x7fcc0c100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:52.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.617+0000 7fcc11496640 1 -- 172.21.15.16:0/410607245 >> 172.21.15.16:0/410607245 conn(0x7fcc0c0fb4d0 msgr2=0x7fcc0c0fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:52.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.617+0000 7fcc11496640 1 -- 172.21.15.16:0/410607245 shutdown_connections 2024-09-13T16:28:52.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.617+0000 7fcc11496640 1 -- 172.21.15.16:0/410607245 wait complete. 2024-09-13T16:28:52.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.617+0000 7fcc11496640 1 Processor -- start 2024-09-13T16:28:52.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.618+0000 7fcc11496640 1 -- start start 2024-09-13T16:28:52.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.618+0000 7fcc11496640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc0c0ffd70 0x7fcc0c074260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:52.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.618+0000 7fcc11496640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc0c100f60 0x7fcc0c0728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:52.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.618+0000 7fcc11496640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc0c101920 0x7fcc0c072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:52.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.618+0000 7fcc11496640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c073540 con 0x7fcc0c101920 2024-09-13T16:28:52.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.618+0000 7fcc11496640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c0736b0 con 0x7fcc0c100f60 2024-09-13T16:28:52.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.618+0000 7fcc11496640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c19fb30 con 0x7fcc0c0ffd70 2024-09-13T16:28:52.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.618+0000 7fcc0b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc0c101920 0x7fcc0c072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:52.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.618+0000 7fcc0b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc0c101920 0x7fcc0c072de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44892/0 (socket says 172.21.15.16:44892) 2024-09-13T16:28:52.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.618+0000 7fcc0b7fe640 1 -- 172.21.15.16:0/2689507158 learned_addr learned my addr 172.21.15.16:0/2689507158 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:52.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.618+0000 7fcc0affd640 1 --2- 172.21.15.16:0/2689507158 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc0c0ffd70 0x7fcc0c074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:52.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.618+0000 7fcc0a7fc640 1 --2- 172.21.15.16:0/2689507158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc0c100f60 0x7fcc0c0728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:52.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.618+0000 7fcc0b7fe640 1 -- 172.21.15.16:0/2689507158 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc0c0ffd70 msgr2=0x7fcc0c074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:52.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.618+0000 7fcc0b7fe640 1 --2- 172.21.15.16:0/2689507158 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc0c0ffd70 0x7fcc0c074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:52.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.618+0000 7fcc0b7fe640 1 -- 172.21.15.16:0/2689507158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc0c100f60 msgr2=0x7fcc0c0728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:52.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.618+0000 7fcc0b7fe640 1 --2- 172.21.15.16:0/2689507158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc0c100f60 0x7fcc0c0728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:52.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.618+0000 7fcc0b7fe640 1 -- 172.21.15.16:0/2689507158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc04009900 con 0x7fcc0c101920 2024-09-13T16:28:52.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.618+0000 7fcc0b7fe640 1 --2- 172.21.15.16:0/2689507158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc0c101920 0x7fcc0c072de0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcc04003bd0 tx=0x7fcc04003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:52.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.619+0000 7fcbf3fff640 1 -- 172.21.15.16:0/2689507158 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc040307c0 con 0x7fcc0c101920 2024-09-13T16:28:52.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.619+0000 7fcbf3fff640 1 -- 172.21.15.16:0/2689507158 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcc04040070 con 0x7fcc0c101920 2024-09-13T16:28:52.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.619+0000 7fcc11496640 1 -- 172.21.15.16:0/2689507158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc0c19fd80 con 0x7fcc0c101920 2024-09-13T16:28:52.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.619+0000 7fcbf3fff640 1 -- 172.21.15.16:0/2689507158 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc04004620 con 0x7fcc0c101920 2024-09-13T16:28:52.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.619+0000 7fcc11496640 1 -- 172.21.15.16:0/2689507158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc0c1a0230 con 0x7fcc0c101920 2024-09-13T16:28:52.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.620+0000 7fcc11496640 1 -- 172.21.15.16:0/2689507158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc0c107160 con 0x7fcc0c101920 2024-09-13T16:28:52.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.621+0000 7fcbf3fff640 1 -- 172.21.15.16:0/2689507158 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fcc04042030 con 0x7fcc0c101920 2024-09-13T16:28:52.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.622+0000 7fcbf3fff640 1 --2- 172.21.15.16:0/2689507158 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fcbf40a1090 0x7fcbf40a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:52.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.622+0000 7fcc0affd640 1 --2- 172.21.15.16:0/2689507158 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fcbf40a1090 0x7fcbf40a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:52.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.622+0000 7fcbf3fff640 1 -- 172.21.15.16:0/2689507158 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fcc040ee370 con 0x7fcc0c101920 2024-09-13T16:28:52.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.622+0000 7fcc0affd640 1 --2- 172.21.15.16:0/2689507158 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fcbf40a1090 0x7fcbf40a3550 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c100dc0 tx=0x7fcbfc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:52.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.626+0000 7fcbf3fff640 1 -- 172.21.15.16:0/2689507158 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fcc040b7910 con 0x7fcc0c101920 2024-09-13T16:28:52.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.801+0000 7fcc11496640 1 -- 172.21.15.16:0/2689507158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcc0c02c480 con 0x7fcc0c101920 2024-09-13T16:28:52.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.801+0000 7fcbf3fff640 1 -- 172.21.15.16:0/2689507158 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcc040baf30 con 0x7fcc0c101920 2024-09-13T16:28:52.803 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-13T16:28:52.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.803+0000 7fcc11496640 1 -- 172.21.15.16:0/2689507158 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fcbf40a1090 msgr2=0x7fcbf40a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:52.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.803+0000 7fcc11496640 1 --2- 172.21.15.16:0/2689507158 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fcbf40a1090 0x7fcbf40a3550 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c100dc0 tx=0x7fcbfc005f90 comp rx=0 tx=0).stop 2024-09-13T16:28:52.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.803+0000 7fcc11496640 1 -- 172.21.15.16:0/2689507158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc0c101920 msgr2=0x7fcc0c072de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:52.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.803+0000 7fcc11496640 1 --2- 172.21.15.16:0/2689507158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc0c101920 0x7fcc0c072de0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcc04003bd0 tx=0x7fcc04003c00 comp rx=0 tx=0).stop 2024-09-13T16:28:52.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.803+0000 7fcc11496640 1 -- 172.21.15.16:0/2689507158 shutdown_connections 2024-09-13T16:28:52.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.803+0000 7fcc11496640 1 --2- 172.21.15.16:0/2689507158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc0c101920 0x7fcc0c072de0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:52.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.803+0000 7fcc11496640 1 --2- 172.21.15.16:0/2689507158 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fcbf40a1090 0x7fcbf40a3550 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:52.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.803+0000 7fcc11496640 1 --2- 172.21.15.16:0/2689507158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc0c100f60 0x7fcc0c0728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:52.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.803+0000 7fcc11496640 1 --2- 172.21.15.16:0/2689507158 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc0c0ffd70 0x7fcc0c074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:52.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.803+0000 7fcc11496640 1 -- 172.21.15.16:0/2689507158 >> 172.21.15.16:0/2689507158 conn(0x7fcc0c0fb4d0 msgr2=0x7fcc0c104330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:52.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.803+0000 7fcc11496640 1 -- 172.21.15.16:0/2689507158 shutdown_connections 2024-09-13T16:28:52.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.804+0000 7fcc11496640 1 -- 172.21.15.16:0/2689507158 wait complete. 2024-09-13T16:28:52.815 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:52 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/4060103254' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T16:28:52.815 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:52 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/4186578390' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T16:28:52.815 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:52 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3601289937' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:28:52.815 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:52 smithi016 ceph-mon[31187]: pgmap v283: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 328 B/s wr, 0 op/s 2024-09-13T16:28:52.817 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T16:28:52.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.919+0000 7f8a9fd33640 1 -- 172.21.15.16:0/2500128294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a980ff8e0 msgr2=0x7f8a980ffd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:52.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.919+0000 7f8a9fd33640 1 --2- 172.21.15.16:0/2500128294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a980ff8e0 0x7f8a980ffd90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8a90009ca0 tx=0x7f8a9002a8c0 comp rx=0 tx=0).stop 2024-09-13T16:28:52.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.919+0000 7f8a9fd33640 1 -- 172.21.15.16:0/2500128294 shutdown_connections 2024-09-13T16:28:52.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.919+0000 7f8a9fd33640 1 --2- 172.21.15.16:0/2500128294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a980ff8e0 0x7f8a980ffd90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:52.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.919+0000 7f8a9fd33640 1 --2- 172.21.15.16:0/2500128294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a980fef40 0x7f8a980ff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:52.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.919+0000 7f8a9fd33640 1 --2- 172.21.15.16:0/2500128294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a980fe600 0x7f8a980fea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:52.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.919+0000 7f8a9fd33640 1 -- 172.21.15.16:0/2500128294 >> 172.21.15.16:0/2500128294 conn(0x7f8a980fa3d0 msgr2=0x7f8a980fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:52.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.920+0000 7f8a9fd33640 1 -- 172.21.15.16:0/2500128294 shutdown_connections 2024-09-13T16:28:52.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.920+0000 7f8a9fd33640 1 -- 172.21.15.16:0/2500128294 wait complete. 2024-09-13T16:28:52.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.920+0000 7f8a9fd33640 1 Processor -- start 2024-09-13T16:28:52.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.920+0000 7f8a9fd33640 1 -- start start 2024-09-13T16:28:52.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.920+0000 7f8a9fd33640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a980fe600 0x7f8a98197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:52.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.920+0000 7f8a9fd33640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a980fef40 0x7f8a981982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:52.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.920+0000 7f8a9fd33640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a980ff8e0 0x7f8a9819f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:52.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.920+0000 7f8a9fd33640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9819f880 con 0x7f8a980ff8e0 2024-09-13T16:28:52.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.920+0000 7f8a9fd33640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9819f9f0 con 0x7f8a980fef40 2024-09-13T16:28:52.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.920+0000 7f8a9fd33640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9819fbb0 con 0x7f8a980fe600 2024-09-13T16:28:52.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.920+0000 7f8a9e2d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a980ff8e0 0x7f8a9819f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:52.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.920+0000 7f8a9e2d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a980ff8e0 0x7f8a9819f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44910/0 (socket says 172.21.15.16:44910) 2024-09-13T16:28:52.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.920+0000 7f8a9e2d0640 1 -- 172.21.15.16:0/3805508310 learned_addr learned my addr 172.21.15.16:0/3805508310 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:52.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.920+0000 7f8a9d2ce640 1 --2- 172.21.15.16:0/3805508310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a980fef40 0x7f8a981982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:52.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.920+0000 7f8a9dacf640 1 --2- 172.21.15.16:0/3805508310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a980fe600 0x7f8a98197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:52.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.921+0000 7f8a9e2d0640 1 -- 172.21.15.16:0/3805508310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a980fe600 msgr2=0x7f8a98197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:52.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.921+0000 7f8a9e2d0640 1 --2- 172.21.15.16:0/3805508310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a980fe600 0x7f8a98197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:52.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.921+0000 7f8a9e2d0640 1 -- 172.21.15.16:0/3805508310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a980fef40 msgr2=0x7f8a981982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:52.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.921+0000 7f8a9e2d0640 1 --2- 172.21.15.16:0/3805508310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a980fef40 0x7f8a981982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:52.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.921+0000 7f8a9e2d0640 1 -- 172.21.15.16:0/3805508310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a90009900 con 0x7f8a980ff8e0 2024-09-13T16:28:52.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.921+0000 7f8a9e2d0640 1 --2- 172.21.15.16:0/3805508310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a980ff8e0 0x7f8a9819f310 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8a9002aeb0 tx=0x7f8a90003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:52.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.921+0000 7f8a8effd640 1 -- 172.21.15.16:0/3805508310 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a900307c0 con 0x7f8a980ff8e0 2024-09-13T16:28:52.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.921+0000 7f8a8effd640 1 -- 172.21.15.16:0/3805508310 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8a90040070 con 0x7f8a980ff8e0 2024-09-13T16:28:52.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.921+0000 7f8a9fd33640 1 -- 172.21.15.16:0/3805508310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a9819fe30 con 0x7f8a980ff8e0 2024-09-13T16:28:52.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.921+0000 7f8a8effd640 1 -- 172.21.15.16:0/3805508310 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a900043c0 con 0x7f8a980ff8e0 2024-09-13T16:28:52.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.921+0000 7f8a9fd33640 1 -- 172.21.15.16:0/3805508310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a981a02b0 con 0x7f8a980ff8e0 2024-09-13T16:28:52.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.922+0000 7f8a9fd33640 1 -- 172.21.15.16:0/3805508310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a981094e0 con 0x7f8a980ff8e0 2024-09-13T16:28:52.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.923+0000 7f8a8effd640 1 -- 172.21.15.16:0/3805508310 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8a90042030 con 0x7f8a980ff8e0 2024-09-13T16:28:52.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.924+0000 7f8a8effd640 1 --2- 172.21.15.16:0/3805508310 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8a840a1110 0x7f8a840a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:52.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.924+0000 7f8a8effd640 1 -- 172.21.15.16:0/3805508310 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8a900ee240 con 0x7f8a980ff8e0 2024-09-13T16:28:52.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.924+0000 7f8a9dacf640 1 --2- 172.21.15.16:0/3805508310 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8a840a1110 0x7f8a840a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:52.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.925+0000 7f8a9dacf640 1 --2- 172.21.15.16:0/3805508310 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8a840a1110 0x7f8a840a35d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8a98199270 tx=0x7f8a9400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:52.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:52.928+0000 7f8a8effd640 1 -- 172.21.15.16:0/3805508310 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f8a900b76f0 con 0x7f8a980ff8e0 2024-09-13T16:28:52.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:52 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/4060103254' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T16:28:52.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:52 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/4186578390' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T16:28:52.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:52 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3601289937' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:28:52.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:52 smithi111 ceph-mon[34600]: pgmap v283: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 328 B/s wr, 0 op/s 2024-09-13T16:28:53.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:52 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/4060103254' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T16:28:53.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:52 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/4186578390' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T16:28:53.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:52 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3601289937' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:28:53.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:52 smithi107 ceph-mon[34139]: pgmap v283: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 328 B/s wr, 0 op/s 2024-09-13T16:28:53.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.099+0000 7f8a9fd33640 1 -- 172.21.15.16:0/3805508310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8a9802c410 con 0x7f8a980ff8e0 2024-09-13T16:28:53.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.099+0000 7f8a8effd640 1 -- 172.21.15.16:0/3805508310 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8a900b9620 con 0x7f8a980ff8e0 2024-09-13T16:28:53.101 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-13T16:28:53.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.101+0000 7f8a9fd33640 1 -- 172.21.15.16:0/3805508310 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8a840a1110 msgr2=0x7f8a840a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:53.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.101+0000 7f8a9fd33640 1 --2- 172.21.15.16:0/3805508310 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8a840a1110 0x7f8a840a35d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8a98199270 tx=0x7f8a9400b390 comp rx=0 tx=0).stop 2024-09-13T16:28:53.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.101+0000 7f8a9fd33640 1 -- 172.21.15.16:0/3805508310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a980ff8e0 msgr2=0x7f8a9819f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:53.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.101+0000 7f8a9fd33640 1 --2- 172.21.15.16:0/3805508310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a980ff8e0 0x7f8a9819f310 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8a9002aeb0 tx=0x7f8a90003c20 comp rx=0 tx=0).stop 2024-09-13T16:28:53.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.101+0000 7f8a9fd33640 1 -- 172.21.15.16:0/3805508310 shutdown_connections 2024-09-13T16:28:53.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.101+0000 7f8a9fd33640 1 --2- 172.21.15.16:0/3805508310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a980ff8e0 0x7f8a9819f310 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:53.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.101+0000 7f8a9fd33640 1 --2- 172.21.15.16:0/3805508310 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8a840a1110 0x7f8a840a35d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:53.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.101+0000 7f8a9fd33640 1 --2- 172.21.15.16:0/3805508310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a980fef40 0x7f8a981982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:53.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.101+0000 7f8a9fd33640 1 --2- 172.21.15.16:0/3805508310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a980fe600 0x7f8a98197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:53.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.101+0000 7f8a9fd33640 1 -- 172.21.15.16:0/3805508310 >> 172.21.15.16:0/3805508310 conn(0x7f8a980fa3d0 msgr2=0x7f8a980fbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:53.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.101+0000 7f8a9fd33640 1 -- 172.21.15.16:0/3805508310 shutdown_connections 2024-09-13T16:28:53.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.101+0000 7f8a9fd33640 1 -- 172.21.15.16:0/3805508310 wait complete. 2024-09-13T16:28:53.116 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T16:28:53.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.219+0000 7f766144f640 1 -- 172.21.15.16:0/3965043183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f765c0fec90 msgr2=0x7f765c0ff090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:53.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.219+0000 7f766144f640 1 --2- 172.21.15.16:0/3965043183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f765c0fec90 0x7f765c0ff090 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f764c009900 tx=0x7f764c02a410 comp rx=0 tx=0).stop 2024-09-13T16:28:53.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.219+0000 7f766144f640 1 -- 172.21.15.16:0/3965043183 shutdown_connections 2024-09-13T16:28:53.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.219+0000 7f766144f640 1 --2- 172.21.15.16:0/3965043183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f765c100840 0x7f765c1070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:53.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.219+0000 7f766144f640 1 --2- 172.21.15.16:0/3965043183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f765c0ffe80 0x7f765c100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:53.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.219+0000 7f766144f640 1 --2- 172.21.15.16:0/3965043183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f765c0fec90 0x7f765c0ff090 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:53.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.219+0000 7f766144f640 1 -- 172.21.15.16:0/3965043183 >> 172.21.15.16:0/3965043183 conn(0x7f765c0fa3f0 msgr2=0x7f765c0fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:53.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.219+0000 7f766144f640 1 -- 172.21.15.16:0/3965043183 shutdown_connections 2024-09-13T16:28:53.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.219+0000 7f766144f640 1 -- 172.21.15.16:0/3965043183 wait complete. 2024-09-13T16:28:53.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.220+0000 7f766144f640 1 Processor -- start 2024-09-13T16:28:53.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.220+0000 7f766144f640 1 -- start start 2024-09-13T16:28:53.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.220+0000 7f766144f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f765c0fec90 0x7f765c102c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:53.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.220+0000 7f766144f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f765c0ffe80 0x7f765c101260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:53.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.220+0000 7f766144f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f765c100840 0x7f765c1017a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:53.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.220+0000 7f766144f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765c101f00 con 0x7f765c0ffe80 2024-09-13T16:28:53.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.220+0000 7f766144f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765c102070 con 0x7f765c0fec90 2024-09-13T16:28:53.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.220+0000 7f766144f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765c19fce0 con 0x7f765c100840 2024-09-13T16:28:53.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.220+0000 7f7653fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f765c0ffe80 0x7f765c101260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.220+0000 7f7653fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f765c0ffe80 0x7f765c101260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44922/0 (socket says 172.21.15.16:44922) 2024-09-13T16:28:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.220+0000 7f7653fff640 1 -- 172.21.15.16:0/3213229433 learned_addr learned my addr 172.21.15.16:0/3213229433 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.220+0000 7f765b7fe640 1 --2- 172.21.15.16:0/3213229433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f765c100840 0x7f765c1017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.221+0000 7f765affd640 1 --2- 172.21.15.16:0/3213229433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f765c0fec90 0x7f765c102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.221+0000 7f7653fff640 1 -- 172.21.15.16:0/3213229433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f765c100840 msgr2=0x7f765c1017a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.221+0000 7f7653fff640 1 --2- 172.21.15.16:0/3213229433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f765c100840 0x7f765c1017a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.221+0000 7f7653fff640 1 -- 172.21.15.16:0/3213229433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f765c0fec90 msgr2=0x7f765c102c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.221+0000 7f7653fff640 1 --2- 172.21.15.16:0/3213229433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f765c0fec90 0x7f765c102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.221+0000 7f7653fff640 1 -- 172.21.15.16:0/3213229433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764c009560 con 0x7f765c0ffe80 2024-09-13T16:28:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.221+0000 7f7653fff640 1 --2- 172.21.15.16:0/3213229433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f765c0ffe80 0x7f765c101260 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f764400d7a0 tx=0x7f764400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.221+0000 7f7658ff9640 1 -- 172.21.15.16:0/3213229433 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7644009c30 con 0x7f765c0ffe80 2024-09-13T16:28:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.221+0000 7f7658ff9640 1 -- 172.21.15.16:0/3213229433 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f764400deb0 con 0x7f765c0ffe80 2024-09-13T16:28:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.221+0000 7f766144f640 1 -- 172.21.15.16:0/3213229433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f765c19ff90 con 0x7f765c0ffe80 2024-09-13T16:28:53.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.221+0000 7f7658ff9640 1 -- 172.21.15.16:0/3213229433 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7644004d60 con 0x7f765c0ffe80 2024-09-13T16:28:53.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.221+0000 7f766144f640 1 -- 172.21.15.16:0/3213229433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f765c1a0410 con 0x7f765c0ffe80 2024-09-13T16:28:53.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.222+0000 7f766144f640 1 -- 172.21.15.16:0/3213229433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f765c10a400 con 0x7f765c0ffe80 2024-09-13T16:28:53.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.223+0000 7f7658ff9640 1 -- 172.21.15.16:0/3213229433 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7644004ec0 con 0x7f765c0ffe80 2024-09-13T16:28:53.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.224+0000 7f7658ff9640 1 --2- 172.21.15.16:0/3213229433 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f76480a1110 0x7f76480a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:53.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.224+0000 7f7658ff9640 1 -- 172.21.15.16:0/3213229433 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f76440cc5a0 con 0x7f765c0ffe80 2024-09-13T16:28:53.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.224+0000 7f765affd640 1 --2- 172.21.15.16:0/3213229433 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f76480a1110 0x7f76480a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:53.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.224+0000 7f765affd640 1 --2- 172.21.15.16:0/3213229433 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f76480a1110 0x7f76480a35d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f764c006010 tx=0x7f764c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:53.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.228+0000 7f7658ff9640 1 -- 172.21.15.16:0/3213229433 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7644095ae0 con 0x7f765c0ffe80 2024-09-13T16:28:53.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.398+0000 7f766144f640 1 -- 172.21.15.16:0/3213229433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f765c1a0830 con 0x7f765c0ffe80 2024-09-13T16:28:53.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.398+0000 7f7658ff9640 1 -- 172.21.15.16:0/3213229433 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7644020020 con 0x7f765c0ffe80 2024-09-13T16:28:53.401 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-09-13T16:28:53.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.400+0000 7f766144f640 1 -- 172.21.15.16:0/3213229433 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f76480a1110 msgr2=0x7f76480a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:53.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.400+0000 7f766144f640 1 --2- 172.21.15.16:0/3213229433 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f76480a1110 0x7f76480a35d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f764c006010 tx=0x7f764c000bc0 comp rx=0 tx=0).stop 2024-09-13T16:28:53.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.400+0000 7f766144f640 1 -- 172.21.15.16:0/3213229433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f765c0ffe80 msgr2=0x7f765c101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:53.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.400+0000 7f766144f640 1 --2- 172.21.15.16:0/3213229433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f765c0ffe80 0x7f765c101260 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f764400d7a0 tx=0x7f764400dc60 comp rx=0 tx=0).stop 2024-09-13T16:28:53.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.400+0000 7f766144f640 1 -- 172.21.15.16:0/3213229433 shutdown_connections 2024-09-13T16:28:53.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.400+0000 7f766144f640 1 --2- 172.21.15.16:0/3213229433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f765c100840 0x7f765c1017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:53.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.400+0000 7f766144f640 1 --2- 172.21.15.16:0/3213229433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f765c0ffe80 0x7f765c101260 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:53.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.400+0000 7f766144f640 1 --2- 172.21.15.16:0/3213229433 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f76480a1110 0x7f76480a35d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:53.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.401+0000 7f766144f640 1 --2- 172.21.15.16:0/3213229433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f765c0fec90 0x7f765c102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:53.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.401+0000 7f766144f640 1 -- 172.21.15.16:0/3213229433 >> 172.21.15.16:0/3213229433 conn(0x7f765c0fa3f0 msgr2=0x7f765c0685e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:53.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.401+0000 7f766144f640 1 -- 172.21.15.16:0/3213229433 shutdown_connections 2024-09-13T16:28:53.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.401+0000 7f766144f640 1 -- 172.21.15.16:0/3213229433 wait complete. 2024-09-13T16:28:53.414 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T16:28:53.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.516+0000 7f816aa0d640 1 -- 172.21.15.16:0/1374436659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81640ffe80 msgr2=0x7f8164100300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:53.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.516+0000 7f816aa0d640 1 --2- 172.21.15.16:0/1374436659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81640ffe80 0x7f8164100300 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8158009900 tx=0x7f815802a410 comp rx=0 tx=0).stop 2024-09-13T16:28:53.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.516+0000 7f816aa0d640 1 -- 172.21.15.16:0/1374436659 shutdown_connections 2024-09-13T16:28:53.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.516+0000 7f816aa0d640 1 --2- 172.21.15.16:0/1374436659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8164100840 0x7f81641070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:53.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.516+0000 7f816aa0d640 1 --2- 172.21.15.16:0/1374436659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81640ffe80 0x7f8164100300 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:53.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.516+0000 7f816aa0d640 1 --2- 172.21.15.16:0/1374436659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81640fec90 0x7f81640ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:53.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.516+0000 7f816aa0d640 1 -- 172.21.15.16:0/1374436659 >> 172.21.15.16:0/1374436659 conn(0x7f81640fa3f0 msgr2=0x7f81640fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:53.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.516+0000 7f816aa0d640 1 -- 172.21.15.16:0/1374436659 shutdown_connections 2024-09-13T16:28:53.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.516+0000 7f816aa0d640 1 -- 172.21.15.16:0/1374436659 wait complete. 2024-09-13T16:28:53.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.517+0000 7f816aa0d640 1 Processor -- start 2024-09-13T16:28:53.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.517+0000 7f816aa0d640 1 -- start start 2024-09-13T16:28:53.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.517+0000 7f816aa0d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81640fec90 0x7f8164195d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:53.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.517+0000 7f816aa0d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81640ffe80 0x7f8164196240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:53.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.517+0000 7f816aa0d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8164100840 0x7f816419d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:53.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.517+0000 7f816aa0d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816419d7e0 con 0x7f81640ffe80 2024-09-13T16:28:53.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.517+0000 7f816aa0d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816419d950 con 0x7f81640fec90 2024-09-13T16:28:53.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.517+0000 7f816aa0d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816419db10 con 0x7f8164100840 2024-09-13T16:28:53.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.517+0000 7f81637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81640ffe80 0x7f8164196240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:53.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.517+0000 7f81637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81640ffe80 0x7f8164196240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41656/0 (socket says 172.21.15.16:41656) 2024-09-13T16:28:53.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.518+0000 7f81637fe640 1 -- 172.21.15.16:0/4256066714 learned_addr learned my addr 172.21.15.16:0/4256066714 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:53.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.518+0000 7f81637fe640 1 -- 172.21.15.16:0/4256066714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8164100840 msgr2=0x7f816419d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:53.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.518+0000 7f81637fe640 1 --2- 172.21.15.16:0/4256066714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8164100840 0x7f816419d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:53.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.518+0000 7f81637fe640 1 -- 172.21.15.16:0/4256066714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81640fec90 msgr2=0x7f8164195d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:53.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.518+0000 7f8163fff640 1 --2- 172.21.15.16:0/4256066714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81640fec90 0x7f8164195d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:53.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.518+0000 7f81637fe640 1 --2- 172.21.15.16:0/4256066714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81640fec90 0x7f8164195d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:53.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.518+0000 7f81637fe640 1 -- 172.21.15.16:0/4256066714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8158009560 con 0x7f81640ffe80 2024-09-13T16:28:53.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.518+0000 7f8163fff640 1 --2- 172.21.15.16:0/4256066714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81640fec90 0x7f8164195d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:28:53.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.518+0000 7f81637fe640 1 --2- 172.21.15.16:0/4256066714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81640ffe80 0x7f8164196240 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8158009230 tx=0x7f8158004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:53.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.518+0000 7f81617fa640 1 -- 172.21.15.16:0/4256066714 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8158031d70 con 0x7f81640ffe80 2024-09-13T16:28:53.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.518+0000 7f816aa0d640 1 -- 172.21.15.16:0/4256066714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f816419dd90 con 0x7f81640ffe80 2024-09-13T16:28:53.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.518+0000 7f81617fa640 1 -- 172.21.15.16:0/4256066714 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8158004b50 con 0x7f81640ffe80 2024-09-13T16:28:53.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.518+0000 7f81617fa640 1 -- 172.21.15.16:0/4256066714 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8158003bf0 con 0x7f81640ffe80 2024-09-13T16:28:53.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.518+0000 7f816aa0d640 1 -- 172.21.15.16:0/4256066714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f816419e1d0 con 0x7f81640ffe80 2024-09-13T16:28:53.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.519+0000 7f816aa0d640 1 -- 172.21.15.16:0/4256066714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8164106160 con 0x7f81640ffe80 2024-09-13T16:28:53.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.520+0000 7f81617fa640 1 -- 172.21.15.16:0/4256066714 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8158003d50 con 0x7f81640ffe80 2024-09-13T16:28:53.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.521+0000 7f81617fa640 1 --2- 172.21.15.16:0/4256066714 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f814c0a1090 0x7f814c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:53.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.521+0000 7f81617fa640 1 -- 172.21.15.16:0/4256066714 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f81580ee1c0 con 0x7f81640ffe80 2024-09-13T16:28:53.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.521+0000 7f8163fff640 1 --2- 172.21.15.16:0/4256066714 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f814c0a1090 0x7f814c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:53.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.521+0000 7f8163fff640 1 --2- 172.21.15.16:0/4256066714 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f814c0a1090 0x7f814c0a3550 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f81641971d0 tx=0x7f8154005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:53.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.525+0000 7f81617fa640 1 -- 172.21.15.16:0/4256066714 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f81580b7670 con 0x7f81640ffe80 2024-09-13T16:28:53.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.696+0000 7f816aa0d640 1 -- 172.21.15.16:0/4256066714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f816410b580 con 0x7f81640ffe80 2024-09-13T16:28:53.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.696+0000 7f81617fa640 1 -- 172.21.15.16:0/4256066714 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7f81580b95a0 con 0x7f81640ffe80 2024-09-13T16:28:53.699 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-13T16:28:53.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.698+0000 7f816aa0d640 1 -- 172.21.15.16:0/4256066714 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f814c0a1090 msgr2=0x7f814c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:53.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.698+0000 7f816aa0d640 1 --2- 172.21.15.16:0/4256066714 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f814c0a1090 0x7f814c0a3550 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f81641971d0 tx=0x7f8154005d50 comp rx=0 tx=0).stop 2024-09-13T16:28:53.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.698+0000 7f816aa0d640 1 -- 172.21.15.16:0/4256066714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81640ffe80 msgr2=0x7f8164196240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:53.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.698+0000 7f816aa0d640 1 --2- 172.21.15.16:0/4256066714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81640ffe80 0x7f8164196240 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8158009230 tx=0x7f8158004380 comp rx=0 tx=0).stop 2024-09-13T16:28:53.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.698+0000 7f816aa0d640 1 -- 172.21.15.16:0/4256066714 shutdown_connections 2024-09-13T16:28:53.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.698+0000 7f816aa0d640 1 --2- 172.21.15.16:0/4256066714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8164100840 0x7f816419d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:53.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.698+0000 7f816aa0d640 1 --2- 172.21.15.16:0/4256066714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81640ffe80 0x7f8164196240 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:53.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.698+0000 7f816aa0d640 1 --2- 172.21.15.16:0/4256066714 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f814c0a1090 0x7f814c0a3550 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:53.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.698+0000 7f816aa0d640 1 --2- 172.21.15.16:0/4256066714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81640fec90 0x7f8164195d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:53.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.698+0000 7f816aa0d640 1 -- 172.21.15.16:0/4256066714 >> 172.21.15.16:0/4256066714 conn(0x7f81640fa3f0 msgr2=0x7f8164103330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:53.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.698+0000 7f816aa0d640 1 -- 172.21.15.16:0/4256066714 shutdown_connections 2024-09-13T16:28:53.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.698+0000 7f816aa0d640 1 -- 172.21.15.16:0/4256066714 wait complete. 2024-09-13T16:28:53.710 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:53 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2689507158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:28:53.710 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:53 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3805508310' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T16:28:53.711 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:53 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3213229433' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T16:28:53.712 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T16:28:53.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.812+0000 7f34d9e87640 1 -- 172.21.15.16:0/4280496531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34d40fec70 msgr2=0x7f34d40ff070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:53.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.812+0000 7f34d9e87640 1 --2- 172.21.15.16:0/4280496531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34d40fec70 0x7f34d40ff070 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009ca0 tx=0x7f34c402a8c0 comp rx=0 tx=0).stop 2024-09-13T16:28:53.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.812+0000 7f34d9e87640 1 -- 172.21.15.16:0/4280496531 shutdown_connections 2024-09-13T16:28:53.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.812+0000 7f34d9e87640 1 --2- 172.21.15.16:0/4280496531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34d4100820 0x7f34d41070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:53.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.812+0000 7f34d9e87640 1 --2- 172.21.15.16:0/4280496531 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34d40ffe60 0x7f34d41002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:53.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.812+0000 7f34d9e87640 1 --2- 172.21.15.16:0/4280496531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34d40fec70 0x7f34d40ff070 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:53.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.812+0000 7f34d9e87640 1 -- 172.21.15.16:0/4280496531 >> 172.21.15.16:0/4280496531 conn(0x7f34d40fa410 msgr2=0x7f34d40fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:53.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.812+0000 7f34d9e87640 1 -- 172.21.15.16:0/4280496531 shutdown_connections 2024-09-13T16:28:53.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.812+0000 7f34d9e87640 1 -- 172.21.15.16:0/4280496531 wait complete. 2024-09-13T16:28:53.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.813+0000 7f34d9e87640 1 Processor -- start 2024-09-13T16:28:53.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.813+0000 7f34d9e87640 1 -- start start 2024-09-13T16:28:53.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.813+0000 7f34d9e87640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34d40fec70 0x7f34d4195c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:53.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.813+0000 7f34d9e87640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34d40ffe60 0x7f34d41961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:53.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.813+0000 7f34d9e87640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34d4100820 0x7f34d419d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:53.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.813+0000 7f34d9e87640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d419d740 con 0x7f34d40ffe60 2024-09-13T16:28:53.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.813+0000 7f34d9e87640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d419d8b0 con 0x7f34d40fec70 2024-09-13T16:28:53.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.813+0000 7f34d9e87640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d419da70 con 0x7f34d4100820 2024-09-13T16:28:53.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.813+0000 7f34d2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34d40ffe60 0x7f34d41961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:53.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.814+0000 7f34d2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34d40ffe60 0x7f34d41961a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41676/0 (socket says 172.21.15.16:41676) 2024-09-13T16:28:53.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.814+0000 7f34d2ffd640 1 -- 172.21.15.16:0/416957732 learned_addr learned my addr 172.21.15.16:0/416957732 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:28:53.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.814+0000 7f34d37fe640 1 --2- 172.21.15.16:0/416957732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34d40fec70 0x7f34d4195c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:53.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.814+0000 7f34d3fff640 1 --2- 172.21.15.16:0/416957732 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34d4100820 0x7f34d419d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:53.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.814+0000 7f34d2ffd640 1 -- 172.21.15.16:0/416957732 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34d4100820 msgr2=0x7f34d419d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:53.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.814+0000 7f34d2ffd640 1 --2- 172.21.15.16:0/416957732 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34d4100820 0x7f34d419d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:53.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.814+0000 7f34d2ffd640 1 -- 172.21.15.16:0/416957732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34d40fec70 msgr2=0x7f34d4195c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:53.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.814+0000 7f34d2ffd640 1 --2- 172.21.15.16:0/416957732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34d40fec70 0x7f34d4195c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:53.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.814+0000 7f34d2ffd640 1 -- 172.21.15.16:0/416957732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c4009900 con 0x7f34d40ffe60 2024-09-13T16:28:53.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.814+0000 7f34d2ffd640 1 --2- 172.21.15.16:0/416957732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34d40ffe60 0x7f34d41961a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f34c800d950 tx=0x7f34c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:53.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.814+0000 7f34d0ff9640 1 -- 172.21.15.16:0/416957732 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c8009a30 con 0x7f34d40ffe60 2024-09-13T16:28:53.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.814+0000 7f34d0ff9640 1 -- 172.21.15.16:0/416957732 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f34c80044b0 con 0x7f34d40ffe60 2024-09-13T16:28:53.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.814+0000 7f34d9e87640 1 -- 172.21.15.16:0/416957732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34d419dd50 con 0x7f34d40ffe60 2024-09-13T16:28:53.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.814+0000 7f34d0ff9640 1 -- 172.21.15.16:0/416957732 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c80047a0 con 0x7f34d40ffe60 2024-09-13T16:28:53.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.814+0000 7f34d9e87640 1 -- 172.21.15.16:0/416957732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d419e1f0 con 0x7f34d40ffe60 2024-09-13T16:28:53.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.815+0000 7f34d9e87640 1 -- 172.21.15.16:0/416957732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34d41061f0 con 0x7f34d40ffe60 2024-09-13T16:28:53.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.816+0000 7f34d0ff9640 1 -- 172.21.15.16:0/416957732 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f34c8004020 con 0x7f34d40ffe60 2024-09-13T16:28:53.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.817+0000 7f34d0ff9640 1 --2- 172.21.15.16:0/416957732 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f34bc0a1110 0x7f34bc0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:28:53.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.817+0000 7f34d37fe640 1 --2- 172.21.15.16:0/416957732 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f34bc0a1110 0x7f34bc0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:28:53.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.817+0000 7f34d0ff9640 1 -- 172.21.15.16:0/416957732 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f34c80cc610 con 0x7f34d40ffe60 2024-09-13T16:28:53.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.818+0000 7f34d37fe640 1 --2- 172.21.15.16:0/416957732 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f34bc0a1110 0x7f34bc0a35d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f34c402aeb0 tx=0x7f34c4009890 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:28:53.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:53.822+0000 7f34d0ff9640 1 -- 172.21.15.16:0/416957732 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f34c8095b20 con 0x7f34d40ffe60 2024-09-13T16:28:53.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:53 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/2689507158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:28:53.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:53 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3805508310' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T16:28:53.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:53 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3213229433' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T16:28:54.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:54.008+0000 7f34d9e87640 1 -- 172.21.15.16:0/416957732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f34d410b560 con 0x7f34d40ffe60 2024-09-13T16:28:54.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:54.009+0000 7f34d0ff9640 1 -- 172.21.15.16:0/416957732 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) v1 ==== 81+0+589 (secure 0 0 0) 0x7f34c80b7e60 con 0x7f34d40ffe60 2024-09-13T16:28:54.011 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2098378667 2024-09-14T16:22:04.485277+0000 2024-09-13T16:28:54.011 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3536229016 2024-09-14T16:22:04.485277+0000 2024-09-13T16:28:54.011 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/1756940569 2024-09-14T16:22:04.485277+0000 2024-09-13T16:28:54.011 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2684687547 2024-09-14T16:22:04.485277+0000 2024-09-13T16:28:54.011 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/1756940569 2024-09-14T16:22:04.485277+0000 2024-09-13T16:28:54.011 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/368410612 2024-09-14T16:21:40.126459+0000 2024-09-13T16:28:54.011 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/12158160 2024-09-14T16:21:40.126459+0000 2024-09-13T16:28:54.011 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/1147765163 2024-09-14T16:21:40.126459+0000 2024-09-13T16:28:54.011 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/1147765163 2024-09-14T16:21:40.126459+0000 2024-09-13T16:28:54.012 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3754423379 2024-09-14T16:21:40.126459+0000 2024-09-13T16:28:54.012 INFO:teuthology.orchestra.run.smithi016.stderr:listed 10 entries 2024-09-13T16:28:54.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:54.011+0000 7f34d9e87640 1 -- 172.21.15.16:0/416957732 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f34bc0a1110 msgr2=0x7f34bc0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:54.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:54.011+0000 7f34d9e87640 1 --2- 172.21.15.16:0/416957732 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f34bc0a1110 0x7f34bc0a35d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f34c402aeb0 tx=0x7f34c4009890 comp rx=0 tx=0).stop 2024-09-13T16:28:54.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:54.011+0000 7f34d9e87640 1 -- 172.21.15.16:0/416957732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34d40ffe60 msgr2=0x7f34d41961a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:28:54.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:54.011+0000 7f34d9e87640 1 --2- 172.21.15.16:0/416957732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34d40ffe60 0x7f34d41961a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f34c800d950 tx=0x7f34c800de10 comp rx=0 tx=0).stop 2024-09-13T16:28:54.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:54.011+0000 7f34d9e87640 1 -- 172.21.15.16:0/416957732 shutdown_connections 2024-09-13T16:28:54.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:54.011+0000 7f34d9e87640 1 --2- 172.21.15.16:0/416957732 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34d4100820 0x7f34d419d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:54.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:54.011+0000 7f34d9e87640 1 --2- 172.21.15.16:0/416957732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34d40ffe60 0x7f34d41961a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:54.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:54.011+0000 7f34d9e87640 1 --2- 172.21.15.16:0/416957732 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f34bc0a1110 0x7f34bc0a35d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:54.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:54.011+0000 7f34d9e87640 1 --2- 172.21.15.16:0/416957732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34d40fec70 0x7f34d4195c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:28:54.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:54.011+0000 7f34d9e87640 1 -- 172.21.15.16:0/416957732 >> 172.21.15.16:0/416957732 conn(0x7f34d40fa410 msgr2=0x7f34d41033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:28:54.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:54.011+0000 7f34d9e87640 1 -- 172.21.15.16:0/416957732 shutdown_connections 2024-09-13T16:28:54.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:28:54.011+0000 7f34d9e87640 1 -- 172.21.15.16:0/416957732 wait complete. 2024-09-13T16:28:54.024 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2024-09-13T16:28:54.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:28:54.024 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-13T16:28:54.048 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[74912]: fuse finished with error 0 and tester_r 0 2024-09-13T16:28:54.050 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:28:54.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:53 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/2689507158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:28:54.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:53 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3805508310' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T16:28:54.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:53 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3213229433' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T16:28:54.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:54 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/4256066714' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T16:28:54.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:54 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/416957732' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T16:28:54.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:54 smithi111 ceph-mon[34600]: pgmap v284: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 279 B/s wr, 0 op/s 2024-09-13T16:28:55.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:54 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/4256066714' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T16:28:55.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:54 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/416957732' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T16:28:55.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:54 smithi107 ceph-mon[34139]: pgmap v284: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 279 B/s wr, 0 op/s 2024-09-13T16:28:55.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:54 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/4256066714' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T16:28:55.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:54 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/416957732' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T16:28:55.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:54 smithi016 ceph-mon[31187]: pgmap v284: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 279 B/s wr, 0 op/s 2024-09-13T16:28:56.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:56 smithi111 ceph-mon[34600]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-13T16:28:57.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:56 smithi107 ceph-mon[34139]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-13T16:28:57.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:56 smithi016 ceph-mon[31187]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-13T16:28:58.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:28:58 smithi016 ceph-mon[31187]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-13T16:28:58.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:28:58 smithi111 ceph-mon[34600]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-13T16:28:59.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:28:58 smithi107 ceph-mon[34139]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-13T16:29:00.151 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-13T16:29:00.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:29:00.152 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-13T16:29:00.174 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:29:00.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.315+0000 7fe3b3ac5640 1 -- 172.21.15.16:0/2809855865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3ac0fe440 msgr2=0x7fe3ac0fe840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:00.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.315+0000 7fe3b3ac5640 1 --2- 172.21.15.16:0/2809855865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3ac0fe440 0x7fe3ac0fe840 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe3a800aa10 tx=0x7fe3a802b720 comp rx=0 tx=0).stop 2024-09-13T16:29:00.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.316+0000 7fe3b3ac5640 1 -- 172.21.15.16:0/2809855865 shutdown_connections 2024-09-13T16:29:00.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.316+0000 7fe3b3ac5640 1 --2- 172.21.15.16:0/2809855865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3ac0ffff0 0x7fe3ac106880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:00.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.316+0000 7fe3b3ac5640 1 --2- 172.21.15.16:0/2809855865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3ac0ff630 0x7fe3ac0ffab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:00.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.316+0000 7fe3b3ac5640 1 --2- 172.21.15.16:0/2809855865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3ac0fe440 0x7fe3ac0fe840 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:00.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.316+0000 7fe3b3ac5640 1 -- 172.21.15.16:0/2809855865 >> 172.21.15.16:0/2809855865 conn(0x7fe3ac0f9bc0 msgr2=0x7fe3ac0fbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:00.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.316+0000 7fe3b3ac5640 1 -- 172.21.15.16:0/2809855865 shutdown_connections 2024-09-13T16:29:00.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.316+0000 7fe3b3ac5640 1 -- 172.21.15.16:0/2809855865 wait complete. 2024-09-13T16:29:00.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.316+0000 7fe3b3ac5640 1 Processor -- start 2024-09-13T16:29:00.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.316+0000 7fe3b3ac5640 1 -- start start 2024-09-13T16:29:00.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.316+0000 7fe3b3ac5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3ac0fe440 0x7fe3ac1023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:00.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.316+0000 7fe3b3ac5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3ac0ff630 0x7fe3ac100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:00.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.316+0000 7fe3b3ac5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3ac0ffff0 0x7fe3ac100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:00.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.316+0000 7fe3b3ac5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ac1016b0 con 0x7fe3ac0fe440 2024-09-13T16:29:00.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.316+0000 7fe3b3ac5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ac101820 con 0x7fe3ac0ffff0 2024-09-13T16:29:00.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.317+0000 7fe3b3ac5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ac19fe30 con 0x7fe3ac0ff630 2024-09-13T16:29:00.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.317+0000 7fe3b1861640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3ac0fe440 0x7fe3ac1023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:00.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.317+0000 7fe3b1861640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3ac0fe440 0x7fe3ac1023d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41694/0 (socket says 172.21.15.16:41694) 2024-09-13T16:29:00.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.317+0000 7fe3b1861640 1 -- 172.21.15.16:0/1323825493 learned_addr learned my addr 172.21.15.16:0/1323825493 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:00.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.317+0000 7fe3b1060640 1 --2- 172.21.15.16:0/1323825493 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3ac0ff630 0x7fe3ac100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:00.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.317+0000 7fe3b2062640 1 --2- 172.21.15.16:0/1323825493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3ac0ffff0 0x7fe3ac100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:00.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.317+0000 7fe3b1861640 1 -- 172.21.15.16:0/1323825493 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3ac0ff630 msgr2=0x7fe3ac100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:00.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.317+0000 7fe3b1861640 1 --2- 172.21.15.16:0/1323825493 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3ac0ff630 0x7fe3ac100a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:00.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.317+0000 7fe3b1861640 1 -- 172.21.15.16:0/1323825493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3ac0ffff0 msgr2=0x7fe3ac100f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:00.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.317+0000 7fe3b1861640 1 --2- 172.21.15.16:0/1323825493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3ac0ffff0 0x7fe3ac100f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:00.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.317+0000 7fe3b1861640 1 -- 172.21.15.16:0/1323825493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a800a670 con 0x7fe3ac0fe440 2024-09-13T16:29:00.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.317+0000 7fe3b2062640 1 --2- 172.21.15.16:0/1323825493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3ac0ffff0 0x7fe3ac100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:29:00.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.317+0000 7fe3b1861640 1 --2- 172.21.15.16:0/1323825493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3ac0fe440 0x7fe3ac1023d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe3a800a340 tx=0x7fe3a8004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:00.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.317+0000 7fe3a2ffd640 1 -- 172.21.15.16:0/1323825493 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8005410 con 0x7fe3ac0fe440 2024-09-13T16:29:00.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.317+0000 7fe3a2ffd640 1 -- 172.21.15.16:0/1323825493 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe3a8004b50 con 0x7fe3ac0fe440 2024-09-13T16:29:00.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.317+0000 7fe3b3ac5640 1 -- 172.21.15.16:0/1323825493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3ac1a0080 con 0x7fe3ac0fe440 2024-09-13T16:29:00.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.317+0000 7fe3a2ffd640 1 -- 172.21.15.16:0/1323825493 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8003be0 con 0x7fe3ac0fe440 2024-09-13T16:29:00.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.317+0000 7fe3b3ac5640 1 -- 172.21.15.16:0/1323825493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3ac1a0530 con 0x7fe3ac0fe440 2024-09-13T16:29:00.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.318+0000 7fe3b3ac5640 1 -- 172.21.15.16:0/1323825493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3ac109380 con 0x7fe3ac0fe440 2024-09-13T16:29:00.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.319+0000 7fe3a2ffd640 1 -- 172.21.15.16:0/1323825493 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe3a8004cc0 con 0x7fe3ac0fe440 2024-09-13T16:29:00.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.320+0000 7fe3a2ffd640 1 --2- 172.21.15.16:0/1323825493 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe3980a1090 0x7fe3980a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:00.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.320+0000 7fe3a2ffd640 1 -- 172.21.15.16:0/1323825493 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe3a80ee060 con 0x7fe3ac0fe440 2024-09-13T16:29:00.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.320+0000 7fe3b1060640 1 --2- 172.21.15.16:0/1323825493 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe3980a1090 0x7fe3980a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:00.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.321+0000 7fe3b1060640 1 --2- 172.21.15.16:0/1323825493 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe3980a1090 0x7fe3980a3550 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac102140 tx=0x7fe39c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:00.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.324+0000 7fe3a2ffd640 1 -- 172.21.15.16:0/1323825493 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fe3a80b7600 con 0x7fe3ac0fe440 2024-09-13T16:29:00.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.498+0000 7fe3b3ac5640 1 -- 172.21.15.16:0/1323825493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3ac109560 con 0x7fe3ac0fe440 2024-09-13T16:29:00.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.500+0000 7fe3a2ffd640 1 -- 172.21.15.16:0/1323825493 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23971 (secure 0 0 0) 0x7fe3a80bac20 con 0x7fe3ac0fe440 2024-09-13T16:29:00.502 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:29:00.502 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":89,"fsid":"0d241f6a-71ec-11ef-bceb-c7b262605968","created":"2024-09-13T16:21:17.971738+0000","modified":"2024-09-13T16:28:43.574106+0000","last_up_change":"2024-09-13T16:27:54.673413+0000","last_in_change":"2024-09-13T16:27:35.718390+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:24:22.981019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:28:37.746027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:28:38.602731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:28:41.633322+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e11ef8f-08d6-4e74-afd6-f49f4bd40032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6803","nonce":2079394186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6805","nonce":2079394186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6809","nonce":2079394186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6807","nonce":2079394186}]},"public_addr":"172.21.15.16:6803/2079394186","cluster_addr":"172.21.15.16:6805/2079394186","heartbeat_back_addr":"172.21.15.16:6809/2079394186","heartbeat_front_addr":"172.21.15.16:6807/2079394186","state":["exists","up"]},{"osd":1,"uuid":"f2cd3694-9c9e-4da7-976e-f80a8df120f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6801","nonce":4021360958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6803","nonce":4021360958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6807","nonce":4021360958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6805","nonce":4021360958}]},"public_addr":"172.21.15.107:6801/4021360958","cluster_addr":"172.21.15.107:6803/4021360958","heartbeat_back_addr":"172.21.15.107:6807/4021360958","heartbeat_front_addr":"172.21.15.107:6805/4021360958","state":["exists","up"]},{"osd":2,"uuid":"a5bbf2eb-6599-44d5-ad96-68bb89ee19f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6801","nonce":3227179825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6803","nonce":3227179825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6807","nonce":3227179825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6805","nonce":3227179825}]},"public_addr":"172.21.15.111:6801/3227179825","cluster_addr":"172.21.15.111:6803/3227179825","heartbeat_back_addr":"172.21.15.111:6807/3227179825","heartbeat_front_addr":"172.21.15.111:6805/3227179825","state":["exists","up"]},{"osd":3,"uuid":"d6295ffe-11bc-4964-a250-9fecca870b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6811","nonce":2390375937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6813","nonce":2390375937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6817","nonce":2390375937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6815","nonce":2390375937}]},"public_addr":"172.21.15.16:6811/2390375937","cluster_addr":"172.21.15.16:6813/2390375937","heartbeat_back_addr":"172.21.15.16:6817/2390375937","heartbeat_front_addr":"172.21.15.16:6815/2390375937","state":["exists","up"]},{"osd":4,"uuid":"0a8446e0-fc86-4646-beb6-a7b9d433093d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6809","nonce":1936582048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6811","nonce":1936582048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6815","nonce":1936582048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6813","nonce":1936582048}]},"public_addr":"172.21.15.107:6809/1936582048","cluster_addr":"172.21.15.107:6811/1936582048","heartbeat_back_addr":"172.21.15.107:6815/1936582048","heartbeat_front_addr":"172.21.15.107:6813/1936582048","state":["exists","up"]},{"osd":5,"uuid":"2c97a446-357e-4975-9e51-9c95fe84a663","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6809","nonce":483858448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6811","nonce":483858448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6815","nonce":483858448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6813","nonce":483858448}]},"public_addr":"172.21.15.111:6809/483858448","cluster_addr":"172.21.15.111:6811/483858448","heartbeat_back_addr":"172.21.15.111:6815/483858448","heartbeat_front_addr":"172.21.15.111:6813/483858448","state":["exists","up"]},{"osd":6,"uuid":"890239e6-e6fb-4f9c-b973-05cc66d199bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6819","nonce":1330214141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6821","nonce":1330214141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6825","nonce":1330214141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6823","nonce":1330214141}]},"public_addr":"172.21.15.16:6819/1330214141","cluster_addr":"172.21.15.16:6821/1330214141","heartbeat_back_addr":"172.21.15.16:6825/1330214141","heartbeat_front_addr":"172.21.15.16:6823/1330214141","state":["exists","up"]},{"osd":7,"uuid":"ee826a9c-f441-4903-9b1b-66b627f7b712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6817","nonce":1133876754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6819","nonce":1133876754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6823","nonce":1133876754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6821","nonce":1133876754}]},"public_addr":"172.21.15.107:6817/1133876754","cluster_addr":"172.21.15.107:6819/1133876754","heartbeat_back_addr":"172.21.15.107:6823/1133876754","heartbeat_front_addr":"172.21.15.107:6821/1133876754","state":["exists","up"]},{"osd":8,"uuid":"e19773f2-dec8-44e0-9d33-0252e07d6415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6817","nonce":837548190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6819","nonce":837548190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6823","nonce":837548190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6821","nonce":837548190}]},"public_addr":"172.21.15.111:6817/837548190","cluster_addr":"172.21.15.111:6819/837548190","heartbeat_back_addr":"172.21.15.111:6823/837548190","heartbeat_front_addr":"172.21.15.111:6821/837548190","state":["exists","up"]},{"osd":9,"uuid":"bae4d6db-8b90-4d02-8e83-7d8f2a6f8fdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6827","nonce":4175383686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6829","nonce":4175383686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6833","nonce":4175383686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6831","nonce":4175383686}]},"public_addr":"172.21.15.16:6827/4175383686","cluster_addr":"172.21.15.16:6829/4175383686","heartbeat_back_addr":"172.21.15.16:6833/4175383686","heartbeat_front_addr":"172.21.15.16:6831/4175383686","state":["exists","up"]},{"osd":10,"uuid":"fd0e7740-76c8-4c99-a349-7077b4905143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6825","nonce":2659550426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6827","nonce":2659550426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6831","nonce":2659550426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6829","nonce":2659550426}]},"public_addr":"172.21.15.107:6825/2659550426","cluster_addr":"172.21.15.107:6827/2659550426","heartbeat_back_addr":"172.21.15.107:6831/2659550426","heartbeat_front_addr":"172.21.15.107:6829/2659550426","state":["exists","up"]},{"osd":11,"uuid":"94a1751d-b00c-4e23-815d-3a8bc3dea146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6825","nonce":1633548683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6827","nonce":1633548683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6831","nonce":1633548683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6829","nonce":1633548683}]},"public_addr":"172.21.15.111:6825/1633548683","cluster_addr":"172.21.15.111:6827/1633548683","heartbeat_back_addr":"172.21.15.111:6831/1633548683","heartbeat_front_addr":"172.21.15.111:6829/1633548683","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:36.359852+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:58.014202+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:19.753672+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:43.404817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:06.001677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:29.361587+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:52.554847+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:15.957553+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:39.581093+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:04.911389+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:28.971280+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:53.473561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/2098378667":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/3536229016":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6801/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/2684687547":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6800/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/368410612":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/12158160":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6800/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6801/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/3754423379":"2024-09-14T16:21:40.126459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:29:00.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.502+0000 7fe3b3ac5640 1 -- 172.21.15.16:0/1323825493 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe3980a1090 msgr2=0x7fe3980a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:00.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.502+0000 7fe3b3ac5640 1 --2- 172.21.15.16:0/1323825493 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe3980a1090 0x7fe3980a3550 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac102140 tx=0x7fe39c008040 comp rx=0 tx=0).stop 2024-09-13T16:29:00.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.502+0000 7fe3b3ac5640 1 -- 172.21.15.16:0/1323825493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3ac0fe440 msgr2=0x7fe3ac1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:00.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.502+0000 7fe3b3ac5640 1 --2- 172.21.15.16:0/1323825493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3ac0fe440 0x7fe3ac1023d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe3a800a340 tx=0x7fe3a8004360 comp rx=0 tx=0).stop 2024-09-13T16:29:00.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.502+0000 7fe3b3ac5640 1 -- 172.21.15.16:0/1323825493 shutdown_connections 2024-09-13T16:29:00.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.502+0000 7fe3b3ac5640 1 --2- 172.21.15.16:0/1323825493 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe3980a1090 0x7fe3980a3550 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:00.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.502+0000 7fe3b3ac5640 1 --2- 172.21.15.16:0/1323825493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3ac0ffff0 0x7fe3ac100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:00.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.502+0000 7fe3b3ac5640 1 --2- 172.21.15.16:0/1323825493 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3ac0ff630 0x7fe3ac100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:00.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.502+0000 7fe3b3ac5640 1 --2- 172.21.15.16:0/1323825493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3ac0fe440 0x7fe3ac1023d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:00.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.502+0000 7fe3b3ac5640 1 -- 172.21.15.16:0/1323825493 >> 172.21.15.16:0/1323825493 conn(0x7fe3ac0f9bc0 msgr2=0x7fe3ac104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:00.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.502+0000 7fe3b3ac5640 1 -- 172.21.15.16:0/1323825493 shutdown_connections 2024-09-13T16:29:00.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.502+0000 7fe3b3ac5640 1 -- 172.21.15.16:0/1323825493 wait complete. 2024-09-13T16:29:00.515 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:29:00.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.616+0000 7f62e2725640 1 -- 172.21.15.16:0/1918700440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62dc100850 msgr2=0x7f62dc102c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:00.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.616+0000 7f62e2725640 1 --2- 172.21.15.16:0/1918700440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62dc100850 0x7f62dc102c40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009900 tx=0x7f62cc02a410 comp rx=0 tx=0).stop 2024-09-13T16:29:00.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.616+0000 7f62e2725640 1 -- 172.21.15.16:0/1918700440 shutdown_connections 2024-09-13T16:29:00.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.616+0000 7f62e2725640 1 --2- 172.21.15.16:0/1918700440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62dc105ab0 0x7f62dc107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:00.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.616+0000 7f62e2725640 1 --2- 172.21.15.16:0/1918700440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62dc103180 0x7f62dc105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:00.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.616+0000 7f62e2725640 1 --2- 172.21.15.16:0/1918700440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62dc100850 0x7f62dc102c40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:00.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.616+0000 7f62e2725640 1 -- 172.21.15.16:0/1918700440 >> 172.21.15.16:0/1918700440 conn(0x7f62dc0fa430 msgr2=0x7f62dc0fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:00.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.616+0000 7f62e2725640 1 -- 172.21.15.16:0/1918700440 shutdown_connections 2024-09-13T16:29:00.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.616+0000 7f62e2725640 1 -- 172.21.15.16:0/1918700440 wait complete. 2024-09-13T16:29:00.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.617+0000 7f62e2725640 1 Processor -- start 2024-09-13T16:29:00.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.617+0000 7f62e2725640 1 -- start start 2024-09-13T16:29:00.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.617+0000 7f62e2725640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62dc100850 0x7f62dc195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:00.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.617+0000 7f62e2725640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62dc103180 0x7f62dc196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:00.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.617+0000 7f62e2725640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62dc105ab0 0x7f62dc19d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:00.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.617+0000 7f62e2725640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc19d820 con 0x7f62dc103180 2024-09-13T16:29:00.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.617+0000 7f62e2725640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc19d990 con 0x7f62dc100850 2024-09-13T16:29:00.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.617+0000 7f62e2725640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc19db50 con 0x7f62dc105ab0 2024-09-13T16:29:00.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.617+0000 7f62db7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62dc103180 0x7f62dc196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:00.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.618+0000 7f62db7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62dc103180 0x7f62dc196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41714/0 (socket says 172.21.15.16:41714) 2024-09-13T16:29:00.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.618+0000 7f62db7fe640 1 -- 172.21.15.16:0/1397456463 learned_addr learned my addr 172.21.15.16:0/1397456463 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:00.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.618+0000 7f62dbfff640 1 --2- 172.21.15.16:0/1397456463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62dc100850 0x7f62dc195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:00.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.618+0000 7f62e0cc2640 1 --2- 172.21.15.16:0/1397456463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62dc105ab0 0x7f62dc19d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:00.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.618+0000 7f62db7fe640 1 -- 172.21.15.16:0/1397456463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62dc105ab0 msgr2=0x7f62dc19d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:00.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.618+0000 7f62db7fe640 1 --2- 172.21.15.16:0/1397456463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62dc105ab0 0x7f62dc19d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:00.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.618+0000 7f62db7fe640 1 -- 172.21.15.16:0/1397456463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62dc100850 msgr2=0x7f62dc195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:00.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.618+0000 7f62db7fe640 1 --2- 172.21.15.16:0/1397456463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62dc100850 0x7f62dc195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:00.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.618+0000 7f62db7fe640 1 -- 172.21.15.16:0/1397456463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62cc009560 con 0x7f62dc103180 2024-09-13T16:29:00.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.618+0000 7f62dbfff640 1 --2- 172.21.15.16:0/1397456463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62dc100850 0x7f62dc195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:29:00.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.618+0000 7f62e0cc2640 1 --2- 172.21.15.16:0/1397456463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62dc105ab0 0x7f62dc19d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:29:00.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.618+0000 7f62db7fe640 1 --2- 172.21.15.16:0/1397456463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62dc103180 0x7f62dc196280 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f62d000c970 tx=0x7f62d000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:00.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.618+0000 7f62d97fa640 1 -- 172.21.15.16:0/1397456463 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0007bb0 con 0x7f62dc103180 2024-09-13T16:29:00.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.618+0000 7f62d97fa640 1 -- 172.21.15.16:0/1397456463 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f62d0007d10 con 0x7f62dc103180 2024-09-13T16:29:00.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.618+0000 7f62e2725640 1 -- 172.21.15.16:0/1397456463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62dc19de30 con 0x7f62dc103180 2024-09-13T16:29:00.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.618+0000 7f62d97fa640 1 -- 172.21.15.16:0/1397456463 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0004da0 con 0x7f62dc103180 2024-09-13T16:29:00.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.618+0000 7f62e2725640 1 -- 172.21.15.16:0/1397456463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62dc19e2b0 con 0x7f62dc103180 2024-09-13T16:29:00.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.619+0000 7f62e2725640 1 -- 172.21.15.16:0/1397456463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62dc0fbf20 con 0x7f62dc103180 2024-09-13T16:29:00.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.621+0000 7f62d97fa640 1 -- 172.21.15.16:0/1397456463 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f62d0005000 con 0x7f62dc103180 2024-09-13T16:29:00.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.621+0000 7f62d97fa640 1 --2- 172.21.15.16:0/1397456463 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f62c40a1110 0x7f62c40a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:00.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.621+0000 7f62d97fa640 1 -- 172.21.15.16:0/1397456463 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f62d00cc880 con 0x7f62dc103180 2024-09-13T16:29:00.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.621+0000 7f62dbfff640 1 --2- 172.21.15.16:0/1397456463 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f62c40a1110 0x7f62c40a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:00.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.622+0000 7f62dbfff640 1 --2- 172.21.15.16:0/1397456463 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f62c40a1110 0x7f62c40a35d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f62cc006010 tx=0x7f62cc005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:00.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.625+0000 7f62d97fa640 1 -- 172.21.15.16:0/1397456463 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f62d0095d90 con 0x7f62dc103180 2024-09-13T16:29:00.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.796+0000 7f62e2725640 1 -- 172.21.15.16:0/1397456463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62dc10b580 con 0x7f62dc103180 2024-09-13T16:29:00.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.797+0000 7f62d97fa640 1 -- 172.21.15.16:0/1397456463 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23971 (secure 0 0 0) 0x7f62d001d020 con 0x7f62dc103180 2024-09-13T16:29:00.800 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:29:00.800 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":89,"fsid":"0d241f6a-71ec-11ef-bceb-c7b262605968","created":"2024-09-13T16:21:17.971738+0000","modified":"2024-09-13T16:28:43.574106+0000","last_up_change":"2024-09-13T16:27:54.673413+0000","last_in_change":"2024-09-13T16:27:35.718390+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:24:22.981019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:28:37.746027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:28:38.602731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:28:41.633322+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e11ef8f-08d6-4e74-afd6-f49f4bd40032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6803","nonce":2079394186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6805","nonce":2079394186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6809","nonce":2079394186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6807","nonce":2079394186}]},"public_addr":"172.21.15.16:6803/2079394186","cluster_addr":"172.21.15.16:6805/2079394186","heartbeat_back_addr":"172.21.15.16:6809/2079394186","heartbeat_front_addr":"172.21.15.16:6807/2079394186","state":["exists","up"]},{"osd":1,"uuid":"f2cd3694-9c9e-4da7-976e-f80a8df120f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6801","nonce":4021360958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6803","nonce":4021360958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6807","nonce":4021360958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6805","nonce":4021360958}]},"public_addr":"172.21.15.107:6801/4021360958","cluster_addr":"172.21.15.107:6803/4021360958","heartbeat_back_addr":"172.21.15.107:6807/4021360958","heartbeat_front_addr":"172.21.15.107:6805/4021360958","state":["exists","up"]},{"osd":2,"uuid":"a5bbf2eb-6599-44d5-ad96-68bb89ee19f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6801","nonce":3227179825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6803","nonce":3227179825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6807","nonce":3227179825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6805","nonce":3227179825}]},"public_addr":"172.21.15.111:6801/3227179825","cluster_addr":"172.21.15.111:6803/3227179825","heartbeat_back_addr":"172.21.15.111:6807/3227179825","heartbeat_front_addr":"172.21.15.111:6805/3227179825","state":["exists","up"]},{"osd":3,"uuid":"d6295ffe-11bc-4964-a250-9fecca870b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6811","nonce":2390375937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6813","nonce":2390375937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6817","nonce":2390375937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6815","nonce":2390375937}]},"public_addr":"172.21.15.16:6811/2390375937","cluster_addr":"172.21.15.16:6813/2390375937","heartbeat_back_addr":"172.21.15.16:6817/2390375937","heartbeat_front_addr":"172.21.15.16:6815/2390375937","state":["exists","up"]},{"osd":4,"uuid":"0a8446e0-fc86-4646-beb6-a7b9d433093d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6809","nonce":1936582048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6811","nonce":1936582048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6815","nonce":1936582048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6813","nonce":1936582048}]},"public_addr":"172.21.15.107:6809/1936582048","cluster_addr":"172.21.15.107:6811/1936582048","heartbeat_back_addr":"172.21.15.107:6815/1936582048","heartbeat_front_addr":"172.21.15.107:6813/1936582048","state":["exists","up"]},{"osd":5,"uuid":"2c97a446-357e-4975-9e51-9c95fe84a663","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6809","nonce":483858448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6811","nonce":483858448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6815","nonce":483858448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6813","nonce":483858448}]},"public_addr":"172.21.15.111:6809/483858448","cluster_addr":"172.21.15.111:6811/483858448","heartbeat_back_addr":"172.21.15.111:6815/483858448","heartbeat_front_addr":"172.21.15.111:6813/483858448","state":["exists","up"]},{"osd":6,"uuid":"890239e6-e6fb-4f9c-b973-05cc66d199bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6819","nonce":1330214141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6821","nonce":1330214141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6825","nonce":1330214141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6823","nonce":1330214141}]},"public_addr":"172.21.15.16:6819/1330214141","cluster_addr":"172.21.15.16:6821/1330214141","heartbeat_back_addr":"172.21.15.16:6825/1330214141","heartbeat_front_addr":"172.21.15.16:6823/1330214141","state":["exists","up"]},{"osd":7,"uuid":"ee826a9c-f441-4903-9b1b-66b627f7b712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6817","nonce":1133876754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6819","nonce":1133876754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6823","nonce":1133876754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6821","nonce":1133876754}]},"public_addr":"172.21.15.107:6817/1133876754","cluster_addr":"172.21.15.107:6819/1133876754","heartbeat_back_addr":"172.21.15.107:6823/1133876754","heartbeat_front_addr":"172.21.15.107:6821/1133876754","state":["exists","up"]},{"osd":8,"uuid":"e19773f2-dec8-44e0-9d33-0252e07d6415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6817","nonce":837548190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6819","nonce":837548190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6823","nonce":837548190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6821","nonce":837548190}]},"public_addr":"172.21.15.111:6817/837548190","cluster_addr":"172.21.15.111:6819/837548190","heartbeat_back_addr":"172.21.15.111:6823/837548190","heartbeat_front_addr":"172.21.15.111:6821/837548190","state":["exists","up"]},{"osd":9,"uuid":"bae4d6db-8b90-4d02-8e83-7d8f2a6f8fdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6827","nonce":4175383686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6829","nonce":4175383686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6833","nonce":4175383686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6831","nonce":4175383686}]},"public_addr":"172.21.15.16:6827/4175383686","cluster_addr":"172.21.15.16:6829/4175383686","heartbeat_back_addr":"172.21.15.16:6833/4175383686","heartbeat_front_addr":"172.21.15.16:6831/4175383686","state":["exists","up"]},{"osd":10,"uuid":"fd0e7740-76c8-4c99-a349-7077b4905143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6825","nonce":2659550426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6827","nonce":2659550426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6831","nonce":2659550426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6829","nonce":2659550426}]},"public_addr":"172.21.15.107:6825/2659550426","cluster_addr":"172.21.15.107:6827/2659550426","heartbeat_back_addr":"172.21.15.107:6831/2659550426","heartbeat_front_addr":"172.21.15.107:6829/2659550426","state":["exists","up"]},{"osd":11,"uuid":"94a1751d-b00c-4e23-815d-3a8bc3dea146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6825","nonce":1633548683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6827","nonce":1633548683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6831","nonce":1633548683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6829","nonce":1633548683}]},"public_addr":"172.21.15.111:6825/1633548683","cluster_addr":"172.21.15.111:6827/1633548683","heartbeat_back_addr":"172.21.15.111:6831/1633548683","heartbeat_front_addr":"172.21.15.111:6829/1633548683","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:36.359852+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:58.014202+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:19.753672+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:43.404817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:06.001677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:29.361587+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:52.554847+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:15.957553+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:39.581093+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:04.911389+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:28.971280+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:53.473561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/2098378667":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/3536229016":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6801/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/2684687547":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6800/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/368410612":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/12158160":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6800/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6801/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/3754423379":"2024-09-14T16:21:40.126459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:29:00.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.799+0000 7f62e2725640 1 -- 172.21.15.16:0/1397456463 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f62c40a1110 msgr2=0x7f62c40a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:00.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.799+0000 7f62e2725640 1 --2- 172.21.15.16:0/1397456463 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f62c40a1110 0x7f62c40a35d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f62cc006010 tx=0x7f62cc005e30 comp rx=0 tx=0).stop 2024-09-13T16:29:00.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.799+0000 7f62e2725640 1 -- 172.21.15.16:0/1397456463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62dc103180 msgr2=0x7f62dc196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:00.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.799+0000 7f62e2725640 1 --2- 172.21.15.16:0/1397456463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62dc103180 0x7f62dc196280 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f62d000c970 tx=0x7f62d000ce30 comp rx=0 tx=0).stop 2024-09-13T16:29:00.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.799+0000 7f62e2725640 1 -- 172.21.15.16:0/1397456463 shutdown_connections 2024-09-13T16:29:00.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.799+0000 7f62e2725640 1 --2- 172.21.15.16:0/1397456463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62dc105ab0 0x7f62dc19d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:00.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.799+0000 7f62e2725640 1 --2- 172.21.15.16:0/1397456463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62dc103180 0x7f62dc196280 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:00.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.799+0000 7f62e2725640 1 --2- 172.21.15.16:0/1397456463 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f62c40a1110 0x7f62c40a35d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:00.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.799+0000 7f62e2725640 1 --2- 172.21.15.16:0/1397456463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62dc100850 0x7f62dc195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:00.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.799+0000 7f62e2725640 1 -- 172.21.15.16:0/1397456463 >> 172.21.15.16:0/1397456463 conn(0x7f62dc0fa430 msgr2=0x7f62dc1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:00.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.800+0000 7f62e2725640 1 -- 172.21.15.16:0/1397456463 shutdown_connections 2024-09-13T16:29:00.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.800+0000 7f62e2725640 1 -- 172.21.15.16:0/1397456463 wait complete. 2024-09-13T16:29:00.811 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:00 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1323825493' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:29:00.813 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-13T16:29:00.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:00 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1323825493' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:29:00.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.916+0000 7f0e96012640 1 -- 172.21.15.16:0/3883507069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e900fe7c0 msgr2=0x7f0e90108370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:00.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.916+0000 7f0e96012640 1 --2- 172.21.15.16:0/3883507069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e900fe7c0 0x7f0e90108370 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0e88009ca0 tx=0x7f0e8802a8c0 comp rx=0 tx=0).stop 2024-09-13T16:29:00.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.916+0000 7f0e96012640 1 -- 172.21.15.16:0/3883507069 shutdown_connections 2024-09-13T16:29:00.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.916+0000 7f0e96012640 1 --2- 172.21.15.16:0/3883507069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e900fe7c0 0x7f0e90108370 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:00.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.916+0000 7f0e96012640 1 --2- 172.21.15.16:0/3883507069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e900fde00 0x7f0e900fe280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:00.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.916+0000 7f0e96012640 1 --2- 172.21.15.16:0/3883507069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e90101390 0x7f0e90101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:00.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.916+0000 7f0e96012640 1 -- 172.21.15.16:0/3883507069 >> 172.21.15.16:0/3883507069 conn(0x7f0e900f9b70 msgr2=0x7f0e900fbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:00.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.916+0000 7f0e96012640 1 -- 172.21.15.16:0/3883507069 shutdown_connections 2024-09-13T16:29:00.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.916+0000 7f0e96012640 1 -- 172.21.15.16:0/3883507069 wait complete. 2024-09-13T16:29:00.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.916+0000 7f0e96012640 1 Processor -- start 2024-09-13T16:29:00.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.917+0000 7f0e96012640 1 -- start start 2024-09-13T16:29:00.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.917+0000 7f0e96012640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e900fde00 0x7f0e90195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:00.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.917+0000 7f0e96012640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e900fe7c0 0x7f0e90196230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:00.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.917+0000 7f0e96012640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e90101390 0x7f0e9019d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:00.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.917+0000 7f0e96012640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9019d7d0 con 0x7f0e900fe7c0 2024-09-13T16:29:00.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.917+0000 7f0e96012640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9019d940 con 0x7f0e900fde00 2024-09-13T16:29:00.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.917+0000 7f0e96012640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9019db00 con 0x7f0e90101390 2024-09-13T16:29:00.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.917+0000 7f0e8effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e900fe7c0 0x7f0e90196230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:00.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.917+0000 7f0e8effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e900fe7c0 0x7f0e90196230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41738/0 (socket says 172.21.15.16:41738) 2024-09-13T16:29:00.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.917+0000 7f0e8effd640 1 -- 172.21.15.16:0/2197860872 learned_addr learned my addr 172.21.15.16:0/2197860872 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:00.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.917+0000 7f0e8ffff640 1 --2- 172.21.15.16:0/2197860872 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e90101390 0x7f0e9019d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:00.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.917+0000 7f0e8f7fe640 1 --2- 172.21.15.16:0/2197860872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e900fde00 0x7f0e90195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:00.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.917+0000 7f0e8effd640 1 -- 172.21.15.16:0/2197860872 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e90101390 msgr2=0x7f0e9019d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:00.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.917+0000 7f0e8effd640 1 --2- 172.21.15.16:0/2197860872 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e90101390 0x7f0e9019d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:00.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.917+0000 7f0e8effd640 1 -- 172.21.15.16:0/2197860872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e900fde00 msgr2=0x7f0e90195cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:00.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.917+0000 7f0e8effd640 1 --2- 172.21.15.16:0/2197860872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e900fde00 0x7f0e90195cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:00.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.917+0000 7f0e8effd640 1 -- 172.21.15.16:0/2197860872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e88009900 con 0x7f0e900fe7c0 2024-09-13T16:29:00.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.917+0000 7f0e8effd640 1 --2- 172.21.15.16:0/2197860872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e900fe7c0 0x7f0e90196230 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400e9f0 tx=0x7f0e8400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:00.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.918+0000 7f0e8cff9640 1 -- 172.21.15.16:0/2197860872 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e84009700 con 0x7f0e900fe7c0 2024-09-13T16:29:00.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.918+0000 7f0e96012640 1 -- 172.21.15.16:0/2197860872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e9019dde0 con 0x7f0e900fe7c0 2024-09-13T16:29:00.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.918+0000 7f0e8cff9640 1 -- 172.21.15.16:0/2197860872 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0e8400cd80 con 0x7f0e900fe7c0 2024-09-13T16:29:00.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.918+0000 7f0e8cff9640 1 -- 172.21.15.16:0/2197860872 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e840187e0 con 0x7f0e900fe7c0 2024-09-13T16:29:00.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.918+0000 7f0e96012640 1 -- 172.21.15.16:0/2197860872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e9019e260 con 0x7f0e900fe7c0 2024-09-13T16:29:00.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.919+0000 7f0e96012640 1 -- 172.21.15.16:0/2197860872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e90107460 con 0x7f0e900fe7c0 2024-09-13T16:29:00.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.920+0000 7f0e8cff9640 1 -- 172.21.15.16:0/2197860872 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f0e84010430 con 0x7f0e900fe7c0 2024-09-13T16:29:00.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.920+0000 7f0e8cff9640 1 --2- 172.21.15.16:0/2197860872 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f0e780a1090 0x7f0e780a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:00.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.920+0000 7f0e8f7fe640 1 --2- 172.21.15.16:0/2197860872 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f0e780a1090 0x7f0e780a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:00.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.920+0000 7f0e8cff9640 1 -- 172.21.15.16:0/2197860872 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0e84014070 con 0x7f0e900fe7c0 2024-09-13T16:29:00.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.921+0000 7f0e8f7fe640 1 --2- 172.21.15.16:0/2197860872 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f0e780a1090 0x7f0e780a3550 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0e901971c0 tx=0x7f0e80005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:00.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:00.924+0000 7f0e8cff9640 1 -- 172.21.15.16:0/2197860872 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0e840968d0 con 0x7f0e900fe7c0 2024-09-13T16:29:00.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:00 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1323825493' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:29:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.131+0000 7f0e96012640 1 -- 172.21.15.16:0/2197860872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0e9010b540 con 0x7f0e900fe7c0 2024-09-13T16:29:01.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.555+0000 7f0e8cff9640 1 -- 172.21.15.16:0/2197860872 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0e84098800 con 0x7f0e900fe7c0 2024-09-13T16:29:01.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.558+0000 7f0e96012640 1 -- 172.21.15.16:0/2197860872 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f0e780a1090 msgr2=0x7f0e780a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:01.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.558+0000 7f0e96012640 1 --2- 172.21.15.16:0/2197860872 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f0e780a1090 0x7f0e780a3550 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0e901971c0 tx=0x7f0e80005f90 comp rx=0 tx=0).stop 2024-09-13T16:29:01.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.558+0000 7f0e96012640 1 -- 172.21.15.16:0/2197860872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e900fe7c0 msgr2=0x7f0e90196230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:01.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.558+0000 7f0e96012640 1 --2- 172.21.15.16:0/2197860872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e900fe7c0 0x7f0e90196230 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400e9f0 tx=0x7f0e8400eeb0 comp rx=0 tx=0).stop 2024-09-13T16:29:01.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.558+0000 7f0e96012640 1 -- 172.21.15.16:0/2197860872 shutdown_connections 2024-09-13T16:29:01.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.558+0000 7f0e96012640 1 --2- 172.21.15.16:0/2197860872 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e90101390 0x7f0e9019d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:01.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.558+0000 7f0e96012640 1 --2- 172.21.15.16:0/2197860872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e900fe7c0 0x7f0e90196230 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:01.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.558+0000 7f0e96012640 1 --2- 172.21.15.16:0/2197860872 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f0e780a1090 0x7f0e780a3550 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:01.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.558+0000 7f0e96012640 1 --2- 172.21.15.16:0/2197860872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e900fde00 0x7f0e90195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:01.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.558+0000 7f0e96012640 1 -- 172.21.15.16:0/2197860872 >> 172.21.15.16:0/2197860872 conn(0x7f0e900f9b70 msgr2=0x7f0e90104630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:01.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.558+0000 7f0e96012640 1 -- 172.21.15.16:0/2197860872 shutdown_connections 2024-09-13T16:29:01.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.558+0000 7f0e96012640 1 -- 172.21.15.16:0/2197860872 wait complete. 2024-09-13T16:29:01.572 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-09-13T16:29:01.572 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-09-13T16:29:01.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.679+0000 7f842079c640 1 -- 172.21.15.16:0/1325710452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8418100840 msgr2=0x7f84181070d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:01.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.679+0000 7f842079c640 1 --2- 172.21.15.16:0/1325710452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8418100840 0x7f84181070d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8410009ca0 tx=0x7f841002a8c0 comp rx=0 tx=0).stop 2024-09-13T16:29:01.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.679+0000 7f842079c640 1 -- 172.21.15.16:0/1325710452 shutdown_connections 2024-09-13T16:29:01.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.679+0000 7f842079c640 1 --2- 172.21.15.16:0/1325710452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8418100840 0x7f84181070d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:01.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.679+0000 7f842079c640 1 --2- 172.21.15.16:0/1325710452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84180ffe80 0x7f8418100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:01.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.679+0000 7f842079c640 1 --2- 172.21.15.16:0/1325710452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84180fec90 0x7f84180ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:01.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.679+0000 7f842079c640 1 -- 172.21.15.16:0/1325710452 >> 172.21.15.16:0/1325710452 conn(0x7f84180fa3f0 msgr2=0x7f84180fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:01.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.679+0000 7f842079c640 1 -- 172.21.15.16:0/1325710452 shutdown_connections 2024-09-13T16:29:01.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.679+0000 7f842079c640 1 -- 172.21.15.16:0/1325710452 wait complete. 2024-09-13T16:29:01.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.680+0000 7f842079c640 1 Processor -- start 2024-09-13T16:29:01.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.680+0000 7f842079c640 1 -- start start 2024-09-13T16:29:01.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.680+0000 7f842079c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84180fec90 0x7f8418102c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:01.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.680+0000 7f842079c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84180ffe80 0x7f8418101260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:01.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.680+0000 7f842079c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8418100840 0x7f84181017a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:01.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.680+0000 7f842079c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8418101f00 con 0x7f8418100840 2024-09-13T16:29:01.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.680+0000 7f842079c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8418102070 con 0x7f84180fec90 2024-09-13T16:29:01.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.680+0000 7f842079c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841819fc70 con 0x7f84180ffe80 2024-09-13T16:29:01.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.680+0000 7f841ed39640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8418100840 0x7f84181017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:01.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.680+0000 7f841ed39640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8418100840 0x7f84181017a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41760/0 (socket says 172.21.15.16:41760) 2024-09-13T16:29:01.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.680+0000 7f841ed39640 1 -- 172.21.15.16:0/3568552701 learned_addr learned my addr 172.21.15.16:0/3568552701 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:01.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.680+0000 7f841e538640 1 --2- 172.21.15.16:0/3568552701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84180fec90 0x7f8418102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:01.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.680+0000 7f841dd37640 1 --2- 172.21.15.16:0/3568552701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84180ffe80 0x7f8418101260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:01.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.680+0000 7f841ed39640 1 -- 172.21.15.16:0/3568552701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84180ffe80 msgr2=0x7f8418101260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:01.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.680+0000 7f841ed39640 1 --2- 172.21.15.16:0/3568552701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84180ffe80 0x7f8418101260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:01.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.680+0000 7f841ed39640 1 -- 172.21.15.16:0/3568552701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84180fec90 msgr2=0x7f8418102c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:01.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.680+0000 7f841ed39640 1 --2- 172.21.15.16:0/3568552701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84180fec90 0x7f8418102c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:01.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.680+0000 7f841ed39640 1 -- 172.21.15.16:0/3568552701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8410009900 con 0x7f8418100840 2024-09-13T16:29:01.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.680+0000 7f841e538640 1 --2- 172.21.15.16:0/3568552701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84180fec90 0x7f8418102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:29:01.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.681+0000 7f841ed39640 1 --2- 172.21.15.16:0/3568552701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8418100840 0x7f84181017a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8410003bd0 tx=0x7f8410003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:01.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.681+0000 7f840f7fe640 1 -- 172.21.15.16:0/3568552701 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84100317c0 con 0x7f8418100840 2024-09-13T16:29:01.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.681+0000 7f840f7fe640 1 -- 172.21.15.16:0/3568552701 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8410040070 con 0x7f8418100840 2024-09-13T16:29:01.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.681+0000 7f842079c640 1 -- 172.21.15.16:0/3568552701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f841819fec0 con 0x7f8418100840 2024-09-13T16:29:01.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.681+0000 7f840f7fe640 1 -- 172.21.15.16:0/3568552701 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84100046b0 con 0x7f8418100840 2024-09-13T16:29:01.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.681+0000 7f842079c640 1 -- 172.21.15.16:0/3568552701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84181a0340 con 0x7f8418100840 2024-09-13T16:29:01.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.682+0000 7f842079c640 1 -- 172.21.15.16:0/3568552701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f841810a430 con 0x7f8418100840 2024-09-13T16:29:01.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.683+0000 7f840f7fe640 1 -- 172.21.15.16:0/3568552701 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8410042030 con 0x7f8418100840 2024-09-13T16:29:01.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.684+0000 7f840f7fe640 1 --2- 172.21.15.16:0/3568552701 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f84040a1090 0x7f84040a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:01.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.684+0000 7f841e538640 1 --2- 172.21.15.16:0/3568552701 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f84040a1090 0x7f84040a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:01.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.684+0000 7f840f7fe640 1 -- 172.21.15.16:0/3568552701 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f84100ee3f0 con 0x7f8418100840 2024-09-13T16:29:01.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.684+0000 7f841e538640 1 --2- 172.21.15.16:0/3568552701 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f84040a1090 0x7f84040a3550 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f84180ffce0 tx=0x7f841400a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:01.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.688+0000 7f840f7fe640 1 -- 172.21.15.16:0/3568552701 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f84100b7930 con 0x7f8418100840 2024-09-13T16:29:01.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:01 smithi107 ceph-mon[34139]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-13T16:29:01.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:01 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1397456463' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:29:01.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:01 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/2197860872' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-13T16:29:01.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:01 smithi016 ceph-mon[31187]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-13T16:29:01.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:01 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1397456463' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:29:01.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:01 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2197860872' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-13T16:29:01.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:01.896+0000 7f842079c640 1 -- 172.21.15.16:0/3568552701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7f84181a07a0 con 0x7f8418100840 2024-09-13T16:29:01.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:01 smithi111 ceph-mon[34600]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-13T16:29:01.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:01 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1397456463' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:29:01.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:01 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/2197860872' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-13T16:29:02.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.565+0000 7f840f7fe640 1 -- 172.21.15.16:0/3568552701 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7f84100baf50 con 0x7f8418100840 2024-09-13T16:29:02.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.568+0000 7f842079c640 1 -- 172.21.15.16:0/3568552701 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f84040a1090 msgr2=0x7f84040a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:02.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.568+0000 7f842079c640 1 --2- 172.21.15.16:0/3568552701 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f84040a1090 0x7f84040a3550 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f84180ffce0 tx=0x7f841400a560 comp rx=0 tx=0).stop 2024-09-13T16:29:02.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.568+0000 7f842079c640 1 -- 172.21.15.16:0/3568552701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8418100840 msgr2=0x7f84181017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:02.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.568+0000 7f842079c640 1 --2- 172.21.15.16:0/3568552701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8418100840 0x7f84181017a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8410003bd0 tx=0x7f8410003c00 comp rx=0 tx=0).stop 2024-09-13T16:29:02.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.568+0000 7f842079c640 1 -- 172.21.15.16:0/3568552701 shutdown_connections 2024-09-13T16:29:02.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.568+0000 7f842079c640 1 --2- 172.21.15.16:0/3568552701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8418100840 0x7f84181017a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:02.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.568+0000 7f842079c640 1 --2- 172.21.15.16:0/3568552701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84180ffe80 0x7f8418101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:02.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.568+0000 7f842079c640 1 --2- 172.21.15.16:0/3568552701 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f84040a1090 0x7f84040a3550 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:02.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.568+0000 7f842079c640 1 --2- 172.21.15.16:0/3568552701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84180fec90 0x7f8418102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:02.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.568+0000 7f842079c640 1 -- 172.21.15.16:0/3568552701 >> 172.21.15.16:0/3568552701 conn(0x7f84180fa3f0 msgr2=0x7f84180685e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:02.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.568+0000 7f842079c640 1 -- 172.21.15.16:0/3568552701 shutdown_connections 2024-09-13T16:29:02.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.568+0000 7f842079c640 1 -- 172.21.15.16:0/3568552701 wait complete. 2024-09-13T16:29:02.584 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-09-13T16:29:02.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.691+0000 7f8e49cc9640 1 -- 172.21.15.16:0/2027344675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e44103140 msgr2=0x7f8e44105530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:02.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.691+0000 7f8e49cc9640 1 --2- 172.21.15.16:0/2027344675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e44103140 0x7f8e44105530 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8e38009900 tx=0x7f8e3802a3f0 comp rx=0 tx=0).stop 2024-09-13T16:29:02.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.691+0000 7f8e49cc9640 1 -- 172.21.15.16:0/2027344675 shutdown_connections 2024-09-13T16:29:02.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.691+0000 7f8e49cc9640 1 --2- 172.21.15.16:0/2027344675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e44105a70 0x7f8e44107ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:02.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.691+0000 7f8e49cc9640 1 --2- 172.21.15.16:0/2027344675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e44103140 0x7f8e44105530 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:02.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.691+0000 7f8e49cc9640 1 --2- 172.21.15.16:0/2027344675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e44100810 0x7f8e44102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:02.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.691+0000 7f8e49cc9640 1 -- 172.21.15.16:0/2027344675 >> 172.21.15.16:0/2027344675 conn(0x7f8e440fa3f0 msgr2=0x7f8e440fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:02.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.691+0000 7f8e49cc9640 1 -- 172.21.15.16:0/2027344675 shutdown_connections 2024-09-13T16:29:02.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.691+0000 7f8e49cc9640 1 -- 172.21.15.16:0/2027344675 wait complete. 2024-09-13T16:29:02.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.691+0000 7f8e49cc9640 1 Processor -- start 2024-09-13T16:29:02.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.691+0000 7f8e49cc9640 1 -- start start 2024-09-13T16:29:02.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.691+0000 7f8e49cc9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e44100810 0x7f8e44195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:02.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.691+0000 7f8e49cc9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e44103140 0x7f8e44196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:02.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.691+0000 7f8e49cc9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e44105a70 0x7f8e4419d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:02.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.691+0000 7f8e49cc9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4419d830 con 0x7f8e44103140 2024-09-13T16:29:02.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.691+0000 7f8e49cc9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4419d9a0 con 0x7f8e44100810 2024-09-13T16:29:02.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.691+0000 7f8e49cc9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4419db60 con 0x7f8e44105a70 2024-09-13T16:29:02.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.691+0000 7f8e42ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e44103140 0x7f8e44196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:02.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.691+0000 7f8e42ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e44103140 0x7f8e44196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41774/0 (socket says 172.21.15.16:41774) 2024-09-13T16:29:02.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.691+0000 7f8e42ffd640 1 -- 172.21.15.16:0/1761211461 learned_addr learned my addr 172.21.15.16:0/1761211461 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:02.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.691+0000 7f8e437fe640 1 --2- 172.21.15.16:0/1761211461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e44100810 0x7f8e44195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:02.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.691+0000 7f8e43fff640 1 --2- 172.21.15.16:0/1761211461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e44105a70 0x7f8e4419d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:02.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.691+0000 7f8e42ffd640 1 -- 172.21.15.16:0/1761211461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e44105a70 msgr2=0x7f8e4419d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:02.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.691+0000 7f8e42ffd640 1 --2- 172.21.15.16:0/1761211461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e44105a70 0x7f8e4419d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:02.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.691+0000 7f8e42ffd640 1 -- 172.21.15.16:0/1761211461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e44100810 msgr2=0x7f8e44195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:02.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.691+0000 7f8e42ffd640 1 --2- 172.21.15.16:0/1761211461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e44100810 0x7f8e44195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:02.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.691+0000 7f8e42ffd640 1 -- 172.21.15.16:0/1761211461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e38009560 con 0x7f8e44103140 2024-09-13T16:29:02.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.692+0000 7f8e42ffd640 1 --2- 172.21.15.16:0/1761211461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e44103140 0x7f8e44196280 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8e38005940 tx=0x7f8e380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:02.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.692+0000 7f8e40ff9640 1 -- 172.21.15.16:0/1761211461 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e38031d70 con 0x7f8e44103140 2024-09-13T16:29:02.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.692+0000 7f8e49cc9640 1 -- 172.21.15.16:0/1761211461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e4419dde0 con 0x7f8e44103140 2024-09-13T16:29:02.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.692+0000 7f8e40ff9640 1 -- 172.21.15.16:0/1761211461 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8e380044e0 con 0x7f8e44103140 2024-09-13T16:29:02.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.692+0000 7f8e40ff9640 1 -- 172.21.15.16:0/1761211461 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e38003b60 con 0x7f8e44103140 2024-09-13T16:29:02.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.692+0000 7f8e49cc9640 1 -- 172.21.15.16:0/1761211461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e4419e230 con 0x7f8e44103140 2024-09-13T16:29:02.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.694+0000 7f8e49cc9640 1 -- 172.21.15.16:0/1761211461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e440fbee0 con 0x7f8e44103140 2024-09-13T16:29:02.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.695+0000 7f8e40ff9640 1 -- 172.21.15.16:0/1761211461 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8e38003cc0 con 0x7f8e44103140 2024-09-13T16:29:02.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.696+0000 7f8e40ff9640 1 --2- 172.21.15.16:0/1761211461 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8e2c0a1090 0x7f8e2c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:02.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.696+0000 7f8e437fe640 1 --2- 172.21.15.16:0/1761211461 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8e2c0a1090 0x7f8e2c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:02.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.696+0000 7f8e40ff9640 1 -- 172.21.15.16:0/1761211461 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8e380edac0 con 0x7f8e44103140 2024-09-13T16:29:02.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.696+0000 7f8e437fe640 1 --2- 172.21.15.16:0/1761211461 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8e2c0a1090 0x7f8e2c0a3550 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8e340058c0 tx=0x7f8e3400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:02.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.699+0000 7f8e40ff9640 1 -- 172.21.15.16:0/1761211461 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f8e380b7110 con 0x7f8e44103140 2024-09-13T16:29:02.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:02 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/2197860872' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-13T16:29:02.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:02 smithi107 ceph-mon[34139]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-13T16:29:02.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:02 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3568552701' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-13T16:29:02.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:02 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2197860872' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-13T16:29:02.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:02 smithi016 ceph-mon[31187]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-13T16:29:02.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:02 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3568552701' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-13T16:29:02.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:02.913+0000 7f8e49cc9640 1 -- 172.21.15.16:0/1761211461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f8e4402c3e0 con 0x7f8e44103140 2024-09-13T16:29:02.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:02 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/2197860872' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-13T16:29:02.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:02 smithi111 ceph-mon[34600]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-13T16:29:02.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:02 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3568552701' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-13T16:29:03.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:03.572+0000 7f8e40ff9640 1 -- 172.21.15.16:0/1761211461 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7f8e380400c0 con 0x7f8e44103140 2024-09-13T16:29:04.082 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:03 smithi111 ceph-mon[34600]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-13T16:29:04.082 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:03 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3568552701' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-13T16:29:04.082 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:03 smithi111 ceph-mon[34600]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-13T16:29:04.082 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:03 smithi111 ceph-mon[34600]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-13T16:29:04.083 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:03 smithi111 ceph-mon[34600]: pgmap v288: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-13T16:29:04.083 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:03 smithi111 ceph-mon[34600]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-13T16:29:04.083 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:03 smithi111 ceph-mon[34600]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-13T16:29:04.083 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:03 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1761211461' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-13T16:29:04.083 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:03 smithi107 ceph-mon[34139]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-13T16:29:04.083 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:03 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3568552701' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-13T16:29:04.084 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:03 smithi107 ceph-mon[34139]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-13T16:29:04.084 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:03 smithi107 ceph-mon[34139]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-13T16:29:04.084 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:03 smithi107 ceph-mon[34139]: pgmap v288: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-13T16:29:04.084 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:03 smithi107 ceph-mon[34139]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-13T16:29:04.084 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:03 smithi107 ceph-mon[34139]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-13T16:29:04.084 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:03 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1761211461' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-13T16:29:04.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:03.575+0000 7f8e49cc9640 1 -- 172.21.15.16:0/1761211461 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8e2c0a1090 msgr2=0x7f8e2c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:04.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:03.575+0000 7f8e49cc9640 1 --2- 172.21.15.16:0/1761211461 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8e2c0a1090 0x7f8e2c0a3550 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8e340058c0 tx=0x7f8e3400a2d0 comp rx=0 tx=0).stop 2024-09-13T16:29:04.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:03.575+0000 7f8e49cc9640 1 -- 172.21.15.16:0/1761211461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e44103140 msgr2=0x7f8e44196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:04.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:03.575+0000 7f8e49cc9640 1 --2- 172.21.15.16:0/1761211461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e44103140 0x7f8e44196280 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8e38005940 tx=0x7f8e380042f0 comp rx=0 tx=0).stop 2024-09-13T16:29:04.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:03.575+0000 7f8e49cc9640 1 -- 172.21.15.16:0/1761211461 shutdown_connections 2024-09-13T16:29:04.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:03.575+0000 7f8e49cc9640 1 --2- 172.21.15.16:0/1761211461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e44105a70 0x7f8e4419d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:04.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:03.575+0000 7f8e49cc9640 1 --2- 172.21.15.16:0/1761211461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e44103140 0x7f8e44196280 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:04.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:03.575+0000 7f8e49cc9640 1 --2- 172.21.15.16:0/1761211461 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8e2c0a1090 0x7f8e2c0a3550 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:04.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:03.575+0000 7f8e49cc9640 1 --2- 172.21.15.16:0/1761211461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e44100810 0x7f8e44195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:04.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:03.575+0000 7f8e49cc9640 1 -- 172.21.15.16:0/1761211461 >> 172.21.15.16:0/1761211461 conn(0x7f8e440fa3f0 msgr2=0x7f8e44101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:04.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:03.575+0000 7f8e49cc9640 1 -- 172.21.15.16:0/1761211461 shutdown_connections 2024-09-13T16:29:04.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:03.575+0000 7f8e49cc9640 1 -- 172.21.15.16:0/1761211461 wait complete. 2024-09-13T16:29:04.088 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-09-13T16:29:04.088 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:03 smithi016 ceph-mon[31187]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-13T16:29:04.088 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:03 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3568552701' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-13T16:29:04.088 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:03 smithi016 ceph-mon[31187]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-13T16:29:04.088 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:03 smithi016 ceph-mon[31187]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-13T16:29:04.088 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:03 smithi016 ceph-mon[31187]: pgmap v288: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-13T16:29:04.089 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:03 smithi016 ceph-mon[31187]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-13T16:29:04.089 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:03 smithi016 ceph-mon[31187]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-13T16:29:04.089 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:03 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1761211461' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-13T16:29:04.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.207+0000 7fdcb2a87640 1 -- 172.21.15.16:0/472415337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcac103140 msgr2=0x7fdcac105530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:04.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.207+0000 7fdcb2a87640 1 --2- 172.21.15.16:0/472415337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcac103140 0x7fdcac105530 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdca0009900 tx=0x7fdca002a410 comp rx=0 tx=0).stop 2024-09-13T16:29:04.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.207+0000 7fdcb2a87640 1 -- 172.21.15.16:0/472415337 shutdown_connections 2024-09-13T16:29:04.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.207+0000 7fdcb2a87640 1 --2- 172.21.15.16:0/472415337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcac105a70 0x7fdcac107ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:04.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.207+0000 7fdcb2a87640 1 --2- 172.21.15.16:0/472415337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcac103140 0x7fdcac105530 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:04.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.207+0000 7fdcb2a87640 1 --2- 172.21.15.16:0/472415337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdcac100810 0x7fdcac102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:04.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.207+0000 7fdcb2a87640 1 -- 172.21.15.16:0/472415337 >> 172.21.15.16:0/472415337 conn(0x7fdcac0fa3f0 msgr2=0x7fdcac0fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:04.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.208+0000 7fdcb2a87640 1 -- 172.21.15.16:0/472415337 shutdown_connections 2024-09-13T16:29:04.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.208+0000 7fdcb2a87640 1 -- 172.21.15.16:0/472415337 wait complete. 2024-09-13T16:29:04.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.208+0000 7fdcb2a87640 1 Processor -- start 2024-09-13T16:29:04.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.208+0000 7fdcb2a87640 1 -- start start 2024-09-13T16:29:04.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.208+0000 7fdcb2a87640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcac100810 0x7fdcac197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:04.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.208+0000 7fdcb2a87640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcac103140 0x7fdcac198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:04.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.208+0000 7fdcb2a87640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdcac105a70 0x7fdcac19f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:04.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.208+0000 7fdcb2a87640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac19fa00 con 0x7fdcac100810 2024-09-13T16:29:04.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.208+0000 7fdcb2a87640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac19fb70 con 0x7fdcac103140 2024-09-13T16:29:04.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.208+0000 7fdcb2a87640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac19fd30 con 0x7fdcac105a70 2024-09-13T16:29:04.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.208+0000 7fdcb0823640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcac100810 0x7fdcac197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:04.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.208+0000 7fdcb0823640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcac100810 0x7fdcac197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49560/0 (socket says 172.21.15.16:49560) 2024-09-13T16:29:04.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.208+0000 7fdcb0823640 1 -- 172.21.15.16:0/741495385 learned_addr learned my addr 172.21.15.16:0/741495385 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:04.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.209+0000 7fdcb1024640 1 --2- 172.21.15.16:0/741495385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdcac105a70 0x7fdcac19f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:04.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.209+0000 7fdcabfff640 1 --2- 172.21.15.16:0/741495385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcac103140 0x7fdcac198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:04.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.209+0000 7fdcb0823640 1 -- 172.21.15.16:0/741495385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdcac105a70 msgr2=0x7fdcac19f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:04.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.209+0000 7fdcb0823640 1 --2- 172.21.15.16:0/741495385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdcac105a70 0x7fdcac19f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:04.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.209+0000 7fdcb0823640 1 -- 172.21.15.16:0/741495385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcac103140 msgr2=0x7fdcac198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:04.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.209+0000 7fdcb0823640 1 --2- 172.21.15.16:0/741495385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcac103140 0x7fdcac198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:04.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.209+0000 7fdcb0823640 1 -- 172.21.15.16:0/741495385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca0009560 con 0x7fdcac100810 2024-09-13T16:29:04.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.209+0000 7fdcabfff640 1 --2- 172.21.15.16:0/741495385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcac103140 0x7fdcac198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:29:04.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.209+0000 7fdcb0823640 1 --2- 172.21.15.16:0/741495385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcac100810 0x7fdcac197f10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00d950 tx=0x7fdc9c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:04.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.209+0000 7fdca9ffb640 1 -- 172.21.15.16:0/741495385 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c009c30 con 0x7fdcac100810 2024-09-13T16:29:04.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.209+0000 7fdca9ffb640 1 -- 172.21.15.16:0/741495385 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdc9c009d90 con 0x7fdcac100810 2024-09-13T16:29:04.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.209+0000 7fdcb2a87640 1 -- 172.21.15.16:0/741495385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcac19ffb0 con 0x7fdcac100810 2024-09-13T16:29:04.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.209+0000 7fdca9ffb640 1 -- 172.21.15.16:0/741495385 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c00f620 con 0x7fdcac100810 2024-09-13T16:29:04.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.210+0000 7fdcb2a87640 1 -- 172.21.15.16:0/741495385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcac1a0480 con 0x7fdcac100810 2024-09-13T16:29:04.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.210+0000 7fdcb2a87640 1 -- 172.21.15.16:0/741495385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcac1075b0 con 0x7fdcac100810 2024-09-13T16:29:04.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.212+0000 7fdca9ffb640 1 -- 172.21.15.16:0/741495385 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fdc9c0109f0 con 0x7fdcac100810 2024-09-13T16:29:04.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.212+0000 7fdca9ffb640 1 --2- 172.21.15.16:0/741495385 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdc940a1110 0x7fdc940a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:04.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.212+0000 7fdca9ffb640 1 -- 172.21.15.16:0/741495385 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fdc9c0cc670 con 0x7fdcac100810 2024-09-13T16:29:04.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.212+0000 7fdcabfff640 1 --2- 172.21.15.16:0/741495385 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdc940a1110 0x7fdc940a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:04.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.213+0000 7fdcabfff640 1 --2- 172.21.15.16:0/741495385 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdc940a1110 0x7fdc940a35d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdca0006010 tx=0x7fdca0005e00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:04.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.217+0000 7fdca9ffb640 1 -- 172.21.15.16:0/741495385 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fdc9c095c10 con 0x7fdcac100810 2024-09-13T16:29:04.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.427+0000 7fdcb2a87640 1 -- 172.21.15.16:0/741495385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fdcac10b510 con 0x7fdcac100810 2024-09-13T16:29:04.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.623+0000 7fdca9ffb640 1 -- 172.21.15.16:0/741495385 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) v1 ==== 115+0+0 (secure 0 0 0) 0x7fdc9c099230 con 0x7fdcac100810 2024-09-13T16:29:04.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.626+0000 7fdcb2a87640 1 -- 172.21.15.16:0/741495385 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdc940a1110 msgr2=0x7fdc940a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:04.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.626+0000 7fdcb2a87640 1 --2- 172.21.15.16:0/741495385 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdc940a1110 0x7fdc940a35d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdca0006010 tx=0x7fdca0005e00 comp rx=0 tx=0).stop 2024-09-13T16:29:04.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.626+0000 7fdcb2a87640 1 -- 172.21.15.16:0/741495385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcac100810 msgr2=0x7fdcac197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:04.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.626+0000 7fdcb2a87640 1 --2- 172.21.15.16:0/741495385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcac100810 0x7fdcac197f10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00d950 tx=0x7fdc9c00de10 comp rx=0 tx=0).stop 2024-09-13T16:29:04.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.626+0000 7fdcb2a87640 1 -- 172.21.15.16:0/741495385 shutdown_connections 2024-09-13T16:29:04.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.626+0000 7fdcb2a87640 1 --2- 172.21.15.16:0/741495385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdcac105a70 0x7fdcac19f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:04.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.626+0000 7fdcb2a87640 1 --2- 172.21.15.16:0/741495385 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdc940a1110 0x7fdc940a35d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:04.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.626+0000 7fdcb2a87640 1 --2- 172.21.15.16:0/741495385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcac103140 0x7fdcac198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:04.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.626+0000 7fdcb2a87640 1 --2- 172.21.15.16:0/741495385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcac100810 0x7fdcac197f10 secure :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00d950 tx=0x7fdc9c00de10 comp rx=0 tx=0).stop 2024-09-13T16:29:04.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.626+0000 7fdcb2a87640 1 -- 172.21.15.16:0/741495385 >> 172.21.15.16:0/741495385 conn(0x7fdcac0fa3f0 msgr2=0x7fdcac0feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:04.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.626+0000 7fdcb2a87640 1 -- 172.21.15.16:0/741495385 shutdown_connections 2024-09-13T16:29:04.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.627+0000 7fdcb2a87640 1 -- 172.21.15.16:0/741495385 wait complete. 2024-09-13T16:29:04.644 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-09-13T16:29:04.644 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-09-13T16:29:04.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.749+0000 7f7384df4640 1 -- 172.21.15.16:0/3033819545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73800ffe80 msgr2=0x7f7380100300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:04.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.749+0000 7f7384df4640 1 --2- 172.21.15.16:0/3033819545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73800ffe80 0x7f7380100300 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7374009900 tx=0x7f737402a410 comp rx=0 tx=0).stop 2024-09-13T16:29:04.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.749+0000 7f7384df4640 1 -- 172.21.15.16:0/3033819545 shutdown_connections 2024-09-13T16:29:04.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.749+0000 7f7384df4640 1 --2- 172.21.15.16:0/3033819545 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7380100840 0x7f73801070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:04.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.749+0000 7f7384df4640 1 --2- 172.21.15.16:0/3033819545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73800ffe80 0x7f7380100300 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:04.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.749+0000 7f7384df4640 1 --2- 172.21.15.16:0/3033819545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73800fec90 0x7f73800ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:04.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.749+0000 7f7384df4640 1 -- 172.21.15.16:0/3033819545 >> 172.21.15.16:0/3033819545 conn(0x7f73800fa3d0 msgr2=0x7f73800fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:04.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.749+0000 7f7384df4640 1 -- 172.21.15.16:0/3033819545 shutdown_connections 2024-09-13T16:29:04.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.749+0000 7f7384df4640 1 -- 172.21.15.16:0/3033819545 wait complete. 2024-09-13T16:29:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.750+0000 7f7384df4640 1 Processor -- start 2024-09-13T16:29:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.750+0000 7f7384df4640 1 -- start start 2024-09-13T16:29:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.750+0000 7f7384df4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73800fec90 0x7f7380195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.750+0000 7f7384df4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73800ffe80 0x7f7380196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.750+0000 7f7384df4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7380100840 0x7f738019d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.750+0000 7f7384df4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738019d830 con 0x7f73800ffe80 2024-09-13T16:29:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.750+0000 7f7384df4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738019d9a0 con 0x7f7380100840 2024-09-13T16:29:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.750+0000 7f7384df4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738019db60 con 0x7f73800fec90 2024-09-13T16:29:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.750+0000 7f737dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73800ffe80 0x7f7380196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.750+0000 7f737dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73800ffe80 0x7f7380196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49588/0 (socket says 172.21.15.16:49588) 2024-09-13T16:29:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.750+0000 7f737dd9b640 1 -- 172.21.15.16:0/1533411220 learned_addr learned my addr 172.21.15.16:0/1533411220 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.750+0000 7f737ed9d640 1 --2- 172.21.15.16:0/1533411220 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7380100840 0x7f738019d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.750+0000 7f737e59c640 1 --2- 172.21.15.16:0/1533411220 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73800fec90 0x7f7380195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.751+0000 7f737dd9b640 1 -- 172.21.15.16:0/1533411220 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73800fec90 msgr2=0x7f7380195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.751+0000 7f737dd9b640 1 --2- 172.21.15.16:0/1533411220 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73800fec90 0x7f7380195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.751+0000 7f737dd9b640 1 -- 172.21.15.16:0/1533411220 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7380100840 msgr2=0x7f738019d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.751+0000 7f737dd9b640 1 --2- 172.21.15.16:0/1533411220 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7380100840 0x7f738019d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.751+0000 7f737dd9b640 1 -- 172.21.15.16:0/1533411220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7374009560 con 0x7f73800ffe80 2024-09-13T16:29:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.751+0000 7f737dd9b640 1 --2- 172.21.15.16:0/1533411220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73800ffe80 0x7f7380196280 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7374009230 tx=0x7f7374004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.751+0000 7f736f7fe640 1 -- 172.21.15.16:0/1533411220 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7374031d70 con 0x7f73800ffe80 2024-09-13T16:29:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.751+0000 7f736f7fe640 1 -- 172.21.15.16:0/1533411220 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7374004b50 con 0x7f73800ffe80 2024-09-13T16:29:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.751+0000 7f7384df4640 1 -- 172.21.15.16:0/1533411220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f738019dde0 con 0x7f73800ffe80 2024-09-13T16:29:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.751+0000 7f736f7fe640 1 -- 172.21.15.16:0/1533411220 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7374003bf0 con 0x7f73800ffe80 2024-09-13T16:29:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.751+0000 7f7384df4640 1 -- 172.21.15.16:0/1533411220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f738019e230 con 0x7f73800ffe80 2024-09-13T16:29:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.752+0000 7f7384df4640 1 -- 172.21.15.16:0/1533411220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f738010a500 con 0x7f73800ffe80 2024-09-13T16:29:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.753+0000 7f736f7fe640 1 -- 172.21.15.16:0/1533411220 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7374003d50 con 0x7f73800ffe80 2024-09-13T16:29:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.754+0000 7f736f7fe640 1 --2- 172.21.15.16:0/1533411220 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f73680a1090 0x7f73680a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.754+0000 7f736f7fe640 1 -- 172.21.15.16:0/1533411220 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f73740ee230 con 0x7f73800ffe80 2024-09-13T16:29:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.754+0000 7f737e59c640 1 --2- 172.21.15.16:0/1533411220 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f73680a1090 0x7f73680a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.754+0000 7f737e59c640 1 --2- 172.21.15.16:0/1533411220 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f73680a1090 0x7f73680a3550 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f73800ffce0 tx=0x7f7370008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:04.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.758+0000 7f736f7fe640 1 -- 172.21.15.16:0/1533411220 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f73740b7650 con 0x7f73800ffe80 2024-09-13T16:29:04.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:04 smithi107 ceph-mon[34139]: mds.? [v2:172.21.15.111:6838/1423842285,v1:172.21.15.111:6839/1423842285] up:active 2024-09-13T16:29:04.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:04 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1761211461' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-13T16:29:04.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:04 smithi107 ceph-mon[34139]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-13T16:29:04.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:04 smithi107 ceph-mon[34139]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-13T16:29:04.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:04 smithi107 ceph-mon[34139]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-13T16:29:04.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:04 smithi107 ceph-mon[34139]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-13T16:29:04.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:04 smithi107 ceph-mon[34139]: mds.? [v2:172.21.15.107:6836/2838457611,v1:172.21.15.107:6837/2838457611] up:active 2024-09-13T16:29:04.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:04 smithi107 ceph-mon[34139]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-13T16:29:04.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:04 smithi107 ceph-mon[34139]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-13T16:29:04.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:04 smithi107 ceph-mon[34139]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-09-13T16:29:04.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:04 smithi107 ceph-mon[34139]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-09-13T16:29:04.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:04 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/741495385' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-13T16:29:04.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-09-13T16:29:04.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-13T16:29:04.851 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-09-13T16:29:04.851 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-09-13T16:29:04.851 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-09-13T16:29:04.851 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-09-13T16:29:04.851 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-13T16:29:04.851 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-13T16:29:04.851 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-09-13T16:29:04.851 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-09-13T16:29:04.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:04 smithi016 ceph-mon[31187]: mds.? [v2:172.21.15.111:6838/1423842285,v1:172.21.15.111:6839/1423842285] up:active 2024-09-13T16:29:04.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:04 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1761211461' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-13T16:29:04.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:04 smithi016 ceph-mon[31187]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-13T16:29:04.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:04 smithi016 ceph-mon[31187]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-13T16:29:04.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:04 smithi016 ceph-mon[31187]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-13T16:29:04.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:04 smithi016 ceph-mon[31187]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-13T16:29:04.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:04 smithi016 ceph-mon[31187]: mds.? [v2:172.21.15.107:6836/2838457611,v1:172.21.15.107:6837/2838457611] up:active 2024-09-13T16:29:04.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:04 smithi016 ceph-mon[31187]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-13T16:29:04.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:04 smithi016 ceph-mon[31187]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-13T16:29:04.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:04 smithi016 ceph-mon[31187]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-09-13T16:29:04.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:04 smithi016 ceph-mon[31187]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-09-13T16:29:04.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:04 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/741495385' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-13T16:29:04.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-09-13T16:29:04.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-13T16:29:04.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-09-13T16:29:04.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-09-13T16:29:04.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-09-13T16:29:04.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-09-13T16:29:04.856 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-13T16:29:04.856 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-13T16:29:04.856 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-09-13T16:29:04.856 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-09-13T16:29:04.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:04 smithi111 ceph-mon[34600]: mds.? [v2:172.21.15.111:6838/1423842285,v1:172.21.15.111:6839/1423842285] up:active 2024-09-13T16:29:04.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:04 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1761211461' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-13T16:29:04.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:04 smithi111 ceph-mon[34600]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-13T16:29:04.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:04 smithi111 ceph-mon[34600]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-13T16:29:04.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:04 smithi111 ceph-mon[34600]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-13T16:29:04.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:04 smithi111 ceph-mon[34600]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-13T16:29:04.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:04 smithi111 ceph-mon[34600]: mds.? [v2:172.21.15.107:6836/2838457611,v1:172.21.15.107:6837/2838457611] up:active 2024-09-13T16:29:04.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:04 smithi111 ceph-mon[34600]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-13T16:29:04.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:04 smithi111 ceph-mon[34600]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-13T16:29:04.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:04 smithi111 ceph-mon[34600]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-09-13T16:29:04.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:04 smithi111 ceph-mon[34600]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-09-13T16:29:04.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:04 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/741495385' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-13T16:29:04.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-09-13T16:29:04.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-13T16:29:04.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-09-13T16:29:04.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-09-13T16:29:04.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-09-13T16:29:04.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-09-13T16:29:04.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-13T16:29:04.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-13T16:29:04.939 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-09-13T16:29:04.939 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-09-13T16:29:04.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:04.953+0000 7f7384df4640 1 -- 172.21.15.16:0/1533411220 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f7380068cf0 con 0x7f73680a1090 2024-09-13T16:29:05.354 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:29:04 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:29:04.958+0000 7fa29c8c8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T16:29:05.355 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:29:04 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:29:04.958+0000 7fa29c8c8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T16:29:05.355 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:29:04 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:29:04.958+0000 7fa29c8c8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T16:29:05.355 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:29:04 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:29:04.958+0000 7fa29c8c8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T16:29:05.355 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:29:04 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:29:04.958+0000 7fa29c8c8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T16:29:05.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:05 smithi107 ceph-mon[34139]: pgmap v289: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 1 op/s 2024-09-13T16:29:05.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:05 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-09-13T16:29:05.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:05 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-13T16:29:05.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:05 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-13T16:29:05.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:05 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-09-13T16:29:05.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:05 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-09-13T16:29:05.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:05 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-09-13T16:29:05.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:05 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-13T16:29:05.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:05 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-13T16:29:05.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:05 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-09-13T16:29:05.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:05 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-09-13T16:29:05.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:05 smithi107 ceph-mon[34139]: osdmap e90: 12 total, 12 up, 12 in 2024-09-13T16:29:05.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:05 smithi107 ceph-mon[34139]: mds.? [v2:172.21.15.107:6832/3797712690,v1:172.21.15.107:6833/3797712690] up:active 2024-09-13T16:29:05.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:05 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/741495385' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-13T16:29:05.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:05 smithi107 ceph-mon[34139]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-13T16:29:05.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:05 smithi107 ceph-mon[34139]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-13T16:29:05.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:05 smithi107 ceph-mon[34139]: Cluster is now healthy 2024-09-13T16:29:05.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:05 smithi107 ceph-mon[34139]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-09-13T16:29:05.851 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:05 smithi107 ceph-mon[34139]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-09-13T16:29:05.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:05 smithi016 ceph-mon[31187]: pgmap v289: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 1 op/s 2024-09-13T16:29:05.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:05 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-09-13T16:29:05.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:05 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-13T16:29:05.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:05 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-13T16:29:05.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:05 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-09-13T16:29:05.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:05 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-09-13T16:29:05.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:05 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-09-13T16:29:05.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:05 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-13T16:29:05.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:05 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-13T16:29:05.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:05 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-09-13T16:29:05.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:05 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-09-13T16:29:05.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:05 smithi016 ceph-mon[31187]: osdmap e90: 12 total, 12 up, 12 in 2024-09-13T16:29:05.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:05 smithi016 ceph-mon[31187]: mds.? [v2:172.21.15.107:6832/3797712690,v1:172.21.15.107:6833/3797712690] up:active 2024-09-13T16:29:05.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:05 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/741495385' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-13T16:29:05.856 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:05 smithi016 ceph-mon[31187]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-13T16:29:05.856 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:05 smithi016 ceph-mon[31187]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-13T16:29:05.856 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:05 smithi016 ceph-mon[31187]: Cluster is now healthy 2024-09-13T16:29:05.856 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:05 smithi016 ceph-mon[31187]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-09-13T16:29:05.856 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:05 smithi016 ceph-mon[31187]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-09-13T16:29:05.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:05 smithi111 ceph-mon[34600]: pgmap v289: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 1 op/s 2024-09-13T16:29:05.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:05 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-09-13T16:29:05.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:05 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-13T16:29:05.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:05 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-13T16:29:05.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:05 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-09-13T16:29:05.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:05 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-09-13T16:29:05.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:05 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-09-13T16:29:05.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:05 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-13T16:29:05.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:05 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-13T16:29:05.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:05 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-09-13T16:29:05.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:05 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-09-13T16:29:05.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:05 smithi111 ceph-mon[34600]: osdmap e90: 12 total, 12 up, 12 in 2024-09-13T16:29:05.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:05 smithi111 ceph-mon[34600]: mds.? [v2:172.21.15.107:6832/3797712690,v1:172.21.15.107:6833/3797712690] up:active 2024-09-13T16:29:05.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:05 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/741495385' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-13T16:29:05.939 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:05 smithi111 ceph-mon[34600]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-13T16:29:05.939 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:05 smithi111 ceph-mon[34600]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-13T16:29:05.939 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:05 smithi111 ceph-mon[34600]: Cluster is now healthy 2024-09-13T16:29:05.939 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:05 smithi111 ceph-mon[34600]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-09-13T16:29:05.939 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:05 smithi111 ceph-mon[34600]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-09-13T16:29:06.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.651+0000 7f736f7fe640 1 -- 172.21.15.16:0/1533411220 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7380068cf0 con 0x7f73680a1090 2024-09-13T16:29:06.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.654+0000 7f7384df4640 1 -- 172.21.15.16:0/1533411220 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f73680a1090 msgr2=0x7f73680a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:06.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.654+0000 7f7384df4640 1 --2- 172.21.15.16:0/1533411220 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f73680a1090 0x7f73680a3550 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f73800ffce0 tx=0x7f7370008040 comp rx=0 tx=0).stop 2024-09-13T16:29:06.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.654+0000 7f7384df4640 1 -- 172.21.15.16:0/1533411220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73800ffe80 msgr2=0x7f7380196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:06.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.654+0000 7f7384df4640 1 --2- 172.21.15.16:0/1533411220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73800ffe80 0x7f7380196280 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7374009230 tx=0x7f7374004380 comp rx=0 tx=0).stop 2024-09-13T16:29:06.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.654+0000 7f7384df4640 1 -- 172.21.15.16:0/1533411220 shutdown_connections 2024-09-13T16:29:06.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.654+0000 7f7384df4640 1 --2- 172.21.15.16:0/1533411220 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f73680a1090 0x7f73680a3550 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:06.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.654+0000 7f7384df4640 1 --2- 172.21.15.16:0/1533411220 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7380100840 0x7f738019d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:06.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.654+0000 7f7384df4640 1 --2- 172.21.15.16:0/1533411220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73800ffe80 0x7f7380196280 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:06.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.654+0000 7f7384df4640 1 --2- 172.21.15.16:0/1533411220 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73800fec90 0x7f7380195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:06.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.654+0000 7f7384df4640 1 -- 172.21.15.16:0/1533411220 >> 172.21.15.16:0/1533411220 conn(0x7f73800fa3d0 msgr2=0x7f73800685e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:06.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.654+0000 7f7384df4640 1 -- 172.21.15.16:0/1533411220 shutdown_connections 2024-09-13T16:29:06.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.654+0000 7f7384df4640 1 -- 172.21.15.16:0/1533411220 wait complete. 2024-09-13T16:29:06.667 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-13T16:29:06.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.777+0000 7fd359baa640 1 -- 172.21.15.16:0/3953036434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd354100820 msgr2=0x7fd3541070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:06.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.777+0000 7fd359baa640 1 --2- 172.21.15.16:0/3953036434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd354100820 0x7fd3541070b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd34c009ca0 tx=0x7fd34c02a8c0 comp rx=0 tx=0).stop 2024-09-13T16:29:06.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.777+0000 7fd359baa640 1 -- 172.21.15.16:0/3953036434 shutdown_connections 2024-09-13T16:29:06.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.777+0000 7fd359baa640 1 --2- 172.21.15.16:0/3953036434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd354100820 0x7fd3541070b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:06.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.777+0000 7fd359baa640 1 --2- 172.21.15.16:0/3953036434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3540ffe60 0x7fd3541002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:06.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.777+0000 7fd359baa640 1 --2- 172.21.15.16:0/3953036434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3540fec70 0x7fd3540ff070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:06.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.777+0000 7fd359baa640 1 -- 172.21.15.16:0/3953036434 >> 172.21.15.16:0/3953036434 conn(0x7fd3540fa3d0 msgr2=0x7fd3540fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:06.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.777+0000 7fd359baa640 1 -- 172.21.15.16:0/3953036434 shutdown_connections 2024-09-13T16:29:06.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.777+0000 7fd359baa640 1 -- 172.21.15.16:0/3953036434 wait complete. 2024-09-13T16:29:06.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.777+0000 7fd359baa640 1 Processor -- start 2024-09-13T16:29:06.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.777+0000 7fd359baa640 1 -- start start 2024-09-13T16:29:06.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.778+0000 7fd359baa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3540fec70 0x7fd354195c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:06.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.778+0000 7fd359baa640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3540ffe60 0x7fd3541961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:06.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.778+0000 7fd359baa640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd354100820 0x7fd35419d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:06.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.778+0000 7fd359baa640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35419d750 con 0x7fd3540fec70 2024-09-13T16:29:06.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.778+0000 7fd359baa640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35419d8c0 con 0x7fd354100820 2024-09-13T16:29:06.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.778+0000 7fd359baa640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35419da80 con 0x7fd3540ffe60 2024-09-13T16:29:06.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.778+0000 7fd3537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3540fec70 0x7fd354195c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:06.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.778+0000 7fd3537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3540fec70 0x7fd354195c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49610/0 (socket says 172.21.15.16:49610) 2024-09-13T16:29:06.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.778+0000 7fd3537fe640 1 -- 172.21.15.16:0/2254034620 learned_addr learned my addr 172.21.15.16:0/2254034620 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:06.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.778+0000 7fd352ffd640 1 --2- 172.21.15.16:0/2254034620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3540ffe60 0x7fd3541961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:06.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.778+0000 7fd353fff640 1 --2- 172.21.15.16:0/2254034620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd354100820 0x7fd35419d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:06.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.778+0000 7fd3537fe640 1 -- 172.21.15.16:0/2254034620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3540ffe60 msgr2=0x7fd3541961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:06.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.778+0000 7fd3537fe640 1 --2- 172.21.15.16:0/2254034620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3540ffe60 0x7fd3541961b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:06.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.778+0000 7fd3537fe640 1 -- 172.21.15.16:0/2254034620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd354100820 msgr2=0x7fd35419d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:06.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.778+0000 7fd3537fe640 1 --2- 172.21.15.16:0/2254034620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd354100820 0x7fd35419d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:06.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.778+0000 7fd3537fe640 1 -- 172.21.15.16:0/2254034620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34c009900 con 0x7fd3540fec70 2024-09-13T16:29:06.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.778+0000 7fd3537fe640 1 --2- 172.21.15.16:0/2254034620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3540fec70 0x7fd354195c70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd34400d950 tx=0x7fd34400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:06.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.778+0000 7fd350ff9640 1 -- 172.21.15.16:0/2254034620 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd344009c30 con 0x7fd3540fec70 2024-09-13T16:29:06.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.778+0000 7fd350ff9640 1 -- 172.21.15.16:0/2254034620 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd344009d90 con 0x7fd3540fec70 2024-09-13T16:29:06.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.778+0000 7fd359baa640 1 -- 172.21.15.16:0/2254034620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd35419dd60 con 0x7fd3540fec70 2024-09-13T16:29:06.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.778+0000 7fd350ff9640 1 -- 172.21.15.16:0/2254034620 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34400f6b0 con 0x7fd3540fec70 2024-09-13T16:29:06.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.779+0000 7fd359baa640 1 -- 172.21.15.16:0/2254034620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd35419e1c0 con 0x7fd3540fec70 2024-09-13T16:29:06.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.779+0000 7fd359baa640 1 -- 172.21.15.16:0/2254034620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd35410a3e0 con 0x7fd3540fec70 2024-09-13T16:29:06.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.781+0000 7fd350ff9640 1 -- 172.21.15.16:0/2254034620 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd344010460 con 0x7fd3540fec70 2024-09-13T16:29:06.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.781+0000 7fd350ff9640 1 --2- 172.21.15.16:0/2254034620 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd33c0a1090 0x7fd33c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:06.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.781+0000 7fd352ffd640 1 --2- 172.21.15.16:0/2254034620 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd33c0a1090 0x7fd33c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:06.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.781+0000 7fd350ff9640 1 -- 172.21.15.16:0/2254034620 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd3440cccc0 con 0x7fd3540fec70 2024-09-13T16:29:06.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.782+0000 7fd352ffd640 1 --2- 172.21.15.16:0/2254034620 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd33c0a1090 0x7fd33c0a3550 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd3480058c0 tx=0x7fd34800a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:06.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.785+0000 7fd350ff9640 1 -- 172.21.15.16:0/2254034620 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd3440960b0 con 0x7fd3540fec70 2024-09-13T16:29:06.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:06 smithi107 ceph-mon[34139]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:29:06.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:06 smithi107 ceph-mon[34139]: osdmap e91: 12 total, 12 up, 12 in 2024-09-13T16:29:06.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:06 smithi107 ceph-mon[34139]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-09-13T16:29:06.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:06 smithi016 ceph-mon[31187]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:29:06.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:06 smithi016 ceph-mon[31187]: osdmap e91: 12 total, 12 up, 12 in 2024-09-13T16:29:06.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:06 smithi016 ceph-mon[31187]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-09-13T16:29:06.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:06 smithi111 ceph-mon[34600]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:29:06.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:06 smithi111 ceph-mon[34600]: osdmap e91: 12 total, 12 up, 12 in 2024-09-13T16:29:06.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:06 smithi111 ceph-mon[34600]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-09-13T16:29:06.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:06.982+0000 7fd359baa640 1 -- 172.21.15.16:0/2254034620 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fd35419e410 con 0x7fd33c0a1090 2024-09-13T16:29:07.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.006+0000 7fd350ff9640 1 -- 172.21.15.16:0/2254034620 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd35419e410 con 0x7fd33c0a1090 2024-09-13T16:29:07.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.008+0000 7fd359baa640 1 -- 172.21.15.16:0/2254034620 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd33c0a1090 msgr2=0x7fd33c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:07.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.008+0000 7fd359baa640 1 --2- 172.21.15.16:0/2254034620 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd33c0a1090 0x7fd33c0a3550 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd3480058c0 tx=0x7fd34800a250 comp rx=0 tx=0).stop 2024-09-13T16:29:07.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.008+0000 7fd359baa640 1 -- 172.21.15.16:0/2254034620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3540fec70 msgr2=0x7fd354195c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:07.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.008+0000 7fd359baa640 1 --2- 172.21.15.16:0/2254034620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3540fec70 0x7fd354195c70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd34400d950 tx=0x7fd34400de10 comp rx=0 tx=0).stop 2024-09-13T16:29:07.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.008+0000 7fd359baa640 1 -- 172.21.15.16:0/2254034620 shutdown_connections 2024-09-13T16:29:07.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.008+0000 7fd359baa640 1 --2- 172.21.15.16:0/2254034620 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd33c0a1090 0x7fd33c0a3550 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:07.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.008+0000 7fd359baa640 1 --2- 172.21.15.16:0/2254034620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd354100820 0x7fd35419d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:07.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.008+0000 7fd359baa640 1 --2- 172.21.15.16:0/2254034620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3540ffe60 0x7fd3541961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:07.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.008+0000 7fd359baa640 1 --2- 172.21.15.16:0/2254034620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3540fec70 0x7fd354195c70 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:07.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.008+0000 7fd359baa640 1 -- 172.21.15.16:0/2254034620 >> 172.21.15.16:0/2254034620 conn(0x7fd3540fa3d0 msgr2=0x7fd354068150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:07.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.008+0000 7fd359baa640 1 -- 172.21.15.16:0/2254034620 shutdown_connections 2024-09-13T16:29:07.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.008+0000 7fd359baa640 1 -- 172.21.15.16:0/2254034620 wait complete. 2024-09-13T16:29:07.020 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-13T16:29:07.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.126+0000 7f2448de2640 1 -- 172.21.15.16:0/89297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24440fec90 msgr2=0x7f24440ff090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:07.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.126+0000 7f2448de2640 1 --2- 172.21.15.16:0/89297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24440fec90 0x7f24440ff090 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2434009900 tx=0x7f243402a410 comp rx=0 tx=0).stop 2024-09-13T16:29:07.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.126+0000 7f2448de2640 1 -- 172.21.15.16:0/89297 shutdown_connections 2024-09-13T16:29:07.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.126+0000 7f2448de2640 1 --2- 172.21.15.16:0/89297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2444100840 0x7f24441070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:07.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.126+0000 7f2448de2640 1 --2- 172.21.15.16:0/89297 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24440ffe80 0x7f2444100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:07.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.126+0000 7f2448de2640 1 --2- 172.21.15.16:0/89297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24440fec90 0x7f24440ff090 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:07.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.126+0000 7f2448de2640 1 -- 172.21.15.16:0/89297 >> 172.21.15.16:0/89297 conn(0x7f24440fa3f0 msgr2=0x7f24440fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.126+0000 7f2448de2640 1 -- 172.21.15.16:0/89297 shutdown_connections 2024-09-13T16:29:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.126+0000 7f2448de2640 1 -- 172.21.15.16:0/89297 wait complete. 2024-09-13T16:29:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.127+0000 7f2448de2640 1 Processor -- start 2024-09-13T16:29:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.127+0000 7f2448de2640 1 -- start start 2024-09-13T16:29:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.127+0000 7f2448de2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24440fec90 0x7f2444195ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.127+0000 7f2448de2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24440ffe80 0x7f24441961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.127+0000 7f2448de2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2444100840 0x7f244419d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.127+0000 7f2448de2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244419d7b0 con 0x7f24440fec90 2024-09-13T16:29:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.127+0000 7f2448de2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244419d920 con 0x7f24440ffe80 2024-09-13T16:29:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.127+0000 7f2448de2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244419dae0 con 0x7f2444100840 2024-09-13T16:29:07.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.127+0000 7f244259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24440fec90 0x7f2444195ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:07.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.127+0000 7f244259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24440fec90 0x7f2444195ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49626/0 (socket says 172.21.15.16:49626) 2024-09-13T16:29:07.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.127+0000 7f244259c640 1 -- 172.21.15.16:0/122960351 learned_addr learned my addr 172.21.15.16:0/122960351 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.127+0000 7f244259c640 1 -- 172.21.15.16:0/122960351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2444100840 msgr2=0x7f244419d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.127+0000 7f244259c640 1 --2- 172.21.15.16:0/122960351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2444100840 0x7f244419d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.127+0000 7f244259c640 1 -- 172.21.15.16:0/122960351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24440ffe80 msgr2=0x7f24441961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.127+0000 7f244259c640 1 --2- 172.21.15.16:0/122960351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24440ffe80 0x7f24441961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.127+0000 7f244259c640 1 -- 172.21.15.16:0/122960351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2434009560 con 0x7f24440fec90 2024-09-13T16:29:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.128+0000 7f244259c640 1 --2- 172.21.15.16:0/122960351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24440fec90 0x7f2444195ca0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f243402a930 tx=0x7f24340048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.128+0000 7f24337fe640 1 -- 172.21.15.16:0/122960351 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24340044d0 con 0x7f24440fec90 2024-09-13T16:29:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.128+0000 7f24337fe640 1 -- 172.21.15.16:0/122960351 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2434003a80 con 0x7f24440fec90 2024-09-13T16:29:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.128+0000 7f2448de2640 1 -- 172.21.15.16:0/122960351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f244419dd60 con 0x7f24440fec90 2024-09-13T16:29:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.128+0000 7f24337fe640 1 -- 172.21.15.16:0/122960351 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243403cb50 con 0x7f24440fec90 2024-09-13T16:29:07.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.128+0000 7f2448de2640 1 -- 172.21.15.16:0/122960351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f244419e200 con 0x7f24440fec90 2024-09-13T16:29:07.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.129+0000 7f2448de2640 1 -- 172.21.15.16:0/122960351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f244410a430 con 0x7f24440fec90 2024-09-13T16:29:07.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.130+0000 7f24337fe640 1 -- 172.21.15.16:0/122960351 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f2434003bf0 con 0x7f24440fec90 2024-09-13T16:29:07.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.130+0000 7f24337fe640 1 --2- 172.21.15.16:0/122960351 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f242c0a1110 0x7f242c0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:07.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.131+0000 7f24337fe640 1 -- 172.21.15.16:0/122960351 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f24340ee340 con 0x7f24440fec90 2024-09-13T16:29:07.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.131+0000 7f2441d9b640 1 --2- 172.21.15.16:0/122960351 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f242c0a1110 0x7f242c0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:07.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.131+0000 7f2441d9b640 1 --2- 172.21.15.16:0/122960351 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f242c0a1110 0x7f242c0a35d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f24441971a0 tx=0x7f2438005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:07.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.134+0000 7f24337fe640 1 -- 172.21.15.16:0/122960351 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f24340b77c0 con 0x7f24440fec90 2024-09-13T16:29:07.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.336+0000 7f2448de2640 1 -- 172.21.15.16:0/122960351 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f24440fc210 con 0x7f242c0a1110 2024-09-13T16:29:07.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.341+0000 7f24337fe640 1 -- 172.21.15.16:0/122960351 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f24440fc210 con 0x7f242c0a1110 2024-09-13T16:29:07.342 INFO:teuthology.orchestra.run.smithi016.stdout:/volumes/qa/sv_0/d317f307-79ab-4bc1-8aca-0d73735f8789 2024-09-13T16:29:07.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.343+0000 7f2448de2640 1 -- 172.21.15.16:0/122960351 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f242c0a1110 msgr2=0x7f242c0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:07.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.343+0000 7f2448de2640 1 --2- 172.21.15.16:0/122960351 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f242c0a1110 0x7f242c0a35d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f24441971a0 tx=0x7f2438005c70 comp rx=0 tx=0).stop 2024-09-13T16:29:07.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.343+0000 7f2448de2640 1 -- 172.21.15.16:0/122960351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24440fec90 msgr2=0x7f2444195ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:07.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.343+0000 7f2448de2640 1 --2- 172.21.15.16:0/122960351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24440fec90 0x7f2444195ca0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f243402a930 tx=0x7f24340048f0 comp rx=0 tx=0).stop 2024-09-13T16:29:07.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.343+0000 7f2448de2640 1 -- 172.21.15.16:0/122960351 shutdown_connections 2024-09-13T16:29:07.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.343+0000 7f2448de2640 1 --2- 172.21.15.16:0/122960351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2444100840 0x7f244419d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:07.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.343+0000 7f2448de2640 1 --2- 172.21.15.16:0/122960351 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f242c0a1110 0x7f242c0a35d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:07.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.343+0000 7f2448de2640 1 --2- 172.21.15.16:0/122960351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24440ffe80 0x7f24441961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:07.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.343+0000 7f2448de2640 1 --2- 172.21.15.16:0/122960351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24440fec90 0x7f2444195ca0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:07.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.343+0000 7f2448de2640 1 -- 172.21.15.16:0/122960351 >> 172.21.15.16:0/122960351 conn(0x7f24440fa3f0 msgr2=0x7f24440fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:07.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.343+0000 7f2448de2640 1 -- 172.21.15.16:0/122960351 shutdown_connections 2024-09-13T16:29:07.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.343+0000 7f2448de2640 1 -- 172.21.15.16:0/122960351 wait complete. 2024-09-13T16:29:07.355 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-13T16:29:07.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.461+0000 7fc3abde2640 1 -- 172.21.15.16:0/2094018982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3a4100840 msgr2=0x7fc3a41070d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:07.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.461+0000 7fc3abde2640 1 --2- 172.21.15.16:0/2094018982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3a4100840 0x7fc3a41070d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc39c009ca0 tx=0x7fc39c02a8c0 comp rx=0 tx=0).stop 2024-09-13T16:29:07.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.461+0000 7fc3abde2640 1 -- 172.21.15.16:0/2094018982 shutdown_connections 2024-09-13T16:29:07.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.461+0000 7fc3abde2640 1 --2- 172.21.15.16:0/2094018982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3a4100840 0x7fc3a41070d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:07.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.461+0000 7fc3abde2640 1 --2- 172.21.15.16:0/2094018982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3a40ffe80 0x7fc3a4100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:07.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.461+0000 7fc3abde2640 1 --2- 172.21.15.16:0/2094018982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3a40fec90 0x7fc3a40ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:07.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.461+0000 7fc3abde2640 1 -- 172.21.15.16:0/2094018982 >> 172.21.15.16:0/2094018982 conn(0x7fc3a40fa3f0 msgr2=0x7fc3a40fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:07.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.461+0000 7fc3abde2640 1 -- 172.21.15.16:0/2094018982 shutdown_connections 2024-09-13T16:29:07.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.461+0000 7fc3abde2640 1 -- 172.21.15.16:0/2094018982 wait complete. 2024-09-13T16:29:07.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.462+0000 7fc3abde2640 1 Processor -- start 2024-09-13T16:29:07.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.462+0000 7fc3abde2640 1 -- start start 2024-09-13T16:29:07.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.462+0000 7fc3abde2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3a40fec90 0x7fc3a4109550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:07.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.462+0000 7fc3abde2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3a40ffe80 0x7fc3a4107b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:07.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.462+0000 7fc3abde2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3a4100840 0x7fc3a41080d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:07.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.462+0000 7fc3abde2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a4108830 con 0x7fc3a40ffe80 2024-09-13T16:29:07.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.462+0000 7fc3abde2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a41089a0 con 0x7fc3a40fec90 2024-09-13T16:29:07.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.462+0000 7fc3abde2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a419fd10 con 0x7fc3a4100840 2024-09-13T16:29:07.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.462+0000 7fc3a937d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3a40ffe80 0x7fc3a4107b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:07.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.462+0000 7fc3a937d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3a40ffe80 0x7fc3a4107b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49642/0 (socket says 172.21.15.16:49642) 2024-09-13T16:29:07.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.462+0000 7fc3a937d640 1 -- 172.21.15.16:0/3867855428 learned_addr learned my addr 172.21.15.16:0/3867855428 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:07.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.462+0000 7fc3a9b7e640 1 --2- 172.21.15.16:0/3867855428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3a40fec90 0x7fc3a4109550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:07.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.462+0000 7fc3aa37f640 1 --2- 172.21.15.16:0/3867855428 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3a4100840 0x7fc3a41080d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:07.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.462+0000 7fc3a937d640 1 -- 172.21.15.16:0/3867855428 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3a4100840 msgr2=0x7fc3a41080d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:07.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.462+0000 7fc3a937d640 1 --2- 172.21.15.16:0/3867855428 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3a4100840 0x7fc3a41080d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:07.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.462+0000 7fc3a937d640 1 -- 172.21.15.16:0/3867855428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3a40fec90 msgr2=0x7fc3a4109550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:07.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.462+0000 7fc3a937d640 1 --2- 172.21.15.16:0/3867855428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3a40fec90 0x7fc3a4109550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:07.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.462+0000 7fc3a937d640 1 -- 172.21.15.16:0/3867855428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc39c009900 con 0x7fc3a40ffe80 2024-09-13T16:29:07.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.462+0000 7fc3a9b7e640 1 --2- 172.21.15.16:0/3867855428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3a40fec90 0x7fc3a4109550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:29:07.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.463+0000 7fc3a937d640 1 --2- 172.21.15.16:0/3867855428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3a40ffe80 0x7fc3a4107b90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc39400d980 tx=0x7fc39400de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:07.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.463+0000 7fc39affd640 1 -- 172.21.15.16:0/3867855428 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc394009c30 con 0x7fc3a40ffe80 2024-09-13T16:29:07.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.463+0000 7fc39affd640 1 -- 172.21.15.16:0/3867855428 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc39400b660 con 0x7fc3a40ffe80 2024-09-13T16:29:07.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.463+0000 7fc3abde2640 1 -- 172.21.15.16:0/3867855428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3a419ffc0 con 0x7fc3a40ffe80 2024-09-13T16:29:07.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.463+0000 7fc39affd640 1 -- 172.21.15.16:0/3867855428 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39400f620 con 0x7fc3a40ffe80 2024-09-13T16:29:07.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.463+0000 7fc3abde2640 1 -- 172.21.15.16:0/3867855428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a41a0440 con 0x7fc3a40ffe80 2024-09-13T16:29:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.464+0000 7fc3abde2640 1 -- 172.21.15.16:0/3867855428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3a41060c0 con 0x7fc3a40ffe80 2024-09-13T16:29:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.465+0000 7fc39affd640 1 -- 172.21.15.16:0/3867855428 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fc394010460 con 0x7fc3a40ffe80 2024-09-13T16:29:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.466+0000 7fc39affd640 1 --2- 172.21.15.16:0/3867855428 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc3900a1090 0x7fc3900a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.466+0000 7fc39affd640 1 -- 172.21.15.16:0/3867855428 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc3940cc890 con 0x7fc3a40ffe80 2024-09-13T16:29:07.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.466+0000 7fc3a9b7e640 1 --2- 172.21.15.16:0/3867855428 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc3900a1090 0x7fc3900a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:07.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.467+0000 7fc3a9b7e640 1 --2- 172.21.15.16:0/3867855428 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc3900a1090 0x7fc3900a3550 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc3a40ffce0 tx=0x7fc3a0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:07.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.469+0000 7fc39affd640 1 -- 172.21.15.16:0/3867855428 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc394095d10 con 0x7fc3a40ffe80 2024-09-13T16:29:07.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:07 smithi016 ceph-mon[31187]: pgmap v292: 129 pgs: 1 peering, 128 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s rd, 2.1 KiB/s wr, 6 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2024-09-13T16:29:07.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:07 smithi016 ceph-mon[31187]: mds.? [v2:172.21.15.111:6832/1984275936,v1:172.21.15.111:6833/1984275936] up:active 2024-09-13T16:29:07.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:07 smithi016 ceph-mon[31187]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-09-13T16:29:07.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:07 smithi016 ceph-mon[31187]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T16:29:07.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.669+0000 7fc3abde2640 1 -- 172.21.15.16:0/3867855428 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3a402c4f0 con 0x7fc3900a1090 2024-09-13T16:29:07.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.694+0000 7fc39affd640 1 -- 172.21.15.16:0/3867855428 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc3a402c4f0 con 0x7fc3900a1090 2024-09-13T16:29:07.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.696+0000 7fc3abde2640 1 -- 172.21.15.16:0/3867855428 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc3900a1090 msgr2=0x7fc3900a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:07.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.696+0000 7fc3abde2640 1 --2- 172.21.15.16:0/3867855428 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc3900a1090 0x7fc3900a3550 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc3a40ffce0 tx=0x7fc3a0008040 comp rx=0 tx=0).stop 2024-09-13T16:29:07.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.696+0000 7fc3abde2640 1 -- 172.21.15.16:0/3867855428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3a40ffe80 msgr2=0x7fc3a4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:07.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.696+0000 7fc3abde2640 1 --2- 172.21.15.16:0/3867855428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3a40ffe80 0x7fc3a4107b90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc39400d980 tx=0x7fc39400de40 comp rx=0 tx=0).stop 2024-09-13T16:29:07.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.696+0000 7fc3abde2640 1 -- 172.21.15.16:0/3867855428 shutdown_connections 2024-09-13T16:29:07.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.696+0000 7fc3abde2640 1 --2- 172.21.15.16:0/3867855428 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3a4100840 0x7fc3a41080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:07.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.696+0000 7fc3abde2640 1 --2- 172.21.15.16:0/3867855428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3a40ffe80 0x7fc3a4107b90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:07.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.696+0000 7fc3abde2640 1 --2- 172.21.15.16:0/3867855428 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc3900a1090 0x7fc3900a3550 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:07.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.696+0000 7fc3abde2640 1 --2- 172.21.15.16:0/3867855428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3a40fec90 0x7fc3a4109550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:07.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.696+0000 7fc3abde2640 1 -- 172.21.15.16:0/3867855428 >> 172.21.15.16:0/3867855428 conn(0x7fc3a40fa3f0 msgr2=0x7fc3a4103290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:07.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.696+0000 7fc3abde2640 1 -- 172.21.15.16:0/3867855428 shutdown_connections 2024-09-13T16:29:07.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.696+0000 7fc3abde2640 1 -- 172.21.15.16:0/3867855428 wait complete. 2024-09-13T16:29:07.707 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-13T16:29:07.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.814+0000 7f7cb1b9d640 1 -- 172.21.15.16:0/4242729416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cac1059d0 msgr2=0x7f7cac107e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:07.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.814+0000 7f7cb1b9d640 1 --2- 172.21.15.16:0/4242729416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cac1059d0 0x7f7cac107e40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4009ca0 tx=0x7f7ca402a8c0 comp rx=0 tx=0).stop 2024-09-13T16:29:07.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.814+0000 7f7cb1b9d640 1 -- 172.21.15.16:0/4242729416 shutdown_connections 2024-09-13T16:29:07.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.814+0000 7f7cb1b9d640 1 --2- 172.21.15.16:0/4242729416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cac1059d0 0x7f7cac107e40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:07.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.814+0000 7f7cb1b9d640 1 --2- 172.21.15.16:0/4242729416 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cac1030a0 0x7f7cac105490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:07.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.814+0000 7f7cb1b9d640 1 --2- 172.21.15.16:0/4242729416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cac100770 0x7f7cac102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:07.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.814+0000 7f7cb1b9d640 1 -- 172.21.15.16:0/4242729416 >> 172.21.15.16:0/4242729416 conn(0x7f7cac0fa330 msgr2=0x7f7cac0fc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:07.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.814+0000 7f7cb1b9d640 1 -- 172.21.15.16:0/4242729416 shutdown_connections 2024-09-13T16:29:07.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.814+0000 7f7cb1b9d640 1 -- 172.21.15.16:0/4242729416 wait complete. 2024-09-13T16:29:07.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.815+0000 7f7cb1b9d640 1 Processor -- start 2024-09-13T16:29:07.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.815+0000 7f7cb1b9d640 1 -- start start 2024-09-13T16:29:07.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.815+0000 7f7cb1b9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cac100770 0x7f7cac195bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:07.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.815+0000 7f7cb1b9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cac1030a0 0x7f7cac1960f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:07.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.815+0000 7f7cb1b9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cac1059d0 0x7f7cac19d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:07.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.815+0000 7f7cb1b9d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac19d6a0 con 0x7f7cac1059d0 2024-09-13T16:29:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.815+0000 7f7cb1b9d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac19d810 con 0x7f7cac1030a0 2024-09-13T16:29:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.815+0000 7f7cb1b9d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac19d9d0 con 0x7f7cac100770 2024-09-13T16:29:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.815+0000 7f7cb139c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cac1059d0 0x7f7cac19d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.815+0000 7f7cb139c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cac1059d0 0x7f7cac19d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49650/0 (socket says 172.21.15.16:49650) 2024-09-13T16:29:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.815+0000 7f7cabfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cac1030a0 0x7f7cac1960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.815+0000 7f7cb139c640 1 -- 172.21.15.16:0/2830775556 learned_addr learned my addr 172.21.15.16:0/2830775556 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.815+0000 7f7cb0b9b640 1 --2- 172.21.15.16:0/2830775556 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cac100770 0x7f7cac195bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.816+0000 7f7cb139c640 1 -- 172.21.15.16:0/2830775556 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cac100770 msgr2=0x7f7cac195bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.816+0000 7f7cb139c640 1 --2- 172.21.15.16:0/2830775556 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cac100770 0x7f7cac195bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.816+0000 7f7cb139c640 1 -- 172.21.15.16:0/2830775556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cac1030a0 msgr2=0x7f7cac1960f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.816+0000 7f7cb139c640 1 --2- 172.21.15.16:0/2830775556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cac1030a0 0x7f7cac1960f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.816+0000 7f7cb139c640 1 -- 172.21.15.16:0/2830775556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca4009900 con 0x7f7cac1059d0 2024-09-13T16:29:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.816+0000 7f7cb0b9b640 1 --2- 172.21.15.16:0/2830775556 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cac100770 0x7f7cac195bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:29:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.816+0000 7f7cabfff640 1 --2- 172.21.15.16:0/2830775556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cac1030a0 0x7f7cac1960f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:29:07.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.816+0000 7f7cb139c640 1 --2- 172.21.15.16:0/2830775556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cac1059d0 0x7f7cac19d160 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7ca402aeb0 tx=0x7f7ca402f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:07.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.816+0000 7f7ca9ffb640 1 -- 172.21.15.16:0/2830775556 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4003bf0 con 0x7f7cac1059d0 2024-09-13T16:29:07.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.816+0000 7f7cb1b9d640 1 -- 172.21.15.16:0/2830775556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cac19dc50 con 0x7f7cac1059d0 2024-09-13T16:29:07.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.816+0000 7f7ca9ffb640 1 -- 172.21.15.16:0/2830775556 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ca4040070 con 0x7f7cac1059d0 2024-09-13T16:29:07.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.816+0000 7f7ca9ffb640 1 -- 172.21.15.16:0/2830775556 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4004440 con 0x7f7cac1059d0 2024-09-13T16:29:07.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.816+0000 7f7cb1b9d640 1 -- 172.21.15.16:0/2830775556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cac19e160 con 0x7f7cac1059d0 2024-09-13T16:29:07.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.817+0000 7f7cb1b9d640 1 -- 172.21.15.16:0/2830775556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cac0fbe20 con 0x7f7cac1059d0 2024-09-13T16:29:07.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.818+0000 7f7ca9ffb640 1 -- 172.21.15.16:0/2830775556 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7ca4042030 con 0x7f7cac1059d0 2024-09-13T16:29:07.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.819+0000 7f7ca9ffb640 1 --2- 172.21.15.16:0/2830775556 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7c9c0a1090 0x7f7c9c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:07.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.820+0000 7f7cb0b9b640 1 --2- 172.21.15.16:0/2830775556 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7c9c0a1090 0x7f7c9c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:07.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.820+0000 7f7ca9ffb640 1 -- 172.21.15.16:0/2830775556 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7ca40ee3b0 con 0x7f7cac1059d0 2024-09-13T16:29:07.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.820+0000 7f7cb0b9b640 1 --2- 172.21.15.16:0/2830775556 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7c9c0a1090 0x7f7c9c0a3550 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7ca00058c0 tx=0x7f7ca0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:07.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:07.824+0000 7f7ca9ffb640 1 -- 172.21.15.16:0/2830775556 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7ca40b77d0 con 0x7f7cac1059d0 2024-09-13T16:29:07.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:07 smithi107 ceph-mon[34139]: pgmap v292: 129 pgs: 1 peering, 128 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s rd, 2.1 KiB/s wr, 6 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2024-09-13T16:29:07.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:07 smithi107 ceph-mon[34139]: mds.? [v2:172.21.15.111:6832/1984275936,v1:172.21.15.111:6833/1984275936] up:active 2024-09-13T16:29:07.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:07 smithi107 ceph-mon[34139]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-09-13T16:29:07.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:07 smithi107 ceph-mon[34139]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T16:29:07.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:07 smithi111 ceph-mon[34600]: pgmap v292: 129 pgs: 1 peering, 128 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s rd, 2.1 KiB/s wr, 6 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2024-09-13T16:29:07.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:07 smithi111 ceph-mon[34600]: mds.? [v2:172.21.15.111:6832/1984275936,v1:172.21.15.111:6833/1984275936] up:active 2024-09-13T16:29:07.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:07 smithi111 ceph-mon[34600]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-09-13T16:29:07.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:07 smithi111 ceph-mon[34600]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T16:29:08.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.021+0000 7f7cb1b9d640 1 -- 172.21.15.16:0/2830775556 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7cac0608f0 con 0x7f7c9c0a1090 2024-09-13T16:29:08.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.027+0000 7f7ca9ffb640 1 -- 172.21.15.16:0/2830775556 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f7cac0608f0 con 0x7f7c9c0a1090 2024-09-13T16:29:08.028 INFO:teuthology.orchestra.run.smithi016.stdout:/volumes/qa/sv_1/da32cc7f-7528-4fb0-93cf-d699d959960a 2024-09-13T16:29:08.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.029+0000 7f7cb1b9d640 1 -- 172.21.15.16:0/2830775556 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7c9c0a1090 msgr2=0x7f7c9c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:08.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.029+0000 7f7cb1b9d640 1 --2- 172.21.15.16:0/2830775556 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7c9c0a1090 0x7f7c9c0a3550 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7ca00058c0 tx=0x7f7ca0005850 comp rx=0 tx=0).stop 2024-09-13T16:29:08.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.029+0000 7f7cb1b9d640 1 -- 172.21.15.16:0/2830775556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cac1059d0 msgr2=0x7f7cac19d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:08.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.029+0000 7f7cb1b9d640 1 --2- 172.21.15.16:0/2830775556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cac1059d0 0x7f7cac19d160 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7ca402aeb0 tx=0x7f7ca402f310 comp rx=0 tx=0).stop 2024-09-13T16:29:08.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.029+0000 7f7cb1b9d640 1 -- 172.21.15.16:0/2830775556 shutdown_connections 2024-09-13T16:29:08.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.029+0000 7f7cb1b9d640 1 --2- 172.21.15.16:0/2830775556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cac1059d0 0x7f7cac19d160 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.029+0000 7f7cb1b9d640 1 --2- 172.21.15.16:0/2830775556 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7c9c0a1090 0x7f7c9c0a3550 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.029+0000 7f7cb1b9d640 1 --2- 172.21.15.16:0/2830775556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cac1030a0 0x7f7cac1960f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.029+0000 7f7cb1b9d640 1 --2- 172.21.15.16:0/2830775556 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cac100770 0x7f7cac195bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.029+0000 7f7cb1b9d640 1 -- 172.21.15.16:0/2830775556 >> 172.21.15.16:0/2830775556 conn(0x7f7cac0fa330 msgr2=0x7f7cac101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:08.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.029+0000 7f7cb1b9d640 1 -- 172.21.15.16:0/2830775556 shutdown_connections 2024-09-13T16:29:08.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.029+0000 7f7cb1b9d640 1 -- 172.21.15.16:0/2830775556 wait complete. 2024-09-13T16:29:08.041 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:29:08.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.142+0000 7f054d02b640 1 -- 172.21.15.16:0/2554190335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0548105a80 msgr2=0x7f0548107ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:08.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.142+0000 7f054d02b640 1 --2- 172.21.15.16:0/2554190335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0548105a80 0x7f0548107ef0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0540009ca0 tx=0x7f054002a8c0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.142+0000 7f054d02b640 1 -- 172.21.15.16:0/2554190335 shutdown_connections 2024-09-13T16:29:08.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.142+0000 7f054d02b640 1 --2- 172.21.15.16:0/2554190335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0548105a80 0x7f0548107ef0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.142+0000 7f054d02b640 1 --2- 172.21.15.16:0/2554190335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0548103150 0x7f0548105540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.142+0000 7f054d02b640 1 --2- 172.21.15.16:0/2554190335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0548100820 0x7f0548102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.142+0000 7f054d02b640 1 -- 172.21.15.16:0/2554190335 >> 172.21.15.16:0/2554190335 conn(0x7f05480fa440 msgr2=0x7f05480fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:08.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.143+0000 7f054d02b640 1 -- 172.21.15.16:0/2554190335 shutdown_connections 2024-09-13T16:29:08.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.143+0000 7f054d02b640 1 -- 172.21.15.16:0/2554190335 wait complete. 2024-09-13T16:29:08.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.143+0000 7f054d02b640 1 Processor -- start 2024-09-13T16:29:08.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.143+0000 7f054d02b640 1 -- start start 2024-09-13T16:29:08.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.144+0000 7f054d02b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0548100820 0x7f0548195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:08.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.144+0000 7f054d02b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0548103150 0x7f0548196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:08.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.144+0000 7f054d02b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0548105a80 0x7f054819d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:08.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.144+0000 7f054d02b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054819d810 con 0x7f0548100820 2024-09-13T16:29:08.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.144+0000 7f054d02b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054819d980 con 0x7f0548103150 2024-09-13T16:29:08.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.144+0000 7f054d02b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054819db40 con 0x7f0548105a80 2024-09-13T16:29:08.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.144+0000 7f0546d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0548100820 0x7f0548195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:08.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.144+0000 7f0546d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0548100820 0x7f0548195d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49676/0 (socket says 172.21.15.16:49676) 2024-09-13T16:29:08.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.144+0000 7f0546d9d640 1 -- 172.21.15.16:0/1594234800 learned_addr learned my addr 172.21.15.16:0/1594234800 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:08.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.144+0000 7f054759e640 1 --2- 172.21.15.16:0/1594234800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0548105a80 0x7f054819d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:08.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.144+0000 7f054659c640 1 --2- 172.21.15.16:0/1594234800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0548103150 0x7f0548196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:08.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.144+0000 7f0546d9d640 1 -- 172.21.15.16:0/1594234800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0548105a80 msgr2=0x7f054819d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:08.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.144+0000 7f0546d9d640 1 --2- 172.21.15.16:0/1594234800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0548105a80 0x7f054819d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.144+0000 7f0546d9d640 1 -- 172.21.15.16:0/1594234800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0548103150 msgr2=0x7f0548196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:08.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.144+0000 7f0546d9d640 1 --2- 172.21.15.16:0/1594234800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0548103150 0x7f0548196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.144+0000 7f0546d9d640 1 -- 172.21.15.16:0/1594234800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0540009900 con 0x7f0548100820 2024-09-13T16:29:08.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.144+0000 7f0546d9d640 1 --2- 172.21.15.16:0/1594234800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0548100820 0x7f0548195d30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f053800d770 tx=0x7f053800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:08.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.144+0000 7f052ffff640 1 -- 172.21.15.16:0/1594234800 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0538009c30 con 0x7f0548100820 2024-09-13T16:29:08.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.144+0000 7f052ffff640 1 -- 172.21.15.16:0/1594234800 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f053800de80 con 0x7f0548100820 2024-09-13T16:29:08.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.144+0000 7f052ffff640 1 -- 172.21.15.16:0/1594234800 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0538004d40 con 0x7f0548100820 2024-09-13T16:29:08.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.145+0000 7f054d02b640 1 -- 172.21.15.16:0/1594234800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f054819de20 con 0x7f0548100820 2024-09-13T16:29:08.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.145+0000 7f054d02b640 1 -- 172.21.15.16:0/1594234800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f054819e1f0 con 0x7f0548100820 2024-09-13T16:29:08.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.146+0000 7f054d02b640 1 -- 172.21.15.16:0/1594234800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05480fbf30 con 0x7f0548100820 2024-09-13T16:29:08.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.147+0000 7f052ffff640 1 -- 172.21.15.16:0/1594234800 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f0538004ea0 con 0x7f0548100820 2024-09-13T16:29:08.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.147+0000 7f052ffff640 1 --2- 172.21.15.16:0/1594234800 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f05300a1090 0x7f05300a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:08.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.147+0000 7f052ffff640 1 -- 172.21.15.16:0/1594234800 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f05380cc6e0 con 0x7f0548100820 2024-09-13T16:29:08.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.147+0000 7f054659c640 1 --2- 172.21.15.16:0/1594234800 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f05300a1090 0x7f05300a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:08.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.148+0000 7f054659c640 1 --2- 172.21.15.16:0/1594234800 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f05300a1090 0x7f05300a3550 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f053c005fd0 tx=0x7f053c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:08.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.152+0000 7f052ffff640 1 -- 172.21.15.16:0/1594234800 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0538095ad0 con 0x7f0548100820 2024-09-13T16:29:08.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.362+0000 7f054d02b640 1 -- 172.21.15.16:0/1594234800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f054810b550 con 0x7f0548100820 2024-09-13T16:29:08.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.363+0000 7f052ffff640 1 -- 172.21.15.16:0/1594234800 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10924 (secure 0 0 0) 0x7f0538097a00 con 0x7f0548100820 2024-09-13T16:29:08.364 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:29:08.364 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3292781664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3292781664},{"type":"v1","addr":"172.21.15.16:6835","nonce":3292781664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4109039027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4109039027},{"type":"v1","addr":"172.21.15.16:6837","nonce":4109039027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T16:28:39.184734+0000","modified":"2024-09-13T16:29:06.636937+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24518,"mds_1":24490,"mds_2":24485,"mds_3":24422,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/2391618693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2391618693},{"type":"v1","addr":"172.21.15.16:6839","nonce":2391618693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/521042681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":521042681},{"type":"v1","addr":"172.21.15.16:6841","nonce":521042681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24418":{"gid":24418,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.111:6833/1984275936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1984275936},{"type":"v1","addr":"172.21.15.111:6833","nonce":1984275936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.107:6833/3797712690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3797712690},{"type":"v1","addr":"172.21.15.107:6833","nonce":3797712690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/338548468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":338548468},{"type":"v1","addr":"172.21.15.107:6835","nonce":338548468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/662156111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":662156111},{"type":"v1","addr":"172.21.15.111:6835","nonce":662156111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/1877128453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1877128453},{"type":"v1","addr":"172.21.15.111:6837","nonce":1877128453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.107:6837/2838457611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2838457611},{"type":"v1","addr":"172.21.15.107:6837","nonce":2838457611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.111:6839/1423842285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":1423842285},{"type":"v1","addr":"172.21.15.111:6839","nonce":1423842285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6839/1914689806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1914689806},{"type":"v1","addr":"172.21.15.107:6839","nonce":1914689806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T16:29:08.364 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-13T16:29:08.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.365+0000 7f054d02b640 1 -- 172.21.15.16:0/1594234800 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f05300a1090 msgr2=0x7f05300a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:08.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.365+0000 7f054d02b640 1 --2- 172.21.15.16:0/1594234800 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f05300a1090 0x7f05300a3550 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f053c005fd0 tx=0x7f053c005850 comp rx=0 tx=0).stop 2024-09-13T16:29:08.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.365+0000 7f054d02b640 1 -- 172.21.15.16:0/1594234800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0548100820 msgr2=0x7f0548195d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:08.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.365+0000 7f054d02b640 1 --2- 172.21.15.16:0/1594234800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0548100820 0x7f0548195d30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f053800d770 tx=0x7f053800dc30 comp rx=0 tx=0).stop 2024-09-13T16:29:08.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.365+0000 7f054d02b640 1 -- 172.21.15.16:0/1594234800 shutdown_connections 2024-09-13T16:29:08.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.365+0000 7f054d02b640 1 --2- 172.21.15.16:0/1594234800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0548105a80 0x7f054819d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.365+0000 7f054d02b640 1 --2- 172.21.15.16:0/1594234800 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f05300a1090 0x7f05300a3550 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.365+0000 7f054d02b640 1 --2- 172.21.15.16:0/1594234800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0548103150 0x7f0548196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.365+0000 7f054d02b640 1 --2- 172.21.15.16:0/1594234800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0548100820 0x7f0548195d30 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.365+0000 7f054d02b640 1 -- 172.21.15.16:0/1594234800 >> 172.21.15.16:0/1594234800 conn(0x7f05480fa440 msgr2=0x7f0548101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:08.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.365+0000 7f054d02b640 1 -- 172.21.15.16:0/1594234800 shutdown_connections 2024-09-13T16:29:08.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.365+0000 7f054d02b640 1 -- 172.21.15.16:0/1594234800 wait complete. 2024-09-13T16:29:08.377 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:29:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.478+0000 7f22ed23a640 1 -- 172.21.15.16:0/3414773625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e80feca0 msgr2=0x7f22e80ff0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.478+0000 7f22ed23a640 1 --2- 172.21.15.16:0/3414773625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e80feca0 0x7f22e80ff0a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009900 tx=0x7f22d802a410 comp rx=0 tx=0).stop 2024-09-13T16:29:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.478+0000 7f22ed23a640 1 -- 172.21.15.16:0/3414773625 shutdown_connections 2024-09-13T16:29:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.478+0000 7f22ed23a640 1 --2- 172.21.15.16:0/3414773625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22e8100850 0x7f22e81070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.478+0000 7f22ed23a640 1 --2- 172.21.15.16:0/3414773625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22e80ffe90 0x7f22e8100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.478+0000 7f22ed23a640 1 --2- 172.21.15.16:0/3414773625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e80feca0 0x7f22e80ff0a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.478+0000 7f22ed23a640 1 -- 172.21.15.16:0/3414773625 >> 172.21.15.16:0/3414773625 conn(0x7f22e80fa440 msgr2=0x7f22e80fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.478+0000 7f22ed23a640 1 -- 172.21.15.16:0/3414773625 shutdown_connections 2024-09-13T16:29:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.478+0000 7f22ed23a640 1 -- 172.21.15.16:0/3414773625 wait complete. 2024-09-13T16:29:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.479+0000 7f22ed23a640 1 Processor -- start 2024-09-13T16:29:08.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.479+0000 7f22ed23a640 1 -- start start 2024-09-13T16:29:08.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.479+0000 7f22ed23a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22e80feca0 0x7f22e8195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:08.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.479+0000 7f22ed23a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e80ffe90 0x7f22e8196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:08.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.479+0000 7f22ed23a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22e8100850 0x7f22e819d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:08.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.479+0000 7f22ed23a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e819d810 con 0x7f22e80ffe90 2024-09-13T16:29:08.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.479+0000 7f22ed23a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e819d980 con 0x7f22e80feca0 2024-09-13T16:29:08.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.479+0000 7f22ed23a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e819db40 con 0x7f22e8100850 2024-09-13T16:29:08.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.480+0000 7f22e659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e80ffe90 0x7f22e8196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:08.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.480+0000 7f22e659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e80ffe90 0x7f22e8196270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49692/0 (socket says 172.21.15.16:49692) 2024-09-13T16:29:08.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.480+0000 7f22e659c640 1 -- 172.21.15.16:0/540127229 learned_addr learned my addr 172.21.15.16:0/540127229 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:08.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.480+0000 7f22e6d9d640 1 --2- 172.21.15.16:0/540127229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22e80feca0 0x7f22e8195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:08.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.480+0000 7f22e759e640 1 --2- 172.21.15.16:0/540127229 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22e8100850 0x7f22e819d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:08.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.480+0000 7f22e659c640 1 -- 172.21.15.16:0/540127229 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22e8100850 msgr2=0x7f22e819d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:08.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.480+0000 7f22e659c640 1 --2- 172.21.15.16:0/540127229 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22e8100850 0x7f22e819d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.480+0000 7f22e659c640 1 -- 172.21.15.16:0/540127229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22e80feca0 msgr2=0x7f22e8195d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:08.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.480+0000 7f22e659c640 1 --2- 172.21.15.16:0/540127229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22e80feca0 0x7f22e8195d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.480+0000 7f22e659c640 1 -- 172.21.15.16:0/540127229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d8009560 con 0x7f22e80ffe90 2024-09-13T16:29:08.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.480+0000 7f22e6d9d640 1 --2- 172.21.15.16:0/540127229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22e80feca0 0x7f22e8195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:29:08.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.480+0000 7f22e659c640 1 --2- 172.21.15.16:0/540127229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e80ffe90 0x7f22e8196270 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00c890 tx=0x7f22dc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:08.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.480+0000 7f22cffff640 1 -- 172.21.15.16:0/540127229 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc004220 con 0x7f22e80ffe90 2024-09-13T16:29:08.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.480+0000 7f22cffff640 1 -- 172.21.15.16:0/540127229 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f22dc007e40 con 0x7f22e80ffe90 2024-09-13T16:29:08.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.480+0000 7f22ed23a640 1 -- 172.21.15.16:0/540127229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e819de20 con 0x7f22e80ffe90 2024-09-13T16:29:08.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.480+0000 7f22cffff640 1 -- 172.21.15.16:0/540127229 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc004a00 con 0x7f22e80ffe90 2024-09-13T16:29:08.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.480+0000 7f22ed23a640 1 -- 172.21.15.16:0/540127229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e819e240 con 0x7f22e80ffe90 2024-09-13T16:29:08.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.481+0000 7f22ed23a640 1 -- 172.21.15.16:0/540127229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e80ff0a0 con 0x7f22e80ffe90 2024-09-13T16:29:08.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.482+0000 7f22cffff640 1 -- 172.21.15.16:0/540127229 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f22dc0079b0 con 0x7f22e80ffe90 2024-09-13T16:29:08.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.483+0000 7f22cffff640 1 --2- 172.21.15.16:0/540127229 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f22d00a1090 0x7f22d00a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:08.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.483+0000 7f22cffff640 1 -- 172.21.15.16:0/540127229 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f22dc0ccdf0 con 0x7f22e80ffe90 2024-09-13T16:29:08.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.483+0000 7f22e6d9d640 1 --2- 172.21.15.16:0/540127229 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f22d00a1090 0x7f22d00a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:08.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.484+0000 7f22e6d9d640 1 --2- 172.21.15.16:0/540127229 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f22d00a1090 0x7f22d00a3550 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f22d802aa00 tx=0x7f22d8005e00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:08.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.487+0000 7f22cffff640 1 -- 172.21.15.16:0/540127229 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f22dc0961e0 con 0x7f22e80ffe90 2024-09-13T16:29:08.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.664+0000 7f22ed23a640 1 -- 172.21.15.16:0/540127229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22e819e690 con 0x7f22e80ffe90 2024-09-13T16:29:08.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.666+0000 7f22cffff640 1 -- 172.21.15.16:0/540127229 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24499 (secure 0 0 0) 0x7f22dc01d020 con 0x7f22e80ffe90 2024-09-13T16:29:08.667 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:29:08.667 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":91,"fsid":"0d241f6a-71ec-11ef-bceb-c7b262605968","created":"2024-09-13T16:21:17.971738+0000","modified":"2024-09-13T16:29:05.626781+0000","last_up_change":"2024-09-13T16:27:54.673413+0000","last_in_change":"2024-09-13T16:27:35.718390+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:24:22.981019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:28:37.746027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:28:38.602731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:28:41.633322+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e11ef8f-08d6-4e74-afd6-f49f4bd40032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6803","nonce":2079394186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6805","nonce":2079394186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6809","nonce":2079394186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6807","nonce":2079394186}]},"public_addr":"172.21.15.16:6803/2079394186","cluster_addr":"172.21.15.16:6805/2079394186","heartbeat_back_addr":"172.21.15.16:6809/2079394186","heartbeat_front_addr":"172.21.15.16:6807/2079394186","state":["exists","up"]},{"osd":1,"uuid":"f2cd3694-9c9e-4da7-976e-f80a8df120f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6801","nonce":4021360958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6803","nonce":4021360958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6807","nonce":4021360958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6805","nonce":4021360958}]},"public_addr":"172.21.15.107:6801/4021360958","cluster_addr":"172.21.15.107:6803/4021360958","heartbeat_back_addr":"172.21.15.107:6807/4021360958","heartbeat_front_addr":"172.21.15.107:6805/4021360958","state":["exists","up"]},{"osd":2,"uuid":"a5bbf2eb-6599-44d5-ad96-68bb89ee19f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6801","nonce":3227179825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6803","nonce":3227179825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6807","nonce":3227179825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6805","nonce":3227179825}]},"public_addr":"172.21.15.111:6801/3227179825","cluster_addr":"172.21.15.111:6803/3227179825","heartbeat_back_addr":"172.21.15.111:6807/3227179825","heartbeat_front_addr":"172.21.15.111:6805/3227179825","state":["exists","up"]},{"osd":3,"uuid":"d6295ffe-11bc-4964-a250-9fecca870b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6811","nonce":2390375937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6813","nonce":2390375937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6817","nonce":2390375937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6815","nonce":2390375937}]},"public_addr":"172.21.15.16:6811/2390375937","cluster_addr":"172.21.15.16:6813/2390375937","heartbeat_back_addr":"172.21.15.16:6817/2390375937","heartbeat_front_addr":"172.21.15.16:6815/2390375937","state":["exists","up"]},{"osd":4,"uuid":"0a8446e0-fc86-4646-beb6-a7b9d433093d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6809","nonce":1936582048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6811","nonce":1936582048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6815","nonce":1936582048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6813","nonce":1936582048}]},"public_addr":"172.21.15.107:6809/1936582048","cluster_addr":"172.21.15.107:6811/1936582048","heartbeat_back_addr":"172.21.15.107:6815/1936582048","heartbeat_front_addr":"172.21.15.107:6813/1936582048","state":["exists","up"]},{"osd":5,"uuid":"2c97a446-357e-4975-9e51-9c95fe84a663","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6809","nonce":483858448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6811","nonce":483858448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6815","nonce":483858448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6813","nonce":483858448}]},"public_addr":"172.21.15.111:6809/483858448","cluster_addr":"172.21.15.111:6811/483858448","heartbeat_back_addr":"172.21.15.111:6815/483858448","heartbeat_front_addr":"172.21.15.111:6813/483858448","state":["exists","up"]},{"osd":6,"uuid":"890239e6-e6fb-4f9c-b973-05cc66d199bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6819","nonce":1330214141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6821","nonce":1330214141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6825","nonce":1330214141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6823","nonce":1330214141}]},"public_addr":"172.21.15.16:6819/1330214141","cluster_addr":"172.21.15.16:6821/1330214141","heartbeat_back_addr":"172.21.15.16:6825/1330214141","heartbeat_front_addr":"172.21.15.16:6823/1330214141","state":["exists","up"]},{"osd":7,"uuid":"ee826a9c-f441-4903-9b1b-66b627f7b712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6817","nonce":1133876754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6819","nonce":1133876754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6823","nonce":1133876754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6821","nonce":1133876754}]},"public_addr":"172.21.15.107:6817/1133876754","cluster_addr":"172.21.15.107:6819/1133876754","heartbeat_back_addr":"172.21.15.107:6823/1133876754","heartbeat_front_addr":"172.21.15.107:6821/1133876754","state":["exists","up"]},{"osd":8,"uuid":"e19773f2-dec8-44e0-9d33-0252e07d6415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6817","nonce":837548190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6819","nonce":837548190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6823","nonce":837548190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6821","nonce":837548190}]},"public_addr":"172.21.15.111:6817/837548190","cluster_addr":"172.21.15.111:6819/837548190","heartbeat_back_addr":"172.21.15.111:6823/837548190","heartbeat_front_addr":"172.21.15.111:6821/837548190","state":["exists","up"]},{"osd":9,"uuid":"bae4d6db-8b90-4d02-8e83-7d8f2a6f8fdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6827","nonce":4175383686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6829","nonce":4175383686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6833","nonce":4175383686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6831","nonce":4175383686}]},"public_addr":"172.21.15.16:6827/4175383686","cluster_addr":"172.21.15.16:6829/4175383686","heartbeat_back_addr":"172.21.15.16:6833/4175383686","heartbeat_front_addr":"172.21.15.16:6831/4175383686","state":["exists","up"]},{"osd":10,"uuid":"fd0e7740-76c8-4c99-a349-7077b4905143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6825","nonce":2659550426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6827","nonce":2659550426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6831","nonce":2659550426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6829","nonce":2659550426}]},"public_addr":"172.21.15.107:6825/2659550426","cluster_addr":"172.21.15.107:6827/2659550426","heartbeat_back_addr":"172.21.15.107:6831/2659550426","heartbeat_front_addr":"172.21.15.107:6829/2659550426","state":["exists","up"]},{"osd":11,"uuid":"94a1751d-b00c-4e23-815d-3a8bc3dea146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6825","nonce":1633548683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6827","nonce":1633548683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6831","nonce":1633548683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6829","nonce":1633548683}]},"public_addr":"172.21.15.111:6825/1633548683","cluster_addr":"172.21.15.111:6827/1633548683","heartbeat_back_addr":"172.21.15.111:6831/1633548683","heartbeat_front_addr":"172.21.15.111:6829/1633548683","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:36.359852+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:58.014202+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:19.753672+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:43.404817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:06.001677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:29.361587+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:52.554847+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:15.957553+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:39.581093+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:04.911389+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:28.971280+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:53.473561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/2098378667":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/3536229016":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6801/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/2684687547":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6800/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/368410612":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/12158160":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6800/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6801/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/3754423379":"2024-09-14T16:21:40.126459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:29:08.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.668+0000 7f22ed23a640 1 -- 172.21.15.16:0/540127229 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f22d00a1090 msgr2=0x7f22d00a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:08.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.668+0000 7f22ed23a640 1 --2- 172.21.15.16:0/540127229 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f22d00a1090 0x7f22d00a3550 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f22d802aa00 tx=0x7f22d8005e00 comp rx=0 tx=0).stop 2024-09-13T16:29:08.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.668+0000 7f22ed23a640 1 -- 172.21.15.16:0/540127229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e80ffe90 msgr2=0x7f22e8196270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:08.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.668+0000 7f22ed23a640 1 --2- 172.21.15.16:0/540127229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e80ffe90 0x7f22e8196270 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00c890 tx=0x7f22dc00cd50 comp rx=0 tx=0).stop 2024-09-13T16:29:08.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.668+0000 7f22ed23a640 1 -- 172.21.15.16:0/540127229 shutdown_connections 2024-09-13T16:29:08.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.668+0000 7f22ed23a640 1 --2- 172.21.15.16:0/540127229 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22e8100850 0x7f22e819d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.668+0000 7f22ed23a640 1 --2- 172.21.15.16:0/540127229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e80ffe90 0x7f22e8196270 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.668+0000 7f22ed23a640 1 --2- 172.21.15.16:0/540127229 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f22d00a1090 0x7f22d00a3550 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.668+0000 7f22ed23a640 1 --2- 172.21.15.16:0/540127229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22e80feca0 0x7f22e8195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.668+0000 7f22ed23a640 1 -- 172.21.15.16:0/540127229 >> 172.21.15.16:0/540127229 conn(0x7f22e80fa440 msgr2=0x7f22e8067610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:08.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.668+0000 7f22ed23a640 1 -- 172.21.15.16:0/540127229 shutdown_connections 2024-09-13T16:29:08.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.668+0000 7f22ed23a640 1 -- 172.21.15.16:0/540127229 wait complete. 2024-09-13T16:29:08.678 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:08 smithi016 ceph-mon[31187]: from='client.14862 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:29:08.678 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:08 smithi016 ceph-mon[31187]: from='client.14868 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:29:08.678 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:08 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1594234800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:29:08.680 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-09-13T16:29:08.680 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T16:29:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.782+0000 7feb7a93e640 1 -- 172.21.15.16:0/1414250386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb740ffec0 msgr2=0x7feb74100340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.782+0000 7feb7a93e640 1 --2- 172.21.15.16:0/1414250386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb740ffec0 0x7feb74100340 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7feb5c009900 tx=0x7feb5c02a410 comp rx=0 tx=0).stop 2024-09-13T16:29:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.782+0000 7feb7a93e640 1 -- 172.21.15.16:0/1414250386 shutdown_connections 2024-09-13T16:29:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.782+0000 7feb7a93e640 1 --2- 172.21.15.16:0/1414250386 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb74100880 0x7feb74107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.782+0000 7feb7a93e640 1 --2- 172.21.15.16:0/1414250386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb740ffec0 0x7feb74100340 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.782+0000 7feb7a93e640 1 --2- 172.21.15.16:0/1414250386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb740fecd0 0x7feb740ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.782+0000 7feb7a93e640 1 -- 172.21.15.16:0/1414250386 >> 172.21.15.16:0/1414250386 conn(0x7feb740fa430 msgr2=0x7feb740fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:08.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.782+0000 7feb7a93e640 1 -- 172.21.15.16:0/1414250386 shutdown_connections 2024-09-13T16:29:08.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.782+0000 7feb7a93e640 1 -- 172.21.15.16:0/1414250386 wait complete. 2024-09-13T16:29:08.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.782+0000 7feb7a93e640 1 Processor -- start 2024-09-13T16:29:08.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.782+0000 7feb7a93e640 1 -- start start 2024-09-13T16:29:08.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.783+0000 7feb7a93e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb740fecd0 0x7feb74069160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:08.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.783+0000 7feb7a93e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb740ffec0 0x7feb7419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:08.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.783+0000 7feb7a93e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb74100880 0x7feb7419d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:08.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.783+0000 7feb7a93e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7419d870 con 0x7feb740fecd0 2024-09-13T16:29:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.783+0000 7feb7a93e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7419d9e0 con 0x7feb74100880 2024-09-13T16:29:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.783+0000 7feb7a93e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7419dba0 con 0x7feb740ffec0 2024-09-13T16:29:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.783+0000 7feb73fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb740fecd0 0x7feb74069160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.783+0000 7feb73fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb740fecd0 0x7feb74069160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49700/0 (socket says 172.21.15.16:49700) 2024-09-13T16:29:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.783+0000 7feb73fff640 1 -- 172.21.15.16:0/2242720643 learned_addr learned my addr 172.21.15.16:0/2242720643 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.783+0000 7feb78edb640 1 --2- 172.21.15.16:0/2242720643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb74100880 0x7feb7419d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.783+0000 7feb73fff640 1 -- 172.21.15.16:0/2242720643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb740ffec0 msgr2=0x7feb7419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.783+0000 7feb6bfff640 1 --2- 172.21.15.16:0/2242720643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb740ffec0 0x7feb7419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.783+0000 7feb73fff640 1 --2- 172.21.15.16:0/2242720643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb740ffec0 0x7feb7419aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.783+0000 7feb73fff640 1 -- 172.21.15.16:0/2242720643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb74100880 msgr2=0x7feb7419d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.783+0000 7feb73fff640 1 --2- 172.21.15.16:0/2242720643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb74100880 0x7feb7419d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.783+0000 7feb73fff640 1 -- 172.21.15.16:0/2242720643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb5c009560 con 0x7feb740fecd0 2024-09-13T16:29:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.783+0000 7feb73fff640 1 --2- 172.21.15.16:0/2242720643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb740fecd0 0x7feb74069160 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7feb6400d950 tx=0x7feb6400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.783+0000 7feb71ffb640 1 -- 172.21.15.16:0/2242720643 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb64009c30 con 0x7feb740fecd0 2024-09-13T16:29:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.783+0000 7feb71ffb640 1 -- 172.21.15.16:0/2242720643 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feb64009d90 con 0x7feb740fecd0 2024-09-13T16:29:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.783+0000 7feb7a93e640 1 -- 172.21.15.16:0/2242720643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb7419de20 con 0x7feb740fecd0 2024-09-13T16:29:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.783+0000 7feb71ffb640 1 -- 172.21.15.16:0/2242720643 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6400f6b0 con 0x7feb740fecd0 2024-09-13T16:29:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.783+0000 7feb7a93e640 1 -- 172.21.15.16:0/2242720643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb7419e2a0 con 0x7feb740fecd0 2024-09-13T16:29:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.784+0000 7feb7a93e640 1 -- 172.21.15.16:0/2242720643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb74106270 con 0x7feb740fecd0 2024-09-13T16:29:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.786+0000 7feb71ffb640 1 -- 172.21.15.16:0/2242720643 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7feb64010460 con 0x7feb740fecd0 2024-09-13T16:29:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.786+0000 7feb71ffb640 1 --2- 172.21.15.16:0/2242720643 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7feb600a1110 0x7feb600a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.786+0000 7feb71ffb640 1 -- 172.21.15.16:0/2242720643 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7feb640cc990 con 0x7feb740fecd0 2024-09-13T16:29:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.786+0000 7feb6bfff640 1 --2- 172.21.15.16:0/2242720643 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7feb600a1110 0x7feb600a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.787+0000 7feb6bfff640 1 --2- 172.21.15.16:0/2242720643 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7feb600a1110 0x7feb600a35d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7feb5c006010 tx=0x7feb5c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.790+0000 7feb71ffb640 1 -- 172.21.15.16:0/2242720643 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7feb64095db0 con 0x7feb740fecd0 2024-09-13T16:29:08.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:08 smithi107 ceph-mon[34139]: from='client.14862 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:29:08.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:08 smithi107 ceph-mon[34139]: from='client.14868 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:29:08.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:08 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1594234800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:29:08.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:08 smithi111 ceph-mon[34600]: from='client.14862 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:29:08.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:08 smithi111 ceph-mon[34600]: from='client.14868 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:29:08.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:08 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1594234800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:29:08.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.960+0000 7feb7a93e640 1 -- 172.21.15.16:0/2242720643 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feb7402c4f0 con 0x7feb600a1110 2024-09-13T16:29:08.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.972+0000 7feb71ffb640 1 -- 172.21.15.16:0/2242720643 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422976 (secure 0 0 0) 0x7feb7402c4f0 con 0x7feb600a1110 2024-09-13T16:29:08.978 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:29:08.981 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-13T16:29:08.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.976+0000 7feb7a93e640 1 -- 172.21.15.16:0/2242720643 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7feb600a1110 msgr2=0x7feb600a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:08.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.976+0000 7feb7a93e640 1 --2- 172.21.15.16:0/2242720643 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7feb600a1110 0x7feb600a35d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7feb5c006010 tx=0x7feb5c000bc0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.976+0000 7feb7a93e640 1 -- 172.21.15.16:0/2242720643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb740fecd0 msgr2=0x7feb74069160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:08.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.976+0000 7feb7a93e640 1 --2- 172.21.15.16:0/2242720643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb740fecd0 0x7feb74069160 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7feb6400d950 tx=0x7feb6400de10 comp rx=0 tx=0).stop 2024-09-13T16:29:08.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.976+0000 7feb7a93e640 1 -- 172.21.15.16:0/2242720643 shutdown_connections 2024-09-13T16:29:08.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.976+0000 7feb7a93e640 1 --2- 172.21.15.16:0/2242720643 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7feb600a1110 0x7feb600a35d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.976+0000 7feb7a93e640 1 --2- 172.21.15.16:0/2242720643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb74100880 0x7feb7419d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.976+0000 7feb7a93e640 1 --2- 172.21.15.16:0/2242720643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb740ffec0 0x7feb7419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.976+0000 7feb7a93e640 1 --2- 172.21.15.16:0/2242720643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb740fecd0 0x7feb74069160 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:08.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.976+0000 7feb7a93e640 1 -- 172.21.15.16:0/2242720643 >> 172.21.15.16:0/2242720643 conn(0x7feb740fa430 msgr2=0x7feb74103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:08.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.976+0000 7feb7a93e640 1 -- 172.21.15.16:0/2242720643 shutdown_connections 2024-09-13T16:29:08.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:08.976+0000 7feb7a93e640 1 -- 172.21.15.16:0/2242720643 wait complete. 2024-09-13T16:29:08.989 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-13T16:29:08.571055+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":608404,"num_objects":100,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":183,"num_read_kb":155,"num_write":396,"num_write_kb":3018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1181498,"num_keys_recovered":10,"num_objects_omap":64,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":417,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364888,"kb_used_data":11980,"kb_used_omap":18,"kb_used_meta":352749,"kb_avail":1124576936,"statfs":{"total":1151940427776,"available":1151566782464,"internally_reserved":0,"allocated":12267520,"data_stored":7711739,"data_compressed":303615,"data_compressed_allocated":405504,"data_compressed_original":2310144,"omap_allocated":19131,"internal_metadata":361215301},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13443,"num_objects":76,"num_object_clones":0,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":57,"num_read_kb":46,"num_write":167,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":762,"num_keys_recovered":10,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002208"},"pg_stats":[{"pgid":"2.1f","version":"91'13","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.648333+0000","last_change":"2024-09-13T16:29:05.647872+0000","last_active":"2024-09-13T16:29:05.648333+0000","last_peered":"2024-09-13T16:29:05.648333+0000","last_clean":"2024-09-13T16:29:05.648333+0000","last_became_active":"2024-09-13T16:29:05.637050+0000","last_became_peered":"2024-09-13T16:29:05.637050+0000","last_unstale":"2024-09-13T16:29:05.648333+0000","last_undegraded":"2024-09-13T16:29:05.648333+0000","last_fullsized":"2024-09-13T16:29:05.648333+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:04.113840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.045098+0000","last_change":"2024-09-13T16:28:39.210024+0000","last_active":"2024-09-13T16:28:44.045098+0000","last_peered":"2024-09-13T16:28:44.045098+0000","last_clean":"2024-09-13T16:28:44.045098+0000","last_became_active":"2024-09-13T16:28:39.209751+0000","last_became_peered":"2024-09-13T16:28:39.209751+0000","last_unstale":"2024-09-13T16:28:44.045098+0000","last_undegraded":"2024-09-13T16:28:44.045098+0000","last_fullsized":"2024-09-13T16:28:44.045098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:34:24.357779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.632148+0000","last_change":"2024-09-13T16:28:43.230223+0000","last_active":"2024-09-13T16:29:04.632148+0000","last_peered":"2024-09-13T16:29:04.632148+0000","last_clean":"2024-09-13T16:29:04.632148+0000","last_became_active":"2024-09-13T16:28:43.228992+0000","last_became_peered":"2024-09-13T16:28:43.228992+0000","last_unstale":"2024-09-13T16:29:04.632148+0000","last_undegraded":"2024-09-13T16:29:04.632148+0000","last_fullsized":"2024-09-13T16:29:04.632148+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:45:37.252865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:29:03.591838+0000","last_change":"2024-09-13T16:28:38.850198+0000","last_active":"2024-09-13T16:29:03.591838+0000","last_peered":"2024-09-13T16:29:03.591838+0000","last_clean":"2024-09-13T16:29:03.591838+0000","last_became_active":"2024-09-13T16:28:38.849780+0000","last_became_peered":"2024-09-13T16:28:38.849780+0000","last_unstale":"2024-09-13T16:29:03.591838+0000","last_undegraded":"2024-09-13T16:29:03.591838+0000","last_fullsized":"2024-09-13T16:29:03.591838+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:09.653474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T16:28:43.218832+0000","last_change":"2024-09-13T16:28:39.203030+0000","last_active":"2024-09-13T16:28:43.218832+0000","last_peered":"2024-09-13T16:28:43.218832+0000","last_clean":"2024-09-13T16:28:43.218832+0000","last_became_active":"2024-09-13T16:28:39.202464+0000","last_became_peered":"2024-09-13T16:28:39.202464+0000","last_unstale":"2024-09-13T16:28:43.218832+0000","last_undegraded":"2024-09-13T16:28:43.218832+0000","last_fullsized":"2024-09-13T16:28:43.218832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:57.528836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.632329+0000","last_change":"2024-09-13T16:28:43.224358+0000","last_active":"2024-09-13T16:29:04.632329+0000","last_peered":"2024-09-13T16:29:04.632329+0000","last_clean":"2024-09-13T16:29:04.632329+0000","last_became_active":"2024-09-13T16:28:43.223955+0000","last_became_peered":"2024-09-13T16:28:43.223955+0000","last_unstale":"2024-09-13T16:29:04.632329+0000","last_undegraded":"2024-09-13T16:29:04.632329+0000","last_fullsized":"2024-09-13T16:29:04.632329+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:53:09.169704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'11","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.634720+0000","last_change":"2024-09-13T16:28:38.846051+0000","last_active":"2024-09-13T16:29:05.634720+0000","last_peered":"2024-09-13T16:29:05.634720+0000","last_clean":"2024-09-13T16:29:05.634720+0000","last_became_active":"2024-09-13T16:28:38.845762+0000","last_became_peered":"2024-09-13T16:28:38.845762+0000","last_unstale":"2024-09-13T16:29:05.634720+0000","last_undegraded":"2024-09-13T16:29:05.634720+0000","last_fullsized":"2024-09-13T16:29:05.634720+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:33:47.048582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044544+0000","last_change":"2024-09-13T16:28:39.210516+0000","last_active":"2024-09-13T16:28:44.044544+0000","last_peered":"2024-09-13T16:28:44.044544+0000","last_clean":"2024-09-13T16:28:44.044544+0000","last_became_active":"2024-09-13T16:28:39.210214+0000","last_became_peered":"2024-09-13T16:28:39.210214+0000","last_unstale":"2024-09-13T16:28:44.044544+0000","last_undegraded":"2024-09-13T16:28:44.044544+0000","last_fullsized":"2024-09-13T16:28:44.044544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:51:58.212685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.630396+0000","last_change":"2024-09-13T16:28:43.230530+0000","last_active":"2024-09-13T16:29:04.630396+0000","last_peered":"2024-09-13T16:29:04.630396+0000","last_clean":"2024-09-13T16:29:04.630396+0000","last_became_active":"2024-09-13T16:28:43.230154+0000","last_became_peered":"2024-09-13T16:28:43.230154+0000","last_unstale":"2024-09-13T16:29:04.630396+0000","last_undegraded":"2024-09-13T16:29:04.630396+0000","last_fullsized":"2024-09-13T16:29:04.630396+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:39:52.362177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.653834+0000","last_change":"2024-09-13T16:29:05.653460+0000","last_active":"2024-09-13T16:29:05.653834+0000","last_peered":"2024-09-13T16:29:05.653834+0000","last_clean":"2024-09-13T16:29:05.653834+0000","last_became_active":"2024-09-13T16:29:05.641836+0000","last_became_peered":"2024-09-13T16:29:05.641836+0000","last_unstale":"2024-09-13T16:29:05.653834+0000","last_undegraded":"2024-09-13T16:29:05.653834+0000","last_fullsized":"2024-09-13T16:29:05.653834+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:22:30.051033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.624846+0000","last_change":"2024-09-13T16:28:39.202381+0000","last_active":"2024-09-13T16:29:04.624846+0000","last_peered":"2024-09-13T16:29:04.624846+0000","last_clean":"2024-09-13T16:29:04.624846+0000","last_became_active":"2024-09-13T16:28:39.202023+0000","last_became_peered":"2024-09-13T16:28:39.202023+0000","last_unstale":"2024-09-13T16:29:04.624846+0000","last_undegraded":"2024-09-13T16:29:04.624846+0000","last_fullsized":"2024-09-13T16:29:04.624846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:55:46.445085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628398+0000","last_change":"2024-09-13T16:28:43.230623+0000","last_active":"2024-09-13T16:29:04.628398+0000","last_peered":"2024-09-13T16:29:04.628398+0000","last_clean":"2024-09-13T16:29:04.628398+0000","last_became_active":"2024-09-13T16:28:43.230240+0000","last_became_peered":"2024-09-13T16:28:43.230240+0000","last_unstale":"2024-09-13T16:29:04.628398+0000","last_undegraded":"2024-09-13T16:29:04.628398+0000","last_fullsized":"2024-09-13T16:29:04.628398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:04:07.723269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625334+0000","last_change":"2024-09-13T16:28:39.201896+0000","last_active":"2024-09-13T16:29:04.625334+0000","last_peered":"2024-09-13T16:29:04.625334+0000","last_clean":"2024-09-13T16:29:04.625334+0000","last_became_active":"2024-09-13T16:28:39.201570+0000","last_became_peered":"2024-09-13T16:28:39.201570+0000","last_unstale":"2024-09-13T16:29:04.625334+0000","last_undegraded":"2024-09-13T16:29:04.625334+0000","last_fullsized":"2024-09-13T16:29:04.625334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:07.532058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627167+0000","last_change":"2024-09-13T16:28:38.845998+0000","last_active":"2024-09-13T16:29:04.627167+0000","last_peered":"2024-09-13T16:29:04.627167+0000","last_clean":"2024-09-13T16:29:04.627167+0000","last_became_active":"2024-09-13T16:28:38.845451+0000","last_became_peered":"2024-09-13T16:28:38.845451+0000","last_unstale":"2024-09-13T16:29:04.627167+0000","last_undegraded":"2024-09-13T16:29:04.627167+0000","last_fullsized":"2024-09-13T16:29:04.627167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:15.696276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628820+0000","last_change":"2024-09-13T16:28:43.293598+0000","last_active":"2024-09-13T16:29:04.628820+0000","last_peered":"2024-09-13T16:29:04.628820+0000","last_clean":"2024-09-13T16:29:04.628820+0000","last_became_active":"2024-09-13T16:28:43.293045+0000","last_became_peered":"2024-09-13T16:28:43.293045+0000","last_unstale":"2024-09-13T16:29:04.628820+0000","last_undegraded":"2024-09-13T16:29:04.628820+0000","last_fullsized":"2024-09-13T16:29:04.628820+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:01.839626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044938+0000","last_change":"2024-09-13T16:28:39.214302+0000","last_active":"2024-09-13T16:28:44.044938+0000","last_peered":"2024-09-13T16:28:44.044938+0000","last_clean":"2024-09-13T16:28:44.044938+0000","last_became_active":"2024-09-13T16:28:39.213672+0000","last_became_peered":"2024-09-13T16:28:39.213672+0000","last_unstale":"2024-09-13T16:28:44.044938+0000","last_undegraded":"2024-09-13T16:28:44.044938+0000","last_fullsized":"2024-09-13T16:28:44.044938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:03:25.440603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"91'2","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.631445+0000","last_change":"2024-09-13T16:28:38.840626+0000","last_active":"2024-09-13T16:29:05.631445+0000","last_peered":"2024-09-13T16:29:05.631445+0000","last_clean":"2024-09-13T16:29:05.631445+0000","last_became_active":"2024-09-13T16:28:38.840344+0000","last_became_peered":"2024-09-13T16:28:38.840344+0000","last_unstale":"2024-09-13T16:29:05.631445+0000","last_undegraded":"2024-09-13T16:29:05.631445+0000","last_fullsized":"2024-09-13T16:29:05.631445+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:03:10.850301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.633381+0000","last_change":"2024-09-13T16:28:43.229163+0000","last_active":"2024-09-13T16:29:05.633381+0000","last_peered":"2024-09-13T16:29:05.633381+0000","last_clean":"2024-09-13T16:29:05.633381+0000","last_became_active":"2024-09-13T16:28:43.228783+0000","last_became_peered":"2024-09-13T16:28:43.228783+0000","last_unstale":"2024-09-13T16:29:05.633381+0000","last_undegraded":"2024-09-13T16:29:05.633381+0000","last_fullsized":"2024-09-13T16:29:05.633381+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:48.183837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627877+0000","last_change":"2024-09-13T16:28:39.209247+0000","last_active":"2024-09-13T16:29:04.627877+0000","last_peered":"2024-09-13T16:29:04.627877+0000","last_clean":"2024-09-13T16:29:04.627877+0000","last_became_active":"2024-09-13T16:28:39.208921+0000","last_became_peered":"2024-09-13T16:28:39.208921+0000","last_unstale":"2024-09-13T16:29:04.627877+0000","last_undegraded":"2024-09-13T16:29:04.627877+0000","last_fullsized":"2024-09-13T16:29:04.627877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:32:41.383029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.640902+0000","last_change":"2024-09-13T16:28:38.845904+0000","last_active":"2024-09-13T16:29:04.640902+0000","last_peered":"2024-09-13T16:29:04.640902+0000","last_clean":"2024-09-13T16:29:04.640902+0000","last_became_active":"2024-09-13T16:28:38.845639+0000","last_became_peered":"2024-09-13T16:28:38.845639+0000","last_unstale":"2024-09-13T16:29:04.640902+0000","last_undegraded":"2024-09-13T16:29:04.640902+0000","last_fullsized":"2024-09-13T16:29:04.640902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:25:34.148607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.640627+0000","last_change":"2024-09-13T16:29:05.640063+0000","last_active":"2024-09-13T16:29:05.640627+0000","last_peered":"2024-09-13T16:29:05.640627+0000","last_clean":"2024-09-13T16:29:05.640627+0000","last_became_active":"2024-09-13T16:29:05.639596+0000","last_became_peered":"2024-09-13T16:29:05.639596+0000","last_unstale":"2024-09-13T16:29:05.640627+0000","last_undegraded":"2024-09-13T16:29:05.640627+0000","last_fullsized":"2024-09-13T16:29:05.640627+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:41:38.351162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.626859+0000","last_change":"2024-09-13T16:28:39.212947+0000","last_active":"2024-09-13T16:29:04.626859+0000","last_peered":"2024-09-13T16:29:04.626859+0000","last_clean":"2024-09-13T16:29:04.626859+0000","last_became_active":"2024-09-13T16:28:39.212622+0000","last_became_peered":"2024-09-13T16:28:39.212622+0000","last_unstale":"2024-09-13T16:29:04.626859+0000","last_undegraded":"2024-09-13T16:29:04.626859+0000","last_fullsized":"2024-09-13T16:29:04.626859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:16.425154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:29:02.581304+0000","last_change":"2024-09-13T16:28:38.846173+0000","last_active":"2024-09-13T16:29:02.581304+0000","last_peered":"2024-09-13T16:29:02.581304+0000","last_clean":"2024-09-13T16:29:02.581304+0000","last_became_active":"2024-09-13T16:28:38.845873+0000","last_became_peered":"2024-09-13T16:28:38.845873+0000","last_unstale":"2024-09-13T16:29:02.581304+0000","last_undegraded":"2024-09-13T16:29:02.581304+0000","last_fullsized":"2024-09-13T16:29:02.581304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:19:55.817524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629006+0000","last_change":"2024-09-13T16:28:43.294347+0000","last_active":"2024-09-13T16:29:04.629006+0000","last_peered":"2024-09-13T16:29:04.629006+0000","last_clean":"2024-09-13T16:29:04.629006+0000","last_became_active":"2024-09-13T16:28:43.293902+0000","last_became_peered":"2024-09-13T16:28:43.293902+0000","last_unstale":"2024-09-13T16:29:04.629006+0000","last_undegraded":"2024-09-13T16:29:04.629006+0000","last_fullsized":"2024-09-13T16:29:04.629006+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:27:37.470163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T16:28:43.217326+0000","last_change":"2024-09-13T16:28:39.205817+0000","last_active":"2024-09-13T16:28:43.217326+0000","last_peered":"2024-09-13T16:28:43.217326+0000","last_clean":"2024-09-13T16:28:43.217326+0000","last_became_active":"2024-09-13T16:28:39.205534+0000","last_became_peered":"2024-09-13T16:28:39.205534+0000","last_unstale":"2024-09-13T16:28:43.217326+0000","last_undegraded":"2024-09-13T16:28:43.217326+0000","last_fullsized":"2024-09-13T16:28:43.217326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:09:16.294842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.633540+0000","last_change":"2024-09-13T16:28:38.847582+0000","last_active":"2024-09-13T16:29:05.633540+0000","last_peered":"2024-09-13T16:29:05.633540+0000","last_clean":"2024-09-13T16:29:05.633540+0000","last_became_active":"2024-09-13T16:28:38.847353+0000","last_became_peered":"2024-09-13T16:28:38.847353+0000","last_unstale":"2024-09-13T16:29:05.633540+0000","last_undegraded":"2024-09-13T16:29:05.633540+0000","last_fullsized":"2024-09-13T16:29:05.633540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:24.944088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.634008+0000","last_change":"2024-09-13T16:28:43.225202+0000","last_active":"2024-09-13T16:29:04.634008+0000","last_peered":"2024-09-13T16:29:04.634008+0000","last_clean":"2024-09-13T16:29:04.634008+0000","last_became_active":"2024-09-13T16:28:43.224802+0000","last_became_peered":"2024-09-13T16:28:43.224802+0000","last_unstale":"2024-09-13T16:29:04.634008+0000","last_undegraded":"2024-09-13T16:29:04.634008+0000","last_fullsized":"2024-09-13T16:29:04.634008+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:51:19.325859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.045516+0000","last_change":"2024-09-13T16:28:39.209630+0000","last_active":"2024-09-13T16:28:44.045516+0000","last_peered":"2024-09-13T16:28:44.045516+0000","last_clean":"2024-09-13T16:28:44.045516+0000","last_became_active":"2024-09-13T16:28:39.209303+0000","last_became_peered":"2024-09-13T16:28:39.209303+0000","last_unstale":"2024-09-13T16:28:44.045516+0000","last_undegraded":"2024-09-13T16:28:44.045516+0000","last_fullsized":"2024-09-13T16:28:44.045516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:01:31.822673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'7","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628093+0000","last_change":"2024-09-13T16:28:38.837858+0000","last_active":"2024-09-13T16:29:04.628093+0000","last_peered":"2024-09-13T16:29:04.628093+0000","last_clean":"2024-09-13T16:29:04.628093+0000","last_became_active":"2024-09-13T16:28:38.837378+0000","last_became_peered":"2024-09-13T16:28:38.837378+0000","last_unstale":"2024-09-13T16:29:04.628093+0000","last_undegraded":"2024-09-13T16:29:04.628093+0000","last_fullsized":"2024-09-13T16:29:04.628093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:59:41.055902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.754728+0000","last_change":"2024-09-13T16:28:43.294241+0000","last_active":"2024-09-13T16:28:44.754728+0000","last_peered":"2024-09-13T16:28:44.754728+0000","last_clean":"2024-09-13T16:28:44.754728+0000","last_became_active":"2024-09-13T16:28:43.293873+0000","last_became_peered":"2024-09-13T16:28:43.293873+0000","last_unstale":"2024-09-13T16:28:44.754728+0000","last_undegraded":"2024-09-13T16:28:44.754728+0000","last_fullsized":"2024-09-13T16:28:44.754728+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:34.018162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044782+0000","last_change":"2024-09-13T16:28:39.218071+0000","last_active":"2024-09-13T16:28:44.044782+0000","last_peered":"2024-09-13T16:28:44.044782+0000","last_clean":"2024-09-13T16:28:44.044782+0000","last_became_active":"2024-09-13T16:28:39.217798+0000","last_became_peered":"2024-09-13T16:28:39.217798+0000","last_unstale":"2024-09-13T16:28:44.044782+0000","last_undegraded":"2024-09-13T16:28:44.044782+0000","last_fullsized":"2024-09-13T16:28:44.044782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:39:09.769384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.640833+0000","last_change":"2024-09-13T16:28:38.847238+0000","last_active":"2024-09-13T16:29:04.640833+0000","last_peered":"2024-09-13T16:29:04.640833+0000","last_clean":"2024-09-13T16:29:04.640833+0000","last_became_active":"2024-09-13T16:28:38.846741+0000","last_became_peered":"2024-09-13T16:28:38.846741+0000","last_unstale":"2024-09-13T16:29:04.640833+0000","last_undegraded":"2024-09-13T16:29:04.640833+0000","last_fullsized":"2024-09-13T16:29:04.640833+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:52:51.946158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628298+0000","last_change":"2024-09-13T16:28:43.228431+0000","last_active":"2024-09-13T16:29:04.628298+0000","last_peered":"2024-09-13T16:29:04.628298+0000","last_clean":"2024-09-13T16:29:04.628298+0000","last_became_active":"2024-09-13T16:28:43.228030+0000","last_became_peered":"2024-09-13T16:28:43.228030+0000","last_unstale":"2024-09-13T16:29:04.628298+0000","last_undegraded":"2024-09-13T16:29:04.628298+0000","last_fullsized":"2024-09-13T16:29:04.628298+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:14:50.573033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629589+0000","last_change":"2024-09-13T16:28:39.209485+0000","last_active":"2024-09-13T16:29:04.629589+0000","last_peered":"2024-09-13T16:29:04.629589+0000","last_clean":"2024-09-13T16:29:04.629589+0000","last_became_active":"2024-09-13T16:28:39.209157+0000","last_became_peered":"2024-09-13T16:28:39.209157+0000","last_unstale":"2024-09-13T16:29:04.629589+0000","last_undegraded":"2024-09-13T16:29:04.629589+0000","last_fullsized":"2024-09-13T16:29:04.629589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:12:19.398255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628085+0000","last_change":"2024-09-13T16:28:38.842551+0000","last_active":"2024-09-13T16:29:04.628085+0000","last_peered":"2024-09-13T16:29:04.628085+0000","last_clean":"2024-09-13T16:29:04.628085+0000","last_became_active":"2024-09-13T16:28:38.841741+0000","last_became_peered":"2024-09-13T16:28:38.841741+0000","last_unstale":"2024-09-13T16:29:04.628085+0000","last_undegraded":"2024-09-13T16:29:04.628085+0000","last_fullsized":"2024-09-13T16:29:04.628085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:35:25.131693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.626884+0000","last_change":"2024-09-13T16:28:43.294300+0000","last_active":"2024-09-13T16:29:04.626884+0000","last_peered":"2024-09-13T16:29:04.626884+0000","last_clean":"2024-09-13T16:29:04.626884+0000","last_became_active":"2024-09-13T16:28:43.293918+0000","last_became_peered":"2024-09-13T16:28:43.293918+0000","last_unstale":"2024-09-13T16:29:04.626884+0000","last_undegraded":"2024-09-13T16:29:04.626884+0000","last_fullsized":"2024-09-13T16:29:04.626884+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:19:21.959617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.631071+0000","last_change":"2024-09-13T16:28:39.212001+0000","last_active":"2024-09-13T16:29:04.631071+0000","last_peered":"2024-09-13T16:29:04.631071+0000","last_clean":"2024-09-13T16:29:04.631071+0000","last_became_active":"2024-09-13T16:28:39.211716+0000","last_became_peered":"2024-09-13T16:28:39.211716+0000","last_unstale":"2024-09-13T16:29:04.631071+0000","last_undegraded":"2024-09-13T16:29:04.631071+0000","last_fullsized":"2024-09-13T16:29:04.631071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:56.753677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:06.648061+0000","last_change":"2024-09-13T16:28:38.851229+0000","last_active":"2024-09-13T16:29:06.648061+0000","last_peered":"2024-09-13T16:29:06.648061+0000","last_clean":"2024-09-13T16:29:06.648061+0000","last_became_active":"2024-09-13T16:28:38.850937+0000","last_became_peered":"2024-09-13T16:28:38.850937+0000","last_unstale":"2024-09-13T16:29:06.648061+0000","last_undegraded":"2024-09-13T16:29:06.648061+0000","last_fullsized":"2024-09-13T16:29:06.648061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:19.460864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:45.495529+0000","last_change":"2024-09-13T16:28:43.224825+0000","last_active":"2024-09-13T16:28:45.495529+0000","last_peered":"2024-09-13T16:28:45.495529+0000","last_clean":"2024-09-13T16:28:45.495529+0000","last_became_active":"2024-09-13T16:28:43.223903+0000","last_became_peered":"2024-09-13T16:28:43.223903+0000","last_unstale":"2024-09-13T16:28:45.495529+0000","last_undegraded":"2024-09-13T16:28:45.495529+0000","last_fullsized":"2024-09-13T16:28:45.495529+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:08:46.850668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.639142+0000","last_change":"2024-09-13T16:29:05.636787+0000","last_active":"2024-09-13T16:29:05.639142+0000","last_peered":"2024-09-13T16:29:05.639142+0000","last_clean":"2024-09-13T16:29:05.639142+0000","last_became_active":"2024-09-13T16:29:05.636399+0000","last_became_peered":"2024-09-13T16:29:05.636399+0000","last_unstale":"2024-09-13T16:29:05.639142+0000","last_undegraded":"2024-09-13T16:29:05.639142+0000","last_fullsized":"2024-09-13T16:29:05.639142+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:38:25.545325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.622845+0000","last_change":"2024-09-13T16:28:38.845374+0000","last_active":"2024-09-13T16:29:04.622845+0000","last_peered":"2024-09-13T16:29:04.622845+0000","last_clean":"2024-09-13T16:29:04.622845+0000","last_became_active":"2024-09-13T16:28:38.845076+0000","last_became_peered":"2024-09-13T16:28:38.845076+0000","last_unstale":"2024-09-13T16:29:04.622845+0000","last_undegraded":"2024-09-13T16:29:04.622845+0000","last_fullsized":"2024-09-13T16:29:04.622845+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:37:56.225682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:45.495577+0000","last_change":"2024-09-13T16:28:43.293529+0000","last_active":"2024-09-13T16:28:45.495577+0000","last_peered":"2024-09-13T16:28:45.495577+0000","last_clean":"2024-09-13T16:28:45.495577+0000","last_became_active":"2024-09-13T16:28:43.293083+0000","last_became_peered":"2024-09-13T16:28:43.293083+0000","last_unstale":"2024-09-13T16:28:45.495577+0000","last_undegraded":"2024-09-13T16:28:45.495577+0000","last_fullsized":"2024-09-13T16:28:45.495577+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:05:55.572984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.043722+0000","last_change":"2024-09-13T16:28:39.217546+0000","last_active":"2024-09-13T16:28:44.043722+0000","last_peered":"2024-09-13T16:28:44.043722+0000","last_clean":"2024-09-13T16:28:44.043722+0000","last_became_active":"2024-09-13T16:28:39.217242+0000","last_became_peered":"2024-09-13T16:28:39.217242+0000","last_unstale":"2024-09-13T16:28:44.043722+0000","last_undegraded":"2024-09-13T16:28:44.043722+0000","last_fullsized":"2024-09-13T16:28:44.043722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:23:18.183616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.641545+0000","last_change":"2024-09-13T16:28:38.846436+0000","last_active":"2024-09-13T16:29:04.641545+0000","last_peered":"2024-09-13T16:29:04.641545+0000","last_clean":"2024-09-13T16:29:04.641545+0000","last_became_active":"2024-09-13T16:28:38.845953+0000","last_became_peered":"2024-09-13T16:28:38.845953+0000","last_unstale":"2024-09-13T16:29:04.641545+0000","last_undegraded":"2024-09-13T16:29:04.641545+0000","last_fullsized":"2024-09-13T16:29:04.641545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:33.998462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.624750+0000","last_change":"2024-09-13T16:28:43.294504+0000","last_active":"2024-09-13T16:29:04.624750+0000","last_peered":"2024-09-13T16:29:04.624750+0000","last_clean":"2024-09-13T16:29:04.624750+0000","last_became_active":"2024-09-13T16:28:43.294063+0000","last_became_peered":"2024-09-13T16:28:43.294063+0000","last_unstale":"2024-09-13T16:29:04.624750+0000","last_undegraded":"2024-09-13T16:29:04.624750+0000","last_fullsized":"2024-09-13T16:29:04.624750+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:51:35.774467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044939+0000","last_change":"2024-09-13T16:28:39.206491+0000","last_active":"2024-09-13T16:28:44.044939+0000","last_peered":"2024-09-13T16:28:44.044939+0000","last_clean":"2024-09-13T16:28:44.044939+0000","last_became_active":"2024-09-13T16:28:39.206187+0000","last_became_peered":"2024-09-13T16:28:39.206187+0000","last_unstale":"2024-09-13T16:28:44.044939+0000","last_undegraded":"2024-09-13T16:28:44.044939+0000","last_fullsized":"2024-09-13T16:28:44.044939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:20:28.457994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.635563+0000","last_change":"2024-09-13T16:28:38.836270+0000","last_active":"2024-09-13T16:29:05.635563+0000","last_peered":"2024-09-13T16:29:05.635563+0000","last_clean":"2024-09-13T16:29:05.635563+0000","last_became_active":"2024-09-13T16:28:38.835966+0000","last_became_peered":"2024-09-13T16:28:38.835966+0000","last_unstale":"2024-09-13T16:29:05.635563+0000","last_undegraded":"2024-09-13T16:29:05.635563+0000","last_fullsized":"2024-09-13T16:29:05.635563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:03:11.388654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":331,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625402+0000","last_change":"2024-09-13T16:27:34.264484+0000","last_active":"2024-09-13T16:29:04.625402+0000","last_peered":"2024-09-13T16:29:04.625402+0000","last_clean":"2024-09-13T16:29:04.625402+0000","last_became_active":"2024-09-13T16:27:34.263995+0000","last_became_peered":"2024-09-13T16:27:34.263995+0000","last_unstale":"2024-09-13T16:29:04.625402+0000","last_undegraded":"2024-09-13T16:29:04.625402+0000","last_fullsized":"2024-09-13T16:29:04.625402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:24:23.015578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:24:23.015578+0000","last_clean_scrub_stamp":"2024-09-13T16:24:23.015578+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:25:04.376312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.634973+0000","last_change":"2024-09-13T16:28:43.229347+0000","last_active":"2024-09-13T16:29:05.634973+0000","last_peered":"2024-09-13T16:29:05.634973+0000","last_clean":"2024-09-13T16:29:05.634973+0000","last_became_active":"2024-09-13T16:28:43.228981+0000","last_became_peered":"2024-09-13T16:28:43.228981+0000","last_unstale":"2024-09-13T16:29:05.634973+0000","last_undegraded":"2024-09-13T16:29:05.634973+0000","last_fullsized":"2024-09-13T16:29:05.634973+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:46:15.151295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625877+0000","last_change":"2024-09-13T16:28:39.203195+0000","last_active":"2024-09-13T16:29:04.625877+0000","last_peered":"2024-09-13T16:29:04.625877+0000","last_clean":"2024-09-13T16:29:04.625877+0000","last_became_active":"2024-09-13T16:28:39.202878+0000","last_became_peered":"2024-09-13T16:28:39.202878+0000","last_unstale":"2024-09-13T16:29:04.625877+0000","last_undegraded":"2024-09-13T16:29:04.625877+0000","last_fullsized":"2024-09-13T16:29:04.625877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:17.615531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:06.646697+0000","last_change":"2024-09-13T16:28:38.844856+0000","last_active":"2024-09-13T16:29:06.646697+0000","last_peered":"2024-09-13T16:29:06.646697+0000","last_clean":"2024-09-13T16:29:06.646697+0000","last_became_active":"2024-09-13T16:28:38.844578+0000","last_became_peered":"2024-09-13T16:28:38.844578+0000","last_unstale":"2024-09-13T16:29:06.646697+0000","last_undegraded":"2024-09-13T16:29:06.646697+0000","last_fullsized":"2024-09-13T16:29:06.646697+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:41:48.253174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:45.572882+0000","last_change":"2024-09-13T16:28:43.228472+0000","last_active":"2024-09-13T16:28:45.572882+0000","last_peered":"2024-09-13T16:28:45.572882+0000","last_clean":"2024-09-13T16:28:45.572882+0000","last_became_active":"2024-09-13T16:28:43.228056+0000","last_became_peered":"2024-09-13T16:28:43.228056+0000","last_unstale":"2024-09-13T16:28:45.572882+0000","last_undegraded":"2024-09-13T16:28:45.572882+0000","last_fullsized":"2024-09-13T16:28:45.572882+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:07:55.980691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625637+0000","last_change":"2024-09-13T16:28:39.203278+0000","last_active":"2024-09-13T16:29:04.625637+0000","last_peered":"2024-09-13T16:29:04.625637+0000","last_clean":"2024-09-13T16:29:04.625637+0000","last_became_active":"2024-09-13T16:28:39.202936+0000","last_became_peered":"2024-09-13T16:28:39.202936+0000","last_unstale":"2024-09-13T16:29:04.625637+0000","last_undegraded":"2024-09-13T16:29:04.625637+0000","last_fullsized":"2024-09-13T16:29:04.625637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:44.210612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.640824+0000","last_change":"2024-09-13T16:28:38.851085+0000","last_active":"2024-09-13T16:29:04.640824+0000","last_peered":"2024-09-13T16:29:04.640824+0000","last_clean":"2024-09-13T16:29:04.640824+0000","last_became_active":"2024-09-13T16:28:38.850785+0000","last_became_peered":"2024-09-13T16:28:38.850785+0000","last_unstale":"2024-09-13T16:29:04.640824+0000","last_undegraded":"2024-09-13T16:29:04.640824+0000","last_fullsized":"2024-09-13T16:29:04.640824+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:52.713431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044053+0000","last_change":"2024-09-13T16:28:43.229980+0000","last_active":"2024-09-13T16:28:44.044053+0000","last_peered":"2024-09-13T16:28:44.044053+0000","last_clean":"2024-09-13T16:28:44.044053+0000","last_became_active":"2024-09-13T16:28:43.229647+0000","last_became_peered":"2024-09-13T16:28:43.229647+0000","last_unstale":"2024-09-13T16:28:44.044053+0000","last_undegraded":"2024-09-13T16:28:44.044053+0000","last_fullsized":"2024-09-13T16:28:44.044053+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:39:42.871613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625770+0000","last_change":"2024-09-13T16:28:39.204167+0000","last_active":"2024-09-13T16:29:04.625770+0000","last_peered":"2024-09-13T16:29:04.625770+0000","last_clean":"2024-09-13T16:29:04.625770+0000","last_became_active":"2024-09-13T16:28:39.203776+0000","last_became_peered":"2024-09-13T16:28:39.203776+0000","last_unstale":"2024-09-13T16:29:04.625770+0000","last_undegraded":"2024-09-13T16:29:04.625770+0000","last_fullsized":"2024-09-13T16:29:04.625770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:53:36.410365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.639297+0000","last_change":"2024-09-13T16:28:38.846331+0000","last_active":"2024-09-13T16:29:04.639297+0000","last_peered":"2024-09-13T16:29:04.639297+0000","last_clean":"2024-09-13T16:29:04.639297+0000","last_became_active":"2024-09-13T16:28:38.846012+0000","last_became_peered":"2024-09-13T16:28:38.846012+0000","last_unstale":"2024-09-13T16:29:04.639297+0000","last_undegraded":"2024-09-13T16:29:04.639297+0000","last_fullsized":"2024-09-13T16:29:04.639297+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:23:54.282736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.623954+0000","last_change":"2024-09-13T16:28:43.227886+0000","last_active":"2024-09-13T16:29:04.623954+0000","last_peered":"2024-09-13T16:29:04.623954+0000","last_clean":"2024-09-13T16:29:04.623954+0000","last_became_active":"2024-09-13T16:28:43.227346+0000","last_became_peered":"2024-09-13T16:28:43.227346+0000","last_unstale":"2024-09-13T16:29:04.623954+0000","last_undegraded":"2024-09-13T16:29:04.623954+0000","last_fullsized":"2024-09-13T16:29:04.623954+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:41:36.559064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629086+0000","last_change":"2024-09-13T16:28:39.205975+0000","last_active":"2024-09-13T16:29:04.629086+0000","last_peered":"2024-09-13T16:29:04.629086+0000","last_clean":"2024-09-13T16:29:04.629086+0000","last_became_active":"2024-09-13T16:28:39.205657+0000","last_became_peered":"2024-09-13T16:28:39.205657+0000","last_unstale":"2024-09-13T16:29:04.629086+0000","last_undegraded":"2024-09-13T16:29:04.629086+0000","last_fullsized":"2024-09-13T16:29:04.629086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:31:47.309482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629013+0000","last_change":"2024-09-13T16:28:38.841589+0000","last_active":"2024-09-13T16:29:04.629013+0000","last_peered":"2024-09-13T16:29:04.629013+0000","last_clean":"2024-09-13T16:29:04.629013+0000","last_became_active":"2024-09-13T16:28:38.841298+0000","last_became_peered":"2024-09-13T16:28:38.841298+0000","last_unstale":"2024-09-13T16:29:04.629013+0000","last_undegraded":"2024-09-13T16:29:04.629013+0000","last_fullsized":"2024-09-13T16:29:04.629013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:27:38.358320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.623986+0000","last_change":"2024-09-13T16:28:43.233790+0000","last_active":"2024-09-13T16:29:04.623986+0000","last_peered":"2024-09-13T16:29:04.623986+0000","last_clean":"2024-09-13T16:29:04.623986+0000","last_became_active":"2024-09-13T16:28:43.231963+0000","last_became_peered":"2024-09-13T16:28:43.231963+0000","last_unstale":"2024-09-13T16:29:04.623986+0000","last_undegraded":"2024-09-13T16:29:04.623986+0000","last_fullsized":"2024-09-13T16:29:04.623986+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:45:23.406395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.624233+0000","last_change":"2024-09-13T16:28:39.201476+0000","last_active":"2024-09-13T16:29:04.624233+0000","last_peered":"2024-09-13T16:29:04.624233+0000","last_clean":"2024-09-13T16:29:04.624233+0000","last_became_active":"2024-09-13T16:28:39.201168+0000","last_became_peered":"2024-09-13T16:28:39.201168+0000","last_unstale":"2024-09-13T16:29:04.624233+0000","last_undegraded":"2024-09-13T16:29:04.624233+0000","last_fullsized":"2024-09-13T16:29:04.624233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:26:43.570295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":62,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.652225+0000","last_change":"2024-09-13T16:29:05.652225+0000","last_active":"2024-09-13T16:29:05.652225+0000","last_peered":"2024-09-13T16:29:05.652225+0000","last_clean":"2024-09-13T16:29:05.652225+0000","last_became_active":"2024-09-13T16:29:05.639325+0000","last_became_peered":"2024-09-13T16:29:05.639325+0000","last_unstale":"2024-09-13T16:29:05.652225+0000","last_undegraded":"2024-09-13T16:29:05.652225+0000","last_fullsized":"2024-09-13T16:29:05.652225+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:18:30.083447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:45.495557+0000","last_change":"2024-09-13T16:28:43.228563+0000","last_active":"2024-09-13T16:28:45.495557+0000","last_peered":"2024-09-13T16:28:45.495557+0000","last_clean":"2024-09-13T16:28:45.495557+0000","last_became_active":"2024-09-13T16:28:43.228184+0000","last_became_peered":"2024-09-13T16:28:43.228184+0000","last_unstale":"2024-09-13T16:28:45.495557+0000","last_undegraded":"2024-09-13T16:28:45.495557+0000","last_fullsized":"2024-09-13T16:28:45.495557+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:05:33.368302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625221+0000","last_change":"2024-09-13T16:28:39.210750+0000","last_active":"2024-09-13T16:29:04.625221+0000","last_peered":"2024-09-13T16:29:04.625221+0000","last_clean":"2024-09-13T16:29:04.625221+0000","last_became_active":"2024-09-13T16:28:39.210401+0000","last_became_peered":"2024-09-13T16:28:39.210401+0000","last_unstale":"2024-09-13T16:29:04.625221+0000","last_undegraded":"2024-09-13T16:29:04.625221+0000","last_fullsized":"2024-09-13T16:29:04.625221+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:36.670420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:06.638428+0000","last_change":"2024-09-13T16:28:38.843573+0000","last_active":"2024-09-13T16:29:06.638428+0000","last_peered":"2024-09-13T16:29:06.638428+0000","last_clean":"2024-09-13T16:29:06.638428+0000","last_became_active":"2024-09-13T16:28:38.843245+0000","last_became_peered":"2024-09-13T16:28:38.843245+0000","last_unstale":"2024-09-13T16:29:06.638428+0000","last_undegraded":"2024-09-13T16:29:06.638428+0000","last_fullsized":"2024-09-13T16:29:06.638428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:45.983123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628374+0000","last_change":"2024-09-13T16:28:43.231002+0000","last_active":"2024-09-13T16:29:04.628374+0000","last_peered":"2024-09-13T16:29:04.628374+0000","last_clean":"2024-09-13T16:29:04.628374+0000","last_became_active":"2024-09-13T16:28:43.230237+0000","last_became_peered":"2024-09-13T16:28:43.230237+0000","last_unstale":"2024-09-13T16:29:04.628374+0000","last_undegraded":"2024-09-13T16:29:04.628374+0000","last_fullsized":"2024-09-13T16:29:04.628374+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:11:47.494738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625559+0000","last_change":"2024-09-13T16:28:39.201471+0000","last_active":"2024-09-13T16:29:04.625559+0000","last_peered":"2024-09-13T16:29:04.625559+0000","last_clean":"2024-09-13T16:29:04.625559+0000","last_became_active":"2024-09-13T16:28:39.201140+0000","last_became_peered":"2024-09-13T16:28:39.201140+0000","last_unstale":"2024-09-13T16:29:04.625559+0000","last_undegraded":"2024-09-13T16:29:04.625559+0000","last_fullsized":"2024-09-13T16:29:04.625559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:37:25.037490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.641008+0000","last_change":"2024-09-13T16:28:38.845703+0000","last_active":"2024-09-13T16:29:04.641008+0000","last_peered":"2024-09-13T16:29:04.641008+0000","last_clean":"2024-09-13T16:29:04.641008+0000","last_became_active":"2024-09-13T16:28:38.845354+0000","last_became_peered":"2024-09-13T16:28:38.845354+0000","last_unstale":"2024-09-13T16:29:04.641008+0000","last_undegraded":"2024-09-13T16:29:04.641008+0000","last_fullsized":"2024-09-13T16:29:04.641008+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:18.208723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629807+0000","last_change":"2024-09-13T16:28:43.223924+0000","last_active":"2024-09-13T16:29:04.629807+0000","last_peered":"2024-09-13T16:29:04.629807+0000","last_clean":"2024-09-13T16:29:04.629807+0000","last_became_active":"2024-09-13T16:28:43.223535+0000","last_became_peered":"2024-09-13T16:28:43.223535+0000","last_unstale":"2024-09-13T16:29:04.629807+0000","last_undegraded":"2024-09-13T16:29:04.629807+0000","last_fullsized":"2024-09-13T16:29:04.629807+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:10.622436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044425+0000","last_change":"2024-09-13T16:28:39.203943+0000","last_active":"2024-09-13T16:28:44.044425+0000","last_peered":"2024-09-13T16:28:44.044425+0000","last_clean":"2024-09-13T16:28:44.044425+0000","last_became_active":"2024-09-13T16:28:39.203657+0000","last_became_peered":"2024-09-13T16:28:39.203657+0000","last_unstale":"2024-09-13T16:28:44.044425+0000","last_undegraded":"2024-09-13T16:28:44.044425+0000","last_fullsized":"2024-09-13T16:28:44.044425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:52:48.523939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.651351+0000","last_change":"2024-09-13T16:29:05.651015+0000","last_active":"2024-09-13T16:29:05.651351+0000","last_peered":"2024-09-13T16:29:05.651351+0000","last_clean":"2024-09-13T16:29:05.651351+0000","last_became_active":"2024-09-13T16:29:05.640006+0000","last_became_peered":"2024-09-13T16:29:05.640006+0000","last_unstale":"2024-09-13T16:29:05.651351+0000","last_undegraded":"2024-09-13T16:29:05.651351+0000","last_fullsized":"2024-09-13T16:29:05.651351+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:12:25.508262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.624674+0000","last_change":"2024-09-13T16:28:43.232634+0000","last_active":"2024-09-13T16:29:04.624674+0000","last_peered":"2024-09-13T16:29:04.624674+0000","last_clean":"2024-09-13T16:29:04.624674+0000","last_became_active":"2024-09-13T16:28:43.232107+0000","last_became_peered":"2024-09-13T16:28:43.232107+0000","last_unstale":"2024-09-13T16:29:04.624674+0000","last_undegraded":"2024-09-13T16:29:04.624674+0000","last_fullsized":"2024-09-13T16:29:04.624674+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:45.952933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627482+0000","last_change":"2024-09-13T16:28:39.212883+0000","last_active":"2024-09-13T16:29:04.627482+0000","last_peered":"2024-09-13T16:29:04.627482+0000","last_clean":"2024-09-13T16:29:04.627482+0000","last_became_active":"2024-09-13T16:28:39.212591+0000","last_became_peered":"2024-09-13T16:28:39.212591+0000","last_unstale":"2024-09-13T16:29:04.627482+0000","last_undegraded":"2024-09-13T16:29:04.627482+0000","last_fullsized":"2024-09-13T16:29:04.627482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:17:30.926940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'17","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:06.645117+0000","last_change":"2024-09-13T16:28:38.846564+0000","last_active":"2024-09-13T16:29:06.645117+0000","last_peered":"2024-09-13T16:29:06.645117+0000","last_clean":"2024-09-13T16:29:06.645117+0000","last_became_active":"2024-09-13T16:28:38.846165+0000","last_became_peered":"2024-09-13T16:28:38.846165+0000","last_unstale":"2024-09-13T16:29:06.645117+0000","last_undegraded":"2024-09-13T16:29:06.645117+0000","last_fullsized":"2024-09-13T16:29:06.645117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:33:06.124592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10396,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":19,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627853+0000","last_change":"2024-09-13T16:28:43.231464+0000","last_active":"2024-09-13T16:29:04.627853+0000","last_peered":"2024-09-13T16:29:04.627853+0000","last_clean":"2024-09-13T16:29:04.627853+0000","last_became_active":"2024-09-13T16:28:43.231040+0000","last_became_peered":"2024-09-13T16:28:43.231040+0000","last_unstale":"2024-09-13T16:29:04.627853+0000","last_undegraded":"2024-09-13T16:29:04.627853+0000","last_fullsized":"2024-09-13T16:29:04.627853+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:11:21.847375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"peering","last_fresh":"2024-09-13T16:29:04.630957+0000","last_change":"2024-09-13T16:29:04.631158+0000","last_active":"2024-09-13T16:29:04.630957+0000","last_peered":"2024-09-13T16:28:44.044616+0000","last_clean":"2024-09-13T16:28:44.044616+0000","last_became_active":"2024-09-13T16:28:39.204382+0000","last_became_peered":"2024-09-13T16:28:39.204382+0000","last_unstale":"2024-09-13T16:29:04.630957+0000","last_undegraded":"2024-09-13T16:29:04.630957+0000","last_fullsized":"2024-09-13T16:29:04.630957+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:58.816998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6,8],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625089+0000","last_change":"2024-09-13T16:28:38.841637+0000","last_active":"2024-09-13T16:29:04.625089+0000","last_peered":"2024-09-13T16:29:04.625089+0000","last_clean":"2024-09-13T16:29:04.625089+0000","last_became_active":"2024-09-13T16:28:38.841359+0000","last_became_peered":"2024-09-13T16:28:38.841359+0000","last_unstale":"2024-09-13T16:29:04.625089+0000","last_undegraded":"2024-09-13T16:29:04.625089+0000","last_fullsized":"2024-09-13T16:29:04.625089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:47:05.284987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.638870+0000","last_change":"2024-09-13T16:29:05.638870+0000","last_active":"2024-09-13T16:29:05.638870+0000","last_peered":"2024-09-13T16:29:05.638870+0000","last_clean":"2024-09-13T16:29:05.638870+0000","last_became_active":"2024-09-13T16:29:05.638422+0000","last_became_peered":"2024-09-13T16:29:05.638422+0000","last_unstale":"2024-09-13T16:29:05.638870+0000","last_undegraded":"2024-09-13T16:29:05.638870+0000","last_fullsized":"2024-09-13T16:29:05.638870+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:41.263661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T16:28:43.218920+0000","last_change":"2024-09-13T16:28:39.203595+0000","last_active":"2024-09-13T16:28:43.218920+0000","last_peered":"2024-09-13T16:28:43.218920+0000","last_clean":"2024-09-13T16:28:43.218920+0000","last_became_active":"2024-09-13T16:28:39.203283+0000","last_became_peered":"2024-09-13T16:28:39.203283+0000","last_unstale":"2024-09-13T16:28:43.218920+0000","last_undegraded":"2024-09-13T16:28:43.218920+0000","last_fullsized":"2024-09-13T16:28:43.218920+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:58:45.970761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.642267+0000","last_change":"2024-09-13T16:28:38.850203+0000","last_active":"2024-09-13T16:29:04.642267+0000","last_peered":"2024-09-13T16:29:04.642267+0000","last_clean":"2024-09-13T16:29:04.642267+0000","last_became_active":"2024-09-13T16:28:38.849963+0000","last_became_peered":"2024-09-13T16:28:38.849963+0000","last_unstale":"2024-09-13T16:29:04.642267+0000","last_undegraded":"2024-09-13T16:29:04.642267+0000","last_fullsized":"2024-09-13T16:29:04.642267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:15.849627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627470+0000","last_change":"2024-09-13T16:28:43.227679+0000","last_active":"2024-09-13T16:29:04.627470+0000","last_peered":"2024-09-13T16:29:04.627470+0000","last_clean":"2024-09-13T16:29:04.627470+0000","last_became_active":"2024-09-13T16:28:43.227269+0000","last_became_peered":"2024-09-13T16:28:43.227269+0000","last_unstale":"2024-09-13T16:29:04.627470+0000","last_undegraded":"2024-09-13T16:29:04.627470+0000","last_fullsized":"2024-09-13T16:29:04.627470+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:05:00.186305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044938+0000","last_change":"2024-09-13T16:28:39.211401+0000","last_active":"2024-09-13T16:28:44.044938+0000","last_peered":"2024-09-13T16:28:44.044938+0000","last_clean":"2024-09-13T16:28:44.044938+0000","last_became_active":"2024-09-13T16:28:39.211133+0000","last_became_peered":"2024-09-13T16:28:39.211133+0000","last_unstale":"2024-09-13T16:28:44.044938+0000","last_undegraded":"2024-09-13T16:28:44.044938+0000","last_fullsized":"2024-09-13T16:28:44.044938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:36.468666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:06.642523+0000","last_change":"2024-09-13T16:28:38.845132+0000","last_active":"2024-09-13T16:29:06.642523+0000","last_peered":"2024-09-13T16:29:06.642523+0000","last_clean":"2024-09-13T16:29:06.642523+0000","last_became_active":"2024-09-13T16:28:38.844804+0000","last_became_peered":"2024-09-13T16:28:38.844804+0000","last_unstale":"2024-09-13T16:29:06.642523+0000","last_undegraded":"2024-09-13T16:29:06.642523+0000","last_fullsized":"2024-09-13T16:29:06.642523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:15:08.643227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.754760+0000","last_change":"2024-09-13T16:28:43.230829+0000","last_active":"2024-09-13T16:28:44.754760+0000","last_peered":"2024-09-13T16:28:44.754760+0000","last_clean":"2024-09-13T16:28:44.754760+0000","last_became_active":"2024-09-13T16:28:43.230405+0000","last_became_peered":"2024-09-13T16:28:43.230405+0000","last_unstale":"2024-09-13T16:28:44.754760+0000","last_undegraded":"2024-09-13T16:28:44.754760+0000","last_fullsized":"2024-09-13T16:28:44.754760+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:04:56.083193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044006+0000","last_change":"2024-09-13T16:28:43.226662+0000","last_active":"2024-09-13T16:28:44.044006+0000","last_peered":"2024-09-13T16:28:44.044006+0000","last_clean":"2024-09-13T16:28:44.044006+0000","last_became_active":"2024-09-13T16:28:43.226232+0000","last_became_peered":"2024-09-13T16:28:43.226232+0000","last_unstale":"2024-09-13T16:28:44.044006+0000","last_undegraded":"2024-09-13T16:28:44.044006+0000","last_fullsized":"2024-09-13T16:28:44.044006+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:03.568500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627636+0000","last_change":"2024-09-13T16:28:39.208909+0000","last_active":"2024-09-13T16:29:04.627636+0000","last_peered":"2024-09-13T16:29:04.627636+0000","last_clean":"2024-09-13T16:29:04.627636+0000","last_became_active":"2024-09-13T16:28:39.208572+0000","last_became_peered":"2024-09-13T16:28:39.208572+0000","last_unstale":"2024-09-13T16:29:04.627636+0000","last_undegraded":"2024-09-13T16:29:04.627636+0000","last_fullsized":"2024-09-13T16:29:04.627636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:01:40.173308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.641240+0000","last_change":"2024-09-13T16:28:38.844259+0000","last_active":"2024-09-13T16:29:04.641240+0000","last_peered":"2024-09-13T16:29:04.641240+0000","last_clean":"2024-09-13T16:29:04.641240+0000","last_became_active":"2024-09-13T16:28:38.843956+0000","last_became_peered":"2024-09-13T16:28:38.843956+0000","last_unstale":"2024-09-13T16:29:04.641240+0000","last_undegraded":"2024-09-13T16:29:04.641240+0000","last_fullsized":"2024-09-13T16:29:04.641240+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:05.652852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629221+0000","last_change":"2024-09-13T16:28:43.229385+0000","last_active":"2024-09-13T16:29:04.629221+0000","last_peered":"2024-09-13T16:29:04.629221+0000","last_clean":"2024-09-13T16:29:04.629221+0000","last_became_active":"2024-09-13T16:28:43.229033+0000","last_became_peered":"2024-09-13T16:28:43.229033+0000","last_unstale":"2024-09-13T16:29:04.629221+0000","last_undegraded":"2024-09-13T16:29:04.629221+0000","last_fullsized":"2024-09-13T16:29:04.629221+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:46:41.611758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044455+0000","last_change":"2024-09-13T16:28:39.208100+0000","last_active":"2024-09-13T16:28:44.044455+0000","last_peered":"2024-09-13T16:28:44.044455+0000","last_clean":"2024-09-13T16:28:44.044455+0000","last_became_active":"2024-09-13T16:28:39.207808+0000","last_became_peered":"2024-09-13T16:28:39.207808+0000","last_unstale":"2024-09-13T16:28:44.044455+0000","last_undegraded":"2024-09-13T16:28:44.044455+0000","last_fullsized":"2024-09-13T16:28:44.044455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:55.446568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.622326+0000","last_change":"2024-09-13T16:28:38.844689+0000","last_active":"2024-09-13T16:29:04.622326+0000","last_peered":"2024-09-13T16:29:04.622326+0000","last_clean":"2024-09-13T16:29:04.622326+0000","last_became_active":"2024-09-13T16:28:38.844326+0000","last_became_peered":"2024-09-13T16:28:38.844326+0000","last_unstale":"2024-09-13T16:29:04.622326+0000","last_undegraded":"2024-09-13T16:29:04.622326+0000","last_fullsized":"2024-09-13T16:29:04.622326+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:02.616518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629530+0000","last_change":"2024-09-13T16:28:38.849233+0000","last_active":"2024-09-13T16:29:04.629530+0000","last_peered":"2024-09-13T16:29:04.629530+0000","last_clean":"2024-09-13T16:29:04.629530+0000","last_became_active":"2024-09-13T16:28:38.848911+0000","last_became_peered":"2024-09-13T16:28:38.848911+0000","last_unstale":"2024-09-13T16:29:04.629530+0000","last_undegraded":"2024-09-13T16:29:04.629530+0000","last_fullsized":"2024-09-13T16:29:04.629530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:02:35.951263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.624339+0000","last_change":"2024-09-13T16:28:39.203855+0000","last_active":"2024-09-13T16:29:04.624339+0000","last_peered":"2024-09-13T16:29:04.624339+0000","last_clean":"2024-09-13T16:29:04.624339+0000","last_became_active":"2024-09-13T16:28:39.203532+0000","last_became_peered":"2024-09-13T16:28:39.203532+0000","last_unstale":"2024-09-13T16:29:04.624339+0000","last_undegraded":"2024-09-13T16:29:04.624339+0000","last_fullsized":"2024-09-13T16:29:04.624339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:15.048930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.633730+0000","last_change":"2024-09-13T16:28:43.228554+0000","last_active":"2024-09-13T16:29:04.633730+0000","last_peered":"2024-09-13T16:29:04.633730+0000","last_clean":"2024-09-13T16:29:04.633730+0000","last_became_active":"2024-09-13T16:28:43.228204+0000","last_became_peered":"2024-09-13T16:28:43.228204+0000","last_unstale":"2024-09-13T16:29:04.633730+0000","last_undegraded":"2024-09-13T16:29:04.633730+0000","last_fullsized":"2024-09-13T16:29:04.633730+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:16:15.383932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.639374+0000","last_change":"2024-09-13T16:28:38.848425+0000","last_active":"2024-09-13T16:29:05.639374+0000","last_peered":"2024-09-13T16:29:05.639374+0000","last_clean":"2024-09-13T16:29:05.639374+0000","last_became_active":"2024-09-13T16:28:38.848136+0000","last_became_peered":"2024-09-13T16:28:38.848136+0000","last_unstale":"2024-09-13T16:29:05.639374+0000","last_undegraded":"2024-09-13T16:29:05.639374+0000","last_fullsized":"2024-09-13T16:29:05.639374+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:47:03.904315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627249+0000","last_change":"2024-09-13T16:28:39.210565+0000","last_active":"2024-09-13T16:29:04.627249+0000","last_peered":"2024-09-13T16:29:04.627249+0000","last_clean":"2024-09-13T16:29:04.627249+0000","last_became_active":"2024-09-13T16:28:39.210272+0000","last_became_peered":"2024-09-13T16:28:39.210272+0000","last_unstale":"2024-09-13T16:29:04.627249+0000","last_undegraded":"2024-09-13T16:29:04.627249+0000","last_fullsized":"2024-09-13T16:29:04.627249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:12.746087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.634077+0000","last_change":"2024-09-13T16:28:43.231623+0000","last_active":"2024-09-13T16:29:04.634077+0000","last_peered":"2024-09-13T16:29:04.634077+0000","last_clean":"2024-09-13T16:29:04.634077+0000","last_became_active":"2024-09-13T16:28:43.231255+0000","last_became_peered":"2024-09-13T16:28:43.231255+0000","last_unstale":"2024-09-13T16:29:04.634077+0000","last_undegraded":"2024-09-13T16:29:04.634077+0000","last_fullsized":"2024-09-13T16:29:04.634077+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:29.755772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627783+0000","last_change":"2024-09-13T16:28:39.208438+0000","last_active":"2024-09-13T16:29:04.627783+0000","last_peered":"2024-09-13T16:29:04.627783+0000","last_clean":"2024-09-13T16:29:04.627783+0000","last_became_active":"2024-09-13T16:28:39.208156+0000","last_became_peered":"2024-09-13T16:28:39.208156+0000","last_unstale":"2024-09-13T16:29:04.627783+0000","last_undegraded":"2024-09-13T16:29:04.627783+0000","last_fullsized":"2024-09-13T16:29:04.627783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:53:46.923166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.633806+0000","last_change":"2024-09-13T16:28:39.208529+0000","last_active":"2024-09-13T16:29:04.633806+0000","last_peered":"2024-09-13T16:29:04.633806+0000","last_clean":"2024-09-13T16:29:04.633806+0000","last_became_active":"2024-09-13T16:28:39.208202+0000","last_became_peered":"2024-09-13T16:28:39.208202+0000","last_unstale":"2024-09-13T16:29:04.633806+0000","last_undegraded":"2024-09-13T16:29:04.633806+0000","last_fullsized":"2024-09-13T16:29:04.633806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:17:11.357410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.632611+0000","last_change":"2024-09-13T16:28:39.212958+0000","last_active":"2024-09-13T16:29:04.632611+0000","last_peered":"2024-09-13T16:29:04.632611+0000","last_clean":"2024-09-13T16:29:04.632611+0000","last_became_active":"2024-09-13T16:28:39.212646+0000","last_became_peered":"2024-09-13T16:28:39.212646+0000","last_unstale":"2024-09-13T16:29:04.632611+0000","last_undegraded":"2024-09-13T16:29:04.632611+0000","last_fullsized":"2024-09-13T16:29:04.632611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:31:04.047778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T16:28:43.218683+0000","last_change":"2024-09-13T16:28:39.200449+0000","last_active":"2024-09-13T16:28:43.218683+0000","last_peered":"2024-09-13T16:28:43.218683+0000","last_clean":"2024-09-13T16:28:43.218683+0000","last_became_active":"2024-09-13T16:28:39.200133+0000","last_became_peered":"2024-09-13T16:28:39.200133+0000","last_unstale":"2024-09-13T16:28:43.218683+0000","last_undegraded":"2024-09-13T16:28:43.218683+0000","last_fullsized":"2024-09-13T16:28:43.218683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:18:06.480691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044668+0000","last_change":"2024-09-13T16:28:39.200274+0000","last_active":"2024-09-13T16:28:44.044668+0000","last_peered":"2024-09-13T16:28:44.044668+0000","last_clean":"2024-09-13T16:28:44.044668+0000","last_became_active":"2024-09-13T16:28:39.199951+0000","last_became_peered":"2024-09-13T16:28:39.199951+0000","last_unstale":"2024-09-13T16:28:44.044668+0000","last_undegraded":"2024-09-13T16:28:44.044668+0000","last_fullsized":"2024-09-13T16:28:44.044668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:13:31.199857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.630000+0000","last_change":"2024-09-13T16:28:39.213591+0000","last_active":"2024-09-13T16:29:04.630000+0000","last_peered":"2024-09-13T16:29:04.630000+0000","last_clean":"2024-09-13T16:29:04.630000+0000","last_became_active":"2024-09-13T16:28:39.213324+0000","last_became_peered":"2024-09-13T16:28:39.213324+0000","last_unstale":"2024-09-13T16:29:04.630000+0000","last_undegraded":"2024-09-13T16:29:04.630000+0000","last_fullsized":"2024-09-13T16:29:04.630000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:54:03.172919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.633921+0000","last_change":"2024-09-13T16:28:39.217442+0000","last_active":"2024-09-13T16:29:04.633921+0000","last_peered":"2024-09-13T16:29:04.633921+0000","last_clean":"2024-09-13T16:29:04.633921+0000","last_became_active":"2024-09-13T16:28:39.217151+0000","last_became_peered":"2024-09-13T16:28:39.217151+0000","last_unstale":"2024-09-13T16:29:04.633921+0000","last_undegraded":"2024-09-13T16:29:04.633921+0000","last_fullsized":"2024-09-13T16:29:04.633921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:17.132278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.045476+0000","last_change":"2024-09-13T16:28:39.203938+0000","last_active":"2024-09-13T16:28:44.045476+0000","last_peered":"2024-09-13T16:28:44.045476+0000","last_clean":"2024-09-13T16:28:44.045476+0000","last_became_active":"2024-09-13T16:28:39.203655+0000","last_became_peered":"2024-09-13T16:28:39.203655+0000","last_unstale":"2024-09-13T16:28:44.045476+0000","last_undegraded":"2024-09-13T16:28:44.045476+0000","last_fullsized":"2024-09-13T16:28:44.045476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:03:25.415256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.624165+0000","last_change":"2024-09-13T16:28:39.209592+0000","last_active":"2024-09-13T16:29:04.624165+0000","last_peered":"2024-09-13T16:29:04.624165+0000","last_clean":"2024-09-13T16:29:04.624165+0000","last_became_active":"2024-09-13T16:28:39.209299+0000","last_became_peered":"2024-09-13T16:28:39.209299+0000","last_unstale":"2024-09-13T16:29:04.624165+0000","last_undegraded":"2024-09-13T16:29:04.624165+0000","last_fullsized":"2024-09-13T16:29:04.624165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:04.053405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044501+0000","last_change":"2024-09-13T16:28:39.208437+0000","last_active":"2024-09-13T16:28:44.044501+0000","last_peered":"2024-09-13T16:28:44.044501+0000","last_clean":"2024-09-13T16:28:44.044501+0000","last_became_active":"2024-09-13T16:28:39.208136+0000","last_became_peered":"2024-09-13T16:28:39.208136+0000","last_unstale":"2024-09-13T16:28:44.044501+0000","last_undegraded":"2024-09-13T16:28:44.044501+0000","last_fullsized":"2024-09-13T16:28:44.044501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:21.304037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.045412+0000","last_change":"2024-09-13T16:28:39.203970+0000","last_active":"2024-09-13T16:28:44.045412+0000","last_peered":"2024-09-13T16:28:44.045412+0000","last_clean":"2024-09-13T16:28:44.045412+0000","last_became_active":"2024-09-13T16:28:39.203559+0000","last_became_peered":"2024-09-13T16:28:39.203559+0000","last_unstale":"2024-09-13T16:28:44.045412+0000","last_undegraded":"2024-09-13T16:28:44.045412+0000","last_fullsized":"2024-09-13T16:28:44.045412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:26:25.158393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044158+0000","last_change":"2024-09-13T16:28:39.205584+0000","last_active":"2024-09-13T16:28:44.044158+0000","last_peered":"2024-09-13T16:28:44.044158+0000","last_clean":"2024-09-13T16:28:44.044158+0000","last_became_active":"2024-09-13T16:28:39.205281+0000","last_became_peered":"2024-09-13T16:28:39.205281+0000","last_unstale":"2024-09-13T16:28:44.044158+0000","last_undegraded":"2024-09-13T16:28:44.044158+0000","last_fullsized":"2024-09-13T16:28:44.044158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:58:10.109647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629903+0000","last_change":"2024-09-13T16:28:39.214479+0000","last_active":"2024-09-13T16:29:04.629903+0000","last_peered":"2024-09-13T16:29:04.629903+0000","last_clean":"2024-09-13T16:29:04.629903+0000","last_became_active":"2024-09-13T16:28:39.213984+0000","last_became_peered":"2024-09-13T16:28:39.213984+0000","last_unstale":"2024-09-13T16:29:04.629903+0000","last_undegraded":"2024-09-13T16:29:04.629903+0000","last_fullsized":"2024-09-13T16:29:04.629903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:53:43.137726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.634317+0000","last_change":"2024-09-13T16:28:39.209595+0000","last_active":"2024-09-13T16:29:04.634317+0000","last_peered":"2024-09-13T16:29:04.634317+0000","last_clean":"2024-09-13T16:29:04.634317+0000","last_became_active":"2024-09-13T16:28:39.209282+0000","last_became_peered":"2024-09-13T16:28:39.209282+0000","last_unstale":"2024-09-13T16:29:04.634317+0000","last_undegraded":"2024-09-13T16:29:04.634317+0000","last_fullsized":"2024-09-13T16:29:04.634317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:26:41.452462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628010+0000","last_change":"2024-09-13T16:28:39.210610+0000","last_active":"2024-09-13T16:29:04.628010+0000","last_peered":"2024-09-13T16:29:04.628010+0000","last_clean":"2024-09-13T16:29:04.628010+0000","last_became_active":"2024-09-13T16:28:39.210296+0000","last_became_peered":"2024-09-13T16:28:39.210296+0000","last_unstale":"2024-09-13T16:29:04.628010+0000","last_undegraded":"2024-09-13T16:29:04.628010+0000","last_fullsized":"2024-09-13T16:29:04.628010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:06.385733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629594+0000","last_change":"2024-09-13T16:28:39.207609+0000","last_active":"2024-09-13T16:29:04.629594+0000","last_peered":"2024-09-13T16:29:04.629594+0000","last_clean":"2024-09-13T16:29:04.629594+0000","last_became_active":"2024-09-13T16:28:39.206893+0000","last_became_peered":"2024-09-13T16:28:39.206893+0000","last_unstale":"2024-09-13T16:29:04.629594+0000","last_undegraded":"2024-09-13T16:29:04.629594+0000","last_fullsized":"2024-09-13T16:29:04.629594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:51.582666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.045110+0000","last_change":"2024-09-13T16:28:39.201856+0000","last_active":"2024-09-13T16:28:44.045110+0000","last_peered":"2024-09-13T16:28:44.045110+0000","last_clean":"2024-09-13T16:28:44.045110+0000","last_became_active":"2024-09-13T16:28:39.201511+0000","last_became_peered":"2024-09-13T16:28:39.201511+0000","last_unstale":"2024-09-13T16:28:44.045110+0000","last_undegraded":"2024-09-13T16:28:44.045110+0000","last_fullsized":"2024-09-13T16:28:44.045110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:05:19.722354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T16:28:43.215322+0000","last_change":"2024-09-13T16:28:39.216106+0000","last_active":"2024-09-13T16:28:43.215322+0000","last_peered":"2024-09-13T16:28:43.215322+0000","last_clean":"2024-09-13T16:28:43.215322+0000","last_became_active":"2024-09-13T16:28:39.215432+0000","last_became_peered":"2024-09-13T16:28:39.215432+0000","last_unstale":"2024-09-13T16:28:43.215322+0000","last_undegraded":"2024-09-13T16:28:43.215322+0000","last_fullsized":"2024-09-13T16:28:43.215322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:21:18.439891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T16:28:43.215353+0000","last_change":"2024-09-13T16:28:39.210578+0000","last_active":"2024-09-13T16:28:43.215353+0000","last_peered":"2024-09-13T16:28:43.215353+0000","last_clean":"2024-09-13T16:28:43.215353+0000","last_became_active":"2024-09-13T16:28:39.210269+0000","last_became_peered":"2024-09-13T16:28:39.210269+0000","last_unstale":"2024-09-13T16:28:43.215353+0000","last_undegraded":"2024-09-13T16:28:43.215353+0000","last_fullsized":"2024-09-13T16:28:43.215353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:27:27.078880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044081+0000","last_change":"2024-09-13T16:28:39.213409+0000","last_active":"2024-09-13T16:28:44.044081+0000","last_peered":"2024-09-13T16:28:44.044081+0000","last_clean":"2024-09-13T16:28:44.044081+0000","last_became_active":"2024-09-13T16:28:39.213077+0000","last_became_peered":"2024-09-13T16:28:39.213077+0000","last_unstale":"2024-09-13T16:28:44.044081+0000","last_undegraded":"2024-09-13T16:28:44.044081+0000","last_fullsized":"2024-09-13T16:28:44.044081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:32:24.747516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.632419+0000","last_change":"2024-09-13T16:28:39.215216+0000","last_active":"2024-09-13T16:29:04.632419+0000","last_peered":"2024-09-13T16:29:04.632419+0000","last_clean":"2024-09-13T16:29:04.632419+0000","last_became_active":"2024-09-13T16:28:39.214920+0000","last_became_peered":"2024-09-13T16:28:39.214920+0000","last_unstale":"2024-09-13T16:29:04.632419+0000","last_undegraded":"2024-09-13T16:29:04.632419+0000","last_fullsized":"2024-09-13T16:29:04.632419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:25:38.132833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625826+0000","last_change":"2024-09-13T16:28:39.202506+0000","last_active":"2024-09-13T16:29:04.625826+0000","last_peered":"2024-09-13T16:29:04.625826+0000","last_clean":"2024-09-13T16:29:04.625826+0000","last_became_active":"2024-09-13T16:28:39.202224+0000","last_became_peered":"2024-09-13T16:28:39.202224+0000","last_unstale":"2024-09-13T16:29:04.625826+0000","last_undegraded":"2024-09-13T16:29:04.625826+0000","last_fullsized":"2024-09-13T16:29:04.625826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:00:37.208442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.640477+0000","last_change":"2024-09-13T16:29:05.640477+0000","last_active":"2024-09-13T16:29:05.640477+0000","last_peered":"2024-09-13T16:29:05.640477+0000","last_clean":"2024-09-13T16:29:05.640477+0000","last_became_active":"2024-09-13T16:29:05.640014+0000","last_became_peered":"2024-09-13T16:29:05.640014+0000","last_unstale":"2024-09-13T16:29:05.640477+0000","last_undegraded":"2024-09-13T16:29:05.640477+0000","last_fullsized":"2024-09-13T16:29:05.640477+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:47:45.202113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.045134+0000","last_change":"2024-09-13T16:28:39.200217+0000","last_active":"2024-09-13T16:28:44.045134+0000","last_peered":"2024-09-13T16:28:44.045134+0000","last_clean":"2024-09-13T16:28:44.045134+0000","last_became_active":"2024-09-13T16:28:39.199946+0000","last_became_peered":"2024-09-13T16:28:39.199946+0000","last_unstale":"2024-09-13T16:28:44.045134+0000","last_undegraded":"2024-09-13T16:28:44.045134+0000","last_fullsized":"2024-09-13T16:28:44.045134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:20:02.906225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044376+0000","last_change":"2024-09-13T16:28:39.215013+0000","last_active":"2024-09-13T16:28:44.044376+0000","last_peered":"2024-09-13T16:28:44.044376+0000","last_clean":"2024-09-13T16:28:44.044376+0000","last_became_active":"2024-09-13T16:28:39.214731+0000","last_became_peered":"2024-09-13T16:28:39.214731+0000","last_unstale":"2024-09-13T16:28:44.044376+0000","last_undegraded":"2024-09-13T16:28:44.044376+0000","last_fullsized":"2024-09-13T16:28:44.044376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:35:01.836001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625268+0000","last_change":"2024-09-13T16:28:39.200300+0000","last_active":"2024-09-13T16:29:04.625268+0000","last_peered":"2024-09-13T16:29:04.625268+0000","last_clean":"2024-09-13T16:29:04.625268+0000","last_became_active":"2024-09-13T16:28:39.200006+0000","last_became_peered":"2024-09-13T16:28:39.200006+0000","last_unstale":"2024-09-13T16:29:04.625268+0000","last_undegraded":"2024-09-13T16:29:04.625268+0000","last_fullsized":"2024-09-13T16:29:04.625268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:35:14.783905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629430+0000","last_change":"2024-09-13T16:28:39.213552+0000","last_active":"2024-09-13T16:29:04.629430+0000","last_peered":"2024-09-13T16:29:04.629430+0000","last_clean":"2024-09-13T16:29:04.629430+0000","last_became_active":"2024-09-13T16:28:39.213287+0000","last_became_peered":"2024-09-13T16:28:39.213287+0000","last_unstale":"2024-09-13T16:29:04.629430+0000","last_undegraded":"2024-09-13T16:29:04.629430+0000","last_fullsized":"2024-09-13T16:29:04.629430+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:30.588770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627025+0000","last_change":"2024-09-13T16:28:39.211126+0000","last_active":"2024-09-13T16:29:04.627025+0000","last_peered":"2024-09-13T16:29:04.627025+0000","last_clean":"2024-09-13T16:29:04.627025+0000","last_became_active":"2024-09-13T16:28:39.210810+0000","last_became_peered":"2024-09-13T16:28:39.210810+0000","last_unstale":"2024-09-13T16:29:04.627025+0000","last_undegraded":"2024-09-13T16:29:04.627025+0000","last_fullsized":"2024-09-13T16:29:04.627025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:52:45.249309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.639070+0000","last_change":"2024-09-13T16:29:05.636535+0000","last_active":"2024-09-13T16:29:05.639070+0000","last_peered":"2024-09-13T16:29:05.639070+0000","last_clean":"2024-09-13T16:29:05.639070+0000","last_became_active":"2024-09-13T16:29:05.636161+0000","last_became_peered":"2024-09-13T16:29:05.636161+0000","last_unstale":"2024-09-13T16:29:05.639070+0000","last_undegraded":"2024-09-13T16:29:05.639070+0000","last_fullsized":"2024-09-13T16:29:05.639070+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:09.606922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T16:28:43.215475+0000","last_change":"2024-09-13T16:28:39.202222+0000","last_active":"2024-09-13T16:28:43.215475+0000","last_peered":"2024-09-13T16:28:43.215475+0000","last_clean":"2024-09-13T16:28:43.215475+0000","last_became_active":"2024-09-13T16:28:39.201897+0000","last_became_peered":"2024-09-13T16:28:39.201897+0000","last_unstale":"2024-09-13T16:28:43.215475+0000","last_undegraded":"2024-09-13T16:28:43.215475+0000","last_fullsized":"2024-09-13T16:28:43.215475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:41:09.087185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T16:28:43.217236+0000","last_change":"2024-09-13T16:28:39.211164+0000","last_active":"2024-09-13T16:28:43.217236+0000","last_peered":"2024-09-13T16:28:43.217236+0000","last_clean":"2024-09-13T16:28:43.217236+0000","last_became_active":"2024-09-13T16:28:39.210826+0000","last_became_peered":"2024-09-13T16:28:39.210826+0000","last_unstale":"2024-09-13T16:28:43.217236+0000","last_undegraded":"2024-09-13T16:28:43.217236+0000","last_fullsized":"2024-09-13T16:28:43.217236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:40:30.663867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629508+0000","last_change":"2024-09-13T16:28:39.207991+0000","last_active":"2024-09-13T16:29:04.629508+0000","last_peered":"2024-09-13T16:29:04.629508+0000","last_clean":"2024-09-13T16:29:04.629508+0000","last_became_active":"2024-09-13T16:28:39.207332+0000","last_became_peered":"2024-09-13T16:28:39.207332+0000","last_unstale":"2024-09-13T16:29:04.629508+0000","last_undegraded":"2024-09-13T16:29:04.629508+0000","last_fullsized":"2024-09-13T16:29:04.629508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:30:54.271361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":16384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":17802,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":0,"num_read":57,"num_read_kb":46,"num_write":213,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":762,"num_keys_recovered":10,"num_objects_omap":64,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":57746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":187,"ondisk_log_size":187,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250749,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383696,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":487217,"data_compressed":14739,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.32600000000000001}]},{"osd":1,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.376},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.30399999999999999}]},{"osd":2,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.439},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.627}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]},{"osd":4,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.47699999999999998}]},{"osd":5,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.59099999999999997}]},{"osd":6,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.56599999999999995}]},{"osd":7,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.54000000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":9,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.73299999999999998}]},{"osd":10,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.70999999999999996}]}]},{"osd":10,"up_from":72,"seq":309237645334,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1089536,"data_stored":1091645,"data_compressed":54008,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":1,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.50800000000000001}]},{"osd":2,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.626},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.68799999999999994}]},{"osd":3,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":7.2320000000000002,"5min":7.2320000000000002,"15min":7.2320000000000002},"last":0.61099999999999999}]},{"osd":4,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.57699999999999996}]},{"osd":5,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.75},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.64100000000000001}]},{"osd":6,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.70199999999999996}]},{"osd":7,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.53000000000000003}]},{"osd":8,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":7.2569999999999997,"5min":7.2569999999999997,"15min":7.2569999999999997},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.70999999999999996}]},{"osd":9,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.67300000000000004}]},{"osd":11,"last update":"Fri Sep 13 16:28:58 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":7.2199999999999998,"5min":7.2199999999999998,"15min":7.2199999999999998},"last":0.79400000000000004}]}]},{"osd":9,"up_from":67,"seq":287762808858,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":496463,"data_compressed":17719,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.35399999999999998}]},{"osd":1,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.52900000000000003}]},{"osd":3,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.46500000000000002}]},{"osd":4,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.628}]},{"osd":5,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.81100000000000005}]},{"osd":6,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.45700000000000002}]},{"osd":7,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.76500000000000001}]},{"osd":8,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.745}]},{"osd":10,"last update":"Fri Sep 13 16:28:33 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.85299999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":8,"up_from":59,"seq":253403070495,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":1007616,"data_stored":495587,"data_compressed":17719,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.50600000000000001,"15min":0.50800000000000001},"min":{"1min":0.28100000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.70999999999999996,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51700000000000002,"15min":0.51200000000000001},"min":{"1min":0.32800000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.48599999999999999}]},{"osd":1,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.41299999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.40400000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82399999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.60799999999999998}]},{"osd":2,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.41699999999999998,"15min":0.41199999999999998},"min":{"1min":0.28299999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.435}]},{"osd":3,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.78700000000000003,"5min":0.871,"15min":0.871},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.58199999999999996}]},{"osd":4,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56799999999999995,"15min":0.56399999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.747}]},{"osd":5,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53800000000000003,"15min":0.54300000000000004},"min":{"1min":0.29699999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.622}]},{"osd":6,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.628,"15min":0.63},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.65900000000000003,"15min":0.66300000000000003},"min":{"1min":0.38300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68500000000000005}]},{"osd":7,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.66600000000000004,"15min":0.67000000000000004},"min":{"1min":0.47899999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.97699999999999998,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.65800000000000003,"15min":0.66200000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.92200000000000004,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.69699999999999995}]},{"osd":9,"last update":"Fri Sep 13 16:28:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.79500000000000004}]},{"osd":10,"last update":"Fri Sep 13 16:28:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.77600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]}]},{"osd":7,"up_from":51,"seq":219043332132,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":1093979,"data_compressed":54008,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.39600000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.41199999999999998}]},{"osd":1,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.39200000000000002,"15min":0.38900000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.312},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.68600000000000005,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.68999999999999995}]},{"osd":2,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.54400000000000004,"15min":0.54900000000000004},"min":{"1min":0.40699999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.66700000000000004,"5min":0.748,"15min":0.748},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54700000000000004,"15min":0.54800000000000004},"min":{"1min":0.40000000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.60299999999999998}]},{"osd":3,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.42499999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.441},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56799999999999995,"15min":0.56399999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.57499999999999996}]},{"osd":4,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45200000000000001,"15min":0.45100000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.45800000000000002,"15min":0.46300000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.32700000000000001}]},{"osd":5,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57699999999999996,"15min":0.57799999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.496},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.44600000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.54200000000000004}]},{"osd":6,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60899999999999999,"15min":0.60999999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.41699999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.373}]},{"osd":8,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.627,"15min":0.627},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.624},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.66100000000000003,"15min":0.66500000000000004},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.65800000000000003}]},{"osd":9,"last update":"Fri Sep 13 16:28:08 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.64200000000000002}]},{"osd":10,"last update":"Fri Sep 13 16:28:33 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.67600000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]}]},{"osd":6,"up_from":46,"seq":197568495657,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":991232,"data_stored":486747,"data_compressed":14739,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.38300000000000001,"15min":0.372},"min":{"1min":0.26900000000000002,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.39700000000000002,"15min":0.39000000000000001},"min":{"1min":0.28000000000000003,"5min":0.217,"15min":0.217},"max":{"1min":0.61599999999999999,"5min":0.625,"15min":0.625},"last":0.33700000000000002}]},{"osd":1,"last update":"Fri Sep 13 16:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53400000000000003,"15min":0.51200000000000001},"min":{"1min":0.373,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.753,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56100000000000005,"15min":0.54500000000000004},"min":{"1min":0.39700000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.77700000000000002,"5min":0.873,"15min":0.873},"last":0.53300000000000003}]},{"osd":2,"last update":"Fri Sep 13 16:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55300000000000005,"15min":0.54900000000000004},"min":{"1min":0.35199999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83499999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59699999999999998,"15min":0.58799999999999997},"min":{"1min":0.36899999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81399999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.623}]},{"osd":3,"last update":"Fri Sep 13 16:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.73699999999999999,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.499,"15min":0.48899999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.44400000000000001}]},{"osd":4,"last update":"Fri Sep 13 16:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.30299999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.73599999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.42299999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.92300000000000004,"5min":1.079,"15min":1.079},"last":0.49099999999999999}]},{"osd":5,"last update":"Fri Sep 13 16:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61799999999999999,"15min":0.60599999999999998},"min":{"1min":0.47199999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96599999999999997,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.45800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94999999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.54400000000000004}]},{"osd":7,"last update":"Fri Sep 13 16:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63800000000000001,"15min":0.63200000000000001},"min":{"1min":0.40600000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.02,"5min":1.048,"15min":1.048},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.66300000000000003,"15min":0.65200000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.60299999999999998}]},{"osd":8,"last update":"Fri Sep 13 16:28:43 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.69299999999999995,"15min":0.68899999999999995},"min":{"1min":0.505,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72699999999999998,"15min":0.72799999999999998},"min":{"1min":0.46600000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.64000000000000001}]},{"osd":9,"last update":"Fri Sep 13 16:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.66600000000000004}]},{"osd":10,"last update":"Fri Sep 13 16:28:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.70499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]}]},{"osd":5,"up_from":38,"seq":163208757294,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":1088542,"data_compressed":54008,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.52200000000000002,"15min":0.51600000000000001},"min":{"1min":0.377,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76700000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54000000000000004,"15min":0.53500000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.80700000000000005,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.46500000000000002}]},{"osd":1,"last update":"Fri Sep 13 16:28:40 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.48699999999999999,"15min":0.47699999999999998},"min":{"1min":0.308,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.51900000000000002,"15min":0.505},"min":{"1min":0.34399999999999997,"5min":0.245,"15min":0.245},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.41299999999999998}]},{"osd":2,"last update":"Fri Sep 13 16:28:40 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.375,"15min":0.35999999999999999},"min":{"1min":0.254,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.65000000000000002,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.379,"15min":0.36699999999999999},"min":{"1min":0.23100000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.36899999999999999}]},{"osd":3,"last update":"Fri Sep 13 16:28:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.80000000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.55000000000000004}]},{"osd":4,"last update":"Fri Sep 13 16:28:40 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52400000000000002,"15min":0.51100000000000001},"min":{"1min":0.41799999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53400000000000003,"15min":0.52800000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.54000000000000004}]},{"osd":6,"last update":"Fri Sep 13 16:29:03 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.45800000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.434,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.78900000000000003,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.49399999999999999}]},{"osd":7,"last update":"Fri Sep 13 16:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.60799999999999998,"15min":0.60499999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59999999999999998,"15min":0.59699999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.626}]},{"osd":8,"last update":"Fri Sep 13 16:28:46 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52800000000000002,"15min":0.53300000000000003},"min":{"1min":0.34499999999999997,"5min":0.318,"15min":0.318},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.56200000000000006,"15min":0.56899999999999995},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.56200000000000006}]},{"osd":9,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.65700000000000003}]},{"osd":10,"last update":"Fri Sep 13 16:28:31 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.68500000000000005}]}]},{"osd":4,"up_from":33,"seq":141733920819,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":496497,"data_compressed":17719,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.48699999999999999,"15min":0.46300000000000002},"min":{"1min":0.32200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.751,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.504,"15min":0.48199999999999998},"min":{"1min":0.41399999999999998,"5min":0.25,"15min":0.25},"max":{"1min":0.80800000000000005,"5min":0.874,"15min":0.874},"last":0.60499999999999998}]},{"osd":1,"last update":"Fri Sep 13 16:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.34699999999999998,"15min":0.33100000000000002},"min":{"1min":0.24099999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.34300000000000003,"15min":0.33200000000000002},"min":{"1min":0.27800000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.372}]},{"osd":2,"last update":"Fri Sep 13 16:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.495,"15min":0.48599999999999999},"min":{"1min":0.37,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71599999999999997,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.47399999999999998,"15min":0.45500000000000002},"min":{"1min":0.34399999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67200000000000004,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.59299999999999997}]},{"osd":3,"last update":"Fri Sep 13 16:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.504,"15min":0.48599999999999999},"min":{"1min":0.38700000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.505},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.52100000000000002,"15min":0.496},"min":{"1min":0.39900000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.79300000000000004,"5min":0.875,"15min":0.875},"last":0.56799999999999995}]},{"osd":5,"last update":"Fri Sep 13 16:28:38 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.33800000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54500000000000004,"15min":0.53100000000000003},"min":{"1min":0.39000000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80700000000000005,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Sep 13 16:28:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.54900000000000004,"15min":0.53500000000000003},"min":{"1min":0.47599999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.73299999999999998,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57699999999999996,"15min":0.56599999999999995},"min":{"1min":0.41699999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80400000000000005,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.63}]},{"osd":7,"last update":"Fri Sep 13 16:28:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.495,"15min":0.48099999999999998},"min":{"1min":0.33600000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.85099999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.49399999999999999,"15min":0.48999999999999999},"min":{"1min":0.34999999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.81799999999999995,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.621}]},{"osd":8,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.44,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86899999999999999,"5min":0.872,"15min":0.872},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.61499999999999999,"15min":0.60999999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.872,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.64500000000000002}]},{"osd":9,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.72599999999999998}]},{"osd":10,"last update":"Fri Sep 13 16:28:32 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.65400000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36188,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708964,"statfs":{"total":95995035648,"available":95957979136,"internally_reserved":0,"allocated":999424,"data_stored":486172,"data_compressed":14739,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:56 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.34899999999999998,"15min":0.32500000000000001},"min":{"1min":0.252,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.63,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.35699999999999998,"15min":0.32900000000000001},"min":{"1min":0.27600000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52300000000000002,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.36599999999999999}]},{"osd":1,"last update":"Fri Sep 13 16:28:56 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.54600000000000004,"15min":0.48199999999999998},"min":{"1min":0.48899999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.84299999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53500000000000003,"15min":0.48699999999999999},"min":{"1min":0.42399999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.77000000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.68799999999999994}]},{"osd":2,"last update":"Fri Sep 13 16:28:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52200000000000002,"15min":0.51400000000000001},"min":{"1min":0.375,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79100000000000004,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55300000000000005,"15min":0.52600000000000002},"min":{"1min":0.38600000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.94599999999999995,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.53600000000000003}]},{"osd":4,"last update":"Fri Sep 13 16:28:09 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.51500000000000001,"15min":0.49099999999999999},"min":{"1min":0.40400000000000003,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.56699999999999995,"15min":0.54000000000000004},"min":{"1min":0.44700000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.62}]},{"osd":5,"last update":"Fri Sep 13 16:28:36 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58299999999999996,"15min":0.57199999999999995},"min":{"1min":0.433,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.86299999999999999,"5min":1.038,"15min":1.038},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59699999999999998,"15min":0.59599999999999997},"min":{"1min":0.45900000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.73799999999999999,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.72099999999999997}]},{"osd":6,"last update":"Fri Sep 13 16:29:01 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.505,"15min":0.496},"min":{"1min":0.33600000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.48599999999999999,"15min":0.47999999999999998},"min":{"1min":0.34599999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74099999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.498}]},{"osd":7,"last update":"Fri Sep 13 16:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.49399999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.60699999999999998,"15min":0.59299999999999997},"min":{"1min":0.52100000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.64100000000000001}]},{"osd":8,"last update":"Fri Sep 13 16:27:45 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.71499999999999997}]},{"osd":9,"last update":"Fri Sep 13 16:28:08 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.56499999999999995}]},{"osd":10,"last update":"Fri Sep 13 16:28:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.70299999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":1032192,"data_stored":520251,"data_compressed":17719,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.60399999999999998,"15min":0.498},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.90900000000000003,"5min":16.027999999999999,"15min":16.027999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.47299999999999998,"15min":0.46500000000000002},"min":{"1min":0.309,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.69599999999999995}]},{"osd":1,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.48399999999999999,"15min":0.46500000000000002},"min":{"1min":0.372,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.51500000000000001,"15min":0.49299999999999999},"min":{"1min":0.44500000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.83399999999999996}]},{"osd":3,"last update":"Fri Sep 13 16:28:52 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.54000000000000004,"15min":0.51400000000000001},"min":{"1min":0.43099999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54000000000000004,"15min":0.51500000000000001},"min":{"1min":0.36799999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.59099999999999997}]},{"osd":4,"last update":"Fri Sep 13 16:28:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":0.623,"15min":0.55500000000000005},"min":{"1min":0.36399999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":9.6790000000000003,"5min":9.6790000000000003,"15min":9.6790000000000003},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.54500000000000004,"15min":0.53600000000000003},"min":{"1min":0.34399999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.75900000000000001}]},{"osd":5,"last update":"Fri Sep 13 16:28:39 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.40999999999999998,"15min":0.40200000000000002},"min":{"1min":0.33700000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.432},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.40300000000000002,"15min":0.39200000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.46100000000000002}]},{"osd":6,"last update":"Fri Sep 13 16:29:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.57699999999999996,"15min":0.55200000000000005},"min":{"1min":0.437,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.60599999999999998,"15min":0.58399999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.79400000000000004}]},{"osd":7,"last update":"Fri Sep 13 16:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.379,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59099999999999997,"15min":0.58299999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.748}]},{"osd":8,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.52600000000000002,"15min":0.51800000000000002},"min":{"1min":0.33500000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.501},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.51700000000000002,"15min":0.50800000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.55100000000000005}]},{"osd":9,"last update":"Fri Sep 13 16:29:06 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.67900000000000005,"15min":0.66900000000000004},"min":{"1min":0.45100000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.016,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.68100000000000005,"15min":0.67100000000000004},"min":{"1min":0.50600000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.85299999999999998}]},{"osd":10,"last update":"Fri Sep 13 16:28:34 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.89400000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":491382,"data_compressed":14739,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:09 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.47999999999999998,"15min":0.47799999999999998},"min":{"1min":0.34100000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.497},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.48399999999999999,"15min":0.48099999999999998},"min":{"1min":0.38100000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.435}]},{"osd":2,"last update":"Fri Sep 13 16:28:27 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.67100000000000004,"15min":0.54100000000000004},"min":{"1min":0.34100000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.81200000000000006,"5min":17.655000000000001,"15min":17.655000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.49099999999999999,"15min":0.47899999999999998},"min":{"1min":0.29299999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.76200000000000001}]},{"osd":3,"last update":"Fri Sep 13 16:28:51 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52500000000000002,"15min":0.50700000000000001},"min":{"1min":0.379,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72599999999999998,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.55800000000000005,"15min":0.52200000000000002},"min":{"1min":0.42199999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.73899999999999999}]},{"osd":4,"last update":"Fri Sep 13 16:28:13 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.40400000000000003,"15min":0.38},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.433,"5min":0.40899999999999997,"15min":0.39100000000000001},"min":{"1min":0.28299999999999997,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.57899999999999996}]},{"osd":5,"last update":"Fri Sep 13 16:28:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":1.0609999999999999,"15min":1.2829999999999999},"min":{"1min":0.38400000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.82499999999999996,"5min":17.640000000000001,"15min":17.640000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54600000000000004,"15min":0.54300000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.72499999999999998}]},{"osd":6,"last update":"Fri Sep 13 16:29:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59299999999999997,"15min":0.59399999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.72799999999999998,"5min":0.872,"15min":0.872},"last":0.60899999999999999}]},{"osd":7,"last update":"Fri Sep 13 16:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.372,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.78900000000000003}]},{"osd":8,"last update":"Fri Sep 13 16:28:43 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.60399999999999998,"15min":0.61099999999999999},"min":{"1min":0.40699999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.45500000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.83499999999999996}]},{"osd":9,"last update":"Fri Sep 13 16:28:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.751}]},{"osd":10,"last update":"Fri Sep 13 16:28:34 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.82199999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":477257,"data_compressed":11759,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 16:28:15 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.46999999999999997,"15min":0.46899999999999997},"min":{"1min":0.37,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.755,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.50700000000000001,"15min":0.495},"min":{"1min":0.41199999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81000000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.60899999999999999}]},{"osd":2,"last update":"Fri Sep 13 16:28:33 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.55000000000000004,"15min":0.52600000000000002},"min":{"1min":0.42699999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73799999999999999,"5min":1.111,"15min":1.111},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.54800000000000004,"15min":0.56100000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.621,"5min":1.698,"15min":1.698},"last":0.40000000000000002}]},{"osd":3,"last update":"Fri Sep 13 16:27:47 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.438,"15min":0.42399999999999999},"min":{"1min":0.38200000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.42299999999999999,"15min":0.39800000000000002},"min":{"1min":0.22500000000000001,"5min":0.217,"15min":0.217},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.46500000000000002}]},{"osd":4,"last update":"Fri Sep 13 16:28:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.48199999999999998,"15min":0.45400000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.625},{"interface":"front","average":{"1min":0.629,"5min":0.51500000000000001,"15min":0.48399999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.65400000000000003}]},{"osd":5,"last update":"Fri Sep 13 16:28:33 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.55300000000000005,"15min":0.54100000000000004},"min":{"1min":0.44,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56799999999999995,"15min":0.55600000000000005},"min":{"1min":0.46899999999999997,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77800000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.36299999999999999}]},{"osd":6,"last update":"Fri Sep 13 16:28:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.504,"15min":0.48199999999999998},"min":{"1min":0.29899999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.66500000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55300000000000005,"15min":0.52100000000000002},"min":{"1min":0.34399999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.74299999999999999,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.501}]},{"osd":7,"last update":"Fri Sep 13 16:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.68300000000000005,"15min":0.67500000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96999999999999997,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.63700000000000001,"15min":0.63100000000000001},"min":{"1min":0.43099999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.58499999999999996}]},{"osd":8,"last update":"Fri Sep 13 16:28:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1,"5min":1,"15min":1},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69899999999999995,"15min":0.70299999999999996},"min":{"1min":0.55000000000000004,"5min":0.497,"15min":0.497},"max":{"1min":0.81799999999999995,"5min":1.004,"15min":1.004},"last":0.54900000000000004}]},{"osd":9,"last update":"Fri Sep 13 16:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.53500000000000003}]},{"osd":10,"last update":"Fri Sep 13 16:28:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.68899999999999995}]},{"osd":11,"last update":"Fri Sep 13 16:28:58 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.66900000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":25183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":12476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T16:29:08.997 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T16:29:09.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.130+0000 7fb316e19640 1 -- 172.21.15.16:0/2122111039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3100fecd0 msgr2=0x7fb3100ff0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:09.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.130+0000 7fb316e19640 1 --2- 172.21.15.16:0/2122111039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3100fecd0 0x7fb3100ff0d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb300009ca0 tx=0x7fb30002a8c0 comp rx=0 tx=0).stop 2024-09-13T16:29:09.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.130+0000 7fb316e19640 1 -- 172.21.15.16:0/2122111039 shutdown_connections 2024-09-13T16:29:09.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.130+0000 7fb316e19640 1 --2- 172.21.15.16:0/2122111039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb310100880 0x7fb310107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:09.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.130+0000 7fb316e19640 1 --2- 172.21.15.16:0/2122111039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3100ffec0 0x7fb310100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:09.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.130+0000 7fb316e19640 1 --2- 172.21.15.16:0/2122111039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3100fecd0 0x7fb3100ff0d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:09.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.130+0000 7fb316e19640 1 -- 172.21.15.16:0/2122111039 >> 172.21.15.16:0/2122111039 conn(0x7fb3100fa430 msgr2=0x7fb3100fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:09.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.130+0000 7fb316e19640 1 -- 172.21.15.16:0/2122111039 shutdown_connections 2024-09-13T16:29:09.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.130+0000 7fb316e19640 1 -- 172.21.15.16:0/2122111039 wait complete. 2024-09-13T16:29:09.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.131+0000 7fb316e19640 1 Processor -- start 2024-09-13T16:29:09.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.131+0000 7fb316e19640 1 -- start start 2024-09-13T16:29:09.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.131+0000 7fb316e19640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3100fecd0 0x7fb310195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:09.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.131+0000 7fb316e19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3100ffec0 0x7fb3101962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:09.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.131+0000 7fb316e19640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb310100880 0x7fb31019d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:09.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.131+0000 7fb316e19640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31019d890 con 0x7fb3100ffec0 2024-09-13T16:29:09.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.131+0000 7fb316e19640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31019da00 con 0x7fb310100880 2024-09-13T16:29:09.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.131+0000 7fb316e19640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31019dbc0 con 0x7fb3100fecd0 2024-09-13T16:29:09.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.131+0000 7fb30ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3100ffec0 0x7fb3101962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:09.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.131+0000 7fb30ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3100ffec0 0x7fb3101962f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49720/0 (socket says 172.21.15.16:49720) 2024-09-13T16:29:09.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.131+0000 7fb30ffff640 1 -- 172.21.15.16:0/1672368387 learned_addr learned my addr 172.21.15.16:0/1672368387 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:09.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.131+0000 7fb3153b6640 1 --2- 172.21.15.16:0/1672368387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb310100880 0x7fb31019d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:09.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.131+0000 7fb314bb5640 1 --2- 172.21.15.16:0/1672368387 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3100fecd0 0x7fb310195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:09.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.132+0000 7fb30ffff640 1 -- 172.21.15.16:0/1672368387 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3100fecd0 msgr2=0x7fb310195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:09.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.132+0000 7fb30ffff640 1 --2- 172.21.15.16:0/1672368387 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3100fecd0 0x7fb310195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:09.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.132+0000 7fb30ffff640 1 -- 172.21.15.16:0/1672368387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb310100880 msgr2=0x7fb31019d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:09.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.132+0000 7fb30ffff640 1 --2- 172.21.15.16:0/1672368387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb310100880 0x7fb31019d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:09.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.132+0000 7fb30ffff640 1 -- 172.21.15.16:0/1672368387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb300009900 con 0x7fb3100ffec0 2024-09-13T16:29:09.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.132+0000 7fb30ffff640 1 --2- 172.21.15.16:0/1672368387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3100ffec0 0x7fb3101962f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb30400b4a0 tx=0x7fb30400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:09.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.132+0000 7fb30dffb640 1 -- 172.21.15.16:0/1672368387 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb304010610 con 0x7fb3100ffec0 2024-09-13T16:29:09.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.132+0000 7fb30dffb640 1 -- 172.21.15.16:0/1672368387 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb304010c00 con 0x7fb3100ffec0 2024-09-13T16:29:09.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.132+0000 7fb316e19640 1 -- 172.21.15.16:0/1672368387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb31019dea0 con 0x7fb3100ffec0 2024-09-13T16:29:09.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.132+0000 7fb30dffb640 1 -- 172.21.15.16:0/1672368387 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb304017ca0 con 0x7fb3100ffec0 2024-09-13T16:29:09.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.132+0000 7fb316e19640 1 -- 172.21.15.16:0/1672368387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb31019e2c0 con 0x7fb3100ffec0 2024-09-13T16:29:09.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.133+0000 7fb316e19640 1 -- 172.21.15.16:0/1672368387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3101062f0 con 0x7fb3100ffec0 2024-09-13T16:29:09.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.134+0000 7fb30dffb640 1 -- 172.21.15.16:0/1672368387 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb304010770 con 0x7fb3100ffec0 2024-09-13T16:29:09.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.135+0000 7fb30dffb640 1 --2- 172.21.15.16:0/1672368387 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb2f80a1090 0x7fb2f80a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:09.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.135+0000 7fb314bb5640 1 --2- 172.21.15.16:0/1672368387 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb2f80a1090 0x7fb2f80a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:09.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.135+0000 7fb30dffb640 1 -- 172.21.15.16:0/1672368387 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb3040cc6c0 con 0x7fb3100ffec0 2024-09-13T16:29:09.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.135+0000 7fb314bb5640 1 --2- 172.21.15.16:0/1672368387 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb2f80a1090 0x7fb2f80a3550 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb300006010 tx=0x7fb300000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:09.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.139+0000 7fb30dffb640 1 -- 172.21.15.16:0/1672368387 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb304095a50 con 0x7fb3100ffec0 2024-09-13T16:29:09.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.306+0000 7fb316e19640 1 -- 172.21.15.16:0/1672368387 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb31002c520 con 0x7fb2f80a1090 2024-09-13T16:29:09.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.317+0000 7fb30dffb640 1 -- 172.21.15.16:0/1672368387 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422976 (secure 0 0 0) 0x7fb31002c520 con 0x7fb2f80a1090 2024-09-13T16:29:09.323 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:29:09.326 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-13T16:29:09.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.321+0000 7fb316e19640 1 -- 172.21.15.16:0/1672368387 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb2f80a1090 msgr2=0x7fb2f80a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:09.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.321+0000 7fb316e19640 1 --2- 172.21.15.16:0/1672368387 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb2f80a1090 0x7fb2f80a3550 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb300006010 tx=0x7fb300000f40 comp rx=0 tx=0).stop 2024-09-13T16:29:09.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.321+0000 7fb316e19640 1 -- 172.21.15.16:0/1672368387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3100ffec0 msgr2=0x7fb3101962f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:09.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.321+0000 7fb316e19640 1 --2- 172.21.15.16:0/1672368387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3100ffec0 0x7fb3101962f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb30400b4a0 tx=0x7fb30400b960 comp rx=0 tx=0).stop 2024-09-13T16:29:09.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.321+0000 7fb316e19640 1 -- 172.21.15.16:0/1672368387 shutdown_connections 2024-09-13T16:29:09.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.321+0000 7fb316e19640 1 --2- 172.21.15.16:0/1672368387 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb2f80a1090 0x7fb2f80a3550 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:09.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.321+0000 7fb316e19640 1 --2- 172.21.15.16:0/1672368387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb310100880 0x7fb31019d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:09.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.321+0000 7fb316e19640 1 --2- 172.21.15.16:0/1672368387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3100ffec0 0x7fb3101962f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:09.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.321+0000 7fb316e19640 1 --2- 172.21.15.16:0/1672368387 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3100fecd0 0x7fb310195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:09.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.321+0000 7fb316e19640 1 -- 172.21.15.16:0/1672368387 >> 172.21.15.16:0/1672368387 conn(0x7fb3100fa430 msgr2=0x7fb3101034c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:09.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.321+0000 7fb316e19640 1 -- 172.21.15.16:0/1672368387 shutdown_connections 2024-09-13T16:29:09.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.321+0000 7fb316e19640 1 -- 172.21.15.16:0/1672368387 wait complete. 2024-09-13T16:29:09.333 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-13T16:29:08.571055+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":608404,"num_objects":100,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":183,"num_read_kb":155,"num_write":396,"num_write_kb":3018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1181498,"num_keys_recovered":10,"num_objects_omap":64,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":417,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364888,"kb_used_data":11980,"kb_used_omap":18,"kb_used_meta":352749,"kb_avail":1124576936,"statfs":{"total":1151940427776,"available":1151566782464,"internally_reserved":0,"allocated":12267520,"data_stored":7711739,"data_compressed":303615,"data_compressed_allocated":405504,"data_compressed_original":2310144,"omap_allocated":19131,"internal_metadata":361215301},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13443,"num_objects":76,"num_object_clones":0,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":57,"num_read_kb":46,"num_write":167,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":762,"num_keys_recovered":10,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002208"},"pg_stats":[{"pgid":"2.1f","version":"91'13","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.648333+0000","last_change":"2024-09-13T16:29:05.647872+0000","last_active":"2024-09-13T16:29:05.648333+0000","last_peered":"2024-09-13T16:29:05.648333+0000","last_clean":"2024-09-13T16:29:05.648333+0000","last_became_active":"2024-09-13T16:29:05.637050+0000","last_became_peered":"2024-09-13T16:29:05.637050+0000","last_unstale":"2024-09-13T16:29:05.648333+0000","last_undegraded":"2024-09-13T16:29:05.648333+0000","last_fullsized":"2024-09-13T16:29:05.648333+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:04.113840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.045098+0000","last_change":"2024-09-13T16:28:39.210024+0000","last_active":"2024-09-13T16:28:44.045098+0000","last_peered":"2024-09-13T16:28:44.045098+0000","last_clean":"2024-09-13T16:28:44.045098+0000","last_became_active":"2024-09-13T16:28:39.209751+0000","last_became_peered":"2024-09-13T16:28:39.209751+0000","last_unstale":"2024-09-13T16:28:44.045098+0000","last_undegraded":"2024-09-13T16:28:44.045098+0000","last_fullsized":"2024-09-13T16:28:44.045098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:34:24.357779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.632148+0000","last_change":"2024-09-13T16:28:43.230223+0000","last_active":"2024-09-13T16:29:04.632148+0000","last_peered":"2024-09-13T16:29:04.632148+0000","last_clean":"2024-09-13T16:29:04.632148+0000","last_became_active":"2024-09-13T16:28:43.228992+0000","last_became_peered":"2024-09-13T16:28:43.228992+0000","last_unstale":"2024-09-13T16:29:04.632148+0000","last_undegraded":"2024-09-13T16:29:04.632148+0000","last_fullsized":"2024-09-13T16:29:04.632148+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:45:37.252865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:29:03.591838+0000","last_change":"2024-09-13T16:28:38.850198+0000","last_active":"2024-09-13T16:29:03.591838+0000","last_peered":"2024-09-13T16:29:03.591838+0000","last_clean":"2024-09-13T16:29:03.591838+0000","last_became_active":"2024-09-13T16:28:38.849780+0000","last_became_peered":"2024-09-13T16:28:38.849780+0000","last_unstale":"2024-09-13T16:29:03.591838+0000","last_undegraded":"2024-09-13T16:29:03.591838+0000","last_fullsized":"2024-09-13T16:29:03.591838+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:09.653474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T16:28:43.218832+0000","last_change":"2024-09-13T16:28:39.203030+0000","last_active":"2024-09-13T16:28:43.218832+0000","last_peered":"2024-09-13T16:28:43.218832+0000","last_clean":"2024-09-13T16:28:43.218832+0000","last_became_active":"2024-09-13T16:28:39.202464+0000","last_became_peered":"2024-09-13T16:28:39.202464+0000","last_unstale":"2024-09-13T16:28:43.218832+0000","last_undegraded":"2024-09-13T16:28:43.218832+0000","last_fullsized":"2024-09-13T16:28:43.218832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:57.528836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.632329+0000","last_change":"2024-09-13T16:28:43.224358+0000","last_active":"2024-09-13T16:29:04.632329+0000","last_peered":"2024-09-13T16:29:04.632329+0000","last_clean":"2024-09-13T16:29:04.632329+0000","last_became_active":"2024-09-13T16:28:43.223955+0000","last_became_peered":"2024-09-13T16:28:43.223955+0000","last_unstale":"2024-09-13T16:29:04.632329+0000","last_undegraded":"2024-09-13T16:29:04.632329+0000","last_fullsized":"2024-09-13T16:29:04.632329+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:53:09.169704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'11","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.634720+0000","last_change":"2024-09-13T16:28:38.846051+0000","last_active":"2024-09-13T16:29:05.634720+0000","last_peered":"2024-09-13T16:29:05.634720+0000","last_clean":"2024-09-13T16:29:05.634720+0000","last_became_active":"2024-09-13T16:28:38.845762+0000","last_became_peered":"2024-09-13T16:28:38.845762+0000","last_unstale":"2024-09-13T16:29:05.634720+0000","last_undegraded":"2024-09-13T16:29:05.634720+0000","last_fullsized":"2024-09-13T16:29:05.634720+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:33:47.048582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044544+0000","last_change":"2024-09-13T16:28:39.210516+0000","last_active":"2024-09-13T16:28:44.044544+0000","last_peered":"2024-09-13T16:28:44.044544+0000","last_clean":"2024-09-13T16:28:44.044544+0000","last_became_active":"2024-09-13T16:28:39.210214+0000","last_became_peered":"2024-09-13T16:28:39.210214+0000","last_unstale":"2024-09-13T16:28:44.044544+0000","last_undegraded":"2024-09-13T16:28:44.044544+0000","last_fullsized":"2024-09-13T16:28:44.044544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:51:58.212685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.630396+0000","last_change":"2024-09-13T16:28:43.230530+0000","last_active":"2024-09-13T16:29:04.630396+0000","last_peered":"2024-09-13T16:29:04.630396+0000","last_clean":"2024-09-13T16:29:04.630396+0000","last_became_active":"2024-09-13T16:28:43.230154+0000","last_became_peered":"2024-09-13T16:28:43.230154+0000","last_unstale":"2024-09-13T16:29:04.630396+0000","last_undegraded":"2024-09-13T16:29:04.630396+0000","last_fullsized":"2024-09-13T16:29:04.630396+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:39:52.362177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.653834+0000","last_change":"2024-09-13T16:29:05.653460+0000","last_active":"2024-09-13T16:29:05.653834+0000","last_peered":"2024-09-13T16:29:05.653834+0000","last_clean":"2024-09-13T16:29:05.653834+0000","last_became_active":"2024-09-13T16:29:05.641836+0000","last_became_peered":"2024-09-13T16:29:05.641836+0000","last_unstale":"2024-09-13T16:29:05.653834+0000","last_undegraded":"2024-09-13T16:29:05.653834+0000","last_fullsized":"2024-09-13T16:29:05.653834+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:22:30.051033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.624846+0000","last_change":"2024-09-13T16:28:39.202381+0000","last_active":"2024-09-13T16:29:04.624846+0000","last_peered":"2024-09-13T16:29:04.624846+0000","last_clean":"2024-09-13T16:29:04.624846+0000","last_became_active":"2024-09-13T16:28:39.202023+0000","last_became_peered":"2024-09-13T16:28:39.202023+0000","last_unstale":"2024-09-13T16:29:04.624846+0000","last_undegraded":"2024-09-13T16:29:04.624846+0000","last_fullsized":"2024-09-13T16:29:04.624846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:55:46.445085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628398+0000","last_change":"2024-09-13T16:28:43.230623+0000","last_active":"2024-09-13T16:29:04.628398+0000","last_peered":"2024-09-13T16:29:04.628398+0000","last_clean":"2024-09-13T16:29:04.628398+0000","last_became_active":"2024-09-13T16:28:43.230240+0000","last_became_peered":"2024-09-13T16:28:43.230240+0000","last_unstale":"2024-09-13T16:29:04.628398+0000","last_undegraded":"2024-09-13T16:29:04.628398+0000","last_fullsized":"2024-09-13T16:29:04.628398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:04:07.723269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625334+0000","last_change":"2024-09-13T16:28:39.201896+0000","last_active":"2024-09-13T16:29:04.625334+0000","last_peered":"2024-09-13T16:29:04.625334+0000","last_clean":"2024-09-13T16:29:04.625334+0000","last_became_active":"2024-09-13T16:28:39.201570+0000","last_became_peered":"2024-09-13T16:28:39.201570+0000","last_unstale":"2024-09-13T16:29:04.625334+0000","last_undegraded":"2024-09-13T16:29:04.625334+0000","last_fullsized":"2024-09-13T16:29:04.625334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:07.532058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627167+0000","last_change":"2024-09-13T16:28:38.845998+0000","last_active":"2024-09-13T16:29:04.627167+0000","last_peered":"2024-09-13T16:29:04.627167+0000","last_clean":"2024-09-13T16:29:04.627167+0000","last_became_active":"2024-09-13T16:28:38.845451+0000","last_became_peered":"2024-09-13T16:28:38.845451+0000","last_unstale":"2024-09-13T16:29:04.627167+0000","last_undegraded":"2024-09-13T16:29:04.627167+0000","last_fullsized":"2024-09-13T16:29:04.627167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:15.696276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628820+0000","last_change":"2024-09-13T16:28:43.293598+0000","last_active":"2024-09-13T16:29:04.628820+0000","last_peered":"2024-09-13T16:29:04.628820+0000","last_clean":"2024-09-13T16:29:04.628820+0000","last_became_active":"2024-09-13T16:28:43.293045+0000","last_became_peered":"2024-09-13T16:28:43.293045+0000","last_unstale":"2024-09-13T16:29:04.628820+0000","last_undegraded":"2024-09-13T16:29:04.628820+0000","last_fullsized":"2024-09-13T16:29:04.628820+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:01.839626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044938+0000","last_change":"2024-09-13T16:28:39.214302+0000","last_active":"2024-09-13T16:28:44.044938+0000","last_peered":"2024-09-13T16:28:44.044938+0000","last_clean":"2024-09-13T16:28:44.044938+0000","last_became_active":"2024-09-13T16:28:39.213672+0000","last_became_peered":"2024-09-13T16:28:39.213672+0000","last_unstale":"2024-09-13T16:28:44.044938+0000","last_undegraded":"2024-09-13T16:28:44.044938+0000","last_fullsized":"2024-09-13T16:28:44.044938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:03:25.440603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"91'2","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.631445+0000","last_change":"2024-09-13T16:28:38.840626+0000","last_active":"2024-09-13T16:29:05.631445+0000","last_peered":"2024-09-13T16:29:05.631445+0000","last_clean":"2024-09-13T16:29:05.631445+0000","last_became_active":"2024-09-13T16:28:38.840344+0000","last_became_peered":"2024-09-13T16:28:38.840344+0000","last_unstale":"2024-09-13T16:29:05.631445+0000","last_undegraded":"2024-09-13T16:29:05.631445+0000","last_fullsized":"2024-09-13T16:29:05.631445+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:03:10.850301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.633381+0000","last_change":"2024-09-13T16:28:43.229163+0000","last_active":"2024-09-13T16:29:05.633381+0000","last_peered":"2024-09-13T16:29:05.633381+0000","last_clean":"2024-09-13T16:29:05.633381+0000","last_became_active":"2024-09-13T16:28:43.228783+0000","last_became_peered":"2024-09-13T16:28:43.228783+0000","last_unstale":"2024-09-13T16:29:05.633381+0000","last_undegraded":"2024-09-13T16:29:05.633381+0000","last_fullsized":"2024-09-13T16:29:05.633381+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:48.183837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627877+0000","last_change":"2024-09-13T16:28:39.209247+0000","last_active":"2024-09-13T16:29:04.627877+0000","last_peered":"2024-09-13T16:29:04.627877+0000","last_clean":"2024-09-13T16:29:04.627877+0000","last_became_active":"2024-09-13T16:28:39.208921+0000","last_became_peered":"2024-09-13T16:28:39.208921+0000","last_unstale":"2024-09-13T16:29:04.627877+0000","last_undegraded":"2024-09-13T16:29:04.627877+0000","last_fullsized":"2024-09-13T16:29:04.627877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:32:41.383029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.640902+0000","last_change":"2024-09-13T16:28:38.845904+0000","last_active":"2024-09-13T16:29:04.640902+0000","last_peered":"2024-09-13T16:29:04.640902+0000","last_clean":"2024-09-13T16:29:04.640902+0000","last_became_active":"2024-09-13T16:28:38.845639+0000","last_became_peered":"2024-09-13T16:28:38.845639+0000","last_unstale":"2024-09-13T16:29:04.640902+0000","last_undegraded":"2024-09-13T16:29:04.640902+0000","last_fullsized":"2024-09-13T16:29:04.640902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:25:34.148607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.640627+0000","last_change":"2024-09-13T16:29:05.640063+0000","last_active":"2024-09-13T16:29:05.640627+0000","last_peered":"2024-09-13T16:29:05.640627+0000","last_clean":"2024-09-13T16:29:05.640627+0000","last_became_active":"2024-09-13T16:29:05.639596+0000","last_became_peered":"2024-09-13T16:29:05.639596+0000","last_unstale":"2024-09-13T16:29:05.640627+0000","last_undegraded":"2024-09-13T16:29:05.640627+0000","last_fullsized":"2024-09-13T16:29:05.640627+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:41:38.351162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.626859+0000","last_change":"2024-09-13T16:28:39.212947+0000","last_active":"2024-09-13T16:29:04.626859+0000","last_peered":"2024-09-13T16:29:04.626859+0000","last_clean":"2024-09-13T16:29:04.626859+0000","last_became_active":"2024-09-13T16:28:39.212622+0000","last_became_peered":"2024-09-13T16:28:39.212622+0000","last_unstale":"2024-09-13T16:29:04.626859+0000","last_undegraded":"2024-09-13T16:29:04.626859+0000","last_fullsized":"2024-09-13T16:29:04.626859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:16.425154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:29:02.581304+0000","last_change":"2024-09-13T16:28:38.846173+0000","last_active":"2024-09-13T16:29:02.581304+0000","last_peered":"2024-09-13T16:29:02.581304+0000","last_clean":"2024-09-13T16:29:02.581304+0000","last_became_active":"2024-09-13T16:28:38.845873+0000","last_became_peered":"2024-09-13T16:28:38.845873+0000","last_unstale":"2024-09-13T16:29:02.581304+0000","last_undegraded":"2024-09-13T16:29:02.581304+0000","last_fullsized":"2024-09-13T16:29:02.581304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:19:55.817524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629006+0000","last_change":"2024-09-13T16:28:43.294347+0000","last_active":"2024-09-13T16:29:04.629006+0000","last_peered":"2024-09-13T16:29:04.629006+0000","last_clean":"2024-09-13T16:29:04.629006+0000","last_became_active":"2024-09-13T16:28:43.293902+0000","last_became_peered":"2024-09-13T16:28:43.293902+0000","last_unstale":"2024-09-13T16:29:04.629006+0000","last_undegraded":"2024-09-13T16:29:04.629006+0000","last_fullsized":"2024-09-13T16:29:04.629006+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:27:37.470163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T16:28:43.217326+0000","last_change":"2024-09-13T16:28:39.205817+0000","last_active":"2024-09-13T16:28:43.217326+0000","last_peered":"2024-09-13T16:28:43.217326+0000","last_clean":"2024-09-13T16:28:43.217326+0000","last_became_active":"2024-09-13T16:28:39.205534+0000","last_became_peered":"2024-09-13T16:28:39.205534+0000","last_unstale":"2024-09-13T16:28:43.217326+0000","last_undegraded":"2024-09-13T16:28:43.217326+0000","last_fullsized":"2024-09-13T16:28:43.217326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:09:16.294842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.633540+0000","last_change":"2024-09-13T16:28:38.847582+0000","last_active":"2024-09-13T16:29:05.633540+0000","last_peered":"2024-09-13T16:29:05.633540+0000","last_clean":"2024-09-13T16:29:05.633540+0000","last_became_active":"2024-09-13T16:28:38.847353+0000","last_became_peered":"2024-09-13T16:28:38.847353+0000","last_unstale":"2024-09-13T16:29:05.633540+0000","last_undegraded":"2024-09-13T16:29:05.633540+0000","last_fullsized":"2024-09-13T16:29:05.633540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:24.944088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.634008+0000","last_change":"2024-09-13T16:28:43.225202+0000","last_active":"2024-09-13T16:29:04.634008+0000","last_peered":"2024-09-13T16:29:04.634008+0000","last_clean":"2024-09-13T16:29:04.634008+0000","last_became_active":"2024-09-13T16:28:43.224802+0000","last_became_peered":"2024-09-13T16:28:43.224802+0000","last_unstale":"2024-09-13T16:29:04.634008+0000","last_undegraded":"2024-09-13T16:29:04.634008+0000","last_fullsized":"2024-09-13T16:29:04.634008+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:51:19.325859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.045516+0000","last_change":"2024-09-13T16:28:39.209630+0000","last_active":"2024-09-13T16:28:44.045516+0000","last_peered":"2024-09-13T16:28:44.045516+0000","last_clean":"2024-09-13T16:28:44.045516+0000","last_became_active":"2024-09-13T16:28:39.209303+0000","last_became_peered":"2024-09-13T16:28:39.209303+0000","last_unstale":"2024-09-13T16:28:44.045516+0000","last_undegraded":"2024-09-13T16:28:44.045516+0000","last_fullsized":"2024-09-13T16:28:44.045516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:01:31.822673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'7","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628093+0000","last_change":"2024-09-13T16:28:38.837858+0000","last_active":"2024-09-13T16:29:04.628093+0000","last_peered":"2024-09-13T16:29:04.628093+0000","last_clean":"2024-09-13T16:29:04.628093+0000","last_became_active":"2024-09-13T16:28:38.837378+0000","last_became_peered":"2024-09-13T16:28:38.837378+0000","last_unstale":"2024-09-13T16:29:04.628093+0000","last_undegraded":"2024-09-13T16:29:04.628093+0000","last_fullsized":"2024-09-13T16:29:04.628093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:59:41.055902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.754728+0000","last_change":"2024-09-13T16:28:43.294241+0000","last_active":"2024-09-13T16:28:44.754728+0000","last_peered":"2024-09-13T16:28:44.754728+0000","last_clean":"2024-09-13T16:28:44.754728+0000","last_became_active":"2024-09-13T16:28:43.293873+0000","last_became_peered":"2024-09-13T16:28:43.293873+0000","last_unstale":"2024-09-13T16:28:44.754728+0000","last_undegraded":"2024-09-13T16:28:44.754728+0000","last_fullsized":"2024-09-13T16:28:44.754728+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:34.018162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044782+0000","last_change":"2024-09-13T16:28:39.218071+0000","last_active":"2024-09-13T16:28:44.044782+0000","last_peered":"2024-09-13T16:28:44.044782+0000","last_clean":"2024-09-13T16:28:44.044782+0000","last_became_active":"2024-09-13T16:28:39.217798+0000","last_became_peered":"2024-09-13T16:28:39.217798+0000","last_unstale":"2024-09-13T16:28:44.044782+0000","last_undegraded":"2024-09-13T16:28:44.044782+0000","last_fullsized":"2024-09-13T16:28:44.044782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:39:09.769384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.640833+0000","last_change":"2024-09-13T16:28:38.847238+0000","last_active":"2024-09-13T16:29:04.640833+0000","last_peered":"2024-09-13T16:29:04.640833+0000","last_clean":"2024-09-13T16:29:04.640833+0000","last_became_active":"2024-09-13T16:28:38.846741+0000","last_became_peered":"2024-09-13T16:28:38.846741+0000","last_unstale":"2024-09-13T16:29:04.640833+0000","last_undegraded":"2024-09-13T16:29:04.640833+0000","last_fullsized":"2024-09-13T16:29:04.640833+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:52:51.946158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628298+0000","last_change":"2024-09-13T16:28:43.228431+0000","last_active":"2024-09-13T16:29:04.628298+0000","last_peered":"2024-09-13T16:29:04.628298+0000","last_clean":"2024-09-13T16:29:04.628298+0000","last_became_active":"2024-09-13T16:28:43.228030+0000","last_became_peered":"2024-09-13T16:28:43.228030+0000","last_unstale":"2024-09-13T16:29:04.628298+0000","last_undegraded":"2024-09-13T16:29:04.628298+0000","last_fullsized":"2024-09-13T16:29:04.628298+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:14:50.573033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629589+0000","last_change":"2024-09-13T16:28:39.209485+0000","last_active":"2024-09-13T16:29:04.629589+0000","last_peered":"2024-09-13T16:29:04.629589+0000","last_clean":"2024-09-13T16:29:04.629589+0000","last_became_active":"2024-09-13T16:28:39.209157+0000","last_became_peered":"2024-09-13T16:28:39.209157+0000","last_unstale":"2024-09-13T16:29:04.629589+0000","last_undegraded":"2024-09-13T16:29:04.629589+0000","last_fullsized":"2024-09-13T16:29:04.629589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:12:19.398255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628085+0000","last_change":"2024-09-13T16:28:38.842551+0000","last_active":"2024-09-13T16:29:04.628085+0000","last_peered":"2024-09-13T16:29:04.628085+0000","last_clean":"2024-09-13T16:29:04.628085+0000","last_became_active":"2024-09-13T16:28:38.841741+0000","last_became_peered":"2024-09-13T16:28:38.841741+0000","last_unstale":"2024-09-13T16:29:04.628085+0000","last_undegraded":"2024-09-13T16:29:04.628085+0000","last_fullsized":"2024-09-13T16:29:04.628085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:35:25.131693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.626884+0000","last_change":"2024-09-13T16:28:43.294300+0000","last_active":"2024-09-13T16:29:04.626884+0000","last_peered":"2024-09-13T16:29:04.626884+0000","last_clean":"2024-09-13T16:29:04.626884+0000","last_became_active":"2024-09-13T16:28:43.293918+0000","last_became_peered":"2024-09-13T16:28:43.293918+0000","last_unstale":"2024-09-13T16:29:04.626884+0000","last_undegraded":"2024-09-13T16:29:04.626884+0000","last_fullsized":"2024-09-13T16:29:04.626884+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:19:21.959617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.631071+0000","last_change":"2024-09-13T16:28:39.212001+0000","last_active":"2024-09-13T16:29:04.631071+0000","last_peered":"2024-09-13T16:29:04.631071+0000","last_clean":"2024-09-13T16:29:04.631071+0000","last_became_active":"2024-09-13T16:28:39.211716+0000","last_became_peered":"2024-09-13T16:28:39.211716+0000","last_unstale":"2024-09-13T16:29:04.631071+0000","last_undegraded":"2024-09-13T16:29:04.631071+0000","last_fullsized":"2024-09-13T16:29:04.631071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:56.753677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:06.648061+0000","last_change":"2024-09-13T16:28:38.851229+0000","last_active":"2024-09-13T16:29:06.648061+0000","last_peered":"2024-09-13T16:29:06.648061+0000","last_clean":"2024-09-13T16:29:06.648061+0000","last_became_active":"2024-09-13T16:28:38.850937+0000","last_became_peered":"2024-09-13T16:28:38.850937+0000","last_unstale":"2024-09-13T16:29:06.648061+0000","last_undegraded":"2024-09-13T16:29:06.648061+0000","last_fullsized":"2024-09-13T16:29:06.648061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:19.460864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:45.495529+0000","last_change":"2024-09-13T16:28:43.224825+0000","last_active":"2024-09-13T16:28:45.495529+0000","last_peered":"2024-09-13T16:28:45.495529+0000","last_clean":"2024-09-13T16:28:45.495529+0000","last_became_active":"2024-09-13T16:28:43.223903+0000","last_became_peered":"2024-09-13T16:28:43.223903+0000","last_unstale":"2024-09-13T16:28:45.495529+0000","last_undegraded":"2024-09-13T16:28:45.495529+0000","last_fullsized":"2024-09-13T16:28:45.495529+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:08:46.850668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.639142+0000","last_change":"2024-09-13T16:29:05.636787+0000","last_active":"2024-09-13T16:29:05.639142+0000","last_peered":"2024-09-13T16:29:05.639142+0000","last_clean":"2024-09-13T16:29:05.639142+0000","last_became_active":"2024-09-13T16:29:05.636399+0000","last_became_peered":"2024-09-13T16:29:05.636399+0000","last_unstale":"2024-09-13T16:29:05.639142+0000","last_undegraded":"2024-09-13T16:29:05.639142+0000","last_fullsized":"2024-09-13T16:29:05.639142+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:38:25.545325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.622845+0000","last_change":"2024-09-13T16:28:38.845374+0000","last_active":"2024-09-13T16:29:04.622845+0000","last_peered":"2024-09-13T16:29:04.622845+0000","last_clean":"2024-09-13T16:29:04.622845+0000","last_became_active":"2024-09-13T16:28:38.845076+0000","last_became_peered":"2024-09-13T16:28:38.845076+0000","last_unstale":"2024-09-13T16:29:04.622845+0000","last_undegraded":"2024-09-13T16:29:04.622845+0000","last_fullsized":"2024-09-13T16:29:04.622845+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:37:56.225682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:45.495577+0000","last_change":"2024-09-13T16:28:43.293529+0000","last_active":"2024-09-13T16:28:45.495577+0000","last_peered":"2024-09-13T16:28:45.495577+0000","last_clean":"2024-09-13T16:28:45.495577+0000","last_became_active":"2024-09-13T16:28:43.293083+0000","last_became_peered":"2024-09-13T16:28:43.293083+0000","last_unstale":"2024-09-13T16:28:45.495577+0000","last_undegraded":"2024-09-13T16:28:45.495577+0000","last_fullsized":"2024-09-13T16:28:45.495577+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:05:55.572984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.043722+0000","last_change":"2024-09-13T16:28:39.217546+0000","last_active":"2024-09-13T16:28:44.043722+0000","last_peered":"2024-09-13T16:28:44.043722+0000","last_clean":"2024-09-13T16:28:44.043722+0000","last_became_active":"2024-09-13T16:28:39.217242+0000","last_became_peered":"2024-09-13T16:28:39.217242+0000","last_unstale":"2024-09-13T16:28:44.043722+0000","last_undegraded":"2024-09-13T16:28:44.043722+0000","last_fullsized":"2024-09-13T16:28:44.043722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:23:18.183616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.641545+0000","last_change":"2024-09-13T16:28:38.846436+0000","last_active":"2024-09-13T16:29:04.641545+0000","last_peered":"2024-09-13T16:29:04.641545+0000","last_clean":"2024-09-13T16:29:04.641545+0000","last_became_active":"2024-09-13T16:28:38.845953+0000","last_became_peered":"2024-09-13T16:28:38.845953+0000","last_unstale":"2024-09-13T16:29:04.641545+0000","last_undegraded":"2024-09-13T16:29:04.641545+0000","last_fullsized":"2024-09-13T16:29:04.641545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:33.998462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.624750+0000","last_change":"2024-09-13T16:28:43.294504+0000","last_active":"2024-09-13T16:29:04.624750+0000","last_peered":"2024-09-13T16:29:04.624750+0000","last_clean":"2024-09-13T16:29:04.624750+0000","last_became_active":"2024-09-13T16:28:43.294063+0000","last_became_peered":"2024-09-13T16:28:43.294063+0000","last_unstale":"2024-09-13T16:29:04.624750+0000","last_undegraded":"2024-09-13T16:29:04.624750+0000","last_fullsized":"2024-09-13T16:29:04.624750+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:51:35.774467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044939+0000","last_change":"2024-09-13T16:28:39.206491+0000","last_active":"2024-09-13T16:28:44.044939+0000","last_peered":"2024-09-13T16:28:44.044939+0000","last_clean":"2024-09-13T16:28:44.044939+0000","last_became_active":"2024-09-13T16:28:39.206187+0000","last_became_peered":"2024-09-13T16:28:39.206187+0000","last_unstale":"2024-09-13T16:28:44.044939+0000","last_undegraded":"2024-09-13T16:28:44.044939+0000","last_fullsized":"2024-09-13T16:28:44.044939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:20:28.457994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.635563+0000","last_change":"2024-09-13T16:28:38.836270+0000","last_active":"2024-09-13T16:29:05.635563+0000","last_peered":"2024-09-13T16:29:05.635563+0000","last_clean":"2024-09-13T16:29:05.635563+0000","last_became_active":"2024-09-13T16:28:38.835966+0000","last_became_peered":"2024-09-13T16:28:38.835966+0000","last_unstale":"2024-09-13T16:29:05.635563+0000","last_undegraded":"2024-09-13T16:29:05.635563+0000","last_fullsized":"2024-09-13T16:29:05.635563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:03:11.388654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":331,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625402+0000","last_change":"2024-09-13T16:27:34.264484+0000","last_active":"2024-09-13T16:29:04.625402+0000","last_peered":"2024-09-13T16:29:04.625402+0000","last_clean":"2024-09-13T16:29:04.625402+0000","last_became_active":"2024-09-13T16:27:34.263995+0000","last_became_peered":"2024-09-13T16:27:34.263995+0000","last_unstale":"2024-09-13T16:29:04.625402+0000","last_undegraded":"2024-09-13T16:29:04.625402+0000","last_fullsized":"2024-09-13T16:29:04.625402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:24:23.015578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:24:23.015578+0000","last_clean_scrub_stamp":"2024-09-13T16:24:23.015578+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:25:04.376312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.634973+0000","last_change":"2024-09-13T16:28:43.229347+0000","last_active":"2024-09-13T16:29:05.634973+0000","last_peered":"2024-09-13T16:29:05.634973+0000","last_clean":"2024-09-13T16:29:05.634973+0000","last_became_active":"2024-09-13T16:28:43.228981+0000","last_became_peered":"2024-09-13T16:28:43.228981+0000","last_unstale":"2024-09-13T16:29:05.634973+0000","last_undegraded":"2024-09-13T16:29:05.634973+0000","last_fullsized":"2024-09-13T16:29:05.634973+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:46:15.151295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625877+0000","last_change":"2024-09-13T16:28:39.203195+0000","last_active":"2024-09-13T16:29:04.625877+0000","last_peered":"2024-09-13T16:29:04.625877+0000","last_clean":"2024-09-13T16:29:04.625877+0000","last_became_active":"2024-09-13T16:28:39.202878+0000","last_became_peered":"2024-09-13T16:28:39.202878+0000","last_unstale":"2024-09-13T16:29:04.625877+0000","last_undegraded":"2024-09-13T16:29:04.625877+0000","last_fullsized":"2024-09-13T16:29:04.625877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:17.615531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:06.646697+0000","last_change":"2024-09-13T16:28:38.844856+0000","last_active":"2024-09-13T16:29:06.646697+0000","last_peered":"2024-09-13T16:29:06.646697+0000","last_clean":"2024-09-13T16:29:06.646697+0000","last_became_active":"2024-09-13T16:28:38.844578+0000","last_became_peered":"2024-09-13T16:28:38.844578+0000","last_unstale":"2024-09-13T16:29:06.646697+0000","last_undegraded":"2024-09-13T16:29:06.646697+0000","last_fullsized":"2024-09-13T16:29:06.646697+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:41:48.253174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:45.572882+0000","last_change":"2024-09-13T16:28:43.228472+0000","last_active":"2024-09-13T16:28:45.572882+0000","last_peered":"2024-09-13T16:28:45.572882+0000","last_clean":"2024-09-13T16:28:45.572882+0000","last_became_active":"2024-09-13T16:28:43.228056+0000","last_became_peered":"2024-09-13T16:28:43.228056+0000","last_unstale":"2024-09-13T16:28:45.572882+0000","last_undegraded":"2024-09-13T16:28:45.572882+0000","last_fullsized":"2024-09-13T16:28:45.572882+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:07:55.980691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625637+0000","last_change":"2024-09-13T16:28:39.203278+0000","last_active":"2024-09-13T16:29:04.625637+0000","last_peered":"2024-09-13T16:29:04.625637+0000","last_clean":"2024-09-13T16:29:04.625637+0000","last_became_active":"2024-09-13T16:28:39.202936+0000","last_became_peered":"2024-09-13T16:28:39.202936+0000","last_unstale":"2024-09-13T16:29:04.625637+0000","last_undegraded":"2024-09-13T16:29:04.625637+0000","last_fullsized":"2024-09-13T16:29:04.625637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:44.210612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.640824+0000","last_change":"2024-09-13T16:28:38.851085+0000","last_active":"2024-09-13T16:29:04.640824+0000","last_peered":"2024-09-13T16:29:04.640824+0000","last_clean":"2024-09-13T16:29:04.640824+0000","last_became_active":"2024-09-13T16:28:38.850785+0000","last_became_peered":"2024-09-13T16:28:38.850785+0000","last_unstale":"2024-09-13T16:29:04.640824+0000","last_undegraded":"2024-09-13T16:29:04.640824+0000","last_fullsized":"2024-09-13T16:29:04.640824+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:52.713431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044053+0000","last_change":"2024-09-13T16:28:43.229980+0000","last_active":"2024-09-13T16:28:44.044053+0000","last_peered":"2024-09-13T16:28:44.044053+0000","last_clean":"2024-09-13T16:28:44.044053+0000","last_became_active":"2024-09-13T16:28:43.229647+0000","last_became_peered":"2024-09-13T16:28:43.229647+0000","last_unstale":"2024-09-13T16:28:44.044053+0000","last_undegraded":"2024-09-13T16:28:44.044053+0000","last_fullsized":"2024-09-13T16:28:44.044053+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:39:42.871613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625770+0000","last_change":"2024-09-13T16:28:39.204167+0000","last_active":"2024-09-13T16:29:04.625770+0000","last_peered":"2024-09-13T16:29:04.625770+0000","last_clean":"2024-09-13T16:29:04.625770+0000","last_became_active":"2024-09-13T16:28:39.203776+0000","last_became_peered":"2024-09-13T16:28:39.203776+0000","last_unstale":"2024-09-13T16:29:04.625770+0000","last_undegraded":"2024-09-13T16:29:04.625770+0000","last_fullsized":"2024-09-13T16:29:04.625770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:53:36.410365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.639297+0000","last_change":"2024-09-13T16:28:38.846331+0000","last_active":"2024-09-13T16:29:04.639297+0000","last_peered":"2024-09-13T16:29:04.639297+0000","last_clean":"2024-09-13T16:29:04.639297+0000","last_became_active":"2024-09-13T16:28:38.846012+0000","last_became_peered":"2024-09-13T16:28:38.846012+0000","last_unstale":"2024-09-13T16:29:04.639297+0000","last_undegraded":"2024-09-13T16:29:04.639297+0000","last_fullsized":"2024-09-13T16:29:04.639297+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:23:54.282736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.623954+0000","last_change":"2024-09-13T16:28:43.227886+0000","last_active":"2024-09-13T16:29:04.623954+0000","last_peered":"2024-09-13T16:29:04.623954+0000","last_clean":"2024-09-13T16:29:04.623954+0000","last_became_active":"2024-09-13T16:28:43.227346+0000","last_became_peered":"2024-09-13T16:28:43.227346+0000","last_unstale":"2024-09-13T16:29:04.623954+0000","last_undegraded":"2024-09-13T16:29:04.623954+0000","last_fullsized":"2024-09-13T16:29:04.623954+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:41:36.559064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629086+0000","last_change":"2024-09-13T16:28:39.205975+0000","last_active":"2024-09-13T16:29:04.629086+0000","last_peered":"2024-09-13T16:29:04.629086+0000","last_clean":"2024-09-13T16:29:04.629086+0000","last_became_active":"2024-09-13T16:28:39.205657+0000","last_became_peered":"2024-09-13T16:28:39.205657+0000","last_unstale":"2024-09-13T16:29:04.629086+0000","last_undegraded":"2024-09-13T16:29:04.629086+0000","last_fullsized":"2024-09-13T16:29:04.629086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:31:47.309482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629013+0000","last_change":"2024-09-13T16:28:38.841589+0000","last_active":"2024-09-13T16:29:04.629013+0000","last_peered":"2024-09-13T16:29:04.629013+0000","last_clean":"2024-09-13T16:29:04.629013+0000","last_became_active":"2024-09-13T16:28:38.841298+0000","last_became_peered":"2024-09-13T16:28:38.841298+0000","last_unstale":"2024-09-13T16:29:04.629013+0000","last_undegraded":"2024-09-13T16:29:04.629013+0000","last_fullsized":"2024-09-13T16:29:04.629013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:27:38.358320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.623986+0000","last_change":"2024-09-13T16:28:43.233790+0000","last_active":"2024-09-13T16:29:04.623986+0000","last_peered":"2024-09-13T16:29:04.623986+0000","last_clean":"2024-09-13T16:29:04.623986+0000","last_became_active":"2024-09-13T16:28:43.231963+0000","last_became_peered":"2024-09-13T16:28:43.231963+0000","last_unstale":"2024-09-13T16:29:04.623986+0000","last_undegraded":"2024-09-13T16:29:04.623986+0000","last_fullsized":"2024-09-13T16:29:04.623986+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:45:23.406395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.624233+0000","last_change":"2024-09-13T16:28:39.201476+0000","last_active":"2024-09-13T16:29:04.624233+0000","last_peered":"2024-09-13T16:29:04.624233+0000","last_clean":"2024-09-13T16:29:04.624233+0000","last_became_active":"2024-09-13T16:28:39.201168+0000","last_became_peered":"2024-09-13T16:28:39.201168+0000","last_unstale":"2024-09-13T16:29:04.624233+0000","last_undegraded":"2024-09-13T16:29:04.624233+0000","last_fullsized":"2024-09-13T16:29:04.624233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:26:43.570295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":62,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.652225+0000","last_change":"2024-09-13T16:29:05.652225+0000","last_active":"2024-09-13T16:29:05.652225+0000","last_peered":"2024-09-13T16:29:05.652225+0000","last_clean":"2024-09-13T16:29:05.652225+0000","last_became_active":"2024-09-13T16:29:05.639325+0000","last_became_peered":"2024-09-13T16:29:05.639325+0000","last_unstale":"2024-09-13T16:29:05.652225+0000","last_undegraded":"2024-09-13T16:29:05.652225+0000","last_fullsized":"2024-09-13T16:29:05.652225+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:18:30.083447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:45.495557+0000","last_change":"2024-09-13T16:28:43.228563+0000","last_active":"2024-09-13T16:28:45.495557+0000","last_peered":"2024-09-13T16:28:45.495557+0000","last_clean":"2024-09-13T16:28:45.495557+0000","last_became_active":"2024-09-13T16:28:43.228184+0000","last_became_peered":"2024-09-13T16:28:43.228184+0000","last_unstale":"2024-09-13T16:28:45.495557+0000","last_undegraded":"2024-09-13T16:28:45.495557+0000","last_fullsized":"2024-09-13T16:28:45.495557+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:05:33.368302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625221+0000","last_change":"2024-09-13T16:28:39.210750+0000","last_active":"2024-09-13T16:29:04.625221+0000","last_peered":"2024-09-13T16:29:04.625221+0000","last_clean":"2024-09-13T16:29:04.625221+0000","last_became_active":"2024-09-13T16:28:39.210401+0000","last_became_peered":"2024-09-13T16:28:39.210401+0000","last_unstale":"2024-09-13T16:29:04.625221+0000","last_undegraded":"2024-09-13T16:29:04.625221+0000","last_fullsized":"2024-09-13T16:29:04.625221+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:36.670420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:06.638428+0000","last_change":"2024-09-13T16:28:38.843573+0000","last_active":"2024-09-13T16:29:06.638428+0000","last_peered":"2024-09-13T16:29:06.638428+0000","last_clean":"2024-09-13T16:29:06.638428+0000","last_became_active":"2024-09-13T16:28:38.843245+0000","last_became_peered":"2024-09-13T16:28:38.843245+0000","last_unstale":"2024-09-13T16:29:06.638428+0000","last_undegraded":"2024-09-13T16:29:06.638428+0000","last_fullsized":"2024-09-13T16:29:06.638428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:45.983123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628374+0000","last_change":"2024-09-13T16:28:43.231002+0000","last_active":"2024-09-13T16:29:04.628374+0000","last_peered":"2024-09-13T16:29:04.628374+0000","last_clean":"2024-09-13T16:29:04.628374+0000","last_became_active":"2024-09-13T16:28:43.230237+0000","last_became_peered":"2024-09-13T16:28:43.230237+0000","last_unstale":"2024-09-13T16:29:04.628374+0000","last_undegraded":"2024-09-13T16:29:04.628374+0000","last_fullsized":"2024-09-13T16:29:04.628374+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:11:47.494738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625559+0000","last_change":"2024-09-13T16:28:39.201471+0000","last_active":"2024-09-13T16:29:04.625559+0000","last_peered":"2024-09-13T16:29:04.625559+0000","last_clean":"2024-09-13T16:29:04.625559+0000","last_became_active":"2024-09-13T16:28:39.201140+0000","last_became_peered":"2024-09-13T16:28:39.201140+0000","last_unstale":"2024-09-13T16:29:04.625559+0000","last_undegraded":"2024-09-13T16:29:04.625559+0000","last_fullsized":"2024-09-13T16:29:04.625559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:37:25.037490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.641008+0000","last_change":"2024-09-13T16:28:38.845703+0000","last_active":"2024-09-13T16:29:04.641008+0000","last_peered":"2024-09-13T16:29:04.641008+0000","last_clean":"2024-09-13T16:29:04.641008+0000","last_became_active":"2024-09-13T16:28:38.845354+0000","last_became_peered":"2024-09-13T16:28:38.845354+0000","last_unstale":"2024-09-13T16:29:04.641008+0000","last_undegraded":"2024-09-13T16:29:04.641008+0000","last_fullsized":"2024-09-13T16:29:04.641008+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:18.208723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629807+0000","last_change":"2024-09-13T16:28:43.223924+0000","last_active":"2024-09-13T16:29:04.629807+0000","last_peered":"2024-09-13T16:29:04.629807+0000","last_clean":"2024-09-13T16:29:04.629807+0000","last_became_active":"2024-09-13T16:28:43.223535+0000","last_became_peered":"2024-09-13T16:28:43.223535+0000","last_unstale":"2024-09-13T16:29:04.629807+0000","last_undegraded":"2024-09-13T16:29:04.629807+0000","last_fullsized":"2024-09-13T16:29:04.629807+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:10.622436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044425+0000","last_change":"2024-09-13T16:28:39.203943+0000","last_active":"2024-09-13T16:28:44.044425+0000","last_peered":"2024-09-13T16:28:44.044425+0000","last_clean":"2024-09-13T16:28:44.044425+0000","last_became_active":"2024-09-13T16:28:39.203657+0000","last_became_peered":"2024-09-13T16:28:39.203657+0000","last_unstale":"2024-09-13T16:28:44.044425+0000","last_undegraded":"2024-09-13T16:28:44.044425+0000","last_fullsized":"2024-09-13T16:28:44.044425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:52:48.523939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.651351+0000","last_change":"2024-09-13T16:29:05.651015+0000","last_active":"2024-09-13T16:29:05.651351+0000","last_peered":"2024-09-13T16:29:05.651351+0000","last_clean":"2024-09-13T16:29:05.651351+0000","last_became_active":"2024-09-13T16:29:05.640006+0000","last_became_peered":"2024-09-13T16:29:05.640006+0000","last_unstale":"2024-09-13T16:29:05.651351+0000","last_undegraded":"2024-09-13T16:29:05.651351+0000","last_fullsized":"2024-09-13T16:29:05.651351+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:12:25.508262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.624674+0000","last_change":"2024-09-13T16:28:43.232634+0000","last_active":"2024-09-13T16:29:04.624674+0000","last_peered":"2024-09-13T16:29:04.624674+0000","last_clean":"2024-09-13T16:29:04.624674+0000","last_became_active":"2024-09-13T16:28:43.232107+0000","last_became_peered":"2024-09-13T16:28:43.232107+0000","last_unstale":"2024-09-13T16:29:04.624674+0000","last_undegraded":"2024-09-13T16:29:04.624674+0000","last_fullsized":"2024-09-13T16:29:04.624674+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:45.952933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627482+0000","last_change":"2024-09-13T16:28:39.212883+0000","last_active":"2024-09-13T16:29:04.627482+0000","last_peered":"2024-09-13T16:29:04.627482+0000","last_clean":"2024-09-13T16:29:04.627482+0000","last_became_active":"2024-09-13T16:28:39.212591+0000","last_became_peered":"2024-09-13T16:28:39.212591+0000","last_unstale":"2024-09-13T16:29:04.627482+0000","last_undegraded":"2024-09-13T16:29:04.627482+0000","last_fullsized":"2024-09-13T16:29:04.627482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:17:30.926940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'17","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:06.645117+0000","last_change":"2024-09-13T16:28:38.846564+0000","last_active":"2024-09-13T16:29:06.645117+0000","last_peered":"2024-09-13T16:29:06.645117+0000","last_clean":"2024-09-13T16:29:06.645117+0000","last_became_active":"2024-09-13T16:28:38.846165+0000","last_became_peered":"2024-09-13T16:28:38.846165+0000","last_unstale":"2024-09-13T16:29:06.645117+0000","last_undegraded":"2024-09-13T16:29:06.645117+0000","last_fullsized":"2024-09-13T16:29:06.645117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:33:06.124592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10396,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":19,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627853+0000","last_change":"2024-09-13T16:28:43.231464+0000","last_active":"2024-09-13T16:29:04.627853+0000","last_peered":"2024-09-13T16:29:04.627853+0000","last_clean":"2024-09-13T16:29:04.627853+0000","last_became_active":"2024-09-13T16:28:43.231040+0000","last_became_peered":"2024-09-13T16:28:43.231040+0000","last_unstale":"2024-09-13T16:29:04.627853+0000","last_undegraded":"2024-09-13T16:29:04.627853+0000","last_fullsized":"2024-09-13T16:29:04.627853+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:11:21.847375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"peering","last_fresh":"2024-09-13T16:29:04.630957+0000","last_change":"2024-09-13T16:29:04.631158+0000","last_active":"2024-09-13T16:29:04.630957+0000","last_peered":"2024-09-13T16:28:44.044616+0000","last_clean":"2024-09-13T16:28:44.044616+0000","last_became_active":"2024-09-13T16:28:39.204382+0000","last_became_peered":"2024-09-13T16:28:39.204382+0000","last_unstale":"2024-09-13T16:29:04.630957+0000","last_undegraded":"2024-09-13T16:29:04.630957+0000","last_fullsized":"2024-09-13T16:29:04.630957+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:58.816998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6,8],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625089+0000","last_change":"2024-09-13T16:28:38.841637+0000","last_active":"2024-09-13T16:29:04.625089+0000","last_peered":"2024-09-13T16:29:04.625089+0000","last_clean":"2024-09-13T16:29:04.625089+0000","last_became_active":"2024-09-13T16:28:38.841359+0000","last_became_peered":"2024-09-13T16:28:38.841359+0000","last_unstale":"2024-09-13T16:29:04.625089+0000","last_undegraded":"2024-09-13T16:29:04.625089+0000","last_fullsized":"2024-09-13T16:29:04.625089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:47:05.284987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.638870+0000","last_change":"2024-09-13T16:29:05.638870+0000","last_active":"2024-09-13T16:29:05.638870+0000","last_peered":"2024-09-13T16:29:05.638870+0000","last_clean":"2024-09-13T16:29:05.638870+0000","last_became_active":"2024-09-13T16:29:05.638422+0000","last_became_peered":"2024-09-13T16:29:05.638422+0000","last_unstale":"2024-09-13T16:29:05.638870+0000","last_undegraded":"2024-09-13T16:29:05.638870+0000","last_fullsized":"2024-09-13T16:29:05.638870+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:41.263661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T16:28:43.218920+0000","last_change":"2024-09-13T16:28:39.203595+0000","last_active":"2024-09-13T16:28:43.218920+0000","last_peered":"2024-09-13T16:28:43.218920+0000","last_clean":"2024-09-13T16:28:43.218920+0000","last_became_active":"2024-09-13T16:28:39.203283+0000","last_became_peered":"2024-09-13T16:28:39.203283+0000","last_unstale":"2024-09-13T16:28:43.218920+0000","last_undegraded":"2024-09-13T16:28:43.218920+0000","last_fullsized":"2024-09-13T16:28:43.218920+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:58:45.970761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.642267+0000","last_change":"2024-09-13T16:28:38.850203+0000","last_active":"2024-09-13T16:29:04.642267+0000","last_peered":"2024-09-13T16:29:04.642267+0000","last_clean":"2024-09-13T16:29:04.642267+0000","last_became_active":"2024-09-13T16:28:38.849963+0000","last_became_peered":"2024-09-13T16:28:38.849963+0000","last_unstale":"2024-09-13T16:29:04.642267+0000","last_undegraded":"2024-09-13T16:29:04.642267+0000","last_fullsized":"2024-09-13T16:29:04.642267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:15.849627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627470+0000","last_change":"2024-09-13T16:28:43.227679+0000","last_active":"2024-09-13T16:29:04.627470+0000","last_peered":"2024-09-13T16:29:04.627470+0000","last_clean":"2024-09-13T16:29:04.627470+0000","last_became_active":"2024-09-13T16:28:43.227269+0000","last_became_peered":"2024-09-13T16:28:43.227269+0000","last_unstale":"2024-09-13T16:29:04.627470+0000","last_undegraded":"2024-09-13T16:29:04.627470+0000","last_fullsized":"2024-09-13T16:29:04.627470+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:05:00.186305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044938+0000","last_change":"2024-09-13T16:28:39.211401+0000","last_active":"2024-09-13T16:28:44.044938+0000","last_peered":"2024-09-13T16:28:44.044938+0000","last_clean":"2024-09-13T16:28:44.044938+0000","last_became_active":"2024-09-13T16:28:39.211133+0000","last_became_peered":"2024-09-13T16:28:39.211133+0000","last_unstale":"2024-09-13T16:28:44.044938+0000","last_undegraded":"2024-09-13T16:28:44.044938+0000","last_fullsized":"2024-09-13T16:28:44.044938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:36.468666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:06.642523+0000","last_change":"2024-09-13T16:28:38.845132+0000","last_active":"2024-09-13T16:29:06.642523+0000","last_peered":"2024-09-13T16:29:06.642523+0000","last_clean":"2024-09-13T16:29:06.642523+0000","last_became_active":"2024-09-13T16:28:38.844804+0000","last_became_peered":"2024-09-13T16:28:38.844804+0000","last_unstale":"2024-09-13T16:29:06.642523+0000","last_undegraded":"2024-09-13T16:29:06.642523+0000","last_fullsized":"2024-09-13T16:29:06.642523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:15:08.643227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.754760+0000","last_change":"2024-09-13T16:28:43.230829+0000","last_active":"2024-09-13T16:28:44.754760+0000","last_peered":"2024-09-13T16:28:44.754760+0000","last_clean":"2024-09-13T16:28:44.754760+0000","last_became_active":"2024-09-13T16:28:43.230405+0000","last_became_peered":"2024-09-13T16:28:43.230405+0000","last_unstale":"2024-09-13T16:28:44.754760+0000","last_undegraded":"2024-09-13T16:28:44.754760+0000","last_fullsized":"2024-09-13T16:28:44.754760+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:04:56.083193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044006+0000","last_change":"2024-09-13T16:28:43.226662+0000","last_active":"2024-09-13T16:28:44.044006+0000","last_peered":"2024-09-13T16:28:44.044006+0000","last_clean":"2024-09-13T16:28:44.044006+0000","last_became_active":"2024-09-13T16:28:43.226232+0000","last_became_peered":"2024-09-13T16:28:43.226232+0000","last_unstale":"2024-09-13T16:28:44.044006+0000","last_undegraded":"2024-09-13T16:28:44.044006+0000","last_fullsized":"2024-09-13T16:28:44.044006+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:03.568500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627636+0000","last_change":"2024-09-13T16:28:39.208909+0000","last_active":"2024-09-13T16:29:04.627636+0000","last_peered":"2024-09-13T16:29:04.627636+0000","last_clean":"2024-09-13T16:29:04.627636+0000","last_became_active":"2024-09-13T16:28:39.208572+0000","last_became_peered":"2024-09-13T16:28:39.208572+0000","last_unstale":"2024-09-13T16:29:04.627636+0000","last_undegraded":"2024-09-13T16:29:04.627636+0000","last_fullsized":"2024-09-13T16:29:04.627636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:01:40.173308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.641240+0000","last_change":"2024-09-13T16:28:38.844259+0000","last_active":"2024-09-13T16:29:04.641240+0000","last_peered":"2024-09-13T16:29:04.641240+0000","last_clean":"2024-09-13T16:29:04.641240+0000","last_became_active":"2024-09-13T16:28:38.843956+0000","last_became_peered":"2024-09-13T16:28:38.843956+0000","last_unstale":"2024-09-13T16:29:04.641240+0000","last_undegraded":"2024-09-13T16:29:04.641240+0000","last_fullsized":"2024-09-13T16:29:04.641240+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:05.652852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629221+0000","last_change":"2024-09-13T16:28:43.229385+0000","last_active":"2024-09-13T16:29:04.629221+0000","last_peered":"2024-09-13T16:29:04.629221+0000","last_clean":"2024-09-13T16:29:04.629221+0000","last_became_active":"2024-09-13T16:28:43.229033+0000","last_became_peered":"2024-09-13T16:28:43.229033+0000","last_unstale":"2024-09-13T16:29:04.629221+0000","last_undegraded":"2024-09-13T16:29:04.629221+0000","last_fullsized":"2024-09-13T16:29:04.629221+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:46:41.611758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044455+0000","last_change":"2024-09-13T16:28:39.208100+0000","last_active":"2024-09-13T16:28:44.044455+0000","last_peered":"2024-09-13T16:28:44.044455+0000","last_clean":"2024-09-13T16:28:44.044455+0000","last_became_active":"2024-09-13T16:28:39.207808+0000","last_became_peered":"2024-09-13T16:28:39.207808+0000","last_unstale":"2024-09-13T16:28:44.044455+0000","last_undegraded":"2024-09-13T16:28:44.044455+0000","last_fullsized":"2024-09-13T16:28:44.044455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:55.446568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.622326+0000","last_change":"2024-09-13T16:28:38.844689+0000","last_active":"2024-09-13T16:29:04.622326+0000","last_peered":"2024-09-13T16:29:04.622326+0000","last_clean":"2024-09-13T16:29:04.622326+0000","last_became_active":"2024-09-13T16:28:38.844326+0000","last_became_peered":"2024-09-13T16:28:38.844326+0000","last_unstale":"2024-09-13T16:29:04.622326+0000","last_undegraded":"2024-09-13T16:29:04.622326+0000","last_fullsized":"2024-09-13T16:29:04.622326+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:02.616518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629530+0000","last_change":"2024-09-13T16:28:38.849233+0000","last_active":"2024-09-13T16:29:04.629530+0000","last_peered":"2024-09-13T16:29:04.629530+0000","last_clean":"2024-09-13T16:29:04.629530+0000","last_became_active":"2024-09-13T16:28:38.848911+0000","last_became_peered":"2024-09-13T16:28:38.848911+0000","last_unstale":"2024-09-13T16:29:04.629530+0000","last_undegraded":"2024-09-13T16:29:04.629530+0000","last_fullsized":"2024-09-13T16:29:04.629530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:02:35.951263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.624339+0000","last_change":"2024-09-13T16:28:39.203855+0000","last_active":"2024-09-13T16:29:04.624339+0000","last_peered":"2024-09-13T16:29:04.624339+0000","last_clean":"2024-09-13T16:29:04.624339+0000","last_became_active":"2024-09-13T16:28:39.203532+0000","last_became_peered":"2024-09-13T16:28:39.203532+0000","last_unstale":"2024-09-13T16:29:04.624339+0000","last_undegraded":"2024-09-13T16:29:04.624339+0000","last_fullsized":"2024-09-13T16:29:04.624339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:15.048930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.633730+0000","last_change":"2024-09-13T16:28:43.228554+0000","last_active":"2024-09-13T16:29:04.633730+0000","last_peered":"2024-09-13T16:29:04.633730+0000","last_clean":"2024-09-13T16:29:04.633730+0000","last_became_active":"2024-09-13T16:28:43.228204+0000","last_became_peered":"2024-09-13T16:28:43.228204+0000","last_unstale":"2024-09-13T16:29:04.633730+0000","last_undegraded":"2024-09-13T16:29:04.633730+0000","last_fullsized":"2024-09-13T16:29:04.633730+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:16:15.383932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.639374+0000","last_change":"2024-09-13T16:28:38.848425+0000","last_active":"2024-09-13T16:29:05.639374+0000","last_peered":"2024-09-13T16:29:05.639374+0000","last_clean":"2024-09-13T16:29:05.639374+0000","last_became_active":"2024-09-13T16:28:38.848136+0000","last_became_peered":"2024-09-13T16:28:38.848136+0000","last_unstale":"2024-09-13T16:29:05.639374+0000","last_undegraded":"2024-09-13T16:29:05.639374+0000","last_fullsized":"2024-09-13T16:29:05.639374+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:47:03.904315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627249+0000","last_change":"2024-09-13T16:28:39.210565+0000","last_active":"2024-09-13T16:29:04.627249+0000","last_peered":"2024-09-13T16:29:04.627249+0000","last_clean":"2024-09-13T16:29:04.627249+0000","last_became_active":"2024-09-13T16:28:39.210272+0000","last_became_peered":"2024-09-13T16:28:39.210272+0000","last_unstale":"2024-09-13T16:29:04.627249+0000","last_undegraded":"2024-09-13T16:29:04.627249+0000","last_fullsized":"2024-09-13T16:29:04.627249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:12.746087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.634077+0000","last_change":"2024-09-13T16:28:43.231623+0000","last_active":"2024-09-13T16:29:04.634077+0000","last_peered":"2024-09-13T16:29:04.634077+0000","last_clean":"2024-09-13T16:29:04.634077+0000","last_became_active":"2024-09-13T16:28:43.231255+0000","last_became_peered":"2024-09-13T16:28:43.231255+0000","last_unstale":"2024-09-13T16:29:04.634077+0000","last_undegraded":"2024-09-13T16:29:04.634077+0000","last_fullsized":"2024-09-13T16:29:04.634077+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:29.755772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627783+0000","last_change":"2024-09-13T16:28:39.208438+0000","last_active":"2024-09-13T16:29:04.627783+0000","last_peered":"2024-09-13T16:29:04.627783+0000","last_clean":"2024-09-13T16:29:04.627783+0000","last_became_active":"2024-09-13T16:28:39.208156+0000","last_became_peered":"2024-09-13T16:28:39.208156+0000","last_unstale":"2024-09-13T16:29:04.627783+0000","last_undegraded":"2024-09-13T16:29:04.627783+0000","last_fullsized":"2024-09-13T16:29:04.627783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:53:46.923166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.633806+0000","last_change":"2024-09-13T16:28:39.208529+0000","last_active":"2024-09-13T16:29:04.633806+0000","last_peered":"2024-09-13T16:29:04.633806+0000","last_clean":"2024-09-13T16:29:04.633806+0000","last_became_active":"2024-09-13T16:28:39.208202+0000","last_became_peered":"2024-09-13T16:28:39.208202+0000","last_unstale":"2024-09-13T16:29:04.633806+0000","last_undegraded":"2024-09-13T16:29:04.633806+0000","last_fullsized":"2024-09-13T16:29:04.633806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:17:11.357410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.632611+0000","last_change":"2024-09-13T16:28:39.212958+0000","last_active":"2024-09-13T16:29:04.632611+0000","last_peered":"2024-09-13T16:29:04.632611+0000","last_clean":"2024-09-13T16:29:04.632611+0000","last_became_active":"2024-09-13T16:28:39.212646+0000","last_became_peered":"2024-09-13T16:28:39.212646+0000","last_unstale":"2024-09-13T16:29:04.632611+0000","last_undegraded":"2024-09-13T16:29:04.632611+0000","last_fullsized":"2024-09-13T16:29:04.632611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:31:04.047778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T16:28:43.218683+0000","last_change":"2024-09-13T16:28:39.200449+0000","last_active":"2024-09-13T16:28:43.218683+0000","last_peered":"2024-09-13T16:28:43.218683+0000","last_clean":"2024-09-13T16:28:43.218683+0000","last_became_active":"2024-09-13T16:28:39.200133+0000","last_became_peered":"2024-09-13T16:28:39.200133+0000","last_unstale":"2024-09-13T16:28:43.218683+0000","last_undegraded":"2024-09-13T16:28:43.218683+0000","last_fullsized":"2024-09-13T16:28:43.218683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:18:06.480691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044668+0000","last_change":"2024-09-13T16:28:39.200274+0000","last_active":"2024-09-13T16:28:44.044668+0000","last_peered":"2024-09-13T16:28:44.044668+0000","last_clean":"2024-09-13T16:28:44.044668+0000","last_became_active":"2024-09-13T16:28:39.199951+0000","last_became_peered":"2024-09-13T16:28:39.199951+0000","last_unstale":"2024-09-13T16:28:44.044668+0000","last_undegraded":"2024-09-13T16:28:44.044668+0000","last_fullsized":"2024-09-13T16:28:44.044668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:13:31.199857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.630000+0000","last_change":"2024-09-13T16:28:39.213591+0000","last_active":"2024-09-13T16:29:04.630000+0000","last_peered":"2024-09-13T16:29:04.630000+0000","last_clean":"2024-09-13T16:29:04.630000+0000","last_became_active":"2024-09-13T16:28:39.213324+0000","last_became_peered":"2024-09-13T16:28:39.213324+0000","last_unstale":"2024-09-13T16:29:04.630000+0000","last_undegraded":"2024-09-13T16:29:04.630000+0000","last_fullsized":"2024-09-13T16:29:04.630000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:54:03.172919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.633921+0000","last_change":"2024-09-13T16:28:39.217442+0000","last_active":"2024-09-13T16:29:04.633921+0000","last_peered":"2024-09-13T16:29:04.633921+0000","last_clean":"2024-09-13T16:29:04.633921+0000","last_became_active":"2024-09-13T16:28:39.217151+0000","last_became_peered":"2024-09-13T16:28:39.217151+0000","last_unstale":"2024-09-13T16:29:04.633921+0000","last_undegraded":"2024-09-13T16:29:04.633921+0000","last_fullsized":"2024-09-13T16:29:04.633921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:17.132278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.045476+0000","last_change":"2024-09-13T16:28:39.203938+0000","last_active":"2024-09-13T16:28:44.045476+0000","last_peered":"2024-09-13T16:28:44.045476+0000","last_clean":"2024-09-13T16:28:44.045476+0000","last_became_active":"2024-09-13T16:28:39.203655+0000","last_became_peered":"2024-09-13T16:28:39.203655+0000","last_unstale":"2024-09-13T16:28:44.045476+0000","last_undegraded":"2024-09-13T16:28:44.045476+0000","last_fullsized":"2024-09-13T16:28:44.045476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:03:25.415256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.624165+0000","last_change":"2024-09-13T16:28:39.209592+0000","last_active":"2024-09-13T16:29:04.624165+0000","last_peered":"2024-09-13T16:29:04.624165+0000","last_clean":"2024-09-13T16:29:04.624165+0000","last_became_active":"2024-09-13T16:28:39.209299+0000","last_became_peered":"2024-09-13T16:28:39.209299+0000","last_unstale":"2024-09-13T16:29:04.624165+0000","last_undegraded":"2024-09-13T16:29:04.624165+0000","last_fullsized":"2024-09-13T16:29:04.624165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:04.053405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044501+0000","last_change":"2024-09-13T16:28:39.208437+0000","last_active":"2024-09-13T16:28:44.044501+0000","last_peered":"2024-09-13T16:28:44.044501+0000","last_clean":"2024-09-13T16:28:44.044501+0000","last_became_active":"2024-09-13T16:28:39.208136+0000","last_became_peered":"2024-09-13T16:28:39.208136+0000","last_unstale":"2024-09-13T16:28:44.044501+0000","last_undegraded":"2024-09-13T16:28:44.044501+0000","last_fullsized":"2024-09-13T16:28:44.044501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:21.304037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.045412+0000","last_change":"2024-09-13T16:28:39.203970+0000","last_active":"2024-09-13T16:28:44.045412+0000","last_peered":"2024-09-13T16:28:44.045412+0000","last_clean":"2024-09-13T16:28:44.045412+0000","last_became_active":"2024-09-13T16:28:39.203559+0000","last_became_peered":"2024-09-13T16:28:39.203559+0000","last_unstale":"2024-09-13T16:28:44.045412+0000","last_undegraded":"2024-09-13T16:28:44.045412+0000","last_fullsized":"2024-09-13T16:28:44.045412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:26:25.158393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044158+0000","last_change":"2024-09-13T16:28:39.205584+0000","last_active":"2024-09-13T16:28:44.044158+0000","last_peered":"2024-09-13T16:28:44.044158+0000","last_clean":"2024-09-13T16:28:44.044158+0000","last_became_active":"2024-09-13T16:28:39.205281+0000","last_became_peered":"2024-09-13T16:28:39.205281+0000","last_unstale":"2024-09-13T16:28:44.044158+0000","last_undegraded":"2024-09-13T16:28:44.044158+0000","last_fullsized":"2024-09-13T16:28:44.044158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:58:10.109647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629903+0000","last_change":"2024-09-13T16:28:39.214479+0000","last_active":"2024-09-13T16:29:04.629903+0000","last_peered":"2024-09-13T16:29:04.629903+0000","last_clean":"2024-09-13T16:29:04.629903+0000","last_became_active":"2024-09-13T16:28:39.213984+0000","last_became_peered":"2024-09-13T16:28:39.213984+0000","last_unstale":"2024-09-13T16:29:04.629903+0000","last_undegraded":"2024-09-13T16:29:04.629903+0000","last_fullsized":"2024-09-13T16:29:04.629903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:53:43.137726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.634317+0000","last_change":"2024-09-13T16:28:39.209595+0000","last_active":"2024-09-13T16:29:04.634317+0000","last_peered":"2024-09-13T16:29:04.634317+0000","last_clean":"2024-09-13T16:29:04.634317+0000","last_became_active":"2024-09-13T16:28:39.209282+0000","last_became_peered":"2024-09-13T16:28:39.209282+0000","last_unstale":"2024-09-13T16:29:04.634317+0000","last_undegraded":"2024-09-13T16:29:04.634317+0000","last_fullsized":"2024-09-13T16:29:04.634317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:26:41.452462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628010+0000","last_change":"2024-09-13T16:28:39.210610+0000","last_active":"2024-09-13T16:29:04.628010+0000","last_peered":"2024-09-13T16:29:04.628010+0000","last_clean":"2024-09-13T16:29:04.628010+0000","last_became_active":"2024-09-13T16:28:39.210296+0000","last_became_peered":"2024-09-13T16:28:39.210296+0000","last_unstale":"2024-09-13T16:29:04.628010+0000","last_undegraded":"2024-09-13T16:29:04.628010+0000","last_fullsized":"2024-09-13T16:29:04.628010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:06.385733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629594+0000","last_change":"2024-09-13T16:28:39.207609+0000","last_active":"2024-09-13T16:29:04.629594+0000","last_peered":"2024-09-13T16:29:04.629594+0000","last_clean":"2024-09-13T16:29:04.629594+0000","last_became_active":"2024-09-13T16:28:39.206893+0000","last_became_peered":"2024-09-13T16:28:39.206893+0000","last_unstale":"2024-09-13T16:29:04.629594+0000","last_undegraded":"2024-09-13T16:29:04.629594+0000","last_fullsized":"2024-09-13T16:29:04.629594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:51.582666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.045110+0000","last_change":"2024-09-13T16:28:39.201856+0000","last_active":"2024-09-13T16:28:44.045110+0000","last_peered":"2024-09-13T16:28:44.045110+0000","last_clean":"2024-09-13T16:28:44.045110+0000","last_became_active":"2024-09-13T16:28:39.201511+0000","last_became_peered":"2024-09-13T16:28:39.201511+0000","last_unstale":"2024-09-13T16:28:44.045110+0000","last_undegraded":"2024-09-13T16:28:44.045110+0000","last_fullsized":"2024-09-13T16:28:44.045110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:05:19.722354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T16:28:43.215322+0000","last_change":"2024-09-13T16:28:39.216106+0000","last_active":"2024-09-13T16:28:43.215322+0000","last_peered":"2024-09-13T16:28:43.215322+0000","last_clean":"2024-09-13T16:28:43.215322+0000","last_became_active":"2024-09-13T16:28:39.215432+0000","last_became_peered":"2024-09-13T16:28:39.215432+0000","last_unstale":"2024-09-13T16:28:43.215322+0000","last_undegraded":"2024-09-13T16:28:43.215322+0000","last_fullsized":"2024-09-13T16:28:43.215322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:21:18.439891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T16:28:43.215353+0000","last_change":"2024-09-13T16:28:39.210578+0000","last_active":"2024-09-13T16:28:43.215353+0000","last_peered":"2024-09-13T16:28:43.215353+0000","last_clean":"2024-09-13T16:28:43.215353+0000","last_became_active":"2024-09-13T16:28:39.210269+0000","last_became_peered":"2024-09-13T16:28:39.210269+0000","last_unstale":"2024-09-13T16:28:43.215353+0000","last_undegraded":"2024-09-13T16:28:43.215353+0000","last_fullsized":"2024-09-13T16:28:43.215353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:27:27.078880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044081+0000","last_change":"2024-09-13T16:28:39.213409+0000","last_active":"2024-09-13T16:28:44.044081+0000","last_peered":"2024-09-13T16:28:44.044081+0000","last_clean":"2024-09-13T16:28:44.044081+0000","last_became_active":"2024-09-13T16:28:39.213077+0000","last_became_peered":"2024-09-13T16:28:39.213077+0000","last_unstale":"2024-09-13T16:28:44.044081+0000","last_undegraded":"2024-09-13T16:28:44.044081+0000","last_fullsized":"2024-09-13T16:28:44.044081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:32:24.747516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.632419+0000","last_change":"2024-09-13T16:28:39.215216+0000","last_active":"2024-09-13T16:29:04.632419+0000","last_peered":"2024-09-13T16:29:04.632419+0000","last_clean":"2024-09-13T16:29:04.632419+0000","last_became_active":"2024-09-13T16:28:39.214920+0000","last_became_peered":"2024-09-13T16:28:39.214920+0000","last_unstale":"2024-09-13T16:29:04.632419+0000","last_undegraded":"2024-09-13T16:29:04.632419+0000","last_fullsized":"2024-09-13T16:29:04.632419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:25:38.132833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625826+0000","last_change":"2024-09-13T16:28:39.202506+0000","last_active":"2024-09-13T16:29:04.625826+0000","last_peered":"2024-09-13T16:29:04.625826+0000","last_clean":"2024-09-13T16:29:04.625826+0000","last_became_active":"2024-09-13T16:28:39.202224+0000","last_became_peered":"2024-09-13T16:28:39.202224+0000","last_unstale":"2024-09-13T16:29:04.625826+0000","last_undegraded":"2024-09-13T16:29:04.625826+0000","last_fullsized":"2024-09-13T16:29:04.625826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:00:37.208442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.640477+0000","last_change":"2024-09-13T16:29:05.640477+0000","last_active":"2024-09-13T16:29:05.640477+0000","last_peered":"2024-09-13T16:29:05.640477+0000","last_clean":"2024-09-13T16:29:05.640477+0000","last_became_active":"2024-09-13T16:29:05.640014+0000","last_became_peered":"2024-09-13T16:29:05.640014+0000","last_unstale":"2024-09-13T16:29:05.640477+0000","last_undegraded":"2024-09-13T16:29:05.640477+0000","last_fullsized":"2024-09-13T16:29:05.640477+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:47:45.202113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.045134+0000","last_change":"2024-09-13T16:28:39.200217+0000","last_active":"2024-09-13T16:28:44.045134+0000","last_peered":"2024-09-13T16:28:44.045134+0000","last_clean":"2024-09-13T16:28:44.045134+0000","last_became_active":"2024-09-13T16:28:39.199946+0000","last_became_peered":"2024-09-13T16:28:39.199946+0000","last_unstale":"2024-09-13T16:28:44.045134+0000","last_undegraded":"2024-09-13T16:28:44.045134+0000","last_fullsized":"2024-09-13T16:28:44.045134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:20:02.906225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044376+0000","last_change":"2024-09-13T16:28:39.215013+0000","last_active":"2024-09-13T16:28:44.044376+0000","last_peered":"2024-09-13T16:28:44.044376+0000","last_clean":"2024-09-13T16:28:44.044376+0000","last_became_active":"2024-09-13T16:28:39.214731+0000","last_became_peered":"2024-09-13T16:28:39.214731+0000","last_unstale":"2024-09-13T16:28:44.044376+0000","last_undegraded":"2024-09-13T16:28:44.044376+0000","last_fullsized":"2024-09-13T16:28:44.044376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:35:01.836001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625268+0000","last_change":"2024-09-13T16:28:39.200300+0000","last_active":"2024-09-13T16:29:04.625268+0000","last_peered":"2024-09-13T16:29:04.625268+0000","last_clean":"2024-09-13T16:29:04.625268+0000","last_became_active":"2024-09-13T16:28:39.200006+0000","last_became_peered":"2024-09-13T16:28:39.200006+0000","last_unstale":"2024-09-13T16:29:04.625268+0000","last_undegraded":"2024-09-13T16:29:04.625268+0000","last_fullsized":"2024-09-13T16:29:04.625268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:35:14.783905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629430+0000","last_change":"2024-09-13T16:28:39.213552+0000","last_active":"2024-09-13T16:29:04.629430+0000","last_peered":"2024-09-13T16:29:04.629430+0000","last_clean":"2024-09-13T16:29:04.629430+0000","last_became_active":"2024-09-13T16:28:39.213287+0000","last_became_peered":"2024-09-13T16:28:39.213287+0000","last_unstale":"2024-09-13T16:29:04.629430+0000","last_undegraded":"2024-09-13T16:29:04.629430+0000","last_fullsized":"2024-09-13T16:29:04.629430+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:30.588770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627025+0000","last_change":"2024-09-13T16:28:39.211126+0000","last_active":"2024-09-13T16:29:04.627025+0000","last_peered":"2024-09-13T16:29:04.627025+0000","last_clean":"2024-09-13T16:29:04.627025+0000","last_became_active":"2024-09-13T16:28:39.210810+0000","last_became_peered":"2024-09-13T16:28:39.210810+0000","last_unstale":"2024-09-13T16:29:04.627025+0000","last_undegraded":"2024-09-13T16:29:04.627025+0000","last_fullsized":"2024-09-13T16:29:04.627025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:52:45.249309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.639070+0000","last_change":"2024-09-13T16:29:05.636535+0000","last_active":"2024-09-13T16:29:05.639070+0000","last_peered":"2024-09-13T16:29:05.639070+0000","last_clean":"2024-09-13T16:29:05.639070+0000","last_became_active":"2024-09-13T16:29:05.636161+0000","last_became_peered":"2024-09-13T16:29:05.636161+0000","last_unstale":"2024-09-13T16:29:05.639070+0000","last_undegraded":"2024-09-13T16:29:05.639070+0000","last_fullsized":"2024-09-13T16:29:05.639070+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:09.606922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T16:28:43.215475+0000","last_change":"2024-09-13T16:28:39.202222+0000","last_active":"2024-09-13T16:28:43.215475+0000","last_peered":"2024-09-13T16:28:43.215475+0000","last_clean":"2024-09-13T16:28:43.215475+0000","last_became_active":"2024-09-13T16:28:39.201897+0000","last_became_peered":"2024-09-13T16:28:39.201897+0000","last_unstale":"2024-09-13T16:28:43.215475+0000","last_undegraded":"2024-09-13T16:28:43.215475+0000","last_fullsized":"2024-09-13T16:28:43.215475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:41:09.087185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T16:28:43.217236+0000","last_change":"2024-09-13T16:28:39.211164+0000","last_active":"2024-09-13T16:28:43.217236+0000","last_peered":"2024-09-13T16:28:43.217236+0000","last_clean":"2024-09-13T16:28:43.217236+0000","last_became_active":"2024-09-13T16:28:39.210826+0000","last_became_peered":"2024-09-13T16:28:39.210826+0000","last_unstale":"2024-09-13T16:28:43.217236+0000","last_undegraded":"2024-09-13T16:28:43.217236+0000","last_fullsized":"2024-09-13T16:28:43.217236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:40:30.663867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629508+0000","last_change":"2024-09-13T16:28:39.207991+0000","last_active":"2024-09-13T16:29:04.629508+0000","last_peered":"2024-09-13T16:29:04.629508+0000","last_clean":"2024-09-13T16:29:04.629508+0000","last_became_active":"2024-09-13T16:28:39.207332+0000","last_became_peered":"2024-09-13T16:28:39.207332+0000","last_unstale":"2024-09-13T16:29:04.629508+0000","last_undegraded":"2024-09-13T16:29:04.629508+0000","last_fullsized":"2024-09-13T16:29:04.629508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:30:54.271361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":16384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":17802,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":0,"num_read":57,"num_read_kb":46,"num_write":213,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":762,"num_keys_recovered":10,"num_objects_omap":64,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":57746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":187,"ondisk_log_size":187,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250749,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383696,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":487217,"data_compressed":14739,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.32600000000000001}]},{"osd":1,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.376},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.30399999999999999}]},{"osd":2,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.439},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.627}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]},{"osd":4,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.47699999999999998}]},{"osd":5,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.59099999999999997}]},{"osd":6,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.56599999999999995}]},{"osd":7,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.54000000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":9,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.73299999999999998}]},{"osd":10,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.70999999999999996}]}]},{"osd":10,"up_from":72,"seq":309237645334,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1089536,"data_stored":1091645,"data_compressed":54008,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":1,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.50800000000000001}]},{"osd":2,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.626},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.68799999999999994}]},{"osd":3,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":7.2320000000000002,"5min":7.2320000000000002,"15min":7.2320000000000002},"last":0.61099999999999999}]},{"osd":4,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.57699999999999996}]},{"osd":5,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.75},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.64100000000000001}]},{"osd":6,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.70199999999999996}]},{"osd":7,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.53000000000000003}]},{"osd":8,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":7.2569999999999997,"5min":7.2569999999999997,"15min":7.2569999999999997},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.70999999999999996}]},{"osd":9,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.67300000000000004}]},{"osd":11,"last update":"Fri Sep 13 16:28:58 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":7.2199999999999998,"5min":7.2199999999999998,"15min":7.2199999999999998},"last":0.79400000000000004}]}]},{"osd":9,"up_from":67,"seq":287762808858,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":496463,"data_compressed":17719,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.35399999999999998}]},{"osd":1,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.52900000000000003}]},{"osd":3,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.46500000000000002}]},{"osd":4,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.628}]},{"osd":5,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.81100000000000005}]},{"osd":6,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.45700000000000002}]},{"osd":7,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.76500000000000001}]},{"osd":8,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.745}]},{"osd":10,"last update":"Fri Sep 13 16:28:33 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.85299999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":8,"up_from":59,"seq":253403070495,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":1007616,"data_stored":495587,"data_compressed":17719,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.50600000000000001,"15min":0.50800000000000001},"min":{"1min":0.28100000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.70999999999999996,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51700000000000002,"15min":0.51200000000000001},"min":{"1min":0.32800000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.48599999999999999}]},{"osd":1,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.41299999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.40400000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82399999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.60799999999999998}]},{"osd":2,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.41699999999999998,"15min":0.41199999999999998},"min":{"1min":0.28299999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.435}]},{"osd":3,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.78700000000000003,"5min":0.871,"15min":0.871},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.58199999999999996}]},{"osd":4,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56799999999999995,"15min":0.56399999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.747}]},{"osd":5,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53800000000000003,"15min":0.54300000000000004},"min":{"1min":0.29699999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.622}]},{"osd":6,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.628,"15min":0.63},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.65900000000000003,"15min":0.66300000000000003},"min":{"1min":0.38300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68500000000000005}]},{"osd":7,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.66600000000000004,"15min":0.67000000000000004},"min":{"1min":0.47899999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.97699999999999998,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.65800000000000003,"15min":0.66200000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.92200000000000004,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.69699999999999995}]},{"osd":9,"last update":"Fri Sep 13 16:28:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.79500000000000004}]},{"osd":10,"last update":"Fri Sep 13 16:28:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.77600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]}]},{"osd":7,"up_from":51,"seq":219043332132,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":1093979,"data_compressed":54008,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.39600000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.41199999999999998}]},{"osd":1,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.39200000000000002,"15min":0.38900000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.312},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.68600000000000005,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.68999999999999995}]},{"osd":2,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.54400000000000004,"15min":0.54900000000000004},"min":{"1min":0.40699999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.66700000000000004,"5min":0.748,"15min":0.748},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54700000000000004,"15min":0.54800000000000004},"min":{"1min":0.40000000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.60299999999999998}]},{"osd":3,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.42499999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.441},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56799999999999995,"15min":0.56399999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.57499999999999996}]},{"osd":4,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45200000000000001,"15min":0.45100000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.45800000000000002,"15min":0.46300000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.32700000000000001}]},{"osd":5,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57699999999999996,"15min":0.57799999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.496},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.44600000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.54200000000000004}]},{"osd":6,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60899999999999999,"15min":0.60999999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.41699999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.373}]},{"osd":8,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.627,"15min":0.627},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.624},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.66100000000000003,"15min":0.66500000000000004},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.65800000000000003}]},{"osd":9,"last update":"Fri Sep 13 16:28:08 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.64200000000000002}]},{"osd":10,"last update":"Fri Sep 13 16:28:33 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.67600000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]}]},{"osd":6,"up_from":46,"seq":197568495657,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":991232,"data_stored":486747,"data_compressed":14739,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.38300000000000001,"15min":0.372},"min":{"1min":0.26900000000000002,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.39700000000000002,"15min":0.39000000000000001},"min":{"1min":0.28000000000000003,"5min":0.217,"15min":0.217},"max":{"1min":0.61599999999999999,"5min":0.625,"15min":0.625},"last":0.33700000000000002}]},{"osd":1,"last update":"Fri Sep 13 16:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53400000000000003,"15min":0.51200000000000001},"min":{"1min":0.373,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.753,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56100000000000005,"15min":0.54500000000000004},"min":{"1min":0.39700000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.77700000000000002,"5min":0.873,"15min":0.873},"last":0.53300000000000003}]},{"osd":2,"last update":"Fri Sep 13 16:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55300000000000005,"15min":0.54900000000000004},"min":{"1min":0.35199999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83499999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59699999999999998,"15min":0.58799999999999997},"min":{"1min":0.36899999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81399999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.623}]},{"osd":3,"last update":"Fri Sep 13 16:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.73699999999999999,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.499,"15min":0.48899999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.44400000000000001}]},{"osd":4,"last update":"Fri Sep 13 16:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.30299999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.73599999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.42299999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.92300000000000004,"5min":1.079,"15min":1.079},"last":0.49099999999999999}]},{"osd":5,"last update":"Fri Sep 13 16:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61799999999999999,"15min":0.60599999999999998},"min":{"1min":0.47199999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96599999999999997,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.45800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94999999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.54400000000000004}]},{"osd":7,"last update":"Fri Sep 13 16:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63800000000000001,"15min":0.63200000000000001},"min":{"1min":0.40600000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.02,"5min":1.048,"15min":1.048},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.66300000000000003,"15min":0.65200000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.60299999999999998}]},{"osd":8,"last update":"Fri Sep 13 16:28:43 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.69299999999999995,"15min":0.68899999999999995},"min":{"1min":0.505,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72699999999999998,"15min":0.72799999999999998},"min":{"1min":0.46600000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.64000000000000001}]},{"osd":9,"last update":"Fri Sep 13 16:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.66600000000000004}]},{"osd":10,"last update":"Fri Sep 13 16:28:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.70499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]}]},{"osd":5,"up_from":38,"seq":163208757294,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":1088542,"data_compressed":54008,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.52200000000000002,"15min":0.51600000000000001},"min":{"1min":0.377,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76700000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54000000000000004,"15min":0.53500000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.80700000000000005,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.46500000000000002}]},{"osd":1,"last update":"Fri Sep 13 16:28:40 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.48699999999999999,"15min":0.47699999999999998},"min":{"1min":0.308,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.51900000000000002,"15min":0.505},"min":{"1min":0.34399999999999997,"5min":0.245,"15min":0.245},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.41299999999999998}]},{"osd":2,"last update":"Fri Sep 13 16:28:40 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.375,"15min":0.35999999999999999},"min":{"1min":0.254,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.65000000000000002,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.379,"15min":0.36699999999999999},"min":{"1min":0.23100000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.36899999999999999}]},{"osd":3,"last update":"Fri Sep 13 16:28:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.80000000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.55000000000000004}]},{"osd":4,"last update":"Fri Sep 13 16:28:40 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52400000000000002,"15min":0.51100000000000001},"min":{"1min":0.41799999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53400000000000003,"15min":0.52800000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.54000000000000004}]},{"osd":6,"last update":"Fri Sep 13 16:29:03 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.45800000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.434,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.78900000000000003,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.49399999999999999}]},{"osd":7,"last update":"Fri Sep 13 16:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.60799999999999998,"15min":0.60499999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59999999999999998,"15min":0.59699999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.626}]},{"osd":8,"last update":"Fri Sep 13 16:28:46 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52800000000000002,"15min":0.53300000000000003},"min":{"1min":0.34499999999999997,"5min":0.318,"15min":0.318},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.56200000000000006,"15min":0.56899999999999995},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.56200000000000006}]},{"osd":9,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.65700000000000003}]},{"osd":10,"last update":"Fri Sep 13 16:28:31 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.68500000000000005}]}]},{"osd":4,"up_from":33,"seq":141733920819,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":496497,"data_compressed":17719,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.48699999999999999,"15min":0.46300000000000002},"min":{"1min":0.32200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.751,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.504,"15min":0.48199999999999998},"min":{"1min":0.41399999999999998,"5min":0.25,"15min":0.25},"max":{"1min":0.80800000000000005,"5min":0.874,"15min":0.874},"last":0.60499999999999998}]},{"osd":1,"last update":"Fri Sep 13 16:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.34699999999999998,"15min":0.33100000000000002},"min":{"1min":0.24099999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.34300000000000003,"15min":0.33200000000000002},"min":{"1min":0.27800000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.372}]},{"osd":2,"last update":"Fri Sep 13 16:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.495,"15min":0.48599999999999999},"min":{"1min":0.37,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71599999999999997,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.47399999999999998,"15min":0.45500000000000002},"min":{"1min":0.34399999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67200000000000004,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.59299999999999997}]},{"osd":3,"last update":"Fri Sep 13 16:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.504,"15min":0.48599999999999999},"min":{"1min":0.38700000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.505},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.52100000000000002,"15min":0.496},"min":{"1min":0.39900000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.79300000000000004,"5min":0.875,"15min":0.875},"last":0.56799999999999995}]},{"osd":5,"last update":"Fri Sep 13 16:28:38 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.33800000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54500000000000004,"15min":0.53100000000000003},"min":{"1min":0.39000000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80700000000000005,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Sep 13 16:28:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.54900000000000004,"15min":0.53500000000000003},"min":{"1min":0.47599999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.73299999999999998,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57699999999999996,"15min":0.56599999999999995},"min":{"1min":0.41699999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80400000000000005,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.63}]},{"osd":7,"last update":"Fri Sep 13 16:28:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.495,"15min":0.48099999999999998},"min":{"1min":0.33600000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.85099999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.49399999999999999,"15min":0.48999999999999999},"min":{"1min":0.34999999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.81799999999999995,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.621}]},{"osd":8,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.44,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86899999999999999,"5min":0.872,"15min":0.872},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.61499999999999999,"15min":0.60999999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.872,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.64500000000000002}]},{"osd":9,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.72599999999999998}]},{"osd":10,"last update":"Fri Sep 13 16:28:32 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.65400000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36188,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708964,"statfs":{"total":95995035648,"available":95957979136,"internally_reserved":0,"allocated":999424,"data_stored":486172,"data_compressed":14739,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:56 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.34899999999999998,"15min":0.32500000000000001},"min":{"1min":0.252,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.63,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.35699999999999998,"15min":0.32900000000000001},"min":{"1min":0.27600000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52300000000000002,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.36599999999999999}]},{"osd":1,"last update":"Fri Sep 13 16:28:56 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.54600000000000004,"15min":0.48199999999999998},"min":{"1min":0.48899999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.84299999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53500000000000003,"15min":0.48699999999999999},"min":{"1min":0.42399999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.77000000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.68799999999999994}]},{"osd":2,"last update":"Fri Sep 13 16:28:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52200000000000002,"15min":0.51400000000000001},"min":{"1min":0.375,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79100000000000004,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55300000000000005,"15min":0.52600000000000002},"min":{"1min":0.38600000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.94599999999999995,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.53600000000000003}]},{"osd":4,"last update":"Fri Sep 13 16:28:09 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.51500000000000001,"15min":0.49099999999999999},"min":{"1min":0.40400000000000003,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.56699999999999995,"15min":0.54000000000000004},"min":{"1min":0.44700000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.62}]},{"osd":5,"last update":"Fri Sep 13 16:28:36 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58299999999999996,"15min":0.57199999999999995},"min":{"1min":0.433,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.86299999999999999,"5min":1.038,"15min":1.038},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59699999999999998,"15min":0.59599999999999997},"min":{"1min":0.45900000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.73799999999999999,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.72099999999999997}]},{"osd":6,"last update":"Fri Sep 13 16:29:01 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.505,"15min":0.496},"min":{"1min":0.33600000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.48599999999999999,"15min":0.47999999999999998},"min":{"1min":0.34599999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74099999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.498}]},{"osd":7,"last update":"Fri Sep 13 16:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.49399999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.60699999999999998,"15min":0.59299999999999997},"min":{"1min":0.52100000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.64100000000000001}]},{"osd":8,"last update":"Fri Sep 13 16:27:45 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.71499999999999997}]},{"osd":9,"last update":"Fri Sep 13 16:28:08 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.56499999999999995}]},{"osd":10,"last update":"Fri Sep 13 16:28:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.70299999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":1032192,"data_stored":520251,"data_compressed":17719,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.60399999999999998,"15min":0.498},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.90900000000000003,"5min":16.027999999999999,"15min":16.027999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.47299999999999998,"15min":0.46500000000000002},"min":{"1min":0.309,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.69599999999999995}]},{"osd":1,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.48399999999999999,"15min":0.46500000000000002},"min":{"1min":0.372,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.51500000000000001,"15min":0.49299999999999999},"min":{"1min":0.44500000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.83399999999999996}]},{"osd":3,"last update":"Fri Sep 13 16:28:52 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.54000000000000004,"15min":0.51400000000000001},"min":{"1min":0.43099999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54000000000000004,"15min":0.51500000000000001},"min":{"1min":0.36799999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.59099999999999997}]},{"osd":4,"last update":"Fri Sep 13 16:28:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":0.623,"15min":0.55500000000000005},"min":{"1min":0.36399999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":9.6790000000000003,"5min":9.6790000000000003,"15min":9.6790000000000003},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.54500000000000004,"15min":0.53600000000000003},"min":{"1min":0.34399999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.75900000000000001}]},{"osd":5,"last update":"Fri Sep 13 16:28:39 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.40999999999999998,"15min":0.40200000000000002},"min":{"1min":0.33700000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.432},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.40300000000000002,"15min":0.39200000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.46100000000000002}]},{"osd":6,"last update":"Fri Sep 13 16:29:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.57699999999999996,"15min":0.55200000000000005},"min":{"1min":0.437,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.60599999999999998,"15min":0.58399999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.79400000000000004}]},{"osd":7,"last update":"Fri Sep 13 16:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.379,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59099999999999997,"15min":0.58299999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.748}]},{"osd":8,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.52600000000000002,"15min":0.51800000000000002},"min":{"1min":0.33500000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.501},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.51700000000000002,"15min":0.50800000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.55100000000000005}]},{"osd":9,"last update":"Fri Sep 13 16:29:06 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.67900000000000005,"15min":0.66900000000000004},"min":{"1min":0.45100000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.016,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.68100000000000005,"15min":0.67100000000000004},"min":{"1min":0.50600000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.85299999999999998}]},{"osd":10,"last update":"Fri Sep 13 16:28:34 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.89400000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":491382,"data_compressed":14739,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:09 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.47999999999999998,"15min":0.47799999999999998},"min":{"1min":0.34100000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.497},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.48399999999999999,"15min":0.48099999999999998},"min":{"1min":0.38100000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.435}]},{"osd":2,"last update":"Fri Sep 13 16:28:27 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.67100000000000004,"15min":0.54100000000000004},"min":{"1min":0.34100000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.81200000000000006,"5min":17.655000000000001,"15min":17.655000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.49099999999999999,"15min":0.47899999999999998},"min":{"1min":0.29299999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.76200000000000001}]},{"osd":3,"last update":"Fri Sep 13 16:28:51 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52500000000000002,"15min":0.50700000000000001},"min":{"1min":0.379,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72599999999999998,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.55800000000000005,"15min":0.52200000000000002},"min":{"1min":0.42199999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.73899999999999999}]},{"osd":4,"last update":"Fri Sep 13 16:28:13 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.40400000000000003,"15min":0.38},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.433,"5min":0.40899999999999997,"15min":0.39100000000000001},"min":{"1min":0.28299999999999997,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.57899999999999996}]},{"osd":5,"last update":"Fri Sep 13 16:28:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":1.0609999999999999,"15min":1.2829999999999999},"min":{"1min":0.38400000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.82499999999999996,"5min":17.640000000000001,"15min":17.640000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54600000000000004,"15min":0.54300000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.72499999999999998}]},{"osd":6,"last update":"Fri Sep 13 16:29:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59299999999999997,"15min":0.59399999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.72799999999999998,"5min":0.872,"15min":0.872},"last":0.60899999999999999}]},{"osd":7,"last update":"Fri Sep 13 16:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.372,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.78900000000000003}]},{"osd":8,"last update":"Fri Sep 13 16:28:43 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.60399999999999998,"15min":0.61099999999999999},"min":{"1min":0.40699999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.45500000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.83499999999999996}]},{"osd":9,"last update":"Fri Sep 13 16:28:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.751}]},{"osd":10,"last update":"Fri Sep 13 16:28:34 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.82199999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":477257,"data_compressed":11759,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 16:28:15 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.46999999999999997,"15min":0.46899999999999997},"min":{"1min":0.37,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.755,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.50700000000000001,"15min":0.495},"min":{"1min":0.41199999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81000000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.60899999999999999}]},{"osd":2,"last update":"Fri Sep 13 16:28:33 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.55000000000000004,"15min":0.52600000000000002},"min":{"1min":0.42699999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73799999999999999,"5min":1.111,"15min":1.111},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.54800000000000004,"15min":0.56100000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.621,"5min":1.698,"15min":1.698},"last":0.40000000000000002}]},{"osd":3,"last update":"Fri Sep 13 16:27:47 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.438,"15min":0.42399999999999999},"min":{"1min":0.38200000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.42299999999999999,"15min":0.39800000000000002},"min":{"1min":0.22500000000000001,"5min":0.217,"15min":0.217},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.46500000000000002}]},{"osd":4,"last update":"Fri Sep 13 16:28:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.48199999999999998,"15min":0.45400000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.625},{"interface":"front","average":{"1min":0.629,"5min":0.51500000000000001,"15min":0.48399999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.65400000000000003}]},{"osd":5,"last update":"Fri Sep 13 16:28:33 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.55300000000000005,"15min":0.54100000000000004},"min":{"1min":0.44,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56799999999999995,"15min":0.55600000000000005},"min":{"1min":0.46899999999999997,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77800000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.36299999999999999}]},{"osd":6,"last update":"Fri Sep 13 16:28:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.504,"15min":0.48199999999999998},"min":{"1min":0.29899999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.66500000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55300000000000005,"15min":0.52100000000000002},"min":{"1min":0.34399999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.74299999999999999,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.501}]},{"osd":7,"last update":"Fri Sep 13 16:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.68300000000000005,"15min":0.67500000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96999999999999997,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.63700000000000001,"15min":0.63100000000000001},"min":{"1min":0.43099999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.58499999999999996}]},{"osd":8,"last update":"Fri Sep 13 16:28:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1,"5min":1,"15min":1},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69899999999999995,"15min":0.70299999999999996},"min":{"1min":0.55000000000000004,"5min":0.497,"15min":0.497},"max":{"1min":0.81799999999999995,"5min":1.004,"15min":1.004},"last":0.54900000000000004}]},{"osd":9,"last update":"Fri Sep 13 16:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.53500000000000003}]},{"osd":10,"last update":"Fri Sep 13 16:28:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.68899999999999995}]},{"osd":11,"last update":"Fri Sep 13 16:28:58 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.66900000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":25183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":12476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T16:29:09.340 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T16:29:09.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.476+0000 7f8d3fd5a640 1 -- 172.21.15.16:0/2033282839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d380fecd0 msgr2=0x7f8d380ff0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:09.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.476+0000 7f8d3fd5a640 1 --2- 172.21.15.16:0/2033282839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d380fecd0 0x7f8d380ff0d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8d34009ca0 tx=0x7f8d3402a8c0 comp rx=0 tx=0).stop 2024-09-13T16:29:09.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.476+0000 7f8d3fd5a640 1 -- 172.21.15.16:0/2033282839 shutdown_connections 2024-09-13T16:29:09.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.476+0000 7f8d3fd5a640 1 --2- 172.21.15.16:0/2033282839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d38100880 0x7f8d38107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:09.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.476+0000 7f8d3fd5a640 1 --2- 172.21.15.16:0/2033282839 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d380ffec0 0x7f8d38100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:09.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.476+0000 7f8d3fd5a640 1 --2- 172.21.15.16:0/2033282839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d380fecd0 0x7f8d380ff0d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:09.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.476+0000 7f8d3fd5a640 1 -- 172.21.15.16:0/2033282839 >> 172.21.15.16:0/2033282839 conn(0x7f8d380fa430 msgr2=0x7f8d380fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:09.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.477+0000 7f8d3fd5a640 1 -- 172.21.15.16:0/2033282839 shutdown_connections 2024-09-13T16:29:09.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.477+0000 7f8d3fd5a640 1 -- 172.21.15.16:0/2033282839 wait complete. 2024-09-13T16:29:09.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.477+0000 7f8d3fd5a640 1 Processor -- start 2024-09-13T16:29:09.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.477+0000 7f8d3fd5a640 1 -- start start 2024-09-13T16:29:09.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.477+0000 7f8d3fd5a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d380fecd0 0x7f8d38069160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:09.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.477+0000 7f8d3fd5a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d380ffec0 0x7f8d3819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:09.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.477+0000 7f8d3fd5a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d38100880 0x7f8d3819d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:09.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.477+0000 7f8d3fd5a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3819d8e0 con 0x7f8d380ffec0 2024-09-13T16:29:09.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.477+0000 7f8d3fd5a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3819da50 con 0x7f8d38100880 2024-09-13T16:29:09.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.477+0000 7f8d3fd5a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3819dc10 con 0x7f8d380fecd0 2024-09-13T16:29:09.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.477+0000 7f8d3d2f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d380ffec0 0x7f8d3819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:09.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.477+0000 7f8d3d2f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d380ffec0 0x7f8d3819aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49730/0 (socket says 172.21.15.16:49730) 2024-09-13T16:29:09.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.477+0000 7f8d3d2f5640 1 -- 172.21.15.16:0/1468728648 learned_addr learned my addr 172.21.15.16:0/1468728648 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:09.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.478+0000 7f8d3daf6640 1 --2- 172.21.15.16:0/1468728648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d380fecd0 0x7f8d38069160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:09.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.478+0000 7f8d3e2f7640 1 --2- 172.21.15.16:0/1468728648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d38100880 0x7f8d3819d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:09.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.478+0000 7f8d3d2f5640 1 -- 172.21.15.16:0/1468728648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d380fecd0 msgr2=0x7f8d38069160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:09.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.478+0000 7f8d3d2f5640 1 --2- 172.21.15.16:0/1468728648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d380fecd0 0x7f8d38069160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:09.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.478+0000 7f8d3d2f5640 1 -- 172.21.15.16:0/1468728648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d38100880 msgr2=0x7f8d3819d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:09.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.478+0000 7f8d3d2f5640 1 --2- 172.21.15.16:0/1468728648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d38100880 0x7f8d3819d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:09.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.478+0000 7f8d3d2f5640 1 -- 172.21.15.16:0/1468728648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d34009900 con 0x7f8d380ffec0 2024-09-13T16:29:09.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.478+0000 7f8d3e2f7640 1 --2- 172.21.15.16:0/1468728648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d38100880 0x7f8d3819d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:29:09.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.478+0000 7f8d3d2f5640 1 --2- 172.21.15.16:0/1468728648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d380ffec0 0x7f8d3819aae0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8d2800e9f0 tx=0x7f8d2800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:09.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.478+0000 7f8d2effd640 1 -- 172.21.15.16:0/1468728648 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d28009720 con 0x7f8d380ffec0 2024-09-13T16:29:09.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.478+0000 7f8d2effd640 1 -- 172.21.15.16:0/1468728648 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d2800cd80 con 0x7f8d380ffec0 2024-09-13T16:29:09.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.478+0000 7f8d2effd640 1 -- 172.21.15.16:0/1468728648 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d28010430 con 0x7f8d380ffec0 2024-09-13T16:29:09.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.478+0000 7f8d3fd5a640 1 -- 172.21.15.16:0/1468728648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d3819de90 con 0x7f8d380ffec0 2024-09-13T16:29:09.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.478+0000 7f8d3fd5a640 1 -- 172.21.15.16:0/1468728648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d3819e2c0 con 0x7f8d380ffec0 2024-09-13T16:29:09.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.479+0000 7f8d3fd5a640 1 -- 172.21.15.16:0/1468728648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d381062c0 con 0x7f8d380ffec0 2024-09-13T16:29:09.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.481+0000 7f8d2effd640 1 -- 172.21.15.16:0/1468728648 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8d28004f90 con 0x7f8d380ffec0 2024-09-13T16:29:09.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.482+0000 7f8d2effd640 1 --2- 172.21.15.16:0/1468728648 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8d240a1110 0x7f8d240a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:09.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.482+0000 7f8d3daf6640 1 --2- 172.21.15.16:0/1468728648 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8d240a1110 0x7f8d240a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:09.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.482+0000 7f8d2effd640 1 -- 172.21.15.16:0/1468728648 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8d28014070 con 0x7f8d380ffec0 2024-09-13T16:29:09.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.482+0000 7f8d3daf6640 1 --2- 172.21.15.16:0/1468728648 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8d240a1110 0x7f8d240a35d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8d34000f80 tx=0x7f8d34031ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:09.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.485+0000 7f8d2effd640 1 -- 172.21.15.16:0/1468728648 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f8d28096690 con 0x7f8d380ffec0 2024-09-13T16:29:09.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.596+0000 7f8d2effd640 1 -- 172.21.15.16:0/1468728648 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f8d28096a60 con 0x7f8d380ffec0 2024-09-13T16:29:09.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.663+0000 7f8d3fd5a640 1 -- 172.21.15.16:0/1468728648 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8d3802c4f0 con 0x7f8d240a1110 2024-09-13T16:29:09.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.675+0000 7f8d2effd640 1 -- 172.21.15.16:0/1468728648 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422976 (secure 0 0 0) 0x7f8d3802c4f0 con 0x7f8d240a1110 2024-09-13T16:29:09.680 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:29:09.683 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-13T16:29:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.679+0000 7f8d3fd5a640 1 -- 172.21.15.16:0/1468728648 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8d240a1110 msgr2=0x7f8d240a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.679+0000 7f8d3fd5a640 1 --2- 172.21.15.16:0/1468728648 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8d240a1110 0x7f8d240a35d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8d34000f80 tx=0x7f8d34031ab0 comp rx=0 tx=0).stop 2024-09-13T16:29:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.679+0000 7f8d3fd5a640 1 -- 172.21.15.16:0/1468728648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d380ffec0 msgr2=0x7f8d3819aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.679+0000 7f8d3fd5a640 1 --2- 172.21.15.16:0/1468728648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d380ffec0 0x7f8d3819aae0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8d2800e9f0 tx=0x7f8d2800edf0 comp rx=0 tx=0).stop 2024-09-13T16:29:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.679+0000 7f8d3fd5a640 1 -- 172.21.15.16:0/1468728648 shutdown_connections 2024-09-13T16:29:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.679+0000 7f8d3fd5a640 1 --2- 172.21.15.16:0/1468728648 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8d240a1110 0x7f8d240a35d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.679+0000 7f8d3fd5a640 1 --2- 172.21.15.16:0/1468728648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d38100880 0x7f8d3819d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:09.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.679+0000 7f8d3fd5a640 1 --2- 172.21.15.16:0/1468728648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d380ffec0 0x7f8d3819aae0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:09.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.679+0000 7f8d3fd5a640 1 --2- 172.21.15.16:0/1468728648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d380fecd0 0x7f8d38069160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:09.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.679+0000 7f8d3fd5a640 1 -- 172.21.15.16:0/1468728648 >> 172.21.15.16:0/1468728648 conn(0x7f8d380fa430 msgr2=0x7f8d38103490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:09.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.679+0000 7f8d3fd5a640 1 -- 172.21.15.16:0/1468728648 shutdown_connections 2024-09-13T16:29:09.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.679+0000 7f8d3fd5a640 1 -- 172.21.15.16:0/1468728648 wait complete. 2024-09-13T16:29:09.691 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-13T16:29:08.571055+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":608404,"num_objects":100,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":183,"num_read_kb":155,"num_write":396,"num_write_kb":3018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1181498,"num_keys_recovered":10,"num_objects_omap":64,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":417,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364888,"kb_used_data":11980,"kb_used_omap":18,"kb_used_meta":352749,"kb_avail":1124576936,"statfs":{"total":1151940427776,"available":1151566782464,"internally_reserved":0,"allocated":12267520,"data_stored":7711739,"data_compressed":303615,"data_compressed_allocated":405504,"data_compressed_original":2310144,"omap_allocated":19131,"internal_metadata":361215301},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13443,"num_objects":76,"num_object_clones":0,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":57,"num_read_kb":46,"num_write":167,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":762,"num_keys_recovered":10,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002208"},"pg_stats":[{"pgid":"2.1f","version":"91'13","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.648333+0000","last_change":"2024-09-13T16:29:05.647872+0000","last_active":"2024-09-13T16:29:05.648333+0000","last_peered":"2024-09-13T16:29:05.648333+0000","last_clean":"2024-09-13T16:29:05.648333+0000","last_became_active":"2024-09-13T16:29:05.637050+0000","last_became_peered":"2024-09-13T16:29:05.637050+0000","last_unstale":"2024-09-13T16:29:05.648333+0000","last_undegraded":"2024-09-13T16:29:05.648333+0000","last_fullsized":"2024-09-13T16:29:05.648333+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:04.113840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.045098+0000","last_change":"2024-09-13T16:28:39.210024+0000","last_active":"2024-09-13T16:28:44.045098+0000","last_peered":"2024-09-13T16:28:44.045098+0000","last_clean":"2024-09-13T16:28:44.045098+0000","last_became_active":"2024-09-13T16:28:39.209751+0000","last_became_peered":"2024-09-13T16:28:39.209751+0000","last_unstale":"2024-09-13T16:28:44.045098+0000","last_undegraded":"2024-09-13T16:28:44.045098+0000","last_fullsized":"2024-09-13T16:28:44.045098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:34:24.357779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.632148+0000","last_change":"2024-09-13T16:28:43.230223+0000","last_active":"2024-09-13T16:29:04.632148+0000","last_peered":"2024-09-13T16:29:04.632148+0000","last_clean":"2024-09-13T16:29:04.632148+0000","last_became_active":"2024-09-13T16:28:43.228992+0000","last_became_peered":"2024-09-13T16:28:43.228992+0000","last_unstale":"2024-09-13T16:29:04.632148+0000","last_undegraded":"2024-09-13T16:29:04.632148+0000","last_fullsized":"2024-09-13T16:29:04.632148+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:45:37.252865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:29:03.591838+0000","last_change":"2024-09-13T16:28:38.850198+0000","last_active":"2024-09-13T16:29:03.591838+0000","last_peered":"2024-09-13T16:29:03.591838+0000","last_clean":"2024-09-13T16:29:03.591838+0000","last_became_active":"2024-09-13T16:28:38.849780+0000","last_became_peered":"2024-09-13T16:28:38.849780+0000","last_unstale":"2024-09-13T16:29:03.591838+0000","last_undegraded":"2024-09-13T16:29:03.591838+0000","last_fullsized":"2024-09-13T16:29:03.591838+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:09.653474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T16:28:43.218832+0000","last_change":"2024-09-13T16:28:39.203030+0000","last_active":"2024-09-13T16:28:43.218832+0000","last_peered":"2024-09-13T16:28:43.218832+0000","last_clean":"2024-09-13T16:28:43.218832+0000","last_became_active":"2024-09-13T16:28:39.202464+0000","last_became_peered":"2024-09-13T16:28:39.202464+0000","last_unstale":"2024-09-13T16:28:43.218832+0000","last_undegraded":"2024-09-13T16:28:43.218832+0000","last_fullsized":"2024-09-13T16:28:43.218832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:57.528836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.632329+0000","last_change":"2024-09-13T16:28:43.224358+0000","last_active":"2024-09-13T16:29:04.632329+0000","last_peered":"2024-09-13T16:29:04.632329+0000","last_clean":"2024-09-13T16:29:04.632329+0000","last_became_active":"2024-09-13T16:28:43.223955+0000","last_became_peered":"2024-09-13T16:28:43.223955+0000","last_unstale":"2024-09-13T16:29:04.632329+0000","last_undegraded":"2024-09-13T16:29:04.632329+0000","last_fullsized":"2024-09-13T16:29:04.632329+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:53:09.169704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'11","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.634720+0000","last_change":"2024-09-13T16:28:38.846051+0000","last_active":"2024-09-13T16:29:05.634720+0000","last_peered":"2024-09-13T16:29:05.634720+0000","last_clean":"2024-09-13T16:29:05.634720+0000","last_became_active":"2024-09-13T16:28:38.845762+0000","last_became_peered":"2024-09-13T16:28:38.845762+0000","last_unstale":"2024-09-13T16:29:05.634720+0000","last_undegraded":"2024-09-13T16:29:05.634720+0000","last_fullsized":"2024-09-13T16:29:05.634720+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:33:47.048582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044544+0000","last_change":"2024-09-13T16:28:39.210516+0000","last_active":"2024-09-13T16:28:44.044544+0000","last_peered":"2024-09-13T16:28:44.044544+0000","last_clean":"2024-09-13T16:28:44.044544+0000","last_became_active":"2024-09-13T16:28:39.210214+0000","last_became_peered":"2024-09-13T16:28:39.210214+0000","last_unstale":"2024-09-13T16:28:44.044544+0000","last_undegraded":"2024-09-13T16:28:44.044544+0000","last_fullsized":"2024-09-13T16:28:44.044544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:51:58.212685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.630396+0000","last_change":"2024-09-13T16:28:43.230530+0000","last_active":"2024-09-13T16:29:04.630396+0000","last_peered":"2024-09-13T16:29:04.630396+0000","last_clean":"2024-09-13T16:29:04.630396+0000","last_became_active":"2024-09-13T16:28:43.230154+0000","last_became_peered":"2024-09-13T16:28:43.230154+0000","last_unstale":"2024-09-13T16:29:04.630396+0000","last_undegraded":"2024-09-13T16:29:04.630396+0000","last_fullsized":"2024-09-13T16:29:04.630396+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:39:52.362177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.653834+0000","last_change":"2024-09-13T16:29:05.653460+0000","last_active":"2024-09-13T16:29:05.653834+0000","last_peered":"2024-09-13T16:29:05.653834+0000","last_clean":"2024-09-13T16:29:05.653834+0000","last_became_active":"2024-09-13T16:29:05.641836+0000","last_became_peered":"2024-09-13T16:29:05.641836+0000","last_unstale":"2024-09-13T16:29:05.653834+0000","last_undegraded":"2024-09-13T16:29:05.653834+0000","last_fullsized":"2024-09-13T16:29:05.653834+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:22:30.051033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.624846+0000","last_change":"2024-09-13T16:28:39.202381+0000","last_active":"2024-09-13T16:29:04.624846+0000","last_peered":"2024-09-13T16:29:04.624846+0000","last_clean":"2024-09-13T16:29:04.624846+0000","last_became_active":"2024-09-13T16:28:39.202023+0000","last_became_peered":"2024-09-13T16:28:39.202023+0000","last_unstale":"2024-09-13T16:29:04.624846+0000","last_undegraded":"2024-09-13T16:29:04.624846+0000","last_fullsized":"2024-09-13T16:29:04.624846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:55:46.445085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628398+0000","last_change":"2024-09-13T16:28:43.230623+0000","last_active":"2024-09-13T16:29:04.628398+0000","last_peered":"2024-09-13T16:29:04.628398+0000","last_clean":"2024-09-13T16:29:04.628398+0000","last_became_active":"2024-09-13T16:28:43.230240+0000","last_became_peered":"2024-09-13T16:28:43.230240+0000","last_unstale":"2024-09-13T16:29:04.628398+0000","last_undegraded":"2024-09-13T16:29:04.628398+0000","last_fullsized":"2024-09-13T16:29:04.628398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:04:07.723269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625334+0000","last_change":"2024-09-13T16:28:39.201896+0000","last_active":"2024-09-13T16:29:04.625334+0000","last_peered":"2024-09-13T16:29:04.625334+0000","last_clean":"2024-09-13T16:29:04.625334+0000","last_became_active":"2024-09-13T16:28:39.201570+0000","last_became_peered":"2024-09-13T16:28:39.201570+0000","last_unstale":"2024-09-13T16:29:04.625334+0000","last_undegraded":"2024-09-13T16:29:04.625334+0000","last_fullsized":"2024-09-13T16:29:04.625334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:07.532058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627167+0000","last_change":"2024-09-13T16:28:38.845998+0000","last_active":"2024-09-13T16:29:04.627167+0000","last_peered":"2024-09-13T16:29:04.627167+0000","last_clean":"2024-09-13T16:29:04.627167+0000","last_became_active":"2024-09-13T16:28:38.845451+0000","last_became_peered":"2024-09-13T16:28:38.845451+0000","last_unstale":"2024-09-13T16:29:04.627167+0000","last_undegraded":"2024-09-13T16:29:04.627167+0000","last_fullsized":"2024-09-13T16:29:04.627167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:15.696276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628820+0000","last_change":"2024-09-13T16:28:43.293598+0000","last_active":"2024-09-13T16:29:04.628820+0000","last_peered":"2024-09-13T16:29:04.628820+0000","last_clean":"2024-09-13T16:29:04.628820+0000","last_became_active":"2024-09-13T16:28:43.293045+0000","last_became_peered":"2024-09-13T16:28:43.293045+0000","last_unstale":"2024-09-13T16:29:04.628820+0000","last_undegraded":"2024-09-13T16:29:04.628820+0000","last_fullsized":"2024-09-13T16:29:04.628820+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:01.839626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044938+0000","last_change":"2024-09-13T16:28:39.214302+0000","last_active":"2024-09-13T16:28:44.044938+0000","last_peered":"2024-09-13T16:28:44.044938+0000","last_clean":"2024-09-13T16:28:44.044938+0000","last_became_active":"2024-09-13T16:28:39.213672+0000","last_became_peered":"2024-09-13T16:28:39.213672+0000","last_unstale":"2024-09-13T16:28:44.044938+0000","last_undegraded":"2024-09-13T16:28:44.044938+0000","last_fullsized":"2024-09-13T16:28:44.044938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:03:25.440603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"91'2","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.631445+0000","last_change":"2024-09-13T16:28:38.840626+0000","last_active":"2024-09-13T16:29:05.631445+0000","last_peered":"2024-09-13T16:29:05.631445+0000","last_clean":"2024-09-13T16:29:05.631445+0000","last_became_active":"2024-09-13T16:28:38.840344+0000","last_became_peered":"2024-09-13T16:28:38.840344+0000","last_unstale":"2024-09-13T16:29:05.631445+0000","last_undegraded":"2024-09-13T16:29:05.631445+0000","last_fullsized":"2024-09-13T16:29:05.631445+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:03:10.850301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.633381+0000","last_change":"2024-09-13T16:28:43.229163+0000","last_active":"2024-09-13T16:29:05.633381+0000","last_peered":"2024-09-13T16:29:05.633381+0000","last_clean":"2024-09-13T16:29:05.633381+0000","last_became_active":"2024-09-13T16:28:43.228783+0000","last_became_peered":"2024-09-13T16:28:43.228783+0000","last_unstale":"2024-09-13T16:29:05.633381+0000","last_undegraded":"2024-09-13T16:29:05.633381+0000","last_fullsized":"2024-09-13T16:29:05.633381+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:48.183837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627877+0000","last_change":"2024-09-13T16:28:39.209247+0000","last_active":"2024-09-13T16:29:04.627877+0000","last_peered":"2024-09-13T16:29:04.627877+0000","last_clean":"2024-09-13T16:29:04.627877+0000","last_became_active":"2024-09-13T16:28:39.208921+0000","last_became_peered":"2024-09-13T16:28:39.208921+0000","last_unstale":"2024-09-13T16:29:04.627877+0000","last_undegraded":"2024-09-13T16:29:04.627877+0000","last_fullsized":"2024-09-13T16:29:04.627877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:32:41.383029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.640902+0000","last_change":"2024-09-13T16:28:38.845904+0000","last_active":"2024-09-13T16:29:04.640902+0000","last_peered":"2024-09-13T16:29:04.640902+0000","last_clean":"2024-09-13T16:29:04.640902+0000","last_became_active":"2024-09-13T16:28:38.845639+0000","last_became_peered":"2024-09-13T16:28:38.845639+0000","last_unstale":"2024-09-13T16:29:04.640902+0000","last_undegraded":"2024-09-13T16:29:04.640902+0000","last_fullsized":"2024-09-13T16:29:04.640902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:25:34.148607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.640627+0000","last_change":"2024-09-13T16:29:05.640063+0000","last_active":"2024-09-13T16:29:05.640627+0000","last_peered":"2024-09-13T16:29:05.640627+0000","last_clean":"2024-09-13T16:29:05.640627+0000","last_became_active":"2024-09-13T16:29:05.639596+0000","last_became_peered":"2024-09-13T16:29:05.639596+0000","last_unstale":"2024-09-13T16:29:05.640627+0000","last_undegraded":"2024-09-13T16:29:05.640627+0000","last_fullsized":"2024-09-13T16:29:05.640627+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:41:38.351162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.626859+0000","last_change":"2024-09-13T16:28:39.212947+0000","last_active":"2024-09-13T16:29:04.626859+0000","last_peered":"2024-09-13T16:29:04.626859+0000","last_clean":"2024-09-13T16:29:04.626859+0000","last_became_active":"2024-09-13T16:28:39.212622+0000","last_became_peered":"2024-09-13T16:28:39.212622+0000","last_unstale":"2024-09-13T16:29:04.626859+0000","last_undegraded":"2024-09-13T16:29:04.626859+0000","last_fullsized":"2024-09-13T16:29:04.626859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:16.425154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:29:02.581304+0000","last_change":"2024-09-13T16:28:38.846173+0000","last_active":"2024-09-13T16:29:02.581304+0000","last_peered":"2024-09-13T16:29:02.581304+0000","last_clean":"2024-09-13T16:29:02.581304+0000","last_became_active":"2024-09-13T16:28:38.845873+0000","last_became_peered":"2024-09-13T16:28:38.845873+0000","last_unstale":"2024-09-13T16:29:02.581304+0000","last_undegraded":"2024-09-13T16:29:02.581304+0000","last_fullsized":"2024-09-13T16:29:02.581304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:19:55.817524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629006+0000","last_change":"2024-09-13T16:28:43.294347+0000","last_active":"2024-09-13T16:29:04.629006+0000","last_peered":"2024-09-13T16:29:04.629006+0000","last_clean":"2024-09-13T16:29:04.629006+0000","last_became_active":"2024-09-13T16:28:43.293902+0000","last_became_peered":"2024-09-13T16:28:43.293902+0000","last_unstale":"2024-09-13T16:29:04.629006+0000","last_undegraded":"2024-09-13T16:29:04.629006+0000","last_fullsized":"2024-09-13T16:29:04.629006+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:27:37.470163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T16:28:43.217326+0000","last_change":"2024-09-13T16:28:39.205817+0000","last_active":"2024-09-13T16:28:43.217326+0000","last_peered":"2024-09-13T16:28:43.217326+0000","last_clean":"2024-09-13T16:28:43.217326+0000","last_became_active":"2024-09-13T16:28:39.205534+0000","last_became_peered":"2024-09-13T16:28:39.205534+0000","last_unstale":"2024-09-13T16:28:43.217326+0000","last_undegraded":"2024-09-13T16:28:43.217326+0000","last_fullsized":"2024-09-13T16:28:43.217326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:09:16.294842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.633540+0000","last_change":"2024-09-13T16:28:38.847582+0000","last_active":"2024-09-13T16:29:05.633540+0000","last_peered":"2024-09-13T16:29:05.633540+0000","last_clean":"2024-09-13T16:29:05.633540+0000","last_became_active":"2024-09-13T16:28:38.847353+0000","last_became_peered":"2024-09-13T16:28:38.847353+0000","last_unstale":"2024-09-13T16:29:05.633540+0000","last_undegraded":"2024-09-13T16:29:05.633540+0000","last_fullsized":"2024-09-13T16:29:05.633540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:24.944088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.634008+0000","last_change":"2024-09-13T16:28:43.225202+0000","last_active":"2024-09-13T16:29:04.634008+0000","last_peered":"2024-09-13T16:29:04.634008+0000","last_clean":"2024-09-13T16:29:04.634008+0000","last_became_active":"2024-09-13T16:28:43.224802+0000","last_became_peered":"2024-09-13T16:28:43.224802+0000","last_unstale":"2024-09-13T16:29:04.634008+0000","last_undegraded":"2024-09-13T16:29:04.634008+0000","last_fullsized":"2024-09-13T16:29:04.634008+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:51:19.325859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.045516+0000","last_change":"2024-09-13T16:28:39.209630+0000","last_active":"2024-09-13T16:28:44.045516+0000","last_peered":"2024-09-13T16:28:44.045516+0000","last_clean":"2024-09-13T16:28:44.045516+0000","last_became_active":"2024-09-13T16:28:39.209303+0000","last_became_peered":"2024-09-13T16:28:39.209303+0000","last_unstale":"2024-09-13T16:28:44.045516+0000","last_undegraded":"2024-09-13T16:28:44.045516+0000","last_fullsized":"2024-09-13T16:28:44.045516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:01:31.822673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'7","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628093+0000","last_change":"2024-09-13T16:28:38.837858+0000","last_active":"2024-09-13T16:29:04.628093+0000","last_peered":"2024-09-13T16:29:04.628093+0000","last_clean":"2024-09-13T16:29:04.628093+0000","last_became_active":"2024-09-13T16:28:38.837378+0000","last_became_peered":"2024-09-13T16:28:38.837378+0000","last_unstale":"2024-09-13T16:29:04.628093+0000","last_undegraded":"2024-09-13T16:29:04.628093+0000","last_fullsized":"2024-09-13T16:29:04.628093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:59:41.055902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.754728+0000","last_change":"2024-09-13T16:28:43.294241+0000","last_active":"2024-09-13T16:28:44.754728+0000","last_peered":"2024-09-13T16:28:44.754728+0000","last_clean":"2024-09-13T16:28:44.754728+0000","last_became_active":"2024-09-13T16:28:43.293873+0000","last_became_peered":"2024-09-13T16:28:43.293873+0000","last_unstale":"2024-09-13T16:28:44.754728+0000","last_undegraded":"2024-09-13T16:28:44.754728+0000","last_fullsized":"2024-09-13T16:28:44.754728+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:34.018162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044782+0000","last_change":"2024-09-13T16:28:39.218071+0000","last_active":"2024-09-13T16:28:44.044782+0000","last_peered":"2024-09-13T16:28:44.044782+0000","last_clean":"2024-09-13T16:28:44.044782+0000","last_became_active":"2024-09-13T16:28:39.217798+0000","last_became_peered":"2024-09-13T16:28:39.217798+0000","last_unstale":"2024-09-13T16:28:44.044782+0000","last_undegraded":"2024-09-13T16:28:44.044782+0000","last_fullsized":"2024-09-13T16:28:44.044782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:39:09.769384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.640833+0000","last_change":"2024-09-13T16:28:38.847238+0000","last_active":"2024-09-13T16:29:04.640833+0000","last_peered":"2024-09-13T16:29:04.640833+0000","last_clean":"2024-09-13T16:29:04.640833+0000","last_became_active":"2024-09-13T16:28:38.846741+0000","last_became_peered":"2024-09-13T16:28:38.846741+0000","last_unstale":"2024-09-13T16:29:04.640833+0000","last_undegraded":"2024-09-13T16:29:04.640833+0000","last_fullsized":"2024-09-13T16:29:04.640833+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:52:51.946158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628298+0000","last_change":"2024-09-13T16:28:43.228431+0000","last_active":"2024-09-13T16:29:04.628298+0000","last_peered":"2024-09-13T16:29:04.628298+0000","last_clean":"2024-09-13T16:29:04.628298+0000","last_became_active":"2024-09-13T16:28:43.228030+0000","last_became_peered":"2024-09-13T16:28:43.228030+0000","last_unstale":"2024-09-13T16:29:04.628298+0000","last_undegraded":"2024-09-13T16:29:04.628298+0000","last_fullsized":"2024-09-13T16:29:04.628298+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:14:50.573033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629589+0000","last_change":"2024-09-13T16:28:39.209485+0000","last_active":"2024-09-13T16:29:04.629589+0000","last_peered":"2024-09-13T16:29:04.629589+0000","last_clean":"2024-09-13T16:29:04.629589+0000","last_became_active":"2024-09-13T16:28:39.209157+0000","last_became_peered":"2024-09-13T16:28:39.209157+0000","last_unstale":"2024-09-13T16:29:04.629589+0000","last_undegraded":"2024-09-13T16:29:04.629589+0000","last_fullsized":"2024-09-13T16:29:04.629589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:12:19.398255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628085+0000","last_change":"2024-09-13T16:28:38.842551+0000","last_active":"2024-09-13T16:29:04.628085+0000","last_peered":"2024-09-13T16:29:04.628085+0000","last_clean":"2024-09-13T16:29:04.628085+0000","last_became_active":"2024-09-13T16:28:38.841741+0000","last_became_peered":"2024-09-13T16:28:38.841741+0000","last_unstale":"2024-09-13T16:29:04.628085+0000","last_undegraded":"2024-09-13T16:29:04.628085+0000","last_fullsized":"2024-09-13T16:29:04.628085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:35:25.131693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.626884+0000","last_change":"2024-09-13T16:28:43.294300+0000","last_active":"2024-09-13T16:29:04.626884+0000","last_peered":"2024-09-13T16:29:04.626884+0000","last_clean":"2024-09-13T16:29:04.626884+0000","last_became_active":"2024-09-13T16:28:43.293918+0000","last_became_peered":"2024-09-13T16:28:43.293918+0000","last_unstale":"2024-09-13T16:29:04.626884+0000","last_undegraded":"2024-09-13T16:29:04.626884+0000","last_fullsized":"2024-09-13T16:29:04.626884+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:19:21.959617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.631071+0000","last_change":"2024-09-13T16:28:39.212001+0000","last_active":"2024-09-13T16:29:04.631071+0000","last_peered":"2024-09-13T16:29:04.631071+0000","last_clean":"2024-09-13T16:29:04.631071+0000","last_became_active":"2024-09-13T16:28:39.211716+0000","last_became_peered":"2024-09-13T16:28:39.211716+0000","last_unstale":"2024-09-13T16:29:04.631071+0000","last_undegraded":"2024-09-13T16:29:04.631071+0000","last_fullsized":"2024-09-13T16:29:04.631071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:56.753677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:06.648061+0000","last_change":"2024-09-13T16:28:38.851229+0000","last_active":"2024-09-13T16:29:06.648061+0000","last_peered":"2024-09-13T16:29:06.648061+0000","last_clean":"2024-09-13T16:29:06.648061+0000","last_became_active":"2024-09-13T16:28:38.850937+0000","last_became_peered":"2024-09-13T16:28:38.850937+0000","last_unstale":"2024-09-13T16:29:06.648061+0000","last_undegraded":"2024-09-13T16:29:06.648061+0000","last_fullsized":"2024-09-13T16:29:06.648061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:19.460864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:45.495529+0000","last_change":"2024-09-13T16:28:43.224825+0000","last_active":"2024-09-13T16:28:45.495529+0000","last_peered":"2024-09-13T16:28:45.495529+0000","last_clean":"2024-09-13T16:28:45.495529+0000","last_became_active":"2024-09-13T16:28:43.223903+0000","last_became_peered":"2024-09-13T16:28:43.223903+0000","last_unstale":"2024-09-13T16:28:45.495529+0000","last_undegraded":"2024-09-13T16:28:45.495529+0000","last_fullsized":"2024-09-13T16:28:45.495529+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:08:46.850668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.639142+0000","last_change":"2024-09-13T16:29:05.636787+0000","last_active":"2024-09-13T16:29:05.639142+0000","last_peered":"2024-09-13T16:29:05.639142+0000","last_clean":"2024-09-13T16:29:05.639142+0000","last_became_active":"2024-09-13T16:29:05.636399+0000","last_became_peered":"2024-09-13T16:29:05.636399+0000","last_unstale":"2024-09-13T16:29:05.639142+0000","last_undegraded":"2024-09-13T16:29:05.639142+0000","last_fullsized":"2024-09-13T16:29:05.639142+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:38:25.545325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.622845+0000","last_change":"2024-09-13T16:28:38.845374+0000","last_active":"2024-09-13T16:29:04.622845+0000","last_peered":"2024-09-13T16:29:04.622845+0000","last_clean":"2024-09-13T16:29:04.622845+0000","last_became_active":"2024-09-13T16:28:38.845076+0000","last_became_peered":"2024-09-13T16:28:38.845076+0000","last_unstale":"2024-09-13T16:29:04.622845+0000","last_undegraded":"2024-09-13T16:29:04.622845+0000","last_fullsized":"2024-09-13T16:29:04.622845+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:37:56.225682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:45.495577+0000","last_change":"2024-09-13T16:28:43.293529+0000","last_active":"2024-09-13T16:28:45.495577+0000","last_peered":"2024-09-13T16:28:45.495577+0000","last_clean":"2024-09-13T16:28:45.495577+0000","last_became_active":"2024-09-13T16:28:43.293083+0000","last_became_peered":"2024-09-13T16:28:43.293083+0000","last_unstale":"2024-09-13T16:28:45.495577+0000","last_undegraded":"2024-09-13T16:28:45.495577+0000","last_fullsized":"2024-09-13T16:28:45.495577+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:05:55.572984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.043722+0000","last_change":"2024-09-13T16:28:39.217546+0000","last_active":"2024-09-13T16:28:44.043722+0000","last_peered":"2024-09-13T16:28:44.043722+0000","last_clean":"2024-09-13T16:28:44.043722+0000","last_became_active":"2024-09-13T16:28:39.217242+0000","last_became_peered":"2024-09-13T16:28:39.217242+0000","last_unstale":"2024-09-13T16:28:44.043722+0000","last_undegraded":"2024-09-13T16:28:44.043722+0000","last_fullsized":"2024-09-13T16:28:44.043722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:23:18.183616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.641545+0000","last_change":"2024-09-13T16:28:38.846436+0000","last_active":"2024-09-13T16:29:04.641545+0000","last_peered":"2024-09-13T16:29:04.641545+0000","last_clean":"2024-09-13T16:29:04.641545+0000","last_became_active":"2024-09-13T16:28:38.845953+0000","last_became_peered":"2024-09-13T16:28:38.845953+0000","last_unstale":"2024-09-13T16:29:04.641545+0000","last_undegraded":"2024-09-13T16:29:04.641545+0000","last_fullsized":"2024-09-13T16:29:04.641545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:33.998462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.624750+0000","last_change":"2024-09-13T16:28:43.294504+0000","last_active":"2024-09-13T16:29:04.624750+0000","last_peered":"2024-09-13T16:29:04.624750+0000","last_clean":"2024-09-13T16:29:04.624750+0000","last_became_active":"2024-09-13T16:28:43.294063+0000","last_became_peered":"2024-09-13T16:28:43.294063+0000","last_unstale":"2024-09-13T16:29:04.624750+0000","last_undegraded":"2024-09-13T16:29:04.624750+0000","last_fullsized":"2024-09-13T16:29:04.624750+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:51:35.774467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044939+0000","last_change":"2024-09-13T16:28:39.206491+0000","last_active":"2024-09-13T16:28:44.044939+0000","last_peered":"2024-09-13T16:28:44.044939+0000","last_clean":"2024-09-13T16:28:44.044939+0000","last_became_active":"2024-09-13T16:28:39.206187+0000","last_became_peered":"2024-09-13T16:28:39.206187+0000","last_unstale":"2024-09-13T16:28:44.044939+0000","last_undegraded":"2024-09-13T16:28:44.044939+0000","last_fullsized":"2024-09-13T16:28:44.044939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:20:28.457994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.635563+0000","last_change":"2024-09-13T16:28:38.836270+0000","last_active":"2024-09-13T16:29:05.635563+0000","last_peered":"2024-09-13T16:29:05.635563+0000","last_clean":"2024-09-13T16:29:05.635563+0000","last_became_active":"2024-09-13T16:28:38.835966+0000","last_became_peered":"2024-09-13T16:28:38.835966+0000","last_unstale":"2024-09-13T16:29:05.635563+0000","last_undegraded":"2024-09-13T16:29:05.635563+0000","last_fullsized":"2024-09-13T16:29:05.635563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:03:11.388654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":331,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625402+0000","last_change":"2024-09-13T16:27:34.264484+0000","last_active":"2024-09-13T16:29:04.625402+0000","last_peered":"2024-09-13T16:29:04.625402+0000","last_clean":"2024-09-13T16:29:04.625402+0000","last_became_active":"2024-09-13T16:27:34.263995+0000","last_became_peered":"2024-09-13T16:27:34.263995+0000","last_unstale":"2024-09-13T16:29:04.625402+0000","last_undegraded":"2024-09-13T16:29:04.625402+0000","last_fullsized":"2024-09-13T16:29:04.625402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:24:23.015578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:24:23.015578+0000","last_clean_scrub_stamp":"2024-09-13T16:24:23.015578+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:25:04.376312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.634973+0000","last_change":"2024-09-13T16:28:43.229347+0000","last_active":"2024-09-13T16:29:05.634973+0000","last_peered":"2024-09-13T16:29:05.634973+0000","last_clean":"2024-09-13T16:29:05.634973+0000","last_became_active":"2024-09-13T16:28:43.228981+0000","last_became_peered":"2024-09-13T16:28:43.228981+0000","last_unstale":"2024-09-13T16:29:05.634973+0000","last_undegraded":"2024-09-13T16:29:05.634973+0000","last_fullsized":"2024-09-13T16:29:05.634973+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:46:15.151295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625877+0000","last_change":"2024-09-13T16:28:39.203195+0000","last_active":"2024-09-13T16:29:04.625877+0000","last_peered":"2024-09-13T16:29:04.625877+0000","last_clean":"2024-09-13T16:29:04.625877+0000","last_became_active":"2024-09-13T16:28:39.202878+0000","last_became_peered":"2024-09-13T16:28:39.202878+0000","last_unstale":"2024-09-13T16:29:04.625877+0000","last_undegraded":"2024-09-13T16:29:04.625877+0000","last_fullsized":"2024-09-13T16:29:04.625877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:17.615531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:06.646697+0000","last_change":"2024-09-13T16:28:38.844856+0000","last_active":"2024-09-13T16:29:06.646697+0000","last_peered":"2024-09-13T16:29:06.646697+0000","last_clean":"2024-09-13T16:29:06.646697+0000","last_became_active":"2024-09-13T16:28:38.844578+0000","last_became_peered":"2024-09-13T16:28:38.844578+0000","last_unstale":"2024-09-13T16:29:06.646697+0000","last_undegraded":"2024-09-13T16:29:06.646697+0000","last_fullsized":"2024-09-13T16:29:06.646697+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:41:48.253174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:45.572882+0000","last_change":"2024-09-13T16:28:43.228472+0000","last_active":"2024-09-13T16:28:45.572882+0000","last_peered":"2024-09-13T16:28:45.572882+0000","last_clean":"2024-09-13T16:28:45.572882+0000","last_became_active":"2024-09-13T16:28:43.228056+0000","last_became_peered":"2024-09-13T16:28:43.228056+0000","last_unstale":"2024-09-13T16:28:45.572882+0000","last_undegraded":"2024-09-13T16:28:45.572882+0000","last_fullsized":"2024-09-13T16:28:45.572882+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:07:55.980691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625637+0000","last_change":"2024-09-13T16:28:39.203278+0000","last_active":"2024-09-13T16:29:04.625637+0000","last_peered":"2024-09-13T16:29:04.625637+0000","last_clean":"2024-09-13T16:29:04.625637+0000","last_became_active":"2024-09-13T16:28:39.202936+0000","last_became_peered":"2024-09-13T16:28:39.202936+0000","last_unstale":"2024-09-13T16:29:04.625637+0000","last_undegraded":"2024-09-13T16:29:04.625637+0000","last_fullsized":"2024-09-13T16:29:04.625637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:44.210612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.640824+0000","last_change":"2024-09-13T16:28:38.851085+0000","last_active":"2024-09-13T16:29:04.640824+0000","last_peered":"2024-09-13T16:29:04.640824+0000","last_clean":"2024-09-13T16:29:04.640824+0000","last_became_active":"2024-09-13T16:28:38.850785+0000","last_became_peered":"2024-09-13T16:28:38.850785+0000","last_unstale":"2024-09-13T16:29:04.640824+0000","last_undegraded":"2024-09-13T16:29:04.640824+0000","last_fullsized":"2024-09-13T16:29:04.640824+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:52.713431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044053+0000","last_change":"2024-09-13T16:28:43.229980+0000","last_active":"2024-09-13T16:28:44.044053+0000","last_peered":"2024-09-13T16:28:44.044053+0000","last_clean":"2024-09-13T16:28:44.044053+0000","last_became_active":"2024-09-13T16:28:43.229647+0000","last_became_peered":"2024-09-13T16:28:43.229647+0000","last_unstale":"2024-09-13T16:28:44.044053+0000","last_undegraded":"2024-09-13T16:28:44.044053+0000","last_fullsized":"2024-09-13T16:28:44.044053+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:39:42.871613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625770+0000","last_change":"2024-09-13T16:28:39.204167+0000","last_active":"2024-09-13T16:29:04.625770+0000","last_peered":"2024-09-13T16:29:04.625770+0000","last_clean":"2024-09-13T16:29:04.625770+0000","last_became_active":"2024-09-13T16:28:39.203776+0000","last_became_peered":"2024-09-13T16:28:39.203776+0000","last_unstale":"2024-09-13T16:29:04.625770+0000","last_undegraded":"2024-09-13T16:29:04.625770+0000","last_fullsized":"2024-09-13T16:29:04.625770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:53:36.410365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.639297+0000","last_change":"2024-09-13T16:28:38.846331+0000","last_active":"2024-09-13T16:29:04.639297+0000","last_peered":"2024-09-13T16:29:04.639297+0000","last_clean":"2024-09-13T16:29:04.639297+0000","last_became_active":"2024-09-13T16:28:38.846012+0000","last_became_peered":"2024-09-13T16:28:38.846012+0000","last_unstale":"2024-09-13T16:29:04.639297+0000","last_undegraded":"2024-09-13T16:29:04.639297+0000","last_fullsized":"2024-09-13T16:29:04.639297+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:23:54.282736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.623954+0000","last_change":"2024-09-13T16:28:43.227886+0000","last_active":"2024-09-13T16:29:04.623954+0000","last_peered":"2024-09-13T16:29:04.623954+0000","last_clean":"2024-09-13T16:29:04.623954+0000","last_became_active":"2024-09-13T16:28:43.227346+0000","last_became_peered":"2024-09-13T16:28:43.227346+0000","last_unstale":"2024-09-13T16:29:04.623954+0000","last_undegraded":"2024-09-13T16:29:04.623954+0000","last_fullsized":"2024-09-13T16:29:04.623954+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:41:36.559064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629086+0000","last_change":"2024-09-13T16:28:39.205975+0000","last_active":"2024-09-13T16:29:04.629086+0000","last_peered":"2024-09-13T16:29:04.629086+0000","last_clean":"2024-09-13T16:29:04.629086+0000","last_became_active":"2024-09-13T16:28:39.205657+0000","last_became_peered":"2024-09-13T16:28:39.205657+0000","last_unstale":"2024-09-13T16:29:04.629086+0000","last_undegraded":"2024-09-13T16:29:04.629086+0000","last_fullsized":"2024-09-13T16:29:04.629086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:31:47.309482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629013+0000","last_change":"2024-09-13T16:28:38.841589+0000","last_active":"2024-09-13T16:29:04.629013+0000","last_peered":"2024-09-13T16:29:04.629013+0000","last_clean":"2024-09-13T16:29:04.629013+0000","last_became_active":"2024-09-13T16:28:38.841298+0000","last_became_peered":"2024-09-13T16:28:38.841298+0000","last_unstale":"2024-09-13T16:29:04.629013+0000","last_undegraded":"2024-09-13T16:29:04.629013+0000","last_fullsized":"2024-09-13T16:29:04.629013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:27:38.358320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.623986+0000","last_change":"2024-09-13T16:28:43.233790+0000","last_active":"2024-09-13T16:29:04.623986+0000","last_peered":"2024-09-13T16:29:04.623986+0000","last_clean":"2024-09-13T16:29:04.623986+0000","last_became_active":"2024-09-13T16:28:43.231963+0000","last_became_peered":"2024-09-13T16:28:43.231963+0000","last_unstale":"2024-09-13T16:29:04.623986+0000","last_undegraded":"2024-09-13T16:29:04.623986+0000","last_fullsized":"2024-09-13T16:29:04.623986+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:45:23.406395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.624233+0000","last_change":"2024-09-13T16:28:39.201476+0000","last_active":"2024-09-13T16:29:04.624233+0000","last_peered":"2024-09-13T16:29:04.624233+0000","last_clean":"2024-09-13T16:29:04.624233+0000","last_became_active":"2024-09-13T16:28:39.201168+0000","last_became_peered":"2024-09-13T16:28:39.201168+0000","last_unstale":"2024-09-13T16:29:04.624233+0000","last_undegraded":"2024-09-13T16:29:04.624233+0000","last_fullsized":"2024-09-13T16:29:04.624233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:26:43.570295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":62,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.652225+0000","last_change":"2024-09-13T16:29:05.652225+0000","last_active":"2024-09-13T16:29:05.652225+0000","last_peered":"2024-09-13T16:29:05.652225+0000","last_clean":"2024-09-13T16:29:05.652225+0000","last_became_active":"2024-09-13T16:29:05.639325+0000","last_became_peered":"2024-09-13T16:29:05.639325+0000","last_unstale":"2024-09-13T16:29:05.652225+0000","last_undegraded":"2024-09-13T16:29:05.652225+0000","last_fullsized":"2024-09-13T16:29:05.652225+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:18:30.083447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:45.495557+0000","last_change":"2024-09-13T16:28:43.228563+0000","last_active":"2024-09-13T16:28:45.495557+0000","last_peered":"2024-09-13T16:28:45.495557+0000","last_clean":"2024-09-13T16:28:45.495557+0000","last_became_active":"2024-09-13T16:28:43.228184+0000","last_became_peered":"2024-09-13T16:28:43.228184+0000","last_unstale":"2024-09-13T16:28:45.495557+0000","last_undegraded":"2024-09-13T16:28:45.495557+0000","last_fullsized":"2024-09-13T16:28:45.495557+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:05:33.368302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625221+0000","last_change":"2024-09-13T16:28:39.210750+0000","last_active":"2024-09-13T16:29:04.625221+0000","last_peered":"2024-09-13T16:29:04.625221+0000","last_clean":"2024-09-13T16:29:04.625221+0000","last_became_active":"2024-09-13T16:28:39.210401+0000","last_became_peered":"2024-09-13T16:28:39.210401+0000","last_unstale":"2024-09-13T16:29:04.625221+0000","last_undegraded":"2024-09-13T16:29:04.625221+0000","last_fullsized":"2024-09-13T16:29:04.625221+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:36.670420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:06.638428+0000","last_change":"2024-09-13T16:28:38.843573+0000","last_active":"2024-09-13T16:29:06.638428+0000","last_peered":"2024-09-13T16:29:06.638428+0000","last_clean":"2024-09-13T16:29:06.638428+0000","last_became_active":"2024-09-13T16:28:38.843245+0000","last_became_peered":"2024-09-13T16:28:38.843245+0000","last_unstale":"2024-09-13T16:29:06.638428+0000","last_undegraded":"2024-09-13T16:29:06.638428+0000","last_fullsized":"2024-09-13T16:29:06.638428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:45.983123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628374+0000","last_change":"2024-09-13T16:28:43.231002+0000","last_active":"2024-09-13T16:29:04.628374+0000","last_peered":"2024-09-13T16:29:04.628374+0000","last_clean":"2024-09-13T16:29:04.628374+0000","last_became_active":"2024-09-13T16:28:43.230237+0000","last_became_peered":"2024-09-13T16:28:43.230237+0000","last_unstale":"2024-09-13T16:29:04.628374+0000","last_undegraded":"2024-09-13T16:29:04.628374+0000","last_fullsized":"2024-09-13T16:29:04.628374+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:11:47.494738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625559+0000","last_change":"2024-09-13T16:28:39.201471+0000","last_active":"2024-09-13T16:29:04.625559+0000","last_peered":"2024-09-13T16:29:04.625559+0000","last_clean":"2024-09-13T16:29:04.625559+0000","last_became_active":"2024-09-13T16:28:39.201140+0000","last_became_peered":"2024-09-13T16:28:39.201140+0000","last_unstale":"2024-09-13T16:29:04.625559+0000","last_undegraded":"2024-09-13T16:29:04.625559+0000","last_fullsized":"2024-09-13T16:29:04.625559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:37:25.037490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.641008+0000","last_change":"2024-09-13T16:28:38.845703+0000","last_active":"2024-09-13T16:29:04.641008+0000","last_peered":"2024-09-13T16:29:04.641008+0000","last_clean":"2024-09-13T16:29:04.641008+0000","last_became_active":"2024-09-13T16:28:38.845354+0000","last_became_peered":"2024-09-13T16:28:38.845354+0000","last_unstale":"2024-09-13T16:29:04.641008+0000","last_undegraded":"2024-09-13T16:29:04.641008+0000","last_fullsized":"2024-09-13T16:29:04.641008+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:18.208723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629807+0000","last_change":"2024-09-13T16:28:43.223924+0000","last_active":"2024-09-13T16:29:04.629807+0000","last_peered":"2024-09-13T16:29:04.629807+0000","last_clean":"2024-09-13T16:29:04.629807+0000","last_became_active":"2024-09-13T16:28:43.223535+0000","last_became_peered":"2024-09-13T16:28:43.223535+0000","last_unstale":"2024-09-13T16:29:04.629807+0000","last_undegraded":"2024-09-13T16:29:04.629807+0000","last_fullsized":"2024-09-13T16:29:04.629807+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:10.622436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044425+0000","last_change":"2024-09-13T16:28:39.203943+0000","last_active":"2024-09-13T16:28:44.044425+0000","last_peered":"2024-09-13T16:28:44.044425+0000","last_clean":"2024-09-13T16:28:44.044425+0000","last_became_active":"2024-09-13T16:28:39.203657+0000","last_became_peered":"2024-09-13T16:28:39.203657+0000","last_unstale":"2024-09-13T16:28:44.044425+0000","last_undegraded":"2024-09-13T16:28:44.044425+0000","last_fullsized":"2024-09-13T16:28:44.044425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:52:48.523939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.651351+0000","last_change":"2024-09-13T16:29:05.651015+0000","last_active":"2024-09-13T16:29:05.651351+0000","last_peered":"2024-09-13T16:29:05.651351+0000","last_clean":"2024-09-13T16:29:05.651351+0000","last_became_active":"2024-09-13T16:29:05.640006+0000","last_became_peered":"2024-09-13T16:29:05.640006+0000","last_unstale":"2024-09-13T16:29:05.651351+0000","last_undegraded":"2024-09-13T16:29:05.651351+0000","last_fullsized":"2024-09-13T16:29:05.651351+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:12:25.508262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.624674+0000","last_change":"2024-09-13T16:28:43.232634+0000","last_active":"2024-09-13T16:29:04.624674+0000","last_peered":"2024-09-13T16:29:04.624674+0000","last_clean":"2024-09-13T16:29:04.624674+0000","last_became_active":"2024-09-13T16:28:43.232107+0000","last_became_peered":"2024-09-13T16:28:43.232107+0000","last_unstale":"2024-09-13T16:29:04.624674+0000","last_undegraded":"2024-09-13T16:29:04.624674+0000","last_fullsized":"2024-09-13T16:29:04.624674+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:45.952933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627482+0000","last_change":"2024-09-13T16:28:39.212883+0000","last_active":"2024-09-13T16:29:04.627482+0000","last_peered":"2024-09-13T16:29:04.627482+0000","last_clean":"2024-09-13T16:29:04.627482+0000","last_became_active":"2024-09-13T16:28:39.212591+0000","last_became_peered":"2024-09-13T16:28:39.212591+0000","last_unstale":"2024-09-13T16:29:04.627482+0000","last_undegraded":"2024-09-13T16:29:04.627482+0000","last_fullsized":"2024-09-13T16:29:04.627482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:17:30.926940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'17","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:06.645117+0000","last_change":"2024-09-13T16:28:38.846564+0000","last_active":"2024-09-13T16:29:06.645117+0000","last_peered":"2024-09-13T16:29:06.645117+0000","last_clean":"2024-09-13T16:29:06.645117+0000","last_became_active":"2024-09-13T16:28:38.846165+0000","last_became_peered":"2024-09-13T16:28:38.846165+0000","last_unstale":"2024-09-13T16:29:06.645117+0000","last_undegraded":"2024-09-13T16:29:06.645117+0000","last_fullsized":"2024-09-13T16:29:06.645117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:33:06.124592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10396,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":19,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627853+0000","last_change":"2024-09-13T16:28:43.231464+0000","last_active":"2024-09-13T16:29:04.627853+0000","last_peered":"2024-09-13T16:29:04.627853+0000","last_clean":"2024-09-13T16:29:04.627853+0000","last_became_active":"2024-09-13T16:28:43.231040+0000","last_became_peered":"2024-09-13T16:28:43.231040+0000","last_unstale":"2024-09-13T16:29:04.627853+0000","last_undegraded":"2024-09-13T16:29:04.627853+0000","last_fullsized":"2024-09-13T16:29:04.627853+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:11:21.847375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"peering","last_fresh":"2024-09-13T16:29:04.630957+0000","last_change":"2024-09-13T16:29:04.631158+0000","last_active":"2024-09-13T16:29:04.630957+0000","last_peered":"2024-09-13T16:28:44.044616+0000","last_clean":"2024-09-13T16:28:44.044616+0000","last_became_active":"2024-09-13T16:28:39.204382+0000","last_became_peered":"2024-09-13T16:28:39.204382+0000","last_unstale":"2024-09-13T16:29:04.630957+0000","last_undegraded":"2024-09-13T16:29:04.630957+0000","last_fullsized":"2024-09-13T16:29:04.630957+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:58.816998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6,8],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625089+0000","last_change":"2024-09-13T16:28:38.841637+0000","last_active":"2024-09-13T16:29:04.625089+0000","last_peered":"2024-09-13T16:29:04.625089+0000","last_clean":"2024-09-13T16:29:04.625089+0000","last_became_active":"2024-09-13T16:28:38.841359+0000","last_became_peered":"2024-09-13T16:28:38.841359+0000","last_unstale":"2024-09-13T16:29:04.625089+0000","last_undegraded":"2024-09-13T16:29:04.625089+0000","last_fullsized":"2024-09-13T16:29:04.625089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:47:05.284987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.638870+0000","last_change":"2024-09-13T16:29:05.638870+0000","last_active":"2024-09-13T16:29:05.638870+0000","last_peered":"2024-09-13T16:29:05.638870+0000","last_clean":"2024-09-13T16:29:05.638870+0000","last_became_active":"2024-09-13T16:29:05.638422+0000","last_became_peered":"2024-09-13T16:29:05.638422+0000","last_unstale":"2024-09-13T16:29:05.638870+0000","last_undegraded":"2024-09-13T16:29:05.638870+0000","last_fullsized":"2024-09-13T16:29:05.638870+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:41.263661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T16:28:43.218920+0000","last_change":"2024-09-13T16:28:39.203595+0000","last_active":"2024-09-13T16:28:43.218920+0000","last_peered":"2024-09-13T16:28:43.218920+0000","last_clean":"2024-09-13T16:28:43.218920+0000","last_became_active":"2024-09-13T16:28:39.203283+0000","last_became_peered":"2024-09-13T16:28:39.203283+0000","last_unstale":"2024-09-13T16:28:43.218920+0000","last_undegraded":"2024-09-13T16:28:43.218920+0000","last_fullsized":"2024-09-13T16:28:43.218920+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:58:45.970761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.642267+0000","last_change":"2024-09-13T16:28:38.850203+0000","last_active":"2024-09-13T16:29:04.642267+0000","last_peered":"2024-09-13T16:29:04.642267+0000","last_clean":"2024-09-13T16:29:04.642267+0000","last_became_active":"2024-09-13T16:28:38.849963+0000","last_became_peered":"2024-09-13T16:28:38.849963+0000","last_unstale":"2024-09-13T16:29:04.642267+0000","last_undegraded":"2024-09-13T16:29:04.642267+0000","last_fullsized":"2024-09-13T16:29:04.642267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:15.849627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627470+0000","last_change":"2024-09-13T16:28:43.227679+0000","last_active":"2024-09-13T16:29:04.627470+0000","last_peered":"2024-09-13T16:29:04.627470+0000","last_clean":"2024-09-13T16:29:04.627470+0000","last_became_active":"2024-09-13T16:28:43.227269+0000","last_became_peered":"2024-09-13T16:28:43.227269+0000","last_unstale":"2024-09-13T16:29:04.627470+0000","last_undegraded":"2024-09-13T16:29:04.627470+0000","last_fullsized":"2024-09-13T16:29:04.627470+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:05:00.186305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044938+0000","last_change":"2024-09-13T16:28:39.211401+0000","last_active":"2024-09-13T16:28:44.044938+0000","last_peered":"2024-09-13T16:28:44.044938+0000","last_clean":"2024-09-13T16:28:44.044938+0000","last_became_active":"2024-09-13T16:28:39.211133+0000","last_became_peered":"2024-09-13T16:28:39.211133+0000","last_unstale":"2024-09-13T16:28:44.044938+0000","last_undegraded":"2024-09-13T16:28:44.044938+0000","last_fullsized":"2024-09-13T16:28:44.044938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:36.468666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:06.642523+0000","last_change":"2024-09-13T16:28:38.845132+0000","last_active":"2024-09-13T16:29:06.642523+0000","last_peered":"2024-09-13T16:29:06.642523+0000","last_clean":"2024-09-13T16:29:06.642523+0000","last_became_active":"2024-09-13T16:28:38.844804+0000","last_became_peered":"2024-09-13T16:28:38.844804+0000","last_unstale":"2024-09-13T16:29:06.642523+0000","last_undegraded":"2024-09-13T16:29:06.642523+0000","last_fullsized":"2024-09-13T16:29:06.642523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:15:08.643227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.754760+0000","last_change":"2024-09-13T16:28:43.230829+0000","last_active":"2024-09-13T16:28:44.754760+0000","last_peered":"2024-09-13T16:28:44.754760+0000","last_clean":"2024-09-13T16:28:44.754760+0000","last_became_active":"2024-09-13T16:28:43.230405+0000","last_became_peered":"2024-09-13T16:28:43.230405+0000","last_unstale":"2024-09-13T16:28:44.754760+0000","last_undegraded":"2024-09-13T16:28:44.754760+0000","last_fullsized":"2024-09-13T16:28:44.754760+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:04:56.083193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044006+0000","last_change":"2024-09-13T16:28:43.226662+0000","last_active":"2024-09-13T16:28:44.044006+0000","last_peered":"2024-09-13T16:28:44.044006+0000","last_clean":"2024-09-13T16:28:44.044006+0000","last_became_active":"2024-09-13T16:28:43.226232+0000","last_became_peered":"2024-09-13T16:28:43.226232+0000","last_unstale":"2024-09-13T16:28:44.044006+0000","last_undegraded":"2024-09-13T16:28:44.044006+0000","last_fullsized":"2024-09-13T16:28:44.044006+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:03.568500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627636+0000","last_change":"2024-09-13T16:28:39.208909+0000","last_active":"2024-09-13T16:29:04.627636+0000","last_peered":"2024-09-13T16:29:04.627636+0000","last_clean":"2024-09-13T16:29:04.627636+0000","last_became_active":"2024-09-13T16:28:39.208572+0000","last_became_peered":"2024-09-13T16:28:39.208572+0000","last_unstale":"2024-09-13T16:29:04.627636+0000","last_undegraded":"2024-09-13T16:29:04.627636+0000","last_fullsized":"2024-09-13T16:29:04.627636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:01:40.173308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.641240+0000","last_change":"2024-09-13T16:28:38.844259+0000","last_active":"2024-09-13T16:29:04.641240+0000","last_peered":"2024-09-13T16:29:04.641240+0000","last_clean":"2024-09-13T16:29:04.641240+0000","last_became_active":"2024-09-13T16:28:38.843956+0000","last_became_peered":"2024-09-13T16:28:38.843956+0000","last_unstale":"2024-09-13T16:29:04.641240+0000","last_undegraded":"2024-09-13T16:29:04.641240+0000","last_fullsized":"2024-09-13T16:29:04.641240+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:05.652852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629221+0000","last_change":"2024-09-13T16:28:43.229385+0000","last_active":"2024-09-13T16:29:04.629221+0000","last_peered":"2024-09-13T16:29:04.629221+0000","last_clean":"2024-09-13T16:29:04.629221+0000","last_became_active":"2024-09-13T16:28:43.229033+0000","last_became_peered":"2024-09-13T16:28:43.229033+0000","last_unstale":"2024-09-13T16:29:04.629221+0000","last_undegraded":"2024-09-13T16:29:04.629221+0000","last_fullsized":"2024-09-13T16:29:04.629221+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:46:41.611758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044455+0000","last_change":"2024-09-13T16:28:39.208100+0000","last_active":"2024-09-13T16:28:44.044455+0000","last_peered":"2024-09-13T16:28:44.044455+0000","last_clean":"2024-09-13T16:28:44.044455+0000","last_became_active":"2024-09-13T16:28:39.207808+0000","last_became_peered":"2024-09-13T16:28:39.207808+0000","last_unstale":"2024-09-13T16:28:44.044455+0000","last_undegraded":"2024-09-13T16:28:44.044455+0000","last_fullsized":"2024-09-13T16:28:44.044455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:55.446568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.622326+0000","last_change":"2024-09-13T16:28:38.844689+0000","last_active":"2024-09-13T16:29:04.622326+0000","last_peered":"2024-09-13T16:29:04.622326+0000","last_clean":"2024-09-13T16:29:04.622326+0000","last_became_active":"2024-09-13T16:28:38.844326+0000","last_became_peered":"2024-09-13T16:28:38.844326+0000","last_unstale":"2024-09-13T16:29:04.622326+0000","last_undegraded":"2024-09-13T16:29:04.622326+0000","last_fullsized":"2024-09-13T16:29:04.622326+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:02.616518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629530+0000","last_change":"2024-09-13T16:28:38.849233+0000","last_active":"2024-09-13T16:29:04.629530+0000","last_peered":"2024-09-13T16:29:04.629530+0000","last_clean":"2024-09-13T16:29:04.629530+0000","last_became_active":"2024-09-13T16:28:38.848911+0000","last_became_peered":"2024-09-13T16:28:38.848911+0000","last_unstale":"2024-09-13T16:29:04.629530+0000","last_undegraded":"2024-09-13T16:29:04.629530+0000","last_fullsized":"2024-09-13T16:29:04.629530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:02:35.951263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.624339+0000","last_change":"2024-09-13T16:28:39.203855+0000","last_active":"2024-09-13T16:29:04.624339+0000","last_peered":"2024-09-13T16:29:04.624339+0000","last_clean":"2024-09-13T16:29:04.624339+0000","last_became_active":"2024-09-13T16:28:39.203532+0000","last_became_peered":"2024-09-13T16:28:39.203532+0000","last_unstale":"2024-09-13T16:29:04.624339+0000","last_undegraded":"2024-09-13T16:29:04.624339+0000","last_fullsized":"2024-09-13T16:29:04.624339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:15.048930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.633730+0000","last_change":"2024-09-13T16:28:43.228554+0000","last_active":"2024-09-13T16:29:04.633730+0000","last_peered":"2024-09-13T16:29:04.633730+0000","last_clean":"2024-09-13T16:29:04.633730+0000","last_became_active":"2024-09-13T16:28:43.228204+0000","last_became_peered":"2024-09-13T16:28:43.228204+0000","last_unstale":"2024-09-13T16:29:04.633730+0000","last_undegraded":"2024-09-13T16:29:04.633730+0000","last_fullsized":"2024-09-13T16:29:04.633730+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:16:15.383932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.639374+0000","last_change":"2024-09-13T16:28:38.848425+0000","last_active":"2024-09-13T16:29:05.639374+0000","last_peered":"2024-09-13T16:29:05.639374+0000","last_clean":"2024-09-13T16:29:05.639374+0000","last_became_active":"2024-09-13T16:28:38.848136+0000","last_became_peered":"2024-09-13T16:28:38.848136+0000","last_unstale":"2024-09-13T16:29:05.639374+0000","last_undegraded":"2024-09-13T16:29:05.639374+0000","last_fullsized":"2024-09-13T16:29:05.639374+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:47:03.904315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627249+0000","last_change":"2024-09-13T16:28:39.210565+0000","last_active":"2024-09-13T16:29:04.627249+0000","last_peered":"2024-09-13T16:29:04.627249+0000","last_clean":"2024-09-13T16:29:04.627249+0000","last_became_active":"2024-09-13T16:28:39.210272+0000","last_became_peered":"2024-09-13T16:28:39.210272+0000","last_unstale":"2024-09-13T16:29:04.627249+0000","last_undegraded":"2024-09-13T16:29:04.627249+0000","last_fullsized":"2024-09-13T16:29:04.627249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:12.746087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.634077+0000","last_change":"2024-09-13T16:28:43.231623+0000","last_active":"2024-09-13T16:29:04.634077+0000","last_peered":"2024-09-13T16:29:04.634077+0000","last_clean":"2024-09-13T16:29:04.634077+0000","last_became_active":"2024-09-13T16:28:43.231255+0000","last_became_peered":"2024-09-13T16:28:43.231255+0000","last_unstale":"2024-09-13T16:29:04.634077+0000","last_undegraded":"2024-09-13T16:29:04.634077+0000","last_fullsized":"2024-09-13T16:29:04.634077+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:29.755772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627783+0000","last_change":"2024-09-13T16:28:39.208438+0000","last_active":"2024-09-13T16:29:04.627783+0000","last_peered":"2024-09-13T16:29:04.627783+0000","last_clean":"2024-09-13T16:29:04.627783+0000","last_became_active":"2024-09-13T16:28:39.208156+0000","last_became_peered":"2024-09-13T16:28:39.208156+0000","last_unstale":"2024-09-13T16:29:04.627783+0000","last_undegraded":"2024-09-13T16:29:04.627783+0000","last_fullsized":"2024-09-13T16:29:04.627783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:53:46.923166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.633806+0000","last_change":"2024-09-13T16:28:39.208529+0000","last_active":"2024-09-13T16:29:04.633806+0000","last_peered":"2024-09-13T16:29:04.633806+0000","last_clean":"2024-09-13T16:29:04.633806+0000","last_became_active":"2024-09-13T16:28:39.208202+0000","last_became_peered":"2024-09-13T16:28:39.208202+0000","last_unstale":"2024-09-13T16:29:04.633806+0000","last_undegraded":"2024-09-13T16:29:04.633806+0000","last_fullsized":"2024-09-13T16:29:04.633806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:17:11.357410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.632611+0000","last_change":"2024-09-13T16:28:39.212958+0000","last_active":"2024-09-13T16:29:04.632611+0000","last_peered":"2024-09-13T16:29:04.632611+0000","last_clean":"2024-09-13T16:29:04.632611+0000","last_became_active":"2024-09-13T16:28:39.212646+0000","last_became_peered":"2024-09-13T16:28:39.212646+0000","last_unstale":"2024-09-13T16:29:04.632611+0000","last_undegraded":"2024-09-13T16:29:04.632611+0000","last_fullsized":"2024-09-13T16:29:04.632611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:31:04.047778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T16:28:43.218683+0000","last_change":"2024-09-13T16:28:39.200449+0000","last_active":"2024-09-13T16:28:43.218683+0000","last_peered":"2024-09-13T16:28:43.218683+0000","last_clean":"2024-09-13T16:28:43.218683+0000","last_became_active":"2024-09-13T16:28:39.200133+0000","last_became_peered":"2024-09-13T16:28:39.200133+0000","last_unstale":"2024-09-13T16:28:43.218683+0000","last_undegraded":"2024-09-13T16:28:43.218683+0000","last_fullsized":"2024-09-13T16:28:43.218683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:18:06.480691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044668+0000","last_change":"2024-09-13T16:28:39.200274+0000","last_active":"2024-09-13T16:28:44.044668+0000","last_peered":"2024-09-13T16:28:44.044668+0000","last_clean":"2024-09-13T16:28:44.044668+0000","last_became_active":"2024-09-13T16:28:39.199951+0000","last_became_peered":"2024-09-13T16:28:39.199951+0000","last_unstale":"2024-09-13T16:28:44.044668+0000","last_undegraded":"2024-09-13T16:28:44.044668+0000","last_fullsized":"2024-09-13T16:28:44.044668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:13:31.199857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.630000+0000","last_change":"2024-09-13T16:28:39.213591+0000","last_active":"2024-09-13T16:29:04.630000+0000","last_peered":"2024-09-13T16:29:04.630000+0000","last_clean":"2024-09-13T16:29:04.630000+0000","last_became_active":"2024-09-13T16:28:39.213324+0000","last_became_peered":"2024-09-13T16:28:39.213324+0000","last_unstale":"2024-09-13T16:29:04.630000+0000","last_undegraded":"2024-09-13T16:29:04.630000+0000","last_fullsized":"2024-09-13T16:29:04.630000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:54:03.172919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.633921+0000","last_change":"2024-09-13T16:28:39.217442+0000","last_active":"2024-09-13T16:29:04.633921+0000","last_peered":"2024-09-13T16:29:04.633921+0000","last_clean":"2024-09-13T16:29:04.633921+0000","last_became_active":"2024-09-13T16:28:39.217151+0000","last_became_peered":"2024-09-13T16:28:39.217151+0000","last_unstale":"2024-09-13T16:29:04.633921+0000","last_undegraded":"2024-09-13T16:29:04.633921+0000","last_fullsized":"2024-09-13T16:29:04.633921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:17.132278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.045476+0000","last_change":"2024-09-13T16:28:39.203938+0000","last_active":"2024-09-13T16:28:44.045476+0000","last_peered":"2024-09-13T16:28:44.045476+0000","last_clean":"2024-09-13T16:28:44.045476+0000","last_became_active":"2024-09-13T16:28:39.203655+0000","last_became_peered":"2024-09-13T16:28:39.203655+0000","last_unstale":"2024-09-13T16:28:44.045476+0000","last_undegraded":"2024-09-13T16:28:44.045476+0000","last_fullsized":"2024-09-13T16:28:44.045476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:03:25.415256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.624165+0000","last_change":"2024-09-13T16:28:39.209592+0000","last_active":"2024-09-13T16:29:04.624165+0000","last_peered":"2024-09-13T16:29:04.624165+0000","last_clean":"2024-09-13T16:29:04.624165+0000","last_became_active":"2024-09-13T16:28:39.209299+0000","last_became_peered":"2024-09-13T16:28:39.209299+0000","last_unstale":"2024-09-13T16:29:04.624165+0000","last_undegraded":"2024-09-13T16:29:04.624165+0000","last_fullsized":"2024-09-13T16:29:04.624165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:04.053405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044501+0000","last_change":"2024-09-13T16:28:39.208437+0000","last_active":"2024-09-13T16:28:44.044501+0000","last_peered":"2024-09-13T16:28:44.044501+0000","last_clean":"2024-09-13T16:28:44.044501+0000","last_became_active":"2024-09-13T16:28:39.208136+0000","last_became_peered":"2024-09-13T16:28:39.208136+0000","last_unstale":"2024-09-13T16:28:44.044501+0000","last_undegraded":"2024-09-13T16:28:44.044501+0000","last_fullsized":"2024-09-13T16:28:44.044501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:21.304037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.045412+0000","last_change":"2024-09-13T16:28:39.203970+0000","last_active":"2024-09-13T16:28:44.045412+0000","last_peered":"2024-09-13T16:28:44.045412+0000","last_clean":"2024-09-13T16:28:44.045412+0000","last_became_active":"2024-09-13T16:28:39.203559+0000","last_became_peered":"2024-09-13T16:28:39.203559+0000","last_unstale":"2024-09-13T16:28:44.045412+0000","last_undegraded":"2024-09-13T16:28:44.045412+0000","last_fullsized":"2024-09-13T16:28:44.045412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:26:25.158393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044158+0000","last_change":"2024-09-13T16:28:39.205584+0000","last_active":"2024-09-13T16:28:44.044158+0000","last_peered":"2024-09-13T16:28:44.044158+0000","last_clean":"2024-09-13T16:28:44.044158+0000","last_became_active":"2024-09-13T16:28:39.205281+0000","last_became_peered":"2024-09-13T16:28:39.205281+0000","last_unstale":"2024-09-13T16:28:44.044158+0000","last_undegraded":"2024-09-13T16:28:44.044158+0000","last_fullsized":"2024-09-13T16:28:44.044158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:58:10.109647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629903+0000","last_change":"2024-09-13T16:28:39.214479+0000","last_active":"2024-09-13T16:29:04.629903+0000","last_peered":"2024-09-13T16:29:04.629903+0000","last_clean":"2024-09-13T16:29:04.629903+0000","last_became_active":"2024-09-13T16:28:39.213984+0000","last_became_peered":"2024-09-13T16:28:39.213984+0000","last_unstale":"2024-09-13T16:29:04.629903+0000","last_undegraded":"2024-09-13T16:29:04.629903+0000","last_fullsized":"2024-09-13T16:29:04.629903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:53:43.137726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.634317+0000","last_change":"2024-09-13T16:28:39.209595+0000","last_active":"2024-09-13T16:29:04.634317+0000","last_peered":"2024-09-13T16:29:04.634317+0000","last_clean":"2024-09-13T16:29:04.634317+0000","last_became_active":"2024-09-13T16:28:39.209282+0000","last_became_peered":"2024-09-13T16:28:39.209282+0000","last_unstale":"2024-09-13T16:29:04.634317+0000","last_undegraded":"2024-09-13T16:29:04.634317+0000","last_fullsized":"2024-09-13T16:29:04.634317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:26:41.452462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628010+0000","last_change":"2024-09-13T16:28:39.210610+0000","last_active":"2024-09-13T16:29:04.628010+0000","last_peered":"2024-09-13T16:29:04.628010+0000","last_clean":"2024-09-13T16:29:04.628010+0000","last_became_active":"2024-09-13T16:28:39.210296+0000","last_became_peered":"2024-09-13T16:28:39.210296+0000","last_unstale":"2024-09-13T16:29:04.628010+0000","last_undegraded":"2024-09-13T16:29:04.628010+0000","last_fullsized":"2024-09-13T16:29:04.628010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:06.385733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629594+0000","last_change":"2024-09-13T16:28:39.207609+0000","last_active":"2024-09-13T16:29:04.629594+0000","last_peered":"2024-09-13T16:29:04.629594+0000","last_clean":"2024-09-13T16:29:04.629594+0000","last_became_active":"2024-09-13T16:28:39.206893+0000","last_became_peered":"2024-09-13T16:28:39.206893+0000","last_unstale":"2024-09-13T16:29:04.629594+0000","last_undegraded":"2024-09-13T16:29:04.629594+0000","last_fullsized":"2024-09-13T16:29:04.629594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:51.582666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.045110+0000","last_change":"2024-09-13T16:28:39.201856+0000","last_active":"2024-09-13T16:28:44.045110+0000","last_peered":"2024-09-13T16:28:44.045110+0000","last_clean":"2024-09-13T16:28:44.045110+0000","last_became_active":"2024-09-13T16:28:39.201511+0000","last_became_peered":"2024-09-13T16:28:39.201511+0000","last_unstale":"2024-09-13T16:28:44.045110+0000","last_undegraded":"2024-09-13T16:28:44.045110+0000","last_fullsized":"2024-09-13T16:28:44.045110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:05:19.722354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T16:28:43.215322+0000","last_change":"2024-09-13T16:28:39.216106+0000","last_active":"2024-09-13T16:28:43.215322+0000","last_peered":"2024-09-13T16:28:43.215322+0000","last_clean":"2024-09-13T16:28:43.215322+0000","last_became_active":"2024-09-13T16:28:39.215432+0000","last_became_peered":"2024-09-13T16:28:39.215432+0000","last_unstale":"2024-09-13T16:28:43.215322+0000","last_undegraded":"2024-09-13T16:28:43.215322+0000","last_fullsized":"2024-09-13T16:28:43.215322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:21:18.439891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T16:28:43.215353+0000","last_change":"2024-09-13T16:28:39.210578+0000","last_active":"2024-09-13T16:28:43.215353+0000","last_peered":"2024-09-13T16:28:43.215353+0000","last_clean":"2024-09-13T16:28:43.215353+0000","last_became_active":"2024-09-13T16:28:39.210269+0000","last_became_peered":"2024-09-13T16:28:39.210269+0000","last_unstale":"2024-09-13T16:28:43.215353+0000","last_undegraded":"2024-09-13T16:28:43.215353+0000","last_fullsized":"2024-09-13T16:28:43.215353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:27:27.078880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044081+0000","last_change":"2024-09-13T16:28:39.213409+0000","last_active":"2024-09-13T16:28:44.044081+0000","last_peered":"2024-09-13T16:28:44.044081+0000","last_clean":"2024-09-13T16:28:44.044081+0000","last_became_active":"2024-09-13T16:28:39.213077+0000","last_became_peered":"2024-09-13T16:28:39.213077+0000","last_unstale":"2024-09-13T16:28:44.044081+0000","last_undegraded":"2024-09-13T16:28:44.044081+0000","last_fullsized":"2024-09-13T16:28:44.044081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:32:24.747516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.632419+0000","last_change":"2024-09-13T16:28:39.215216+0000","last_active":"2024-09-13T16:29:04.632419+0000","last_peered":"2024-09-13T16:29:04.632419+0000","last_clean":"2024-09-13T16:29:04.632419+0000","last_became_active":"2024-09-13T16:28:39.214920+0000","last_became_peered":"2024-09-13T16:28:39.214920+0000","last_unstale":"2024-09-13T16:29:04.632419+0000","last_undegraded":"2024-09-13T16:29:04.632419+0000","last_fullsized":"2024-09-13T16:29:04.632419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:25:38.132833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625826+0000","last_change":"2024-09-13T16:28:39.202506+0000","last_active":"2024-09-13T16:29:04.625826+0000","last_peered":"2024-09-13T16:29:04.625826+0000","last_clean":"2024-09-13T16:29:04.625826+0000","last_became_active":"2024-09-13T16:28:39.202224+0000","last_became_peered":"2024-09-13T16:28:39.202224+0000","last_unstale":"2024-09-13T16:29:04.625826+0000","last_undegraded":"2024-09-13T16:29:04.625826+0000","last_fullsized":"2024-09-13T16:29:04.625826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:00:37.208442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.640477+0000","last_change":"2024-09-13T16:29:05.640477+0000","last_active":"2024-09-13T16:29:05.640477+0000","last_peered":"2024-09-13T16:29:05.640477+0000","last_clean":"2024-09-13T16:29:05.640477+0000","last_became_active":"2024-09-13T16:29:05.640014+0000","last_became_peered":"2024-09-13T16:29:05.640014+0000","last_unstale":"2024-09-13T16:29:05.640477+0000","last_undegraded":"2024-09-13T16:29:05.640477+0000","last_fullsized":"2024-09-13T16:29:05.640477+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:47:45.202113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.045134+0000","last_change":"2024-09-13T16:28:39.200217+0000","last_active":"2024-09-13T16:28:44.045134+0000","last_peered":"2024-09-13T16:28:44.045134+0000","last_clean":"2024-09-13T16:28:44.045134+0000","last_became_active":"2024-09-13T16:28:39.199946+0000","last_became_peered":"2024-09-13T16:28:39.199946+0000","last_unstale":"2024-09-13T16:28:44.045134+0000","last_undegraded":"2024-09-13T16:28:44.045134+0000","last_fullsized":"2024-09-13T16:28:44.045134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:20:02.906225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044376+0000","last_change":"2024-09-13T16:28:39.215013+0000","last_active":"2024-09-13T16:28:44.044376+0000","last_peered":"2024-09-13T16:28:44.044376+0000","last_clean":"2024-09-13T16:28:44.044376+0000","last_became_active":"2024-09-13T16:28:39.214731+0000","last_became_peered":"2024-09-13T16:28:39.214731+0000","last_unstale":"2024-09-13T16:28:44.044376+0000","last_undegraded":"2024-09-13T16:28:44.044376+0000","last_fullsized":"2024-09-13T16:28:44.044376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:35:01.836001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625268+0000","last_change":"2024-09-13T16:28:39.200300+0000","last_active":"2024-09-13T16:29:04.625268+0000","last_peered":"2024-09-13T16:29:04.625268+0000","last_clean":"2024-09-13T16:29:04.625268+0000","last_became_active":"2024-09-13T16:28:39.200006+0000","last_became_peered":"2024-09-13T16:28:39.200006+0000","last_unstale":"2024-09-13T16:29:04.625268+0000","last_undegraded":"2024-09-13T16:29:04.625268+0000","last_fullsized":"2024-09-13T16:29:04.625268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:35:14.783905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629430+0000","last_change":"2024-09-13T16:28:39.213552+0000","last_active":"2024-09-13T16:29:04.629430+0000","last_peered":"2024-09-13T16:29:04.629430+0000","last_clean":"2024-09-13T16:29:04.629430+0000","last_became_active":"2024-09-13T16:28:39.213287+0000","last_became_peered":"2024-09-13T16:28:39.213287+0000","last_unstale":"2024-09-13T16:29:04.629430+0000","last_undegraded":"2024-09-13T16:29:04.629430+0000","last_fullsized":"2024-09-13T16:29:04.629430+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:30.588770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627025+0000","last_change":"2024-09-13T16:28:39.211126+0000","last_active":"2024-09-13T16:29:04.627025+0000","last_peered":"2024-09-13T16:29:04.627025+0000","last_clean":"2024-09-13T16:29:04.627025+0000","last_became_active":"2024-09-13T16:28:39.210810+0000","last_became_peered":"2024-09-13T16:28:39.210810+0000","last_unstale":"2024-09-13T16:29:04.627025+0000","last_undegraded":"2024-09-13T16:29:04.627025+0000","last_fullsized":"2024-09-13T16:29:04.627025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:52:45.249309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.639070+0000","last_change":"2024-09-13T16:29:05.636535+0000","last_active":"2024-09-13T16:29:05.639070+0000","last_peered":"2024-09-13T16:29:05.639070+0000","last_clean":"2024-09-13T16:29:05.639070+0000","last_became_active":"2024-09-13T16:29:05.636161+0000","last_became_peered":"2024-09-13T16:29:05.636161+0000","last_unstale":"2024-09-13T16:29:05.639070+0000","last_undegraded":"2024-09-13T16:29:05.639070+0000","last_fullsized":"2024-09-13T16:29:05.639070+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:09.606922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T16:28:43.215475+0000","last_change":"2024-09-13T16:28:39.202222+0000","last_active":"2024-09-13T16:28:43.215475+0000","last_peered":"2024-09-13T16:28:43.215475+0000","last_clean":"2024-09-13T16:28:43.215475+0000","last_became_active":"2024-09-13T16:28:39.201897+0000","last_became_peered":"2024-09-13T16:28:39.201897+0000","last_unstale":"2024-09-13T16:28:43.215475+0000","last_undegraded":"2024-09-13T16:28:43.215475+0000","last_fullsized":"2024-09-13T16:28:43.215475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:41:09.087185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T16:28:43.217236+0000","last_change":"2024-09-13T16:28:39.211164+0000","last_active":"2024-09-13T16:28:43.217236+0000","last_peered":"2024-09-13T16:28:43.217236+0000","last_clean":"2024-09-13T16:28:43.217236+0000","last_became_active":"2024-09-13T16:28:39.210826+0000","last_became_peered":"2024-09-13T16:28:39.210826+0000","last_unstale":"2024-09-13T16:28:43.217236+0000","last_undegraded":"2024-09-13T16:28:43.217236+0000","last_fullsized":"2024-09-13T16:28:43.217236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:40:30.663867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629508+0000","last_change":"2024-09-13T16:28:39.207991+0000","last_active":"2024-09-13T16:29:04.629508+0000","last_peered":"2024-09-13T16:29:04.629508+0000","last_clean":"2024-09-13T16:29:04.629508+0000","last_became_active":"2024-09-13T16:28:39.207332+0000","last_became_peered":"2024-09-13T16:28:39.207332+0000","last_unstale":"2024-09-13T16:29:04.629508+0000","last_undegraded":"2024-09-13T16:29:04.629508+0000","last_fullsized":"2024-09-13T16:29:04.629508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:30:54.271361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":16384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":17802,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":0,"num_read":57,"num_read_kb":46,"num_write":213,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":762,"num_keys_recovered":10,"num_objects_omap":64,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":57746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":187,"ondisk_log_size":187,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250749,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383696,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":487217,"data_compressed":14739,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.32600000000000001}]},{"osd":1,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.376},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.30399999999999999}]},{"osd":2,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.439},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.627}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]},{"osd":4,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.47699999999999998}]},{"osd":5,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.59099999999999997}]},{"osd":6,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.56599999999999995}]},{"osd":7,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.54000000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":9,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.73299999999999998}]},{"osd":10,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.70999999999999996}]}]},{"osd":10,"up_from":72,"seq":309237645334,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1089536,"data_stored":1091645,"data_compressed":54008,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":1,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.50800000000000001}]},{"osd":2,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.626},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.68799999999999994}]},{"osd":3,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":7.2320000000000002,"5min":7.2320000000000002,"15min":7.2320000000000002},"last":0.61099999999999999}]},{"osd":4,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.57699999999999996}]},{"osd":5,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.75},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.64100000000000001}]},{"osd":6,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.70199999999999996}]},{"osd":7,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.53000000000000003}]},{"osd":8,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":7.2569999999999997,"5min":7.2569999999999997,"15min":7.2569999999999997},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.70999999999999996}]},{"osd":9,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.67300000000000004}]},{"osd":11,"last update":"Fri Sep 13 16:28:58 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":7.2199999999999998,"5min":7.2199999999999998,"15min":7.2199999999999998},"last":0.79400000000000004}]}]},{"osd":9,"up_from":67,"seq":287762808858,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":496463,"data_compressed":17719,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.35399999999999998}]},{"osd":1,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.52900000000000003}]},{"osd":3,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.46500000000000002}]},{"osd":4,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.628}]},{"osd":5,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.81100000000000005}]},{"osd":6,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.45700000000000002}]},{"osd":7,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.76500000000000001}]},{"osd":8,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.745}]},{"osd":10,"last update":"Fri Sep 13 16:28:33 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.85299999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":8,"up_from":59,"seq":253403070495,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":1007616,"data_stored":495587,"data_compressed":17719,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.50600000000000001,"15min":0.50800000000000001},"min":{"1min":0.28100000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.70999999999999996,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51700000000000002,"15min":0.51200000000000001},"min":{"1min":0.32800000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.48599999999999999}]},{"osd":1,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.41299999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.40400000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82399999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.60799999999999998}]},{"osd":2,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.41699999999999998,"15min":0.41199999999999998},"min":{"1min":0.28299999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.435}]},{"osd":3,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.78700000000000003,"5min":0.871,"15min":0.871},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.58199999999999996}]},{"osd":4,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56799999999999995,"15min":0.56399999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.747}]},{"osd":5,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53800000000000003,"15min":0.54300000000000004},"min":{"1min":0.29699999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.622}]},{"osd":6,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.628,"15min":0.63},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.65900000000000003,"15min":0.66300000000000003},"min":{"1min":0.38300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68500000000000005}]},{"osd":7,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.66600000000000004,"15min":0.67000000000000004},"min":{"1min":0.47899999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.97699999999999998,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.65800000000000003,"15min":0.66200000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.92200000000000004,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.69699999999999995}]},{"osd":9,"last update":"Fri Sep 13 16:28:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.79500000000000004}]},{"osd":10,"last update":"Fri Sep 13 16:28:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.77600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]}]},{"osd":7,"up_from":51,"seq":219043332132,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":1093979,"data_compressed":54008,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.39600000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.41199999999999998}]},{"osd":1,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.39200000000000002,"15min":0.38900000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.312},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.68600000000000005,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.68999999999999995}]},{"osd":2,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.54400000000000004,"15min":0.54900000000000004},"min":{"1min":0.40699999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.66700000000000004,"5min":0.748,"15min":0.748},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54700000000000004,"15min":0.54800000000000004},"min":{"1min":0.40000000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.60299999999999998}]},{"osd":3,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.42499999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.441},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56799999999999995,"15min":0.56399999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.57499999999999996}]},{"osd":4,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45200000000000001,"15min":0.45100000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.45800000000000002,"15min":0.46300000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.32700000000000001}]},{"osd":5,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57699999999999996,"15min":0.57799999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.496},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.44600000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.54200000000000004}]},{"osd":6,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60899999999999999,"15min":0.60999999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.41699999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.373}]},{"osd":8,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.627,"15min":0.627},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.624},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.66100000000000003,"15min":0.66500000000000004},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.65800000000000003}]},{"osd":9,"last update":"Fri Sep 13 16:28:08 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.64200000000000002}]},{"osd":10,"last update":"Fri Sep 13 16:28:33 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.67600000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]}]},{"osd":6,"up_from":46,"seq":197568495657,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":991232,"data_stored":486747,"data_compressed":14739,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.38300000000000001,"15min":0.372},"min":{"1min":0.26900000000000002,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.39700000000000002,"15min":0.39000000000000001},"min":{"1min":0.28000000000000003,"5min":0.217,"15min":0.217},"max":{"1min":0.61599999999999999,"5min":0.625,"15min":0.625},"last":0.33700000000000002}]},{"osd":1,"last update":"Fri Sep 13 16:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53400000000000003,"15min":0.51200000000000001},"min":{"1min":0.373,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.753,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56100000000000005,"15min":0.54500000000000004},"min":{"1min":0.39700000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.77700000000000002,"5min":0.873,"15min":0.873},"last":0.53300000000000003}]},{"osd":2,"last update":"Fri Sep 13 16:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55300000000000005,"15min":0.54900000000000004},"min":{"1min":0.35199999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83499999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59699999999999998,"15min":0.58799999999999997},"min":{"1min":0.36899999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81399999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.623}]},{"osd":3,"last update":"Fri Sep 13 16:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.73699999999999999,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.499,"15min":0.48899999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.44400000000000001}]},{"osd":4,"last update":"Fri Sep 13 16:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.30299999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.73599999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.42299999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.92300000000000004,"5min":1.079,"15min":1.079},"last":0.49099999999999999}]},{"osd":5,"last update":"Fri Sep 13 16:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61799999999999999,"15min":0.60599999999999998},"min":{"1min":0.47199999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96599999999999997,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.45800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94999999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.54400000000000004}]},{"osd":7,"last update":"Fri Sep 13 16:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63800000000000001,"15min":0.63200000000000001},"min":{"1min":0.40600000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.02,"5min":1.048,"15min":1.048},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.66300000000000003,"15min":0.65200000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.60299999999999998}]},{"osd":8,"last update":"Fri Sep 13 16:28:43 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.69299999999999995,"15min":0.68899999999999995},"min":{"1min":0.505,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72699999999999998,"15min":0.72799999999999998},"min":{"1min":0.46600000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.64000000000000001}]},{"osd":9,"last update":"Fri Sep 13 16:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.66600000000000004}]},{"osd":10,"last update":"Fri Sep 13 16:28:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.70499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]}]},{"osd":5,"up_from":38,"seq":163208757294,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":1088542,"data_compressed":54008,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.52200000000000002,"15min":0.51600000000000001},"min":{"1min":0.377,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76700000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54000000000000004,"15min":0.53500000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.80700000000000005,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.46500000000000002}]},{"osd":1,"last update":"Fri Sep 13 16:28:40 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.48699999999999999,"15min":0.47699999999999998},"min":{"1min":0.308,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.51900000000000002,"15min":0.505},"min":{"1min":0.34399999999999997,"5min":0.245,"15min":0.245},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.41299999999999998}]},{"osd":2,"last update":"Fri Sep 13 16:28:40 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.375,"15min":0.35999999999999999},"min":{"1min":0.254,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.65000000000000002,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.379,"15min":0.36699999999999999},"min":{"1min":0.23100000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.36899999999999999}]},{"osd":3,"last update":"Fri Sep 13 16:28:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.80000000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.55000000000000004}]},{"osd":4,"last update":"Fri Sep 13 16:28:40 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52400000000000002,"15min":0.51100000000000001},"min":{"1min":0.41799999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53400000000000003,"15min":0.52800000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.54000000000000004}]},{"osd":6,"last update":"Fri Sep 13 16:29:03 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.45800000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.434,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.78900000000000003,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.49399999999999999}]},{"osd":7,"last update":"Fri Sep 13 16:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.60799999999999998,"15min":0.60499999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59999999999999998,"15min":0.59699999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.626}]},{"osd":8,"last update":"Fri Sep 13 16:28:46 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52800000000000002,"15min":0.53300000000000003},"min":{"1min":0.34499999999999997,"5min":0.318,"15min":0.318},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.56200000000000006,"15min":0.56899999999999995},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.56200000000000006}]},{"osd":9,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.65700000000000003}]},{"osd":10,"last update":"Fri Sep 13 16:28:31 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.68500000000000005}]}]},{"osd":4,"up_from":33,"seq":141733920819,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":496497,"data_compressed":17719,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.48699999999999999,"15min":0.46300000000000002},"min":{"1min":0.32200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.751,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.504,"15min":0.48199999999999998},"min":{"1min":0.41399999999999998,"5min":0.25,"15min":0.25},"max":{"1min":0.80800000000000005,"5min":0.874,"15min":0.874},"last":0.60499999999999998}]},{"osd":1,"last update":"Fri Sep 13 16:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.34699999999999998,"15min":0.33100000000000002},"min":{"1min":0.24099999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.34300000000000003,"15min":0.33200000000000002},"min":{"1min":0.27800000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.372}]},{"osd":2,"last update":"Fri Sep 13 16:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.495,"15min":0.48599999999999999},"min":{"1min":0.37,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71599999999999997,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.47399999999999998,"15min":0.45500000000000002},"min":{"1min":0.34399999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67200000000000004,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.59299999999999997}]},{"osd":3,"last update":"Fri Sep 13 16:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.504,"15min":0.48599999999999999},"min":{"1min":0.38700000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.505},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.52100000000000002,"15min":0.496},"min":{"1min":0.39900000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.79300000000000004,"5min":0.875,"15min":0.875},"last":0.56799999999999995}]},{"osd":5,"last update":"Fri Sep 13 16:28:38 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.33800000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54500000000000004,"15min":0.53100000000000003},"min":{"1min":0.39000000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80700000000000005,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Sep 13 16:28:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.54900000000000004,"15min":0.53500000000000003},"min":{"1min":0.47599999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.73299999999999998,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57699999999999996,"15min":0.56599999999999995},"min":{"1min":0.41699999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80400000000000005,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.63}]},{"osd":7,"last update":"Fri Sep 13 16:28:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.495,"15min":0.48099999999999998},"min":{"1min":0.33600000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.85099999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.49399999999999999,"15min":0.48999999999999999},"min":{"1min":0.34999999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.81799999999999995,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.621}]},{"osd":8,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.44,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86899999999999999,"5min":0.872,"15min":0.872},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.61499999999999999,"15min":0.60999999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.872,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.64500000000000002}]},{"osd":9,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.72599999999999998}]},{"osd":10,"last update":"Fri Sep 13 16:28:32 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.65400000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36188,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708964,"statfs":{"total":95995035648,"available":95957979136,"internally_reserved":0,"allocated":999424,"data_stored":486172,"data_compressed":14739,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:56 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.34899999999999998,"15min":0.32500000000000001},"min":{"1min":0.252,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.63,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.35699999999999998,"15min":0.32900000000000001},"min":{"1min":0.27600000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52300000000000002,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.36599999999999999}]},{"osd":1,"last update":"Fri Sep 13 16:28:56 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.54600000000000004,"15min":0.48199999999999998},"min":{"1min":0.48899999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.84299999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53500000000000003,"15min":0.48699999999999999},"min":{"1min":0.42399999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.77000000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.68799999999999994}]},{"osd":2,"last update":"Fri Sep 13 16:28:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52200000000000002,"15min":0.51400000000000001},"min":{"1min":0.375,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79100000000000004,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55300000000000005,"15min":0.52600000000000002},"min":{"1min":0.38600000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.94599999999999995,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.53600000000000003}]},{"osd":4,"last update":"Fri Sep 13 16:28:09 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.51500000000000001,"15min":0.49099999999999999},"min":{"1min":0.40400000000000003,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.56699999999999995,"15min":0.54000000000000004},"min":{"1min":0.44700000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.62}]},{"osd":5,"last update":"Fri Sep 13 16:28:36 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58299999999999996,"15min":0.57199999999999995},"min":{"1min":0.433,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.86299999999999999,"5min":1.038,"15min":1.038},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59699999999999998,"15min":0.59599999999999997},"min":{"1min":0.45900000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.73799999999999999,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.72099999999999997}]},{"osd":6,"last update":"Fri Sep 13 16:29:01 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.505,"15min":0.496},"min":{"1min":0.33600000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.48599999999999999,"15min":0.47999999999999998},"min":{"1min":0.34599999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74099999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.498}]},{"osd":7,"last update":"Fri Sep 13 16:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.49399999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.60699999999999998,"15min":0.59299999999999997},"min":{"1min":0.52100000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.64100000000000001}]},{"osd":8,"last update":"Fri Sep 13 16:27:45 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.71499999999999997}]},{"osd":9,"last update":"Fri Sep 13 16:28:08 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.56499999999999995}]},{"osd":10,"last update":"Fri Sep 13 16:28:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.70299999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":1032192,"data_stored":520251,"data_compressed":17719,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.60399999999999998,"15min":0.498},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.90900000000000003,"5min":16.027999999999999,"15min":16.027999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.47299999999999998,"15min":0.46500000000000002},"min":{"1min":0.309,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.69599999999999995}]},{"osd":1,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.48399999999999999,"15min":0.46500000000000002},"min":{"1min":0.372,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.51500000000000001,"15min":0.49299999999999999},"min":{"1min":0.44500000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.83399999999999996}]},{"osd":3,"last update":"Fri Sep 13 16:28:52 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.54000000000000004,"15min":0.51400000000000001},"min":{"1min":0.43099999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54000000000000004,"15min":0.51500000000000001},"min":{"1min":0.36799999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.59099999999999997}]},{"osd":4,"last update":"Fri Sep 13 16:28:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":0.623,"15min":0.55500000000000005},"min":{"1min":0.36399999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":9.6790000000000003,"5min":9.6790000000000003,"15min":9.6790000000000003},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.54500000000000004,"15min":0.53600000000000003},"min":{"1min":0.34399999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.75900000000000001}]},{"osd":5,"last update":"Fri Sep 13 16:28:39 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.40999999999999998,"15min":0.40200000000000002},"min":{"1min":0.33700000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.432},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.40300000000000002,"15min":0.39200000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.46100000000000002}]},{"osd":6,"last update":"Fri Sep 13 16:29:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.57699999999999996,"15min":0.55200000000000005},"min":{"1min":0.437,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.60599999999999998,"15min":0.58399999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.79400000000000004}]},{"osd":7,"last update":"Fri Sep 13 16:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.379,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59099999999999997,"15min":0.58299999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.748}]},{"osd":8,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.52600000000000002,"15min":0.51800000000000002},"min":{"1min":0.33500000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.501},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.51700000000000002,"15min":0.50800000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.55100000000000005}]},{"osd":9,"last update":"Fri Sep 13 16:29:06 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.67900000000000005,"15min":0.66900000000000004},"min":{"1min":0.45100000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.016,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.68100000000000005,"15min":0.67100000000000004},"min":{"1min":0.50600000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.85299999999999998}]},{"osd":10,"last update":"Fri Sep 13 16:28:34 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.89400000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":491382,"data_compressed":14739,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:09 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.47999999999999998,"15min":0.47799999999999998},"min":{"1min":0.34100000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.497},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.48399999999999999,"15min":0.48099999999999998},"min":{"1min":0.38100000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.435}]},{"osd":2,"last update":"Fri Sep 13 16:28:27 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.67100000000000004,"15min":0.54100000000000004},"min":{"1min":0.34100000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.81200000000000006,"5min":17.655000000000001,"15min":17.655000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.49099999999999999,"15min":0.47899999999999998},"min":{"1min":0.29299999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.76200000000000001}]},{"osd":3,"last update":"Fri Sep 13 16:28:51 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52500000000000002,"15min":0.50700000000000001},"min":{"1min":0.379,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72599999999999998,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.55800000000000005,"15min":0.52200000000000002},"min":{"1min":0.42199999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.73899999999999999}]},{"osd":4,"last update":"Fri Sep 13 16:28:13 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.40400000000000003,"15min":0.38},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.433,"5min":0.40899999999999997,"15min":0.39100000000000001},"min":{"1min":0.28299999999999997,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.57899999999999996}]},{"osd":5,"last update":"Fri Sep 13 16:28:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":1.0609999999999999,"15min":1.2829999999999999},"min":{"1min":0.38400000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.82499999999999996,"5min":17.640000000000001,"15min":17.640000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54600000000000004,"15min":0.54300000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.72499999999999998}]},{"osd":6,"last update":"Fri Sep 13 16:29:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59299999999999997,"15min":0.59399999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.72799999999999998,"5min":0.872,"15min":0.872},"last":0.60899999999999999}]},{"osd":7,"last update":"Fri Sep 13 16:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.372,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.78900000000000003}]},{"osd":8,"last update":"Fri Sep 13 16:28:43 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.60399999999999998,"15min":0.61099999999999999},"min":{"1min":0.40699999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.45500000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.83499999999999996}]},{"osd":9,"last update":"Fri Sep 13 16:28:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.751}]},{"osd":10,"last update":"Fri Sep 13 16:28:34 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.82199999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":477257,"data_compressed":11759,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 16:28:15 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.46999999999999997,"15min":0.46899999999999997},"min":{"1min":0.37,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.755,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.50700000000000001,"15min":0.495},"min":{"1min":0.41199999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81000000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.60899999999999999}]},{"osd":2,"last update":"Fri Sep 13 16:28:33 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.55000000000000004,"15min":0.52600000000000002},"min":{"1min":0.42699999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73799999999999999,"5min":1.111,"15min":1.111},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.54800000000000004,"15min":0.56100000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.621,"5min":1.698,"15min":1.698},"last":0.40000000000000002}]},{"osd":3,"last update":"Fri Sep 13 16:27:47 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.438,"15min":0.42399999999999999},"min":{"1min":0.38200000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.42299999999999999,"15min":0.39800000000000002},"min":{"1min":0.22500000000000001,"5min":0.217,"15min":0.217},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.46500000000000002}]},{"osd":4,"last update":"Fri Sep 13 16:28:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.48199999999999998,"15min":0.45400000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.625},{"interface":"front","average":{"1min":0.629,"5min":0.51500000000000001,"15min":0.48399999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.65400000000000003}]},{"osd":5,"last update":"Fri Sep 13 16:28:33 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.55300000000000005,"15min":0.54100000000000004},"min":{"1min":0.44,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56799999999999995,"15min":0.55600000000000005},"min":{"1min":0.46899999999999997,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77800000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.36299999999999999}]},{"osd":6,"last update":"Fri Sep 13 16:28:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.504,"15min":0.48199999999999998},"min":{"1min":0.29899999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.66500000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55300000000000005,"15min":0.52100000000000002},"min":{"1min":0.34399999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.74299999999999999,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.501}]},{"osd":7,"last update":"Fri Sep 13 16:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.68300000000000005,"15min":0.67500000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96999999999999997,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.63700000000000001,"15min":0.63100000000000001},"min":{"1min":0.43099999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.58499999999999996}]},{"osd":8,"last update":"Fri Sep 13 16:28:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1,"5min":1,"15min":1},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69899999999999995,"15min":0.70299999999999996},"min":{"1min":0.55000000000000004,"5min":0.497,"15min":0.497},"max":{"1min":0.81799999999999995,"5min":1.004,"15min":1.004},"last":0.54900000000000004}]},{"osd":9,"last update":"Fri Sep 13 16:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.53500000000000003}]},{"osd":10,"last update":"Fri Sep 13 16:28:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.68899999999999995}]},{"osd":11,"last update":"Fri Sep 13 16:28:58 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.66900000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":25183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":12476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T16:29:09.698 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.14 is not active+clean 2024-09-13T16:29:09.698 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 16, 'reported_epoch': 90, 'state': 'peering', 'last_fresh': '2024-09-13T16:29:04.630957+0000', 'last_change': '2024-09-13T16:29:04.631158+0000', 'last_active': '2024-09-13T16:29:04.630957+0000', 'last_peered': '2024-09-13T16:28:44.044616+0000', 'last_clean': '2024-09-13T16:28:44.044616+0000', 'last_became_active': '2024-09-13T16:28:39.204382+0000', 'last_became_peered': '2024-09-13T16:28:39.204382+0000', 'last_unstale': '2024-09-13T16:29:04.630957+0000', 'last_undegraded': '2024-09-13T16:29:04.630957+0000', 'last_fullsized': '2024-09-13T16:29:04.630957+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 84, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T16:28:38.825121+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T16:28:38.825121+0000', 'last_clean_scrub_stamp': '2024-09-13T16:28:38.825121+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-15T01:46:58.816998+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 8, 6], 'acting': [1, 8, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4, 6, 8], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-13T16:29:09.699 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-13T16:29:09.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.832+0000 7f48fb3e2640 1 -- 172.21.15.16:0/3535341212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f40ffe90 msgr2=0x7f48f4100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:09.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.832+0000 7f48fb3e2640 1 --2- 172.21.15.16:0/3535341212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f40ffe90 0x7f48f4100310 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f48f0009900 tx=0x7f48f002a410 comp rx=0 tx=0).stop 2024-09-13T16:29:09.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.832+0000 7f48fb3e2640 1 -- 172.21.15.16:0/3535341212 shutdown_connections 2024-09-13T16:29:09.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.832+0000 7f48fb3e2640 1 --2- 172.21.15.16:0/3535341212 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48f4100850 0x7f48f41070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:09.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.832+0000 7f48fb3e2640 1 --2- 172.21.15.16:0/3535341212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f40ffe90 0x7f48f4100310 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:09.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.832+0000 7f48fb3e2640 1 --2- 172.21.15.16:0/3535341212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48f40feca0 0x7f48f40ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:09.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.832+0000 7f48fb3e2640 1 -- 172.21.15.16:0/3535341212 >> 172.21.15.16:0/3535341212 conn(0x7f48f40fa400 msgr2=0x7f48f40fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:09.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.832+0000 7f48fb3e2640 1 -- 172.21.15.16:0/3535341212 shutdown_connections 2024-09-13T16:29:09.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.832+0000 7f48fb3e2640 1 -- 172.21.15.16:0/3535341212 wait complete. 2024-09-13T16:29:09.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.832+0000 7f48fb3e2640 1 Processor -- start 2024-09-13T16:29:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.832+0000 7f48fb3e2640 1 -- start start 2024-09-13T16:29:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.832+0000 7f48fb3e2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f40feca0 0x7f48f4102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.832+0000 7f48fb3e2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48f40ffe90 0x7f48f4101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.832+0000 7f48fb3e2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48f4100850 0x7f48f41017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.833+0000 7f48fb3e2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f4101f10 con 0x7f48f40feca0 2024-09-13T16:29:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.833+0000 7f48fb3e2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f4102080 con 0x7f48f4100850 2024-09-13T16:29:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.833+0000 7f48fb3e2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f419fcb0 con 0x7f48f40ffe90 2024-09-13T16:29:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.833+0000 7f48f917e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f40feca0 0x7f48f4102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.833+0000 7f48f917e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f40feca0 0x7f48f4102c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49740/0 (socket says 172.21.15.16:49740) 2024-09-13T16:29:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.833+0000 7f48f917e640 1 -- 172.21.15.16:0/587126362 learned_addr learned my addr 172.21.15.16:0/587126362 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.833+0000 7f48f997f640 1 --2- 172.21.15.16:0/587126362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48f4100850 0x7f48f41017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.833+0000 7f48f897d640 1 --2- 172.21.15.16:0/587126362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48f40ffe90 0x7f48f4101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.833+0000 7f48f917e640 1 -- 172.21.15.16:0/587126362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48f40ffe90 msgr2=0x7f48f4101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.833+0000 7f48f917e640 1 --2- 172.21.15.16:0/587126362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48f40ffe90 0x7f48f4101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.833+0000 7f48f917e640 1 -- 172.21.15.16:0/587126362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48f4100850 msgr2=0x7f48f41017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.833+0000 7f48f917e640 1 --2- 172.21.15.16:0/587126362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48f4100850 0x7f48f41017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.833+0000 7f48f917e640 1 -- 172.21.15.16:0/587126362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f0009560 con 0x7f48f40feca0 2024-09-13T16:29:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.833+0000 7f48f897d640 1 --2- 172.21.15.16:0/587126362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48f40ffe90 0x7f48f4101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:29:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.833+0000 7f48f997f640 1 --2- 172.21.15.16:0/587126362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48f4100850 0x7f48f41017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:29:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.833+0000 7f48f917e640 1 --2- 172.21.15.16:0/587126362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f40feca0 0x7f48f4102c30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f48e800d980 tx=0x7f48e800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.833+0000 7f48e67fc640 1 -- 172.21.15.16:0/587126362 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e8009c30 con 0x7f48f40feca0 2024-09-13T16:29:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.833+0000 7f48e67fc640 1 -- 172.21.15.16:0/587126362 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f48e8009d90 con 0x7f48f40feca0 2024-09-13T16:29:09.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.833+0000 7f48e67fc640 1 -- 172.21.15.16:0/587126362 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e800f620 con 0x7f48f40feca0 2024-09-13T16:29:09.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.833+0000 7f48fb3e2640 1 -- 172.21.15.16:0/587126362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48f419ff60 con 0x7f48f40feca0 2024-09-13T16:29:09.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.833+0000 7f48fb3e2640 1 -- 172.21.15.16:0/587126362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f41a0380 con 0x7f48f40feca0 2024-09-13T16:29:09.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.834+0000 7f48fb3e2640 1 -- 172.21.15.16:0/587126362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48f40ff0a0 con 0x7f48f40feca0 2024-09-13T16:29:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.835+0000 7f48e67fc640 1 -- 172.21.15.16:0/587126362 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f48e8010460 con 0x7f48f40feca0 2024-09-13T16:29:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.836+0000 7f48e67fc640 1 --2- 172.21.15.16:0/587126362 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f48e00a11a0 0x7f48e00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.836+0000 7f48e67fc640 1 -- 172.21.15.16:0/587126362 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f48e80cd010 con 0x7f48f40feca0 2024-09-13T16:29:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.836+0000 7f48f897d640 1 --2- 172.21.15.16:0/587126362 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f48e00a11a0 0x7f48e00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.837+0000 7f48f897d640 1 --2- 172.21.15.16:0/587126362 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f48e00a11a0 0x7f48e00a3660 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f48f0005ab0 tx=0x7f48f0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:09.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:09.840+0000 7f48e67fc640 1 -- 172.21.15.16:0/587126362 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f48e8096330 con 0x7f48f40feca0 2024-09-13T16:29:09.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:09 smithi016 ceph-mon[31187]: from='client.14874 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:29:09.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:09 smithi016 ceph-mon[31187]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:29:09.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:09 smithi016 ceph-mon[31187]: pgmap v293: 129 pgs: 1 peering, 128 active+clean; 594 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 7.7 KiB/s wr, 27 op/s; 95 B/s, 1 keys/s, 2 objects/s recovering 2024-09-13T16:29:09.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:09 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/540127229' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:29:09.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:09 smithi111 ceph-mon[34600]: from='client.14874 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:29:09.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:09 smithi111 ceph-mon[34600]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:29:09.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:09 smithi111 ceph-mon[34600]: pgmap v293: 129 pgs: 1 peering, 128 active+clean; 594 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 7.7 KiB/s wr, 27 op/s; 95 B/s, 1 keys/s, 2 objects/s recovering 2024-09-13T16:29:09.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:09 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/540127229' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:29:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.079+0000 7f48fb3e2640 1 -- 172.21.15.16:0/587126362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f48f410a6c0 con 0x7f48f40feca0 2024-09-13T16:29:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.079+0000 7f48e67fc640 1 -- 172.21.15.16:0/587126362 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+3500 (secure 0 0 0) 0x7f48e8017dc0 con 0x7f48f40feca0 2024-09-13T16:29:10.081 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:29:10.081 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"0d241f6a-71ec-11ef-bceb-c7b262605968","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":368,"monmap":{"epoch":3,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":91,"num_osds":12,"num_up_osds":12,"osd_up_since":1726244874,"num_in_osds":12,"osd_in_since":1726244855,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":128},{"state_name":"peering","count":1}],"num_pgs":129,"num_pools":4,"num_objects":100,"data_bytes":608404,"bytes_used":373645312,"bytes_avail":1151566782464,"bytes_total":1151940427776,"inactive_pgs_ratio":0.0077519379556179047,"recovering_objects_per_sec":2,"recovering_bytes_per_sec":95,"recovering_keys_per_sec":1,"num_objects_recovered":19,"num_bytes_recovered":762,"num_keys_recovered":10,"read_bytes_sec":5886,"write_bytes_sec":7933,"read_op_per_sec":7,"write_op_per_sec":20},"fsmap":{"epoch":28,"id":1,"up":5,"in":5,"max":5,"by_rank":[{"filesystem_id":1,"rank":4,"name":"g","status":"up:standby-replay","gid":14586},{"filesystem_id":1,"rank":3,"name":"a","status":"up:standby-replay","gid":14598},{"filesystem_id":1,"rank":4,"name":"l","status":"up:active","gid":24418},{"filesystem_id":1,"rank":3,"name":"k","status":"up:active","gid":24422},{"filesystem_id":1,"rank":2,"name":"e","status":"up:standby-replay","gid":24446},{"filesystem_id":1,"rank":1,"name":"f","status":"up:standby-replay","gid":24460},{"filesystem_id":1,"rank":0,"name":"c","status":"up:standby-replay","gid":24472},{"filesystem_id":1,"rank":2,"name":"h","status":"up:active","gid":24485},{"filesystem_id":1,"rank":1,"name":"i","status":"up:active","gid":24490},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24518}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.16:8443/"}},"servicemap":{"epoch":19,"modified":"2024-09-13T16:29:06.570654+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"g":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"j":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"k":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-13T16:29:10.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.081+0000 7f48fb3e2640 1 -- 172.21.15.16:0/587126362 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f48e00a11a0 msgr2=0x7f48e00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:10.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.081+0000 7f48fb3e2640 1 --2- 172.21.15.16:0/587126362 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f48e00a11a0 0x7f48e00a3660 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f48f0005ab0 tx=0x7f48f0005a20 comp rx=0 tx=0).stop 2024-09-13T16:29:10.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.081+0000 7f48fb3e2640 1 -- 172.21.15.16:0/587126362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f40feca0 msgr2=0x7f48f4102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:10.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.081+0000 7f48fb3e2640 1 --2- 172.21.15.16:0/587126362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f40feca0 0x7f48f4102c30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f48e800d980 tx=0x7f48e800de40 comp rx=0 tx=0).stop 2024-09-13T16:29:10.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.081+0000 7f48fb3e2640 1 -- 172.21.15.16:0/587126362 shutdown_connections 2024-09-13T16:29:10.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.081+0000 7f48fb3e2640 1 --2- 172.21.15.16:0/587126362 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f48e00a11a0 0x7f48e00a3660 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:10.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.081+0000 7f48fb3e2640 1 --2- 172.21.15.16:0/587126362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48f4100850 0x7f48f41017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:10.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.081+0000 7f48fb3e2640 1 --2- 172.21.15.16:0/587126362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48f40ffe90 0x7f48f4101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:10.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.081+0000 7f48fb3e2640 1 --2- 172.21.15.16:0/587126362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f40feca0 0x7f48f4102c30 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:10.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.081+0000 7f48fb3e2640 1 -- 172.21.15.16:0/587126362 >> 172.21.15.16:0/587126362 conn(0x7f48f40fa400 msgr2=0x7f48f40685c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:10.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.082+0000 7f48fb3e2640 1 -- 172.21.15.16:0/587126362 shutdown_connections 2024-09-13T16:29:10.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.082+0000 7f48fb3e2640 1 -- 172.21.15.16:0/587126362 wait complete. 2024-09-13T16:29:10.093 INFO:tasks.cephfs.filesystem.ceph_manager:making progress, resetting timeout 2024-09-13T16:29:10.093 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T16:29:10.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:09 smithi107 ceph-mon[34139]: from='client.14874 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:29:10.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:09 smithi107 ceph-mon[34139]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:29:10.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:09 smithi107 ceph-mon[34139]: pgmap v293: 129 pgs: 1 peering, 128 active+clean; 594 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 7.7 KiB/s wr, 27 op/s; 95 B/s, 1 keys/s, 2 objects/s recovering 2024-09-13T16:29:10.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:09 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/540127229' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:29:10.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.198+0000 7fb6b20e9640 1 -- 172.21.15.16:0/724216105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ac100850 msgr2=0x7fb6ac1070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:10.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.198+0000 7fb6b20e9640 1 --2- 172.21.15.16:0/724216105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ac100850 0x7fb6ac1070e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4009ca0 tx=0x7fb6a402a8c0 comp rx=0 tx=0).stop 2024-09-13T16:29:10.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.198+0000 7fb6b20e9640 1 -- 172.21.15.16:0/724216105 shutdown_connections 2024-09-13T16:29:10.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.198+0000 7fb6b20e9640 1 --2- 172.21.15.16:0/724216105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ac100850 0x7fb6ac1070e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:10.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.198+0000 7fb6b20e9640 1 --2- 172.21.15.16:0/724216105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6ac0ffe90 0x7fb6ac100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:10.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.198+0000 7fb6b20e9640 1 --2- 172.21.15.16:0/724216105 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6ac0feca0 0x7fb6ac0ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:10.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.198+0000 7fb6b20e9640 1 -- 172.21.15.16:0/724216105 >> 172.21.15.16:0/724216105 conn(0x7fb6ac0fa420 msgr2=0x7fb6ac0fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:10.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.198+0000 7fb6b20e9640 1 -- 172.21.15.16:0/724216105 shutdown_connections 2024-09-13T16:29:10.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.198+0000 7fb6b20e9640 1 -- 172.21.15.16:0/724216105 wait complete. 2024-09-13T16:29:10.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.198+0000 7fb6b20e9640 1 Processor -- start 2024-09-13T16:29:10.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.199+0000 7fb6b20e9640 1 -- start start 2024-09-13T16:29:10.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.199+0000 7fb6b20e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ac0feca0 0x7fb6ac194b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:10.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.199+0000 7fb6b20e9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6ac0ffe90 0x7fb6ac1950c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:10.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.199+0000 7fb6b20e9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6ac100850 0x7fb6ac19c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:10.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.199+0000 7fb6b20e9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac19c670 con 0x7fb6ac0feca0 2024-09-13T16:29:10.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.199+0000 7fb6b20e9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac19c7e0 con 0x7fb6ac0ffe90 2024-09-13T16:29:10.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.199+0000 7fb6b20e9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac19c9a0 con 0x7fb6ac100850 2024-09-13T16:29:10.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.199+0000 7fb6ab7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ac0feca0 0x7fb6ac194b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:10.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.199+0000 7fb6ab7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ac0feca0 0x7fb6ac194b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49754/0 (socket says 172.21.15.16:49754) 2024-09-13T16:29:10.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.199+0000 7fb6ab7fe640 1 -- 172.21.15.16:0/49987368 learned_addr learned my addr 172.21.15.16:0/49987368 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:10.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.199+0000 7fb6aaffd640 1 --2- 172.21.15.16:0/49987368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6ac0ffe90 0x7fb6ac1950c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:10.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.199+0000 7fb6abfff640 1 --2- 172.21.15.16:0/49987368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6ac100850 0x7fb6ac19c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:10.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.199+0000 7fb6ab7fe640 1 -- 172.21.15.16:0/49987368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6ac100850 msgr2=0x7fb6ac19c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:10.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.199+0000 7fb6ab7fe640 1 --2- 172.21.15.16:0/49987368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6ac100850 0x7fb6ac19c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:10.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.199+0000 7fb6ab7fe640 1 -- 172.21.15.16:0/49987368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6ac0ffe90 msgr2=0x7fb6ac1950c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:10.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.199+0000 7fb6ab7fe640 1 --2- 172.21.15.16:0/49987368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6ac0ffe90 0x7fb6ac1950c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:10.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.199+0000 7fb6ab7fe640 1 -- 172.21.15.16:0/49987368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a4009900 con 0x7fb6ac0feca0 2024-09-13T16:29:10.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.200+0000 7fb6ab7fe640 1 --2- 172.21.15.16:0/49987368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ac0feca0 0x7fb6ac194b80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb69400d770 tx=0x7fb69400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:10.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.200+0000 7fb6a8ff9640 1 -- 172.21.15.16:0/49987368 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb694009c30 con 0x7fb6ac0feca0 2024-09-13T16:29:10.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.200+0000 7fb6a8ff9640 1 -- 172.21.15.16:0/49987368 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb69400de80 con 0x7fb6ac0feca0 2024-09-13T16:29:10.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.200+0000 7fb6b20e9640 1 -- 172.21.15.16:0/49987368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6ac19cc80 con 0x7fb6ac0feca0 2024-09-13T16:29:10.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.200+0000 7fb6a8ff9640 1 -- 172.21.15.16:0/49987368 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb694003d40 con 0x7fb6ac0feca0 2024-09-13T16:29:10.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.200+0000 7fb6b20e9640 1 -- 172.21.15.16:0/49987368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6ac19d190 con 0x7fb6ac0feca0 2024-09-13T16:29:10.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.201+0000 7fb6b20e9640 1 -- 172.21.15.16:0/49987368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6ac1061d0 con 0x7fb6ac0feca0 2024-09-13T16:29:10.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.203+0000 7fb6a8ff9640 1 -- 172.21.15.16:0/49987368 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fb694003ea0 con 0x7fb6ac0feca0 2024-09-13T16:29:10.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.203+0000 7fb6a8ff9640 1 --2- 172.21.15.16:0/49987368 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb6980a1120 0x7fb6980a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:10.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.204+0000 7fb6a8ff9640 1 -- 172.21.15.16:0/49987368 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb6940cc720 con 0x7fb6ac0feca0 2024-09-13T16:29:10.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.204+0000 7fb6aaffd640 1 --2- 172.21.15.16:0/49987368 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb6980a1120 0x7fb6980a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:10.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.204+0000 7fb6aaffd640 1 --2- 172.21.15.16:0/49987368 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb6980a1120 0x7fb6980a35e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac196090 tx=0x7fb69c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:10.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.208+0000 7fb6a8ff9640 1 -- 172.21.15.16:0/49987368 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb694095a40 con 0x7fb6ac0feca0 2024-09-13T16:29:10.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.378+0000 7fb6b20e9640 1 -- 172.21.15.16:0/49987368 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb6ac02c450 con 0x7fb6980a1120 2024-09-13T16:29:10.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.389+0000 7fb6a8ff9640 1 -- 172.21.15.16:0/49987368 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422976 (secure 0 0 0) 0x7fb6ac02c450 con 0x7fb6980a1120 2024-09-13T16:29:10.401 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:29:10.405 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-13T16:29:10.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.392+0000 7fb6b20e9640 1 -- 172.21.15.16:0/49987368 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb6980a1120 msgr2=0x7fb6980a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:10.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.392+0000 7fb6b20e9640 1 --2- 172.21.15.16:0/49987368 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb6980a1120 0x7fb6980a35e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac196090 tx=0x7fb69c00b410 comp rx=0 tx=0).stop 2024-09-13T16:29:10.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.393+0000 7fb6b20e9640 1 -- 172.21.15.16:0/49987368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ac0feca0 msgr2=0x7fb6ac194b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:10.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.393+0000 7fb6b20e9640 1 --2- 172.21.15.16:0/49987368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ac0feca0 0x7fb6ac194b80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb69400d770 tx=0x7fb69400dc30 comp rx=0 tx=0).stop 2024-09-13T16:29:10.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.393+0000 7fb6b20e9640 1 -- 172.21.15.16:0/49987368 shutdown_connections 2024-09-13T16:29:10.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.393+0000 7fb6b20e9640 1 --2- 172.21.15.16:0/49987368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6ac100850 0x7fb6ac19c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:10.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.393+0000 7fb6b20e9640 1 --2- 172.21.15.16:0/49987368 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb6980a1120 0x7fb6980a35e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:10.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.393+0000 7fb6b20e9640 1 --2- 172.21.15.16:0/49987368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6ac0ffe90 0x7fb6ac1950c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:10.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.393+0000 7fb6b20e9640 1 --2- 172.21.15.16:0/49987368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ac0feca0 0x7fb6ac194b80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:10.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.393+0000 7fb6b20e9640 1 -- 172.21.15.16:0/49987368 >> 172.21.15.16:0/49987368 conn(0x7fb6ac0fa420 msgr2=0x7fb6ac1033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:10.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.393+0000 7fb6b20e9640 1 -- 172.21.15.16:0/49987368 shutdown_connections 2024-09-13T16:29:10.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:10.393+0000 7fb6b20e9640 1 -- 172.21.15.16:0/49987368 wait complete. 2024-09-13T16:29:10.409 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-13T16:29:08.571055+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":608404,"num_objects":100,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":183,"num_read_kb":155,"num_write":396,"num_write_kb":3018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1181498,"num_keys_recovered":10,"num_objects_omap":64,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":417,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364888,"kb_used_data":11980,"kb_used_omap":18,"kb_used_meta":352749,"kb_avail":1124576936,"statfs":{"total":1151940427776,"available":1151566782464,"internally_reserved":0,"allocated":12267520,"data_stored":7711739,"data_compressed":303615,"data_compressed_allocated":405504,"data_compressed_original":2310144,"omap_allocated":19131,"internal_metadata":361215301},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13443,"num_objects":76,"num_object_clones":0,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":57,"num_read_kb":46,"num_write":167,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":762,"num_keys_recovered":10,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002208"},"pg_stats":[{"pgid":"2.1f","version":"91'13","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.648333+0000","last_change":"2024-09-13T16:29:05.647872+0000","last_active":"2024-09-13T16:29:05.648333+0000","last_peered":"2024-09-13T16:29:05.648333+0000","last_clean":"2024-09-13T16:29:05.648333+0000","last_became_active":"2024-09-13T16:29:05.637050+0000","last_became_peered":"2024-09-13T16:29:05.637050+0000","last_unstale":"2024-09-13T16:29:05.648333+0000","last_undegraded":"2024-09-13T16:29:05.648333+0000","last_fullsized":"2024-09-13T16:29:05.648333+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:04.113840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.045098+0000","last_change":"2024-09-13T16:28:39.210024+0000","last_active":"2024-09-13T16:28:44.045098+0000","last_peered":"2024-09-13T16:28:44.045098+0000","last_clean":"2024-09-13T16:28:44.045098+0000","last_became_active":"2024-09-13T16:28:39.209751+0000","last_became_peered":"2024-09-13T16:28:39.209751+0000","last_unstale":"2024-09-13T16:28:44.045098+0000","last_undegraded":"2024-09-13T16:28:44.045098+0000","last_fullsized":"2024-09-13T16:28:44.045098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:34:24.357779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.632148+0000","last_change":"2024-09-13T16:28:43.230223+0000","last_active":"2024-09-13T16:29:04.632148+0000","last_peered":"2024-09-13T16:29:04.632148+0000","last_clean":"2024-09-13T16:29:04.632148+0000","last_became_active":"2024-09-13T16:28:43.228992+0000","last_became_peered":"2024-09-13T16:28:43.228992+0000","last_unstale":"2024-09-13T16:29:04.632148+0000","last_undegraded":"2024-09-13T16:29:04.632148+0000","last_fullsized":"2024-09-13T16:29:04.632148+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:45:37.252865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:29:03.591838+0000","last_change":"2024-09-13T16:28:38.850198+0000","last_active":"2024-09-13T16:29:03.591838+0000","last_peered":"2024-09-13T16:29:03.591838+0000","last_clean":"2024-09-13T16:29:03.591838+0000","last_became_active":"2024-09-13T16:28:38.849780+0000","last_became_peered":"2024-09-13T16:28:38.849780+0000","last_unstale":"2024-09-13T16:29:03.591838+0000","last_undegraded":"2024-09-13T16:29:03.591838+0000","last_fullsized":"2024-09-13T16:29:03.591838+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:09.653474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T16:28:43.218832+0000","last_change":"2024-09-13T16:28:39.203030+0000","last_active":"2024-09-13T16:28:43.218832+0000","last_peered":"2024-09-13T16:28:43.218832+0000","last_clean":"2024-09-13T16:28:43.218832+0000","last_became_active":"2024-09-13T16:28:39.202464+0000","last_became_peered":"2024-09-13T16:28:39.202464+0000","last_unstale":"2024-09-13T16:28:43.218832+0000","last_undegraded":"2024-09-13T16:28:43.218832+0000","last_fullsized":"2024-09-13T16:28:43.218832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:57.528836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.632329+0000","last_change":"2024-09-13T16:28:43.224358+0000","last_active":"2024-09-13T16:29:04.632329+0000","last_peered":"2024-09-13T16:29:04.632329+0000","last_clean":"2024-09-13T16:29:04.632329+0000","last_became_active":"2024-09-13T16:28:43.223955+0000","last_became_peered":"2024-09-13T16:28:43.223955+0000","last_unstale":"2024-09-13T16:29:04.632329+0000","last_undegraded":"2024-09-13T16:29:04.632329+0000","last_fullsized":"2024-09-13T16:29:04.632329+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:53:09.169704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'11","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.634720+0000","last_change":"2024-09-13T16:28:38.846051+0000","last_active":"2024-09-13T16:29:05.634720+0000","last_peered":"2024-09-13T16:29:05.634720+0000","last_clean":"2024-09-13T16:29:05.634720+0000","last_became_active":"2024-09-13T16:28:38.845762+0000","last_became_peered":"2024-09-13T16:28:38.845762+0000","last_unstale":"2024-09-13T16:29:05.634720+0000","last_undegraded":"2024-09-13T16:29:05.634720+0000","last_fullsized":"2024-09-13T16:29:05.634720+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:33:47.048582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044544+0000","last_change":"2024-09-13T16:28:39.210516+0000","last_active":"2024-09-13T16:28:44.044544+0000","last_peered":"2024-09-13T16:28:44.044544+0000","last_clean":"2024-09-13T16:28:44.044544+0000","last_became_active":"2024-09-13T16:28:39.210214+0000","last_became_peered":"2024-09-13T16:28:39.210214+0000","last_unstale":"2024-09-13T16:28:44.044544+0000","last_undegraded":"2024-09-13T16:28:44.044544+0000","last_fullsized":"2024-09-13T16:28:44.044544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:51:58.212685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.630396+0000","last_change":"2024-09-13T16:28:43.230530+0000","last_active":"2024-09-13T16:29:04.630396+0000","last_peered":"2024-09-13T16:29:04.630396+0000","last_clean":"2024-09-13T16:29:04.630396+0000","last_became_active":"2024-09-13T16:28:43.230154+0000","last_became_peered":"2024-09-13T16:28:43.230154+0000","last_unstale":"2024-09-13T16:29:04.630396+0000","last_undegraded":"2024-09-13T16:29:04.630396+0000","last_fullsized":"2024-09-13T16:29:04.630396+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:39:52.362177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.653834+0000","last_change":"2024-09-13T16:29:05.653460+0000","last_active":"2024-09-13T16:29:05.653834+0000","last_peered":"2024-09-13T16:29:05.653834+0000","last_clean":"2024-09-13T16:29:05.653834+0000","last_became_active":"2024-09-13T16:29:05.641836+0000","last_became_peered":"2024-09-13T16:29:05.641836+0000","last_unstale":"2024-09-13T16:29:05.653834+0000","last_undegraded":"2024-09-13T16:29:05.653834+0000","last_fullsized":"2024-09-13T16:29:05.653834+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:22:30.051033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.624846+0000","last_change":"2024-09-13T16:28:39.202381+0000","last_active":"2024-09-13T16:29:04.624846+0000","last_peered":"2024-09-13T16:29:04.624846+0000","last_clean":"2024-09-13T16:29:04.624846+0000","last_became_active":"2024-09-13T16:28:39.202023+0000","last_became_peered":"2024-09-13T16:28:39.202023+0000","last_unstale":"2024-09-13T16:29:04.624846+0000","last_undegraded":"2024-09-13T16:29:04.624846+0000","last_fullsized":"2024-09-13T16:29:04.624846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:55:46.445085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628398+0000","last_change":"2024-09-13T16:28:43.230623+0000","last_active":"2024-09-13T16:29:04.628398+0000","last_peered":"2024-09-13T16:29:04.628398+0000","last_clean":"2024-09-13T16:29:04.628398+0000","last_became_active":"2024-09-13T16:28:43.230240+0000","last_became_peered":"2024-09-13T16:28:43.230240+0000","last_unstale":"2024-09-13T16:29:04.628398+0000","last_undegraded":"2024-09-13T16:29:04.628398+0000","last_fullsized":"2024-09-13T16:29:04.628398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:04:07.723269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625334+0000","last_change":"2024-09-13T16:28:39.201896+0000","last_active":"2024-09-13T16:29:04.625334+0000","last_peered":"2024-09-13T16:29:04.625334+0000","last_clean":"2024-09-13T16:29:04.625334+0000","last_became_active":"2024-09-13T16:28:39.201570+0000","last_became_peered":"2024-09-13T16:28:39.201570+0000","last_unstale":"2024-09-13T16:29:04.625334+0000","last_undegraded":"2024-09-13T16:29:04.625334+0000","last_fullsized":"2024-09-13T16:29:04.625334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:07.532058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627167+0000","last_change":"2024-09-13T16:28:38.845998+0000","last_active":"2024-09-13T16:29:04.627167+0000","last_peered":"2024-09-13T16:29:04.627167+0000","last_clean":"2024-09-13T16:29:04.627167+0000","last_became_active":"2024-09-13T16:28:38.845451+0000","last_became_peered":"2024-09-13T16:28:38.845451+0000","last_unstale":"2024-09-13T16:29:04.627167+0000","last_undegraded":"2024-09-13T16:29:04.627167+0000","last_fullsized":"2024-09-13T16:29:04.627167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:15.696276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628820+0000","last_change":"2024-09-13T16:28:43.293598+0000","last_active":"2024-09-13T16:29:04.628820+0000","last_peered":"2024-09-13T16:29:04.628820+0000","last_clean":"2024-09-13T16:29:04.628820+0000","last_became_active":"2024-09-13T16:28:43.293045+0000","last_became_peered":"2024-09-13T16:28:43.293045+0000","last_unstale":"2024-09-13T16:29:04.628820+0000","last_undegraded":"2024-09-13T16:29:04.628820+0000","last_fullsized":"2024-09-13T16:29:04.628820+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:01.839626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044938+0000","last_change":"2024-09-13T16:28:39.214302+0000","last_active":"2024-09-13T16:28:44.044938+0000","last_peered":"2024-09-13T16:28:44.044938+0000","last_clean":"2024-09-13T16:28:44.044938+0000","last_became_active":"2024-09-13T16:28:39.213672+0000","last_became_peered":"2024-09-13T16:28:39.213672+0000","last_unstale":"2024-09-13T16:28:44.044938+0000","last_undegraded":"2024-09-13T16:28:44.044938+0000","last_fullsized":"2024-09-13T16:28:44.044938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:03:25.440603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"91'2","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.631445+0000","last_change":"2024-09-13T16:28:38.840626+0000","last_active":"2024-09-13T16:29:05.631445+0000","last_peered":"2024-09-13T16:29:05.631445+0000","last_clean":"2024-09-13T16:29:05.631445+0000","last_became_active":"2024-09-13T16:28:38.840344+0000","last_became_peered":"2024-09-13T16:28:38.840344+0000","last_unstale":"2024-09-13T16:29:05.631445+0000","last_undegraded":"2024-09-13T16:29:05.631445+0000","last_fullsized":"2024-09-13T16:29:05.631445+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:03:10.850301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.633381+0000","last_change":"2024-09-13T16:28:43.229163+0000","last_active":"2024-09-13T16:29:05.633381+0000","last_peered":"2024-09-13T16:29:05.633381+0000","last_clean":"2024-09-13T16:29:05.633381+0000","last_became_active":"2024-09-13T16:28:43.228783+0000","last_became_peered":"2024-09-13T16:28:43.228783+0000","last_unstale":"2024-09-13T16:29:05.633381+0000","last_undegraded":"2024-09-13T16:29:05.633381+0000","last_fullsized":"2024-09-13T16:29:05.633381+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:48.183837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627877+0000","last_change":"2024-09-13T16:28:39.209247+0000","last_active":"2024-09-13T16:29:04.627877+0000","last_peered":"2024-09-13T16:29:04.627877+0000","last_clean":"2024-09-13T16:29:04.627877+0000","last_became_active":"2024-09-13T16:28:39.208921+0000","last_became_peered":"2024-09-13T16:28:39.208921+0000","last_unstale":"2024-09-13T16:29:04.627877+0000","last_undegraded":"2024-09-13T16:29:04.627877+0000","last_fullsized":"2024-09-13T16:29:04.627877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:32:41.383029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.640902+0000","last_change":"2024-09-13T16:28:38.845904+0000","last_active":"2024-09-13T16:29:04.640902+0000","last_peered":"2024-09-13T16:29:04.640902+0000","last_clean":"2024-09-13T16:29:04.640902+0000","last_became_active":"2024-09-13T16:28:38.845639+0000","last_became_peered":"2024-09-13T16:28:38.845639+0000","last_unstale":"2024-09-13T16:29:04.640902+0000","last_undegraded":"2024-09-13T16:29:04.640902+0000","last_fullsized":"2024-09-13T16:29:04.640902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:25:34.148607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.640627+0000","last_change":"2024-09-13T16:29:05.640063+0000","last_active":"2024-09-13T16:29:05.640627+0000","last_peered":"2024-09-13T16:29:05.640627+0000","last_clean":"2024-09-13T16:29:05.640627+0000","last_became_active":"2024-09-13T16:29:05.639596+0000","last_became_peered":"2024-09-13T16:29:05.639596+0000","last_unstale":"2024-09-13T16:29:05.640627+0000","last_undegraded":"2024-09-13T16:29:05.640627+0000","last_fullsized":"2024-09-13T16:29:05.640627+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:41:38.351162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.626859+0000","last_change":"2024-09-13T16:28:39.212947+0000","last_active":"2024-09-13T16:29:04.626859+0000","last_peered":"2024-09-13T16:29:04.626859+0000","last_clean":"2024-09-13T16:29:04.626859+0000","last_became_active":"2024-09-13T16:28:39.212622+0000","last_became_peered":"2024-09-13T16:28:39.212622+0000","last_unstale":"2024-09-13T16:29:04.626859+0000","last_undegraded":"2024-09-13T16:29:04.626859+0000","last_fullsized":"2024-09-13T16:29:04.626859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:16.425154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:29:02.581304+0000","last_change":"2024-09-13T16:28:38.846173+0000","last_active":"2024-09-13T16:29:02.581304+0000","last_peered":"2024-09-13T16:29:02.581304+0000","last_clean":"2024-09-13T16:29:02.581304+0000","last_became_active":"2024-09-13T16:28:38.845873+0000","last_became_peered":"2024-09-13T16:28:38.845873+0000","last_unstale":"2024-09-13T16:29:02.581304+0000","last_undegraded":"2024-09-13T16:29:02.581304+0000","last_fullsized":"2024-09-13T16:29:02.581304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:19:55.817524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629006+0000","last_change":"2024-09-13T16:28:43.294347+0000","last_active":"2024-09-13T16:29:04.629006+0000","last_peered":"2024-09-13T16:29:04.629006+0000","last_clean":"2024-09-13T16:29:04.629006+0000","last_became_active":"2024-09-13T16:28:43.293902+0000","last_became_peered":"2024-09-13T16:28:43.293902+0000","last_unstale":"2024-09-13T16:29:04.629006+0000","last_undegraded":"2024-09-13T16:29:04.629006+0000","last_fullsized":"2024-09-13T16:29:04.629006+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:27:37.470163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T16:28:43.217326+0000","last_change":"2024-09-13T16:28:39.205817+0000","last_active":"2024-09-13T16:28:43.217326+0000","last_peered":"2024-09-13T16:28:43.217326+0000","last_clean":"2024-09-13T16:28:43.217326+0000","last_became_active":"2024-09-13T16:28:39.205534+0000","last_became_peered":"2024-09-13T16:28:39.205534+0000","last_unstale":"2024-09-13T16:28:43.217326+0000","last_undegraded":"2024-09-13T16:28:43.217326+0000","last_fullsized":"2024-09-13T16:28:43.217326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:09:16.294842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.633540+0000","last_change":"2024-09-13T16:28:38.847582+0000","last_active":"2024-09-13T16:29:05.633540+0000","last_peered":"2024-09-13T16:29:05.633540+0000","last_clean":"2024-09-13T16:29:05.633540+0000","last_became_active":"2024-09-13T16:28:38.847353+0000","last_became_peered":"2024-09-13T16:28:38.847353+0000","last_unstale":"2024-09-13T16:29:05.633540+0000","last_undegraded":"2024-09-13T16:29:05.633540+0000","last_fullsized":"2024-09-13T16:29:05.633540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:24.944088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.634008+0000","last_change":"2024-09-13T16:28:43.225202+0000","last_active":"2024-09-13T16:29:04.634008+0000","last_peered":"2024-09-13T16:29:04.634008+0000","last_clean":"2024-09-13T16:29:04.634008+0000","last_became_active":"2024-09-13T16:28:43.224802+0000","last_became_peered":"2024-09-13T16:28:43.224802+0000","last_unstale":"2024-09-13T16:29:04.634008+0000","last_undegraded":"2024-09-13T16:29:04.634008+0000","last_fullsized":"2024-09-13T16:29:04.634008+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:51:19.325859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.045516+0000","last_change":"2024-09-13T16:28:39.209630+0000","last_active":"2024-09-13T16:28:44.045516+0000","last_peered":"2024-09-13T16:28:44.045516+0000","last_clean":"2024-09-13T16:28:44.045516+0000","last_became_active":"2024-09-13T16:28:39.209303+0000","last_became_peered":"2024-09-13T16:28:39.209303+0000","last_unstale":"2024-09-13T16:28:44.045516+0000","last_undegraded":"2024-09-13T16:28:44.045516+0000","last_fullsized":"2024-09-13T16:28:44.045516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:01:31.822673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'7","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628093+0000","last_change":"2024-09-13T16:28:38.837858+0000","last_active":"2024-09-13T16:29:04.628093+0000","last_peered":"2024-09-13T16:29:04.628093+0000","last_clean":"2024-09-13T16:29:04.628093+0000","last_became_active":"2024-09-13T16:28:38.837378+0000","last_became_peered":"2024-09-13T16:28:38.837378+0000","last_unstale":"2024-09-13T16:29:04.628093+0000","last_undegraded":"2024-09-13T16:29:04.628093+0000","last_fullsized":"2024-09-13T16:29:04.628093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:59:41.055902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.754728+0000","last_change":"2024-09-13T16:28:43.294241+0000","last_active":"2024-09-13T16:28:44.754728+0000","last_peered":"2024-09-13T16:28:44.754728+0000","last_clean":"2024-09-13T16:28:44.754728+0000","last_became_active":"2024-09-13T16:28:43.293873+0000","last_became_peered":"2024-09-13T16:28:43.293873+0000","last_unstale":"2024-09-13T16:28:44.754728+0000","last_undegraded":"2024-09-13T16:28:44.754728+0000","last_fullsized":"2024-09-13T16:28:44.754728+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:34.018162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044782+0000","last_change":"2024-09-13T16:28:39.218071+0000","last_active":"2024-09-13T16:28:44.044782+0000","last_peered":"2024-09-13T16:28:44.044782+0000","last_clean":"2024-09-13T16:28:44.044782+0000","last_became_active":"2024-09-13T16:28:39.217798+0000","last_became_peered":"2024-09-13T16:28:39.217798+0000","last_unstale":"2024-09-13T16:28:44.044782+0000","last_undegraded":"2024-09-13T16:28:44.044782+0000","last_fullsized":"2024-09-13T16:28:44.044782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:39:09.769384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.640833+0000","last_change":"2024-09-13T16:28:38.847238+0000","last_active":"2024-09-13T16:29:04.640833+0000","last_peered":"2024-09-13T16:29:04.640833+0000","last_clean":"2024-09-13T16:29:04.640833+0000","last_became_active":"2024-09-13T16:28:38.846741+0000","last_became_peered":"2024-09-13T16:28:38.846741+0000","last_unstale":"2024-09-13T16:29:04.640833+0000","last_undegraded":"2024-09-13T16:29:04.640833+0000","last_fullsized":"2024-09-13T16:29:04.640833+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:52:51.946158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628298+0000","last_change":"2024-09-13T16:28:43.228431+0000","last_active":"2024-09-13T16:29:04.628298+0000","last_peered":"2024-09-13T16:29:04.628298+0000","last_clean":"2024-09-13T16:29:04.628298+0000","last_became_active":"2024-09-13T16:28:43.228030+0000","last_became_peered":"2024-09-13T16:28:43.228030+0000","last_unstale":"2024-09-13T16:29:04.628298+0000","last_undegraded":"2024-09-13T16:29:04.628298+0000","last_fullsized":"2024-09-13T16:29:04.628298+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:14:50.573033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629589+0000","last_change":"2024-09-13T16:28:39.209485+0000","last_active":"2024-09-13T16:29:04.629589+0000","last_peered":"2024-09-13T16:29:04.629589+0000","last_clean":"2024-09-13T16:29:04.629589+0000","last_became_active":"2024-09-13T16:28:39.209157+0000","last_became_peered":"2024-09-13T16:28:39.209157+0000","last_unstale":"2024-09-13T16:29:04.629589+0000","last_undegraded":"2024-09-13T16:29:04.629589+0000","last_fullsized":"2024-09-13T16:29:04.629589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:12:19.398255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628085+0000","last_change":"2024-09-13T16:28:38.842551+0000","last_active":"2024-09-13T16:29:04.628085+0000","last_peered":"2024-09-13T16:29:04.628085+0000","last_clean":"2024-09-13T16:29:04.628085+0000","last_became_active":"2024-09-13T16:28:38.841741+0000","last_became_peered":"2024-09-13T16:28:38.841741+0000","last_unstale":"2024-09-13T16:29:04.628085+0000","last_undegraded":"2024-09-13T16:29:04.628085+0000","last_fullsized":"2024-09-13T16:29:04.628085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:35:25.131693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.626884+0000","last_change":"2024-09-13T16:28:43.294300+0000","last_active":"2024-09-13T16:29:04.626884+0000","last_peered":"2024-09-13T16:29:04.626884+0000","last_clean":"2024-09-13T16:29:04.626884+0000","last_became_active":"2024-09-13T16:28:43.293918+0000","last_became_peered":"2024-09-13T16:28:43.293918+0000","last_unstale":"2024-09-13T16:29:04.626884+0000","last_undegraded":"2024-09-13T16:29:04.626884+0000","last_fullsized":"2024-09-13T16:29:04.626884+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:19:21.959617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.631071+0000","last_change":"2024-09-13T16:28:39.212001+0000","last_active":"2024-09-13T16:29:04.631071+0000","last_peered":"2024-09-13T16:29:04.631071+0000","last_clean":"2024-09-13T16:29:04.631071+0000","last_became_active":"2024-09-13T16:28:39.211716+0000","last_became_peered":"2024-09-13T16:28:39.211716+0000","last_unstale":"2024-09-13T16:29:04.631071+0000","last_undegraded":"2024-09-13T16:29:04.631071+0000","last_fullsized":"2024-09-13T16:29:04.631071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:56.753677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:06.648061+0000","last_change":"2024-09-13T16:28:38.851229+0000","last_active":"2024-09-13T16:29:06.648061+0000","last_peered":"2024-09-13T16:29:06.648061+0000","last_clean":"2024-09-13T16:29:06.648061+0000","last_became_active":"2024-09-13T16:28:38.850937+0000","last_became_peered":"2024-09-13T16:28:38.850937+0000","last_unstale":"2024-09-13T16:29:06.648061+0000","last_undegraded":"2024-09-13T16:29:06.648061+0000","last_fullsized":"2024-09-13T16:29:06.648061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:19.460864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:45.495529+0000","last_change":"2024-09-13T16:28:43.224825+0000","last_active":"2024-09-13T16:28:45.495529+0000","last_peered":"2024-09-13T16:28:45.495529+0000","last_clean":"2024-09-13T16:28:45.495529+0000","last_became_active":"2024-09-13T16:28:43.223903+0000","last_became_peered":"2024-09-13T16:28:43.223903+0000","last_unstale":"2024-09-13T16:28:45.495529+0000","last_undegraded":"2024-09-13T16:28:45.495529+0000","last_fullsized":"2024-09-13T16:28:45.495529+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:08:46.850668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.639142+0000","last_change":"2024-09-13T16:29:05.636787+0000","last_active":"2024-09-13T16:29:05.639142+0000","last_peered":"2024-09-13T16:29:05.639142+0000","last_clean":"2024-09-13T16:29:05.639142+0000","last_became_active":"2024-09-13T16:29:05.636399+0000","last_became_peered":"2024-09-13T16:29:05.636399+0000","last_unstale":"2024-09-13T16:29:05.639142+0000","last_undegraded":"2024-09-13T16:29:05.639142+0000","last_fullsized":"2024-09-13T16:29:05.639142+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:38:25.545325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.622845+0000","last_change":"2024-09-13T16:28:38.845374+0000","last_active":"2024-09-13T16:29:04.622845+0000","last_peered":"2024-09-13T16:29:04.622845+0000","last_clean":"2024-09-13T16:29:04.622845+0000","last_became_active":"2024-09-13T16:28:38.845076+0000","last_became_peered":"2024-09-13T16:28:38.845076+0000","last_unstale":"2024-09-13T16:29:04.622845+0000","last_undegraded":"2024-09-13T16:29:04.622845+0000","last_fullsized":"2024-09-13T16:29:04.622845+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:37:56.225682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:45.495577+0000","last_change":"2024-09-13T16:28:43.293529+0000","last_active":"2024-09-13T16:28:45.495577+0000","last_peered":"2024-09-13T16:28:45.495577+0000","last_clean":"2024-09-13T16:28:45.495577+0000","last_became_active":"2024-09-13T16:28:43.293083+0000","last_became_peered":"2024-09-13T16:28:43.293083+0000","last_unstale":"2024-09-13T16:28:45.495577+0000","last_undegraded":"2024-09-13T16:28:45.495577+0000","last_fullsized":"2024-09-13T16:28:45.495577+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:05:55.572984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.043722+0000","last_change":"2024-09-13T16:28:39.217546+0000","last_active":"2024-09-13T16:28:44.043722+0000","last_peered":"2024-09-13T16:28:44.043722+0000","last_clean":"2024-09-13T16:28:44.043722+0000","last_became_active":"2024-09-13T16:28:39.217242+0000","last_became_peered":"2024-09-13T16:28:39.217242+0000","last_unstale":"2024-09-13T16:28:44.043722+0000","last_undegraded":"2024-09-13T16:28:44.043722+0000","last_fullsized":"2024-09-13T16:28:44.043722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:23:18.183616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.641545+0000","last_change":"2024-09-13T16:28:38.846436+0000","last_active":"2024-09-13T16:29:04.641545+0000","last_peered":"2024-09-13T16:29:04.641545+0000","last_clean":"2024-09-13T16:29:04.641545+0000","last_became_active":"2024-09-13T16:28:38.845953+0000","last_became_peered":"2024-09-13T16:28:38.845953+0000","last_unstale":"2024-09-13T16:29:04.641545+0000","last_undegraded":"2024-09-13T16:29:04.641545+0000","last_fullsized":"2024-09-13T16:29:04.641545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:33.998462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.624750+0000","last_change":"2024-09-13T16:28:43.294504+0000","last_active":"2024-09-13T16:29:04.624750+0000","last_peered":"2024-09-13T16:29:04.624750+0000","last_clean":"2024-09-13T16:29:04.624750+0000","last_became_active":"2024-09-13T16:28:43.294063+0000","last_became_peered":"2024-09-13T16:28:43.294063+0000","last_unstale":"2024-09-13T16:29:04.624750+0000","last_undegraded":"2024-09-13T16:29:04.624750+0000","last_fullsized":"2024-09-13T16:29:04.624750+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:51:35.774467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044939+0000","last_change":"2024-09-13T16:28:39.206491+0000","last_active":"2024-09-13T16:28:44.044939+0000","last_peered":"2024-09-13T16:28:44.044939+0000","last_clean":"2024-09-13T16:28:44.044939+0000","last_became_active":"2024-09-13T16:28:39.206187+0000","last_became_peered":"2024-09-13T16:28:39.206187+0000","last_unstale":"2024-09-13T16:28:44.044939+0000","last_undegraded":"2024-09-13T16:28:44.044939+0000","last_fullsized":"2024-09-13T16:28:44.044939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:20:28.457994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.635563+0000","last_change":"2024-09-13T16:28:38.836270+0000","last_active":"2024-09-13T16:29:05.635563+0000","last_peered":"2024-09-13T16:29:05.635563+0000","last_clean":"2024-09-13T16:29:05.635563+0000","last_became_active":"2024-09-13T16:28:38.835966+0000","last_became_peered":"2024-09-13T16:28:38.835966+0000","last_unstale":"2024-09-13T16:29:05.635563+0000","last_undegraded":"2024-09-13T16:29:05.635563+0000","last_fullsized":"2024-09-13T16:29:05.635563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:03:11.388654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":331,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625402+0000","last_change":"2024-09-13T16:27:34.264484+0000","last_active":"2024-09-13T16:29:04.625402+0000","last_peered":"2024-09-13T16:29:04.625402+0000","last_clean":"2024-09-13T16:29:04.625402+0000","last_became_active":"2024-09-13T16:27:34.263995+0000","last_became_peered":"2024-09-13T16:27:34.263995+0000","last_unstale":"2024-09-13T16:29:04.625402+0000","last_undegraded":"2024-09-13T16:29:04.625402+0000","last_fullsized":"2024-09-13T16:29:04.625402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:24:23.015578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:24:23.015578+0000","last_clean_scrub_stamp":"2024-09-13T16:24:23.015578+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:25:04.376312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.634973+0000","last_change":"2024-09-13T16:28:43.229347+0000","last_active":"2024-09-13T16:29:05.634973+0000","last_peered":"2024-09-13T16:29:05.634973+0000","last_clean":"2024-09-13T16:29:05.634973+0000","last_became_active":"2024-09-13T16:28:43.228981+0000","last_became_peered":"2024-09-13T16:28:43.228981+0000","last_unstale":"2024-09-13T16:29:05.634973+0000","last_undegraded":"2024-09-13T16:29:05.634973+0000","last_fullsized":"2024-09-13T16:29:05.634973+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:46:15.151295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625877+0000","last_change":"2024-09-13T16:28:39.203195+0000","last_active":"2024-09-13T16:29:04.625877+0000","last_peered":"2024-09-13T16:29:04.625877+0000","last_clean":"2024-09-13T16:29:04.625877+0000","last_became_active":"2024-09-13T16:28:39.202878+0000","last_became_peered":"2024-09-13T16:28:39.202878+0000","last_unstale":"2024-09-13T16:29:04.625877+0000","last_undegraded":"2024-09-13T16:29:04.625877+0000","last_fullsized":"2024-09-13T16:29:04.625877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:17.615531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:06.646697+0000","last_change":"2024-09-13T16:28:38.844856+0000","last_active":"2024-09-13T16:29:06.646697+0000","last_peered":"2024-09-13T16:29:06.646697+0000","last_clean":"2024-09-13T16:29:06.646697+0000","last_became_active":"2024-09-13T16:28:38.844578+0000","last_became_peered":"2024-09-13T16:28:38.844578+0000","last_unstale":"2024-09-13T16:29:06.646697+0000","last_undegraded":"2024-09-13T16:29:06.646697+0000","last_fullsized":"2024-09-13T16:29:06.646697+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:41:48.253174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:45.572882+0000","last_change":"2024-09-13T16:28:43.228472+0000","last_active":"2024-09-13T16:28:45.572882+0000","last_peered":"2024-09-13T16:28:45.572882+0000","last_clean":"2024-09-13T16:28:45.572882+0000","last_became_active":"2024-09-13T16:28:43.228056+0000","last_became_peered":"2024-09-13T16:28:43.228056+0000","last_unstale":"2024-09-13T16:28:45.572882+0000","last_undegraded":"2024-09-13T16:28:45.572882+0000","last_fullsized":"2024-09-13T16:28:45.572882+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:07:55.980691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625637+0000","last_change":"2024-09-13T16:28:39.203278+0000","last_active":"2024-09-13T16:29:04.625637+0000","last_peered":"2024-09-13T16:29:04.625637+0000","last_clean":"2024-09-13T16:29:04.625637+0000","last_became_active":"2024-09-13T16:28:39.202936+0000","last_became_peered":"2024-09-13T16:28:39.202936+0000","last_unstale":"2024-09-13T16:29:04.625637+0000","last_undegraded":"2024-09-13T16:29:04.625637+0000","last_fullsized":"2024-09-13T16:29:04.625637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:44.210612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.640824+0000","last_change":"2024-09-13T16:28:38.851085+0000","last_active":"2024-09-13T16:29:04.640824+0000","last_peered":"2024-09-13T16:29:04.640824+0000","last_clean":"2024-09-13T16:29:04.640824+0000","last_became_active":"2024-09-13T16:28:38.850785+0000","last_became_peered":"2024-09-13T16:28:38.850785+0000","last_unstale":"2024-09-13T16:29:04.640824+0000","last_undegraded":"2024-09-13T16:29:04.640824+0000","last_fullsized":"2024-09-13T16:29:04.640824+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:52.713431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044053+0000","last_change":"2024-09-13T16:28:43.229980+0000","last_active":"2024-09-13T16:28:44.044053+0000","last_peered":"2024-09-13T16:28:44.044053+0000","last_clean":"2024-09-13T16:28:44.044053+0000","last_became_active":"2024-09-13T16:28:43.229647+0000","last_became_peered":"2024-09-13T16:28:43.229647+0000","last_unstale":"2024-09-13T16:28:44.044053+0000","last_undegraded":"2024-09-13T16:28:44.044053+0000","last_fullsized":"2024-09-13T16:28:44.044053+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:39:42.871613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625770+0000","last_change":"2024-09-13T16:28:39.204167+0000","last_active":"2024-09-13T16:29:04.625770+0000","last_peered":"2024-09-13T16:29:04.625770+0000","last_clean":"2024-09-13T16:29:04.625770+0000","last_became_active":"2024-09-13T16:28:39.203776+0000","last_became_peered":"2024-09-13T16:28:39.203776+0000","last_unstale":"2024-09-13T16:29:04.625770+0000","last_undegraded":"2024-09-13T16:29:04.625770+0000","last_fullsized":"2024-09-13T16:29:04.625770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:53:36.410365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.639297+0000","last_change":"2024-09-13T16:28:38.846331+0000","last_active":"2024-09-13T16:29:04.639297+0000","last_peered":"2024-09-13T16:29:04.639297+0000","last_clean":"2024-09-13T16:29:04.639297+0000","last_became_active":"2024-09-13T16:28:38.846012+0000","last_became_peered":"2024-09-13T16:28:38.846012+0000","last_unstale":"2024-09-13T16:29:04.639297+0000","last_undegraded":"2024-09-13T16:29:04.639297+0000","last_fullsized":"2024-09-13T16:29:04.639297+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:23:54.282736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.623954+0000","last_change":"2024-09-13T16:28:43.227886+0000","last_active":"2024-09-13T16:29:04.623954+0000","last_peered":"2024-09-13T16:29:04.623954+0000","last_clean":"2024-09-13T16:29:04.623954+0000","last_became_active":"2024-09-13T16:28:43.227346+0000","last_became_peered":"2024-09-13T16:28:43.227346+0000","last_unstale":"2024-09-13T16:29:04.623954+0000","last_undegraded":"2024-09-13T16:29:04.623954+0000","last_fullsized":"2024-09-13T16:29:04.623954+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:41:36.559064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629086+0000","last_change":"2024-09-13T16:28:39.205975+0000","last_active":"2024-09-13T16:29:04.629086+0000","last_peered":"2024-09-13T16:29:04.629086+0000","last_clean":"2024-09-13T16:29:04.629086+0000","last_became_active":"2024-09-13T16:28:39.205657+0000","last_became_peered":"2024-09-13T16:28:39.205657+0000","last_unstale":"2024-09-13T16:29:04.629086+0000","last_undegraded":"2024-09-13T16:29:04.629086+0000","last_fullsized":"2024-09-13T16:29:04.629086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:31:47.309482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629013+0000","last_change":"2024-09-13T16:28:38.841589+0000","last_active":"2024-09-13T16:29:04.629013+0000","last_peered":"2024-09-13T16:29:04.629013+0000","last_clean":"2024-09-13T16:29:04.629013+0000","last_became_active":"2024-09-13T16:28:38.841298+0000","last_became_peered":"2024-09-13T16:28:38.841298+0000","last_unstale":"2024-09-13T16:29:04.629013+0000","last_undegraded":"2024-09-13T16:29:04.629013+0000","last_fullsized":"2024-09-13T16:29:04.629013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:27:38.358320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.623986+0000","last_change":"2024-09-13T16:28:43.233790+0000","last_active":"2024-09-13T16:29:04.623986+0000","last_peered":"2024-09-13T16:29:04.623986+0000","last_clean":"2024-09-13T16:29:04.623986+0000","last_became_active":"2024-09-13T16:28:43.231963+0000","last_became_peered":"2024-09-13T16:28:43.231963+0000","last_unstale":"2024-09-13T16:29:04.623986+0000","last_undegraded":"2024-09-13T16:29:04.623986+0000","last_fullsized":"2024-09-13T16:29:04.623986+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:45:23.406395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.624233+0000","last_change":"2024-09-13T16:28:39.201476+0000","last_active":"2024-09-13T16:29:04.624233+0000","last_peered":"2024-09-13T16:29:04.624233+0000","last_clean":"2024-09-13T16:29:04.624233+0000","last_became_active":"2024-09-13T16:28:39.201168+0000","last_became_peered":"2024-09-13T16:28:39.201168+0000","last_unstale":"2024-09-13T16:29:04.624233+0000","last_undegraded":"2024-09-13T16:29:04.624233+0000","last_fullsized":"2024-09-13T16:29:04.624233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:26:43.570295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":62,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.652225+0000","last_change":"2024-09-13T16:29:05.652225+0000","last_active":"2024-09-13T16:29:05.652225+0000","last_peered":"2024-09-13T16:29:05.652225+0000","last_clean":"2024-09-13T16:29:05.652225+0000","last_became_active":"2024-09-13T16:29:05.639325+0000","last_became_peered":"2024-09-13T16:29:05.639325+0000","last_unstale":"2024-09-13T16:29:05.652225+0000","last_undegraded":"2024-09-13T16:29:05.652225+0000","last_fullsized":"2024-09-13T16:29:05.652225+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:18:30.083447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:45.495557+0000","last_change":"2024-09-13T16:28:43.228563+0000","last_active":"2024-09-13T16:28:45.495557+0000","last_peered":"2024-09-13T16:28:45.495557+0000","last_clean":"2024-09-13T16:28:45.495557+0000","last_became_active":"2024-09-13T16:28:43.228184+0000","last_became_peered":"2024-09-13T16:28:43.228184+0000","last_unstale":"2024-09-13T16:28:45.495557+0000","last_undegraded":"2024-09-13T16:28:45.495557+0000","last_fullsized":"2024-09-13T16:28:45.495557+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:05:33.368302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625221+0000","last_change":"2024-09-13T16:28:39.210750+0000","last_active":"2024-09-13T16:29:04.625221+0000","last_peered":"2024-09-13T16:29:04.625221+0000","last_clean":"2024-09-13T16:29:04.625221+0000","last_became_active":"2024-09-13T16:28:39.210401+0000","last_became_peered":"2024-09-13T16:28:39.210401+0000","last_unstale":"2024-09-13T16:29:04.625221+0000","last_undegraded":"2024-09-13T16:29:04.625221+0000","last_fullsized":"2024-09-13T16:29:04.625221+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:36.670420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:06.638428+0000","last_change":"2024-09-13T16:28:38.843573+0000","last_active":"2024-09-13T16:29:06.638428+0000","last_peered":"2024-09-13T16:29:06.638428+0000","last_clean":"2024-09-13T16:29:06.638428+0000","last_became_active":"2024-09-13T16:28:38.843245+0000","last_became_peered":"2024-09-13T16:28:38.843245+0000","last_unstale":"2024-09-13T16:29:06.638428+0000","last_undegraded":"2024-09-13T16:29:06.638428+0000","last_fullsized":"2024-09-13T16:29:06.638428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:45.983123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628374+0000","last_change":"2024-09-13T16:28:43.231002+0000","last_active":"2024-09-13T16:29:04.628374+0000","last_peered":"2024-09-13T16:29:04.628374+0000","last_clean":"2024-09-13T16:29:04.628374+0000","last_became_active":"2024-09-13T16:28:43.230237+0000","last_became_peered":"2024-09-13T16:28:43.230237+0000","last_unstale":"2024-09-13T16:29:04.628374+0000","last_undegraded":"2024-09-13T16:29:04.628374+0000","last_fullsized":"2024-09-13T16:29:04.628374+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:11:47.494738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625559+0000","last_change":"2024-09-13T16:28:39.201471+0000","last_active":"2024-09-13T16:29:04.625559+0000","last_peered":"2024-09-13T16:29:04.625559+0000","last_clean":"2024-09-13T16:29:04.625559+0000","last_became_active":"2024-09-13T16:28:39.201140+0000","last_became_peered":"2024-09-13T16:28:39.201140+0000","last_unstale":"2024-09-13T16:29:04.625559+0000","last_undegraded":"2024-09-13T16:29:04.625559+0000","last_fullsized":"2024-09-13T16:29:04.625559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:37:25.037490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.641008+0000","last_change":"2024-09-13T16:28:38.845703+0000","last_active":"2024-09-13T16:29:04.641008+0000","last_peered":"2024-09-13T16:29:04.641008+0000","last_clean":"2024-09-13T16:29:04.641008+0000","last_became_active":"2024-09-13T16:28:38.845354+0000","last_became_peered":"2024-09-13T16:28:38.845354+0000","last_unstale":"2024-09-13T16:29:04.641008+0000","last_undegraded":"2024-09-13T16:29:04.641008+0000","last_fullsized":"2024-09-13T16:29:04.641008+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:18.208723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629807+0000","last_change":"2024-09-13T16:28:43.223924+0000","last_active":"2024-09-13T16:29:04.629807+0000","last_peered":"2024-09-13T16:29:04.629807+0000","last_clean":"2024-09-13T16:29:04.629807+0000","last_became_active":"2024-09-13T16:28:43.223535+0000","last_became_peered":"2024-09-13T16:28:43.223535+0000","last_unstale":"2024-09-13T16:29:04.629807+0000","last_undegraded":"2024-09-13T16:29:04.629807+0000","last_fullsized":"2024-09-13T16:29:04.629807+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:10.622436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044425+0000","last_change":"2024-09-13T16:28:39.203943+0000","last_active":"2024-09-13T16:28:44.044425+0000","last_peered":"2024-09-13T16:28:44.044425+0000","last_clean":"2024-09-13T16:28:44.044425+0000","last_became_active":"2024-09-13T16:28:39.203657+0000","last_became_peered":"2024-09-13T16:28:39.203657+0000","last_unstale":"2024-09-13T16:28:44.044425+0000","last_undegraded":"2024-09-13T16:28:44.044425+0000","last_fullsized":"2024-09-13T16:28:44.044425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:52:48.523939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.651351+0000","last_change":"2024-09-13T16:29:05.651015+0000","last_active":"2024-09-13T16:29:05.651351+0000","last_peered":"2024-09-13T16:29:05.651351+0000","last_clean":"2024-09-13T16:29:05.651351+0000","last_became_active":"2024-09-13T16:29:05.640006+0000","last_became_peered":"2024-09-13T16:29:05.640006+0000","last_unstale":"2024-09-13T16:29:05.651351+0000","last_undegraded":"2024-09-13T16:29:05.651351+0000","last_fullsized":"2024-09-13T16:29:05.651351+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:12:25.508262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.624674+0000","last_change":"2024-09-13T16:28:43.232634+0000","last_active":"2024-09-13T16:29:04.624674+0000","last_peered":"2024-09-13T16:29:04.624674+0000","last_clean":"2024-09-13T16:29:04.624674+0000","last_became_active":"2024-09-13T16:28:43.232107+0000","last_became_peered":"2024-09-13T16:28:43.232107+0000","last_unstale":"2024-09-13T16:29:04.624674+0000","last_undegraded":"2024-09-13T16:29:04.624674+0000","last_fullsized":"2024-09-13T16:29:04.624674+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:45.952933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627482+0000","last_change":"2024-09-13T16:28:39.212883+0000","last_active":"2024-09-13T16:29:04.627482+0000","last_peered":"2024-09-13T16:29:04.627482+0000","last_clean":"2024-09-13T16:29:04.627482+0000","last_became_active":"2024-09-13T16:28:39.212591+0000","last_became_peered":"2024-09-13T16:28:39.212591+0000","last_unstale":"2024-09-13T16:29:04.627482+0000","last_undegraded":"2024-09-13T16:29:04.627482+0000","last_fullsized":"2024-09-13T16:29:04.627482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:17:30.926940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'17","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:06.645117+0000","last_change":"2024-09-13T16:28:38.846564+0000","last_active":"2024-09-13T16:29:06.645117+0000","last_peered":"2024-09-13T16:29:06.645117+0000","last_clean":"2024-09-13T16:29:06.645117+0000","last_became_active":"2024-09-13T16:28:38.846165+0000","last_became_peered":"2024-09-13T16:28:38.846165+0000","last_unstale":"2024-09-13T16:29:06.645117+0000","last_undegraded":"2024-09-13T16:29:06.645117+0000","last_fullsized":"2024-09-13T16:29:06.645117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:33:06.124592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10396,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":19,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627853+0000","last_change":"2024-09-13T16:28:43.231464+0000","last_active":"2024-09-13T16:29:04.627853+0000","last_peered":"2024-09-13T16:29:04.627853+0000","last_clean":"2024-09-13T16:29:04.627853+0000","last_became_active":"2024-09-13T16:28:43.231040+0000","last_became_peered":"2024-09-13T16:28:43.231040+0000","last_unstale":"2024-09-13T16:29:04.627853+0000","last_undegraded":"2024-09-13T16:29:04.627853+0000","last_fullsized":"2024-09-13T16:29:04.627853+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:11:21.847375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"peering","last_fresh":"2024-09-13T16:29:04.630957+0000","last_change":"2024-09-13T16:29:04.631158+0000","last_active":"2024-09-13T16:29:04.630957+0000","last_peered":"2024-09-13T16:28:44.044616+0000","last_clean":"2024-09-13T16:28:44.044616+0000","last_became_active":"2024-09-13T16:28:39.204382+0000","last_became_peered":"2024-09-13T16:28:39.204382+0000","last_unstale":"2024-09-13T16:29:04.630957+0000","last_undegraded":"2024-09-13T16:29:04.630957+0000","last_fullsized":"2024-09-13T16:29:04.630957+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:58.816998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6,8],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625089+0000","last_change":"2024-09-13T16:28:38.841637+0000","last_active":"2024-09-13T16:29:04.625089+0000","last_peered":"2024-09-13T16:29:04.625089+0000","last_clean":"2024-09-13T16:29:04.625089+0000","last_became_active":"2024-09-13T16:28:38.841359+0000","last_became_peered":"2024-09-13T16:28:38.841359+0000","last_unstale":"2024-09-13T16:29:04.625089+0000","last_undegraded":"2024-09-13T16:29:04.625089+0000","last_fullsized":"2024-09-13T16:29:04.625089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:47:05.284987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.638870+0000","last_change":"2024-09-13T16:29:05.638870+0000","last_active":"2024-09-13T16:29:05.638870+0000","last_peered":"2024-09-13T16:29:05.638870+0000","last_clean":"2024-09-13T16:29:05.638870+0000","last_became_active":"2024-09-13T16:29:05.638422+0000","last_became_peered":"2024-09-13T16:29:05.638422+0000","last_unstale":"2024-09-13T16:29:05.638870+0000","last_undegraded":"2024-09-13T16:29:05.638870+0000","last_fullsized":"2024-09-13T16:29:05.638870+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:41.263661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T16:28:43.218920+0000","last_change":"2024-09-13T16:28:39.203595+0000","last_active":"2024-09-13T16:28:43.218920+0000","last_peered":"2024-09-13T16:28:43.218920+0000","last_clean":"2024-09-13T16:28:43.218920+0000","last_became_active":"2024-09-13T16:28:39.203283+0000","last_became_peered":"2024-09-13T16:28:39.203283+0000","last_unstale":"2024-09-13T16:28:43.218920+0000","last_undegraded":"2024-09-13T16:28:43.218920+0000","last_fullsized":"2024-09-13T16:28:43.218920+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:58:45.970761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.642267+0000","last_change":"2024-09-13T16:28:38.850203+0000","last_active":"2024-09-13T16:29:04.642267+0000","last_peered":"2024-09-13T16:29:04.642267+0000","last_clean":"2024-09-13T16:29:04.642267+0000","last_became_active":"2024-09-13T16:28:38.849963+0000","last_became_peered":"2024-09-13T16:28:38.849963+0000","last_unstale":"2024-09-13T16:29:04.642267+0000","last_undegraded":"2024-09-13T16:29:04.642267+0000","last_fullsized":"2024-09-13T16:29:04.642267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:15.849627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627470+0000","last_change":"2024-09-13T16:28:43.227679+0000","last_active":"2024-09-13T16:29:04.627470+0000","last_peered":"2024-09-13T16:29:04.627470+0000","last_clean":"2024-09-13T16:29:04.627470+0000","last_became_active":"2024-09-13T16:28:43.227269+0000","last_became_peered":"2024-09-13T16:28:43.227269+0000","last_unstale":"2024-09-13T16:29:04.627470+0000","last_undegraded":"2024-09-13T16:29:04.627470+0000","last_fullsized":"2024-09-13T16:29:04.627470+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:05:00.186305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044938+0000","last_change":"2024-09-13T16:28:39.211401+0000","last_active":"2024-09-13T16:28:44.044938+0000","last_peered":"2024-09-13T16:28:44.044938+0000","last_clean":"2024-09-13T16:28:44.044938+0000","last_became_active":"2024-09-13T16:28:39.211133+0000","last_became_peered":"2024-09-13T16:28:39.211133+0000","last_unstale":"2024-09-13T16:28:44.044938+0000","last_undegraded":"2024-09-13T16:28:44.044938+0000","last_fullsized":"2024-09-13T16:28:44.044938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:36.468666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:06.642523+0000","last_change":"2024-09-13T16:28:38.845132+0000","last_active":"2024-09-13T16:29:06.642523+0000","last_peered":"2024-09-13T16:29:06.642523+0000","last_clean":"2024-09-13T16:29:06.642523+0000","last_became_active":"2024-09-13T16:28:38.844804+0000","last_became_peered":"2024-09-13T16:28:38.844804+0000","last_unstale":"2024-09-13T16:29:06.642523+0000","last_undegraded":"2024-09-13T16:29:06.642523+0000","last_fullsized":"2024-09-13T16:29:06.642523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:15:08.643227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.754760+0000","last_change":"2024-09-13T16:28:43.230829+0000","last_active":"2024-09-13T16:28:44.754760+0000","last_peered":"2024-09-13T16:28:44.754760+0000","last_clean":"2024-09-13T16:28:44.754760+0000","last_became_active":"2024-09-13T16:28:43.230405+0000","last_became_peered":"2024-09-13T16:28:43.230405+0000","last_unstale":"2024-09-13T16:28:44.754760+0000","last_undegraded":"2024-09-13T16:28:44.754760+0000","last_fullsized":"2024-09-13T16:28:44.754760+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:04:56.083193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044006+0000","last_change":"2024-09-13T16:28:43.226662+0000","last_active":"2024-09-13T16:28:44.044006+0000","last_peered":"2024-09-13T16:28:44.044006+0000","last_clean":"2024-09-13T16:28:44.044006+0000","last_became_active":"2024-09-13T16:28:43.226232+0000","last_became_peered":"2024-09-13T16:28:43.226232+0000","last_unstale":"2024-09-13T16:28:44.044006+0000","last_undegraded":"2024-09-13T16:28:44.044006+0000","last_fullsized":"2024-09-13T16:28:44.044006+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:03.568500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627636+0000","last_change":"2024-09-13T16:28:39.208909+0000","last_active":"2024-09-13T16:29:04.627636+0000","last_peered":"2024-09-13T16:29:04.627636+0000","last_clean":"2024-09-13T16:29:04.627636+0000","last_became_active":"2024-09-13T16:28:39.208572+0000","last_became_peered":"2024-09-13T16:28:39.208572+0000","last_unstale":"2024-09-13T16:29:04.627636+0000","last_undegraded":"2024-09-13T16:29:04.627636+0000","last_fullsized":"2024-09-13T16:29:04.627636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:01:40.173308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.641240+0000","last_change":"2024-09-13T16:28:38.844259+0000","last_active":"2024-09-13T16:29:04.641240+0000","last_peered":"2024-09-13T16:29:04.641240+0000","last_clean":"2024-09-13T16:29:04.641240+0000","last_became_active":"2024-09-13T16:28:38.843956+0000","last_became_peered":"2024-09-13T16:28:38.843956+0000","last_unstale":"2024-09-13T16:29:04.641240+0000","last_undegraded":"2024-09-13T16:29:04.641240+0000","last_fullsized":"2024-09-13T16:29:04.641240+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:05.652852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629221+0000","last_change":"2024-09-13T16:28:43.229385+0000","last_active":"2024-09-13T16:29:04.629221+0000","last_peered":"2024-09-13T16:29:04.629221+0000","last_clean":"2024-09-13T16:29:04.629221+0000","last_became_active":"2024-09-13T16:28:43.229033+0000","last_became_peered":"2024-09-13T16:28:43.229033+0000","last_unstale":"2024-09-13T16:29:04.629221+0000","last_undegraded":"2024-09-13T16:29:04.629221+0000","last_fullsized":"2024-09-13T16:29:04.629221+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:46:41.611758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044455+0000","last_change":"2024-09-13T16:28:39.208100+0000","last_active":"2024-09-13T16:28:44.044455+0000","last_peered":"2024-09-13T16:28:44.044455+0000","last_clean":"2024-09-13T16:28:44.044455+0000","last_became_active":"2024-09-13T16:28:39.207808+0000","last_became_peered":"2024-09-13T16:28:39.207808+0000","last_unstale":"2024-09-13T16:28:44.044455+0000","last_undegraded":"2024-09-13T16:28:44.044455+0000","last_fullsized":"2024-09-13T16:28:44.044455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:55.446568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.622326+0000","last_change":"2024-09-13T16:28:38.844689+0000","last_active":"2024-09-13T16:29:04.622326+0000","last_peered":"2024-09-13T16:29:04.622326+0000","last_clean":"2024-09-13T16:29:04.622326+0000","last_became_active":"2024-09-13T16:28:38.844326+0000","last_became_peered":"2024-09-13T16:28:38.844326+0000","last_unstale":"2024-09-13T16:29:04.622326+0000","last_undegraded":"2024-09-13T16:29:04.622326+0000","last_fullsized":"2024-09-13T16:29:04.622326+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:02.616518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629530+0000","last_change":"2024-09-13T16:28:38.849233+0000","last_active":"2024-09-13T16:29:04.629530+0000","last_peered":"2024-09-13T16:29:04.629530+0000","last_clean":"2024-09-13T16:29:04.629530+0000","last_became_active":"2024-09-13T16:28:38.848911+0000","last_became_peered":"2024-09-13T16:28:38.848911+0000","last_unstale":"2024-09-13T16:29:04.629530+0000","last_undegraded":"2024-09-13T16:29:04.629530+0000","last_fullsized":"2024-09-13T16:29:04.629530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:02:35.951263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.624339+0000","last_change":"2024-09-13T16:28:39.203855+0000","last_active":"2024-09-13T16:29:04.624339+0000","last_peered":"2024-09-13T16:29:04.624339+0000","last_clean":"2024-09-13T16:29:04.624339+0000","last_became_active":"2024-09-13T16:28:39.203532+0000","last_became_peered":"2024-09-13T16:28:39.203532+0000","last_unstale":"2024-09-13T16:29:04.624339+0000","last_undegraded":"2024-09-13T16:29:04.624339+0000","last_fullsized":"2024-09-13T16:29:04.624339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:15.048930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.633730+0000","last_change":"2024-09-13T16:28:43.228554+0000","last_active":"2024-09-13T16:29:04.633730+0000","last_peered":"2024-09-13T16:29:04.633730+0000","last_clean":"2024-09-13T16:29:04.633730+0000","last_became_active":"2024-09-13T16:28:43.228204+0000","last_became_peered":"2024-09-13T16:28:43.228204+0000","last_unstale":"2024-09-13T16:29:04.633730+0000","last_undegraded":"2024-09-13T16:29:04.633730+0000","last_fullsized":"2024-09-13T16:29:04.633730+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:16:15.383932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.639374+0000","last_change":"2024-09-13T16:28:38.848425+0000","last_active":"2024-09-13T16:29:05.639374+0000","last_peered":"2024-09-13T16:29:05.639374+0000","last_clean":"2024-09-13T16:29:05.639374+0000","last_became_active":"2024-09-13T16:28:38.848136+0000","last_became_peered":"2024-09-13T16:28:38.848136+0000","last_unstale":"2024-09-13T16:29:05.639374+0000","last_undegraded":"2024-09-13T16:29:05.639374+0000","last_fullsized":"2024-09-13T16:29:05.639374+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:47:03.904315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627249+0000","last_change":"2024-09-13T16:28:39.210565+0000","last_active":"2024-09-13T16:29:04.627249+0000","last_peered":"2024-09-13T16:29:04.627249+0000","last_clean":"2024-09-13T16:29:04.627249+0000","last_became_active":"2024-09-13T16:28:39.210272+0000","last_became_peered":"2024-09-13T16:28:39.210272+0000","last_unstale":"2024-09-13T16:29:04.627249+0000","last_undegraded":"2024-09-13T16:29:04.627249+0000","last_fullsized":"2024-09-13T16:29:04.627249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:12.746087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.634077+0000","last_change":"2024-09-13T16:28:43.231623+0000","last_active":"2024-09-13T16:29:04.634077+0000","last_peered":"2024-09-13T16:29:04.634077+0000","last_clean":"2024-09-13T16:29:04.634077+0000","last_became_active":"2024-09-13T16:28:43.231255+0000","last_became_peered":"2024-09-13T16:28:43.231255+0000","last_unstale":"2024-09-13T16:29:04.634077+0000","last_undegraded":"2024-09-13T16:29:04.634077+0000","last_fullsized":"2024-09-13T16:29:04.634077+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:29.755772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627783+0000","last_change":"2024-09-13T16:28:39.208438+0000","last_active":"2024-09-13T16:29:04.627783+0000","last_peered":"2024-09-13T16:29:04.627783+0000","last_clean":"2024-09-13T16:29:04.627783+0000","last_became_active":"2024-09-13T16:28:39.208156+0000","last_became_peered":"2024-09-13T16:28:39.208156+0000","last_unstale":"2024-09-13T16:29:04.627783+0000","last_undegraded":"2024-09-13T16:29:04.627783+0000","last_fullsized":"2024-09-13T16:29:04.627783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:53:46.923166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.633806+0000","last_change":"2024-09-13T16:28:39.208529+0000","last_active":"2024-09-13T16:29:04.633806+0000","last_peered":"2024-09-13T16:29:04.633806+0000","last_clean":"2024-09-13T16:29:04.633806+0000","last_became_active":"2024-09-13T16:28:39.208202+0000","last_became_peered":"2024-09-13T16:28:39.208202+0000","last_unstale":"2024-09-13T16:29:04.633806+0000","last_undegraded":"2024-09-13T16:29:04.633806+0000","last_fullsized":"2024-09-13T16:29:04.633806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:17:11.357410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.632611+0000","last_change":"2024-09-13T16:28:39.212958+0000","last_active":"2024-09-13T16:29:04.632611+0000","last_peered":"2024-09-13T16:29:04.632611+0000","last_clean":"2024-09-13T16:29:04.632611+0000","last_became_active":"2024-09-13T16:28:39.212646+0000","last_became_peered":"2024-09-13T16:28:39.212646+0000","last_unstale":"2024-09-13T16:29:04.632611+0000","last_undegraded":"2024-09-13T16:29:04.632611+0000","last_fullsized":"2024-09-13T16:29:04.632611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:31:04.047778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T16:28:43.218683+0000","last_change":"2024-09-13T16:28:39.200449+0000","last_active":"2024-09-13T16:28:43.218683+0000","last_peered":"2024-09-13T16:28:43.218683+0000","last_clean":"2024-09-13T16:28:43.218683+0000","last_became_active":"2024-09-13T16:28:39.200133+0000","last_became_peered":"2024-09-13T16:28:39.200133+0000","last_unstale":"2024-09-13T16:28:43.218683+0000","last_undegraded":"2024-09-13T16:28:43.218683+0000","last_fullsized":"2024-09-13T16:28:43.218683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:18:06.480691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044668+0000","last_change":"2024-09-13T16:28:39.200274+0000","last_active":"2024-09-13T16:28:44.044668+0000","last_peered":"2024-09-13T16:28:44.044668+0000","last_clean":"2024-09-13T16:28:44.044668+0000","last_became_active":"2024-09-13T16:28:39.199951+0000","last_became_peered":"2024-09-13T16:28:39.199951+0000","last_unstale":"2024-09-13T16:28:44.044668+0000","last_undegraded":"2024-09-13T16:28:44.044668+0000","last_fullsized":"2024-09-13T16:28:44.044668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:13:31.199857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.630000+0000","last_change":"2024-09-13T16:28:39.213591+0000","last_active":"2024-09-13T16:29:04.630000+0000","last_peered":"2024-09-13T16:29:04.630000+0000","last_clean":"2024-09-13T16:29:04.630000+0000","last_became_active":"2024-09-13T16:28:39.213324+0000","last_became_peered":"2024-09-13T16:28:39.213324+0000","last_unstale":"2024-09-13T16:29:04.630000+0000","last_undegraded":"2024-09-13T16:29:04.630000+0000","last_fullsized":"2024-09-13T16:29:04.630000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:54:03.172919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.633921+0000","last_change":"2024-09-13T16:28:39.217442+0000","last_active":"2024-09-13T16:29:04.633921+0000","last_peered":"2024-09-13T16:29:04.633921+0000","last_clean":"2024-09-13T16:29:04.633921+0000","last_became_active":"2024-09-13T16:28:39.217151+0000","last_became_peered":"2024-09-13T16:28:39.217151+0000","last_unstale":"2024-09-13T16:29:04.633921+0000","last_undegraded":"2024-09-13T16:29:04.633921+0000","last_fullsized":"2024-09-13T16:29:04.633921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:17.132278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.045476+0000","last_change":"2024-09-13T16:28:39.203938+0000","last_active":"2024-09-13T16:28:44.045476+0000","last_peered":"2024-09-13T16:28:44.045476+0000","last_clean":"2024-09-13T16:28:44.045476+0000","last_became_active":"2024-09-13T16:28:39.203655+0000","last_became_peered":"2024-09-13T16:28:39.203655+0000","last_unstale":"2024-09-13T16:28:44.045476+0000","last_undegraded":"2024-09-13T16:28:44.045476+0000","last_fullsized":"2024-09-13T16:28:44.045476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:03:25.415256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.624165+0000","last_change":"2024-09-13T16:28:39.209592+0000","last_active":"2024-09-13T16:29:04.624165+0000","last_peered":"2024-09-13T16:29:04.624165+0000","last_clean":"2024-09-13T16:29:04.624165+0000","last_became_active":"2024-09-13T16:28:39.209299+0000","last_became_peered":"2024-09-13T16:28:39.209299+0000","last_unstale":"2024-09-13T16:29:04.624165+0000","last_undegraded":"2024-09-13T16:29:04.624165+0000","last_fullsized":"2024-09-13T16:29:04.624165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:04.053405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044501+0000","last_change":"2024-09-13T16:28:39.208437+0000","last_active":"2024-09-13T16:28:44.044501+0000","last_peered":"2024-09-13T16:28:44.044501+0000","last_clean":"2024-09-13T16:28:44.044501+0000","last_became_active":"2024-09-13T16:28:39.208136+0000","last_became_peered":"2024-09-13T16:28:39.208136+0000","last_unstale":"2024-09-13T16:28:44.044501+0000","last_undegraded":"2024-09-13T16:28:44.044501+0000","last_fullsized":"2024-09-13T16:28:44.044501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:21.304037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.045412+0000","last_change":"2024-09-13T16:28:39.203970+0000","last_active":"2024-09-13T16:28:44.045412+0000","last_peered":"2024-09-13T16:28:44.045412+0000","last_clean":"2024-09-13T16:28:44.045412+0000","last_became_active":"2024-09-13T16:28:39.203559+0000","last_became_peered":"2024-09-13T16:28:39.203559+0000","last_unstale":"2024-09-13T16:28:44.045412+0000","last_undegraded":"2024-09-13T16:28:44.045412+0000","last_fullsized":"2024-09-13T16:28:44.045412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:26:25.158393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044158+0000","last_change":"2024-09-13T16:28:39.205584+0000","last_active":"2024-09-13T16:28:44.044158+0000","last_peered":"2024-09-13T16:28:44.044158+0000","last_clean":"2024-09-13T16:28:44.044158+0000","last_became_active":"2024-09-13T16:28:39.205281+0000","last_became_peered":"2024-09-13T16:28:39.205281+0000","last_unstale":"2024-09-13T16:28:44.044158+0000","last_undegraded":"2024-09-13T16:28:44.044158+0000","last_fullsized":"2024-09-13T16:28:44.044158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:58:10.109647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629903+0000","last_change":"2024-09-13T16:28:39.214479+0000","last_active":"2024-09-13T16:29:04.629903+0000","last_peered":"2024-09-13T16:29:04.629903+0000","last_clean":"2024-09-13T16:29:04.629903+0000","last_became_active":"2024-09-13T16:28:39.213984+0000","last_became_peered":"2024-09-13T16:28:39.213984+0000","last_unstale":"2024-09-13T16:29:04.629903+0000","last_undegraded":"2024-09-13T16:29:04.629903+0000","last_fullsized":"2024-09-13T16:29:04.629903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:53:43.137726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.634317+0000","last_change":"2024-09-13T16:28:39.209595+0000","last_active":"2024-09-13T16:29:04.634317+0000","last_peered":"2024-09-13T16:29:04.634317+0000","last_clean":"2024-09-13T16:29:04.634317+0000","last_became_active":"2024-09-13T16:28:39.209282+0000","last_became_peered":"2024-09-13T16:28:39.209282+0000","last_unstale":"2024-09-13T16:29:04.634317+0000","last_undegraded":"2024-09-13T16:29:04.634317+0000","last_fullsized":"2024-09-13T16:29:04.634317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:26:41.452462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628010+0000","last_change":"2024-09-13T16:28:39.210610+0000","last_active":"2024-09-13T16:29:04.628010+0000","last_peered":"2024-09-13T16:29:04.628010+0000","last_clean":"2024-09-13T16:29:04.628010+0000","last_became_active":"2024-09-13T16:28:39.210296+0000","last_became_peered":"2024-09-13T16:28:39.210296+0000","last_unstale":"2024-09-13T16:29:04.628010+0000","last_undegraded":"2024-09-13T16:29:04.628010+0000","last_fullsized":"2024-09-13T16:29:04.628010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:06.385733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629594+0000","last_change":"2024-09-13T16:28:39.207609+0000","last_active":"2024-09-13T16:29:04.629594+0000","last_peered":"2024-09-13T16:29:04.629594+0000","last_clean":"2024-09-13T16:29:04.629594+0000","last_became_active":"2024-09-13T16:28:39.206893+0000","last_became_peered":"2024-09-13T16:28:39.206893+0000","last_unstale":"2024-09-13T16:29:04.629594+0000","last_undegraded":"2024-09-13T16:29:04.629594+0000","last_fullsized":"2024-09-13T16:29:04.629594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:51.582666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.045110+0000","last_change":"2024-09-13T16:28:39.201856+0000","last_active":"2024-09-13T16:28:44.045110+0000","last_peered":"2024-09-13T16:28:44.045110+0000","last_clean":"2024-09-13T16:28:44.045110+0000","last_became_active":"2024-09-13T16:28:39.201511+0000","last_became_peered":"2024-09-13T16:28:39.201511+0000","last_unstale":"2024-09-13T16:28:44.045110+0000","last_undegraded":"2024-09-13T16:28:44.045110+0000","last_fullsized":"2024-09-13T16:28:44.045110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:05:19.722354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T16:28:43.215322+0000","last_change":"2024-09-13T16:28:39.216106+0000","last_active":"2024-09-13T16:28:43.215322+0000","last_peered":"2024-09-13T16:28:43.215322+0000","last_clean":"2024-09-13T16:28:43.215322+0000","last_became_active":"2024-09-13T16:28:39.215432+0000","last_became_peered":"2024-09-13T16:28:39.215432+0000","last_unstale":"2024-09-13T16:28:43.215322+0000","last_undegraded":"2024-09-13T16:28:43.215322+0000","last_fullsized":"2024-09-13T16:28:43.215322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:21:18.439891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T16:28:43.215353+0000","last_change":"2024-09-13T16:28:39.210578+0000","last_active":"2024-09-13T16:28:43.215353+0000","last_peered":"2024-09-13T16:28:43.215353+0000","last_clean":"2024-09-13T16:28:43.215353+0000","last_became_active":"2024-09-13T16:28:39.210269+0000","last_became_peered":"2024-09-13T16:28:39.210269+0000","last_unstale":"2024-09-13T16:28:43.215353+0000","last_undegraded":"2024-09-13T16:28:43.215353+0000","last_fullsized":"2024-09-13T16:28:43.215353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:27:27.078880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044081+0000","last_change":"2024-09-13T16:28:39.213409+0000","last_active":"2024-09-13T16:28:44.044081+0000","last_peered":"2024-09-13T16:28:44.044081+0000","last_clean":"2024-09-13T16:28:44.044081+0000","last_became_active":"2024-09-13T16:28:39.213077+0000","last_became_peered":"2024-09-13T16:28:39.213077+0000","last_unstale":"2024-09-13T16:28:44.044081+0000","last_undegraded":"2024-09-13T16:28:44.044081+0000","last_fullsized":"2024-09-13T16:28:44.044081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:32:24.747516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.632419+0000","last_change":"2024-09-13T16:28:39.215216+0000","last_active":"2024-09-13T16:29:04.632419+0000","last_peered":"2024-09-13T16:29:04.632419+0000","last_clean":"2024-09-13T16:29:04.632419+0000","last_became_active":"2024-09-13T16:28:39.214920+0000","last_became_peered":"2024-09-13T16:28:39.214920+0000","last_unstale":"2024-09-13T16:29:04.632419+0000","last_undegraded":"2024-09-13T16:29:04.632419+0000","last_fullsized":"2024-09-13T16:29:04.632419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:25:38.132833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625826+0000","last_change":"2024-09-13T16:28:39.202506+0000","last_active":"2024-09-13T16:29:04.625826+0000","last_peered":"2024-09-13T16:29:04.625826+0000","last_clean":"2024-09-13T16:29:04.625826+0000","last_became_active":"2024-09-13T16:28:39.202224+0000","last_became_peered":"2024-09-13T16:28:39.202224+0000","last_unstale":"2024-09-13T16:29:04.625826+0000","last_undegraded":"2024-09-13T16:29:04.625826+0000","last_fullsized":"2024-09-13T16:29:04.625826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:00:37.208442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.640477+0000","last_change":"2024-09-13T16:29:05.640477+0000","last_active":"2024-09-13T16:29:05.640477+0000","last_peered":"2024-09-13T16:29:05.640477+0000","last_clean":"2024-09-13T16:29:05.640477+0000","last_became_active":"2024-09-13T16:29:05.640014+0000","last_became_peered":"2024-09-13T16:29:05.640014+0000","last_unstale":"2024-09-13T16:29:05.640477+0000","last_undegraded":"2024-09-13T16:29:05.640477+0000","last_fullsized":"2024-09-13T16:29:05.640477+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:47:45.202113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.045134+0000","last_change":"2024-09-13T16:28:39.200217+0000","last_active":"2024-09-13T16:28:44.045134+0000","last_peered":"2024-09-13T16:28:44.045134+0000","last_clean":"2024-09-13T16:28:44.045134+0000","last_became_active":"2024-09-13T16:28:39.199946+0000","last_became_peered":"2024-09-13T16:28:39.199946+0000","last_unstale":"2024-09-13T16:28:44.045134+0000","last_undegraded":"2024-09-13T16:28:44.045134+0000","last_fullsized":"2024-09-13T16:28:44.045134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:20:02.906225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T16:28:44.044376+0000","last_change":"2024-09-13T16:28:39.215013+0000","last_active":"2024-09-13T16:28:44.044376+0000","last_peered":"2024-09-13T16:28:44.044376+0000","last_clean":"2024-09-13T16:28:44.044376+0000","last_became_active":"2024-09-13T16:28:39.214731+0000","last_became_peered":"2024-09-13T16:28:39.214731+0000","last_unstale":"2024-09-13T16:28:44.044376+0000","last_undegraded":"2024-09-13T16:28:44.044376+0000","last_fullsized":"2024-09-13T16:28:44.044376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:35:01.836001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625268+0000","last_change":"2024-09-13T16:28:39.200300+0000","last_active":"2024-09-13T16:29:04.625268+0000","last_peered":"2024-09-13T16:29:04.625268+0000","last_clean":"2024-09-13T16:29:04.625268+0000","last_became_active":"2024-09-13T16:28:39.200006+0000","last_became_peered":"2024-09-13T16:28:39.200006+0000","last_unstale":"2024-09-13T16:29:04.625268+0000","last_undegraded":"2024-09-13T16:29:04.625268+0000","last_fullsized":"2024-09-13T16:29:04.625268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:35:14.783905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629430+0000","last_change":"2024-09-13T16:28:39.213552+0000","last_active":"2024-09-13T16:29:04.629430+0000","last_peered":"2024-09-13T16:29:04.629430+0000","last_clean":"2024-09-13T16:29:04.629430+0000","last_became_active":"2024-09-13T16:28:39.213287+0000","last_became_peered":"2024-09-13T16:28:39.213287+0000","last_unstale":"2024-09-13T16:29:04.629430+0000","last_undegraded":"2024-09-13T16:29:04.629430+0000","last_fullsized":"2024-09-13T16:29:04.629430+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:30.588770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627025+0000","last_change":"2024-09-13T16:28:39.211126+0000","last_active":"2024-09-13T16:29:04.627025+0000","last_peered":"2024-09-13T16:29:04.627025+0000","last_clean":"2024-09-13T16:29:04.627025+0000","last_became_active":"2024-09-13T16:28:39.210810+0000","last_became_peered":"2024-09-13T16:28:39.210810+0000","last_unstale":"2024-09-13T16:29:04.627025+0000","last_undegraded":"2024-09-13T16:29:04.627025+0000","last_fullsized":"2024-09-13T16:29:04.627025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:52:45.249309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.639070+0000","last_change":"2024-09-13T16:29:05.636535+0000","last_active":"2024-09-13T16:29:05.639070+0000","last_peered":"2024-09-13T16:29:05.639070+0000","last_clean":"2024-09-13T16:29:05.639070+0000","last_became_active":"2024-09-13T16:29:05.636161+0000","last_became_peered":"2024-09-13T16:29:05.636161+0000","last_unstale":"2024-09-13T16:29:05.639070+0000","last_undegraded":"2024-09-13T16:29:05.639070+0000","last_fullsized":"2024-09-13T16:29:05.639070+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:09.606922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T16:28:43.215475+0000","last_change":"2024-09-13T16:28:39.202222+0000","last_active":"2024-09-13T16:28:43.215475+0000","last_peered":"2024-09-13T16:28:43.215475+0000","last_clean":"2024-09-13T16:28:43.215475+0000","last_became_active":"2024-09-13T16:28:39.201897+0000","last_became_peered":"2024-09-13T16:28:39.201897+0000","last_unstale":"2024-09-13T16:28:43.215475+0000","last_undegraded":"2024-09-13T16:28:43.215475+0000","last_fullsized":"2024-09-13T16:28:43.215475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:41:09.087185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T16:28:43.217236+0000","last_change":"2024-09-13T16:28:39.211164+0000","last_active":"2024-09-13T16:28:43.217236+0000","last_peered":"2024-09-13T16:28:43.217236+0000","last_clean":"2024-09-13T16:28:43.217236+0000","last_became_active":"2024-09-13T16:28:39.210826+0000","last_became_peered":"2024-09-13T16:28:39.210826+0000","last_unstale":"2024-09-13T16:28:43.217236+0000","last_undegraded":"2024-09-13T16:28:43.217236+0000","last_fullsized":"2024-09-13T16:28:43.217236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:40:30.663867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629508+0000","last_change":"2024-09-13T16:28:39.207991+0000","last_active":"2024-09-13T16:29:04.629508+0000","last_peered":"2024-09-13T16:29:04.629508+0000","last_clean":"2024-09-13T16:29:04.629508+0000","last_became_active":"2024-09-13T16:28:39.207332+0000","last_became_peered":"2024-09-13T16:28:39.207332+0000","last_unstale":"2024-09-13T16:29:04.629508+0000","last_undegraded":"2024-09-13T16:29:04.629508+0000","last_fullsized":"2024-09-13T16:29:04.629508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:30:54.271361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":16384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":17802,"num_objects":96,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":0,"num_read":57,"num_read_kb":46,"num_write":213,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":762,"num_keys_recovered":10,"num_objects_omap":64,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":57746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":187,"ondisk_log_size":187,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250749,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383696,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":487217,"data_compressed":14739,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.32600000000000001}]},{"osd":1,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.376},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.30399999999999999}]},{"osd":2,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.439},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.627}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]},{"osd":4,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.47699999999999998}]},{"osd":5,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.59099999999999997}]},{"osd":6,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.56599999999999995}]},{"osd":7,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.54000000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":9,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.73299999999999998}]},{"osd":10,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.70999999999999996}]}]},{"osd":10,"up_from":72,"seq":309237645334,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1089536,"data_stored":1091645,"data_compressed":54008,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":1,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.50800000000000001}]},{"osd":2,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.626},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.68799999999999994}]},{"osd":3,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":7.2320000000000002,"5min":7.2320000000000002,"15min":7.2320000000000002},"last":0.61099999999999999}]},{"osd":4,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.57699999999999996}]},{"osd":5,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.75},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.64100000000000001}]},{"osd":6,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.70199999999999996}]},{"osd":7,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.53000000000000003}]},{"osd":8,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":7.2569999999999997,"5min":7.2569999999999997,"15min":7.2569999999999997},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.70999999999999996}]},{"osd":9,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.67300000000000004}]},{"osd":11,"last update":"Fri Sep 13 16:28:58 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":7.2199999999999998,"5min":7.2199999999999998,"15min":7.2199999999999998},"last":0.79400000000000004}]}]},{"osd":9,"up_from":67,"seq":287762808858,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":496463,"data_compressed":17719,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.35399999999999998}]},{"osd":1,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.52900000000000003}]},{"osd":3,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.46500000000000002}]},{"osd":4,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.628}]},{"osd":5,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.81100000000000005}]},{"osd":6,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.45700000000000002}]},{"osd":7,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.76500000000000001}]},{"osd":8,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.745}]},{"osd":10,"last update":"Fri Sep 13 16:28:33 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.85299999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":8,"up_from":59,"seq":253403070495,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":1007616,"data_stored":495587,"data_compressed":17719,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.50600000000000001,"15min":0.50800000000000001},"min":{"1min":0.28100000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.70999999999999996,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51700000000000002,"15min":0.51200000000000001},"min":{"1min":0.32800000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.48599999999999999}]},{"osd":1,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.41299999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.40400000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82399999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.60799999999999998}]},{"osd":2,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.41699999999999998,"15min":0.41199999999999998},"min":{"1min":0.28299999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.435}]},{"osd":3,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.78700000000000003,"5min":0.871,"15min":0.871},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.58199999999999996}]},{"osd":4,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56799999999999995,"15min":0.56399999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.747}]},{"osd":5,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53800000000000003,"15min":0.54300000000000004},"min":{"1min":0.29699999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.622}]},{"osd":6,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.628,"15min":0.63},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.65900000000000003,"15min":0.66300000000000003},"min":{"1min":0.38300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68500000000000005}]},{"osd":7,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.66600000000000004,"15min":0.67000000000000004},"min":{"1min":0.47899999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.97699999999999998,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.65800000000000003,"15min":0.66200000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.92200000000000004,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.69699999999999995}]},{"osd":9,"last update":"Fri Sep 13 16:28:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.79500000000000004}]},{"osd":10,"last update":"Fri Sep 13 16:28:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.77600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]}]},{"osd":7,"up_from":51,"seq":219043332132,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":1093979,"data_compressed":54008,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.39600000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.41199999999999998}]},{"osd":1,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.39200000000000002,"15min":0.38900000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.312},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.68600000000000005,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.68999999999999995}]},{"osd":2,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.54400000000000004,"15min":0.54900000000000004},"min":{"1min":0.40699999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.66700000000000004,"5min":0.748,"15min":0.748},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54700000000000004,"15min":0.54800000000000004},"min":{"1min":0.40000000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.60299999999999998}]},{"osd":3,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.42499999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.441},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56799999999999995,"15min":0.56399999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.57499999999999996}]},{"osd":4,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45200000000000001,"15min":0.45100000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.45800000000000002,"15min":0.46300000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.32700000000000001}]},{"osd":5,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57699999999999996,"15min":0.57799999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.496},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.44600000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.54200000000000004}]},{"osd":6,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60899999999999999,"15min":0.60999999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.41699999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.373}]},{"osd":8,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.627,"15min":0.627},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.624},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.66100000000000003,"15min":0.66500000000000004},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.65800000000000003}]},{"osd":9,"last update":"Fri Sep 13 16:28:08 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.64200000000000002}]},{"osd":10,"last update":"Fri Sep 13 16:28:33 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.67600000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]}]},{"osd":6,"up_from":46,"seq":197568495657,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":991232,"data_stored":486747,"data_compressed":14739,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.38300000000000001,"15min":0.372},"min":{"1min":0.26900000000000002,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.39700000000000002,"15min":0.39000000000000001},"min":{"1min":0.28000000000000003,"5min":0.217,"15min":0.217},"max":{"1min":0.61599999999999999,"5min":0.625,"15min":0.625},"last":0.33700000000000002}]},{"osd":1,"last update":"Fri Sep 13 16:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53400000000000003,"15min":0.51200000000000001},"min":{"1min":0.373,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.753,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56100000000000005,"15min":0.54500000000000004},"min":{"1min":0.39700000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.77700000000000002,"5min":0.873,"15min":0.873},"last":0.53300000000000003}]},{"osd":2,"last update":"Fri Sep 13 16:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55300000000000005,"15min":0.54900000000000004},"min":{"1min":0.35199999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83499999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59699999999999998,"15min":0.58799999999999997},"min":{"1min":0.36899999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81399999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.623}]},{"osd":3,"last update":"Fri Sep 13 16:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.73699999999999999,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.499,"15min":0.48899999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.44400000000000001}]},{"osd":4,"last update":"Fri Sep 13 16:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.30299999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.73599999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.42299999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.92300000000000004,"5min":1.079,"15min":1.079},"last":0.49099999999999999}]},{"osd":5,"last update":"Fri Sep 13 16:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61799999999999999,"15min":0.60599999999999998},"min":{"1min":0.47199999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96599999999999997,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.45800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94999999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.54400000000000004}]},{"osd":7,"last update":"Fri Sep 13 16:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63800000000000001,"15min":0.63200000000000001},"min":{"1min":0.40600000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.02,"5min":1.048,"15min":1.048},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.66300000000000003,"15min":0.65200000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.60299999999999998}]},{"osd":8,"last update":"Fri Sep 13 16:28:43 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.69299999999999995,"15min":0.68899999999999995},"min":{"1min":0.505,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72699999999999998,"15min":0.72799999999999998},"min":{"1min":0.46600000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.64000000000000001}]},{"osd":9,"last update":"Fri Sep 13 16:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.66600000000000004}]},{"osd":10,"last update":"Fri Sep 13 16:28:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.70499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]}]},{"osd":5,"up_from":38,"seq":163208757294,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":1088542,"data_compressed":54008,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.52200000000000002,"15min":0.51600000000000001},"min":{"1min":0.377,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76700000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54000000000000004,"15min":0.53500000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.80700000000000005,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.46500000000000002}]},{"osd":1,"last update":"Fri Sep 13 16:28:40 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.48699999999999999,"15min":0.47699999999999998},"min":{"1min":0.308,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.51900000000000002,"15min":0.505},"min":{"1min":0.34399999999999997,"5min":0.245,"15min":0.245},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.41299999999999998}]},{"osd":2,"last update":"Fri Sep 13 16:28:40 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.375,"15min":0.35999999999999999},"min":{"1min":0.254,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.65000000000000002,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.379,"15min":0.36699999999999999},"min":{"1min":0.23100000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.36899999999999999}]},{"osd":3,"last update":"Fri Sep 13 16:28:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.80000000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.55000000000000004}]},{"osd":4,"last update":"Fri Sep 13 16:28:40 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52400000000000002,"15min":0.51100000000000001},"min":{"1min":0.41799999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53400000000000003,"15min":0.52800000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.54000000000000004}]},{"osd":6,"last update":"Fri Sep 13 16:29:03 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.45800000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.434,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.78900000000000003,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.49399999999999999}]},{"osd":7,"last update":"Fri Sep 13 16:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.60799999999999998,"15min":0.60499999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59999999999999998,"15min":0.59699999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.626}]},{"osd":8,"last update":"Fri Sep 13 16:28:46 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52800000000000002,"15min":0.53300000000000003},"min":{"1min":0.34499999999999997,"5min":0.318,"15min":0.318},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.56200000000000006,"15min":0.56899999999999995},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.56200000000000006}]},{"osd":9,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.65700000000000003}]},{"osd":10,"last update":"Fri Sep 13 16:28:31 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.68500000000000005}]}]},{"osd":4,"up_from":33,"seq":141733920819,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":496497,"data_compressed":17719,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.48699999999999999,"15min":0.46300000000000002},"min":{"1min":0.32200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.751,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.504,"15min":0.48199999999999998},"min":{"1min":0.41399999999999998,"5min":0.25,"15min":0.25},"max":{"1min":0.80800000000000005,"5min":0.874,"15min":0.874},"last":0.60499999999999998}]},{"osd":1,"last update":"Fri Sep 13 16:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.34699999999999998,"15min":0.33100000000000002},"min":{"1min":0.24099999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.34300000000000003,"15min":0.33200000000000002},"min":{"1min":0.27800000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.372}]},{"osd":2,"last update":"Fri Sep 13 16:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.495,"15min":0.48599999999999999},"min":{"1min":0.37,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71599999999999997,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.47399999999999998,"15min":0.45500000000000002},"min":{"1min":0.34399999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67200000000000004,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.59299999999999997}]},{"osd":3,"last update":"Fri Sep 13 16:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.504,"15min":0.48599999999999999},"min":{"1min":0.38700000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.505},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.52100000000000002,"15min":0.496},"min":{"1min":0.39900000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.79300000000000004,"5min":0.875,"15min":0.875},"last":0.56799999999999995}]},{"osd":5,"last update":"Fri Sep 13 16:28:38 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.33800000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54500000000000004,"15min":0.53100000000000003},"min":{"1min":0.39000000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80700000000000005,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Sep 13 16:28:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.54900000000000004,"15min":0.53500000000000003},"min":{"1min":0.47599999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.73299999999999998,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57699999999999996,"15min":0.56599999999999995},"min":{"1min":0.41699999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80400000000000005,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.63}]},{"osd":7,"last update":"Fri Sep 13 16:28:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.495,"15min":0.48099999999999998},"min":{"1min":0.33600000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.85099999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.49399999999999999,"15min":0.48999999999999999},"min":{"1min":0.34999999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.81799999999999995,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.621}]},{"osd":8,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.44,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86899999999999999,"5min":0.872,"15min":0.872},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.61499999999999999,"15min":0.60999999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.872,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.64500000000000002}]},{"osd":9,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.72599999999999998}]},{"osd":10,"last update":"Fri Sep 13 16:28:32 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.65400000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36188,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708964,"statfs":{"total":95995035648,"available":95957979136,"internally_reserved":0,"allocated":999424,"data_stored":486172,"data_compressed":14739,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:56 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.34899999999999998,"15min":0.32500000000000001},"min":{"1min":0.252,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.63,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.35699999999999998,"15min":0.32900000000000001},"min":{"1min":0.27600000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52300000000000002,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.36599999999999999}]},{"osd":1,"last update":"Fri Sep 13 16:28:56 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.54600000000000004,"15min":0.48199999999999998},"min":{"1min":0.48899999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.84299999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53500000000000003,"15min":0.48699999999999999},"min":{"1min":0.42399999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.77000000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.68799999999999994}]},{"osd":2,"last update":"Fri Sep 13 16:28:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52200000000000002,"15min":0.51400000000000001},"min":{"1min":0.375,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79100000000000004,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55300000000000005,"15min":0.52600000000000002},"min":{"1min":0.38600000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.94599999999999995,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.53600000000000003}]},{"osd":4,"last update":"Fri Sep 13 16:28:09 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.51500000000000001,"15min":0.49099999999999999},"min":{"1min":0.40400000000000003,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.56699999999999995,"15min":0.54000000000000004},"min":{"1min":0.44700000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.62}]},{"osd":5,"last update":"Fri Sep 13 16:28:36 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58299999999999996,"15min":0.57199999999999995},"min":{"1min":0.433,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.86299999999999999,"5min":1.038,"15min":1.038},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59699999999999998,"15min":0.59599999999999997},"min":{"1min":0.45900000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.73799999999999999,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.72099999999999997}]},{"osd":6,"last update":"Fri Sep 13 16:29:01 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.505,"15min":0.496},"min":{"1min":0.33600000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.48599999999999999,"15min":0.47999999999999998},"min":{"1min":0.34599999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74099999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.498}]},{"osd":7,"last update":"Fri Sep 13 16:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.49399999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.60699999999999998,"15min":0.59299999999999997},"min":{"1min":0.52100000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.64100000000000001}]},{"osd":8,"last update":"Fri Sep 13 16:27:45 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.71499999999999997}]},{"osd":9,"last update":"Fri Sep 13 16:28:08 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.56499999999999995}]},{"osd":10,"last update":"Fri Sep 13 16:28:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.70299999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":1032192,"data_stored":520251,"data_compressed":17719,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.60399999999999998,"15min":0.498},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.90900000000000003,"5min":16.027999999999999,"15min":16.027999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.47299999999999998,"15min":0.46500000000000002},"min":{"1min":0.309,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.69599999999999995}]},{"osd":1,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.48399999999999999,"15min":0.46500000000000002},"min":{"1min":0.372,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.51500000000000001,"15min":0.49299999999999999},"min":{"1min":0.44500000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.83399999999999996}]},{"osd":3,"last update":"Fri Sep 13 16:28:52 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.54000000000000004,"15min":0.51400000000000001},"min":{"1min":0.43099999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54000000000000004,"15min":0.51500000000000001},"min":{"1min":0.36799999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.59099999999999997}]},{"osd":4,"last update":"Fri Sep 13 16:28:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":0.623,"15min":0.55500000000000005},"min":{"1min":0.36399999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":9.6790000000000003,"5min":9.6790000000000003,"15min":9.6790000000000003},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.54500000000000004,"15min":0.53600000000000003},"min":{"1min":0.34399999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.75900000000000001}]},{"osd":5,"last update":"Fri Sep 13 16:28:39 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.40999999999999998,"15min":0.40200000000000002},"min":{"1min":0.33700000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.432},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.40300000000000002,"15min":0.39200000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.46100000000000002}]},{"osd":6,"last update":"Fri Sep 13 16:29:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.57699999999999996,"15min":0.55200000000000005},"min":{"1min":0.437,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.60599999999999998,"15min":0.58399999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.79400000000000004}]},{"osd":7,"last update":"Fri Sep 13 16:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.379,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59099999999999997,"15min":0.58299999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.748}]},{"osd":8,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.52600000000000002,"15min":0.51800000000000002},"min":{"1min":0.33500000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.501},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.51700000000000002,"15min":0.50800000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.55100000000000005}]},{"osd":9,"last update":"Fri Sep 13 16:29:06 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.67900000000000005,"15min":0.66900000000000004},"min":{"1min":0.45100000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.016,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.68100000000000005,"15min":0.67100000000000004},"min":{"1min":0.50600000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.85299999999999998}]},{"osd":10,"last update":"Fri Sep 13 16:28:34 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.89400000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":491382,"data_compressed":14739,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:09 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.47999999999999998,"15min":0.47799999999999998},"min":{"1min":0.34100000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.497},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.48399999999999999,"15min":0.48099999999999998},"min":{"1min":0.38100000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.435}]},{"osd":2,"last update":"Fri Sep 13 16:28:27 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.67100000000000004,"15min":0.54100000000000004},"min":{"1min":0.34100000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.81200000000000006,"5min":17.655000000000001,"15min":17.655000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.49099999999999999,"15min":0.47899999999999998},"min":{"1min":0.29299999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.76200000000000001}]},{"osd":3,"last update":"Fri Sep 13 16:28:51 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52500000000000002,"15min":0.50700000000000001},"min":{"1min":0.379,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72599999999999998,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.55800000000000005,"15min":0.52200000000000002},"min":{"1min":0.42199999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.73899999999999999}]},{"osd":4,"last update":"Fri Sep 13 16:28:13 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.40400000000000003,"15min":0.38},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.433,"5min":0.40899999999999997,"15min":0.39100000000000001},"min":{"1min":0.28299999999999997,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.57899999999999996}]},{"osd":5,"last update":"Fri Sep 13 16:28:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":1.0609999999999999,"15min":1.2829999999999999},"min":{"1min":0.38400000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.82499999999999996,"5min":17.640000000000001,"15min":17.640000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54600000000000004,"15min":0.54300000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.72499999999999998}]},{"osd":6,"last update":"Fri Sep 13 16:29:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59299999999999997,"15min":0.59399999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.72799999999999998,"5min":0.872,"15min":0.872},"last":0.60899999999999999}]},{"osd":7,"last update":"Fri Sep 13 16:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.372,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.78900000000000003}]},{"osd":8,"last update":"Fri Sep 13 16:28:43 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.60399999999999998,"15min":0.61099999999999999},"min":{"1min":0.40699999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.45500000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.83499999999999996}]},{"osd":9,"last update":"Fri Sep 13 16:28:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.751}]},{"osd":10,"last update":"Fri Sep 13 16:28:34 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.82199999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":477257,"data_compressed":11759,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 16:28:15 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.46999999999999997,"15min":0.46899999999999997},"min":{"1min":0.37,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.755,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.50700000000000001,"15min":0.495},"min":{"1min":0.41199999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81000000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.60899999999999999}]},{"osd":2,"last update":"Fri Sep 13 16:28:33 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.55000000000000004,"15min":0.52600000000000002},"min":{"1min":0.42699999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73799999999999999,"5min":1.111,"15min":1.111},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.54800000000000004,"15min":0.56100000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.621,"5min":1.698,"15min":1.698},"last":0.40000000000000002}]},{"osd":3,"last update":"Fri Sep 13 16:27:47 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.438,"15min":0.42399999999999999},"min":{"1min":0.38200000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.42299999999999999,"15min":0.39800000000000002},"min":{"1min":0.22500000000000001,"5min":0.217,"15min":0.217},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.46500000000000002}]},{"osd":4,"last update":"Fri Sep 13 16:28:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.48199999999999998,"15min":0.45400000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.625},{"interface":"front","average":{"1min":0.629,"5min":0.51500000000000001,"15min":0.48399999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.65400000000000003}]},{"osd":5,"last update":"Fri Sep 13 16:28:33 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.55300000000000005,"15min":0.54100000000000004},"min":{"1min":0.44,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56799999999999995,"15min":0.55600000000000005},"min":{"1min":0.46899999999999997,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77800000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.36299999999999999}]},{"osd":6,"last update":"Fri Sep 13 16:28:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.504,"15min":0.48199999999999998},"min":{"1min":0.29899999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.66500000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55300000000000005,"15min":0.52100000000000002},"min":{"1min":0.34399999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.74299999999999999,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.501}]},{"osd":7,"last update":"Fri Sep 13 16:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.68300000000000005,"15min":0.67500000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96999999999999997,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.63700000000000001,"15min":0.63100000000000001},"min":{"1min":0.43099999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.58499999999999996}]},{"osd":8,"last update":"Fri Sep 13 16:28:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1,"5min":1,"15min":1},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69899999999999995,"15min":0.70299999999999996},"min":{"1min":0.55000000000000004,"5min":0.497,"15min":0.497},"max":{"1min":0.81799999999999995,"5min":1.004,"15min":1.004},"last":0.54900000000000004}]},{"osd":9,"last update":"Fri Sep 13 16:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.53500000000000003}]},{"osd":10,"last update":"Fri Sep 13 16:28:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.68899999999999995}]},{"osd":11,"last update":"Fri Sep 13 16:28:58 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.66900000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":25183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":12476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T16:29:10.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:10 smithi016 ceph-mon[31187]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T16:29:10.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:10 smithi016 ceph-mon[31187]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T16:29:10.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:10 smithi016 ceph-mon[31187]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-13T16:29:10.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:10 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/587126362' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-13T16:29:10.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:10 smithi111 ceph-mon[34600]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T16:29:10.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:10 smithi111 ceph-mon[34600]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T16:29:10.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:10 smithi111 ceph-mon[34600]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-13T16:29:10.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:10 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/587126362' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-13T16:29:11.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:10 smithi107 ceph-mon[34139]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T16:29:11.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:10 smithi107 ceph-mon[34139]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T16:29:11.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:10 smithi107 ceph-mon[34139]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-13T16:29:11.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:10 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/587126362' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-13T16:29:11.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:11 smithi111 ceph-mon[34600]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T16:29:11.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:11 smithi111 ceph-mon[34600]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T16:29:11.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:11 smithi111 ceph-mon[34600]: pgmap v294: 129 pgs: 1 peering, 128 active+clean; 594 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 7.7 KiB/s wr, 28 op/s; 95 B/s, 1 keys/s, 2 objects/s recovering 2024-09-13T16:29:12.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:11 smithi107 ceph-mon[34139]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T16:29:12.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:11 smithi107 ceph-mon[34139]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T16:29:12.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:11 smithi107 ceph-mon[34139]: pgmap v294: 129 pgs: 1 peering, 128 active+clean; 594 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 7.7 KiB/s wr, 28 op/s; 95 B/s, 1 keys/s, 2 objects/s recovering 2024-09-13T16:29:12.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:11 smithi016 ceph-mon[31187]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T16:29:12.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:11 smithi016 ceph-mon[31187]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T16:29:12.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:11 smithi016 ceph-mon[31187]: pgmap v294: 129 pgs: 1 peering, 128 active+clean; 594 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 7.7 KiB/s wr, 28 op/s; 95 B/s, 1 keys/s, 2 objects/s recovering 2024-09-13T16:29:12.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:12 smithi111 ceph-mon[34600]: pgmap v295: 129 pgs: 129 active+clean; 629 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 33 op/s; 95 B/s, 1 keys/s, 2 objects/s recovering 2024-09-13T16:29:13.000 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:12 smithi107 ceph-mon[34139]: pgmap v295: 129 pgs: 129 active+clean; 629 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 33 op/s; 95 B/s, 1 keys/s, 2 objects/s recovering 2024-09-13T16:29:13.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:12 smithi016 ceph-mon[31187]: pgmap v295: 129 pgs: 129 active+clean; 629 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 33 op/s; 95 B/s, 1 keys/s, 2 objects/s recovering 2024-09-13T16:29:13.419 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T16:29:13.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.534+0000 7f18cb59c640 1 -- 172.21.15.16:0/544900380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18c4100820 msgr2=0x7f18c4102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:13.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.534+0000 7f18cb59c640 1 --2- 172.21.15.16:0/544900380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18c4100820 0x7f18c4102c10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009900 tx=0x7f18c002a410 comp rx=0 tx=0).stop 2024-09-13T16:29:13.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.534+0000 7f18cb59c640 1 -- 172.21.15.16:0/544900380 shutdown_connections 2024-09-13T16:29:13.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.534+0000 7f18cb59c640 1 --2- 172.21.15.16:0/544900380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18c4105a80 0x7f18c4107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:13.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.534+0000 7f18cb59c640 1 --2- 172.21.15.16:0/544900380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18c4103150 0x7f18c4105540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:13.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.534+0000 7f18cb59c640 1 --2- 172.21.15.16:0/544900380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18c4100820 0x7f18c4102c10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:13.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.534+0000 7f18cb59c640 1 -- 172.21.15.16:0/544900380 >> 172.21.15.16:0/544900380 conn(0x7f18c40fa440 msgr2=0x7f18c40fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:13.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.534+0000 7f18cb59c640 1 -- 172.21.15.16:0/544900380 shutdown_connections 2024-09-13T16:29:13.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.534+0000 7f18cb59c640 1 -- 172.21.15.16:0/544900380 wait complete. 2024-09-13T16:29:13.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.534+0000 7f18cb59c640 1 Processor -- start 2024-09-13T16:29:13.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.535+0000 7f18cb59c640 1 -- start start 2024-09-13T16:29:13.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.535+0000 7f18cb59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18c4100820 0x7f18c4195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:13.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.535+0000 7f18cb59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18c4103150 0x7f18c41962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:13.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.535+0000 7f18cb59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18c4105a80 0x7f18c419d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:13.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.535+0000 7f18cb59c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c419d810 con 0x7f18c4100820 2024-09-13T16:29:13.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.535+0000 7f18cb59c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c419d980 con 0x7f18c4105a80 2024-09-13T16:29:13.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.535+0000 7f18cb59c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c419db40 con 0x7f18c4103150 2024-09-13T16:29:13.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.535+0000 7f18c9338640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18c4100820 0x7f18c4195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:13.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.535+0000 7f18c9338640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18c4100820 0x7f18c4195d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49198/0 (socket says 172.21.15.16:49198) 2024-09-13T16:29:13.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.535+0000 7f18c9338640 1 -- 172.21.15.16:0/3565819517 learned_addr learned my addr 172.21.15.16:0/3565819517 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:13.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.535+0000 7f18c8b37640 1 --2- 172.21.15.16:0/3565819517 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18c4103150 0x7f18c41962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:13.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.535+0000 7f18c9b39640 1 --2- 172.21.15.16:0/3565819517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18c4105a80 0x7f18c419d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:13.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.535+0000 7f18c9338640 1 -- 172.21.15.16:0/3565819517 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18c4103150 msgr2=0x7f18c41962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:13.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.535+0000 7f18c9338640 1 --2- 172.21.15.16:0/3565819517 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18c4103150 0x7f18c41962c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:13.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.535+0000 7f18c9338640 1 -- 172.21.15.16:0/3565819517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18c4105a80 msgr2=0x7f18c419d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:13.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.535+0000 7f18c9338640 1 --2- 172.21.15.16:0/3565819517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18c4105a80 0x7f18c419d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:13.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.535+0000 7f18c9338640 1 -- 172.21.15.16:0/3565819517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c0009560 con 0x7f18c4100820 2024-09-13T16:29:13.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.535+0000 7f18c9338640 1 --2- 172.21.15.16:0/3565819517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18c4100820 0x7f18c4195d80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f18c00057c0 tx=0x7f18c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:13.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.536+0000 7f18ba7fc640 1 -- 172.21.15.16:0/3565819517 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c0031d70 con 0x7f18c4100820 2024-09-13T16:29:13.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.536+0000 7f18ba7fc640 1 -- 172.21.15.16:0/3565819517 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f18c00044e0 con 0x7f18c4100820 2024-09-13T16:29:13.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.536+0000 7f18cb59c640 1 -- 172.21.15.16:0/3565819517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c419ddc0 con 0x7f18c4100820 2024-09-13T16:29:13.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.536+0000 7f18ba7fc640 1 -- 172.21.15.16:0/3565819517 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c0003b80 con 0x7f18c4100820 2024-09-13T16:29:13.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.536+0000 7f18cb59c640 1 -- 172.21.15.16:0/3565819517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c419e1c0 con 0x7f18c4100820 2024-09-13T16:29:13.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.537+0000 7f18cb59c640 1 -- 172.21.15.16:0/3565819517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18c40fbf30 con 0x7f18c4100820 2024-09-13T16:29:13.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.538+0000 7f18ba7fc640 1 -- 172.21.15.16:0/3565819517 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f18c0003ce0 con 0x7f18c4100820 2024-09-13T16:29:13.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.539+0000 7f18ba7fc640 1 --2- 172.21.15.16:0/3565819517 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f18b00a1120 0x7f18b00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:13.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.539+0000 7f18c8b37640 1 --2- 172.21.15.16:0/3565819517 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f18b00a1120 0x7f18b00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:13.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.539+0000 7f18ba7fc640 1 -- 172.21.15.16:0/3565819517 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f18c00ee2e0 con 0x7f18c4100820 2024-09-13T16:29:13.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.540+0000 7f18c8b37640 1 --2- 172.21.15.16:0/3565819517 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f18b00a1120 0x7f18b00a35e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f18b4005d00 tx=0x7f18b400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:13.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.544+0000 7f18ba7fc640 1 -- 172.21.15.16:0/3565819517 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f18c00b7510 con 0x7f18c4100820 2024-09-13T16:29:13.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.716+0000 7f18cb59c640 1 -- 172.21.15.16:0/3565819517 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f18c402c4f0 con 0x7f18b00a1120 2024-09-13T16:29:13.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.727+0000 7f18ba7fc640 1 -- 172.21.15.16:0/3565819517 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423058 (secure 0 0 0) 0x7f18c402c4f0 con 0x7f18b00a1120 2024-09-13T16:29:13.733 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:29:13.735 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-13T16:29:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.731+0000 7f18cb59c640 1 -- 172.21.15.16:0/3565819517 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f18b00a1120 msgr2=0x7f18b00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.731+0000 7f18cb59c640 1 --2- 172.21.15.16:0/3565819517 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f18b00a1120 0x7f18b00a35e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f18b4005d00 tx=0x7f18b400b410 comp rx=0 tx=0).stop 2024-09-13T16:29:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.731+0000 7f18cb59c640 1 -- 172.21.15.16:0/3565819517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18c4100820 msgr2=0x7f18c4195d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.731+0000 7f18cb59c640 1 --2- 172.21.15.16:0/3565819517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18c4100820 0x7f18c4195d80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f18c00057c0 tx=0x7f18c00042f0 comp rx=0 tx=0).stop 2024-09-13T16:29:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.731+0000 7f18cb59c640 1 -- 172.21.15.16:0/3565819517 shutdown_connections 2024-09-13T16:29:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.731+0000 7f18cb59c640 1 --2- 172.21.15.16:0/3565819517 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f18b00a1120 0x7f18b00a35e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.731+0000 7f18cb59c640 1 --2- 172.21.15.16:0/3565819517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18c4105a80 0x7f18c419d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.731+0000 7f18cb59c640 1 --2- 172.21.15.16:0/3565819517 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18c4103150 0x7f18c41962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.731+0000 7f18cb59c640 1 --2- 172.21.15.16:0/3565819517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18c4100820 0x7f18c4195d80 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.731+0000 7f18cb59c640 1 -- 172.21.15.16:0/3565819517 >> 172.21.15.16:0/3565819517 conn(0x7f18c40fa440 msgr2=0x7f18c4101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.731+0000 7f18cb59c640 1 -- 172.21.15.16:0/3565819517 shutdown_connections 2024-09-13T16:29:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:13.731+0000 7f18cb59c640 1 -- 172.21.15.16:0/3565819517 wait complete. 2024-09-13T16:29:13.743 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2024-09-13T16:29:12.572368+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":643633,"num_objects":101,"num_object_clones":0,"num_object_copies":305,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":227,"num_read_kb":226,"num_write":408,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1181498,"num_keys_recovered":10,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":337,"ondisk_log_size":337,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":420,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365096,"kb_used_data":12204,"kb_used_omap":18,"kb_used_meta":352749,"kb_avail":1124576728,"statfs":{"total":1151940427776,"available":1151566569472,"internally_reserved":0,"allocated":12496896,"data_stored":7875069,"data_compressed":321495,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19131,"internal_metadata":361215301},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":48672,"num_objects":72,"num_object_clones":0,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":101,"num_read_kb":117,"num_write":167,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":762,"num_keys_recovered":10,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002356"},"pg_stats":[{"pgid":"2.1f","version":"91'13","reported_seq":59,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:10.651727+0000","last_change":"2024-09-13T16:29:05.647872+0000","last_active":"2024-09-13T16:29:10.651727+0000","last_peered":"2024-09-13T16:29:10.651727+0000","last_clean":"2024-09-13T16:29:10.651727+0000","last_became_active":"2024-09-13T16:29:05.637050+0000","last_became_peered":"2024-09-13T16:29:05.637050+0000","last_unstale":"2024-09-13T16:29:10.651727+0000","last_undegraded":"2024-09-13T16:29:10.651727+0000","last_fullsized":"2024-09-13T16:29:10.651727+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:04.113840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.631052+0000","last_change":"2024-09-13T16:28:39.210024+0000","last_active":"2024-09-13T16:29:04.631052+0000","last_peered":"2024-09-13T16:29:04.631052+0000","last_clean":"2024-09-13T16:29:04.631052+0000","last_became_active":"2024-09-13T16:28:39.209751+0000","last_became_peered":"2024-09-13T16:28:39.209751+0000","last_unstale":"2024-09-13T16:29:04.631052+0000","last_undegraded":"2024-09-13T16:29:04.631052+0000","last_fullsized":"2024-09-13T16:29:04.631052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:34:24.357779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.632148+0000","last_change":"2024-09-13T16:28:43.230223+0000","last_active":"2024-09-13T16:29:04.632148+0000","last_peered":"2024-09-13T16:29:04.632148+0000","last_clean":"2024-09-13T16:29:04.632148+0000","last_became_active":"2024-09-13T16:28:43.228992+0000","last_became_peered":"2024-09-13T16:28:43.228992+0000","last_unstale":"2024-09-13T16:29:04.632148+0000","last_undegraded":"2024-09-13T16:29:04.632148+0000","last_fullsized":"2024-09-13T16:29:04.632148+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:45:37.252865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.631315+0000","last_change":"2024-09-13T16:28:38.850198+0000","last_active":"2024-09-13T16:29:04.631315+0000","last_peered":"2024-09-13T16:29:04.631315+0000","last_clean":"2024-09-13T16:29:04.631315+0000","last_became_active":"2024-09-13T16:28:38.849780+0000","last_became_peered":"2024-09-13T16:28:38.849780+0000","last_unstale":"2024-09-13T16:29:04.631315+0000","last_undegraded":"2024-09-13T16:29:04.631315+0000","last_fullsized":"2024-09-13T16:29:04.631315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:09.653474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.631959+0000","last_change":"2024-09-13T16:28:39.203030+0000","last_active":"2024-09-13T16:29:04.631959+0000","last_peered":"2024-09-13T16:29:04.631959+0000","last_clean":"2024-09-13T16:29:04.631959+0000","last_became_active":"2024-09-13T16:28:39.202464+0000","last_became_peered":"2024-09-13T16:28:39.202464+0000","last_unstale":"2024-09-13T16:29:04.631959+0000","last_undegraded":"2024-09-13T16:29:04.631959+0000","last_fullsized":"2024-09-13T16:29:04.631959+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:57.528836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.632329+0000","last_change":"2024-09-13T16:28:43.224358+0000","last_active":"2024-09-13T16:29:04.632329+0000","last_peered":"2024-09-13T16:29:04.632329+0000","last_clean":"2024-09-13T16:29:04.632329+0000","last_became_active":"2024-09-13T16:28:43.223955+0000","last_became_peered":"2024-09-13T16:28:43.223955+0000","last_unstale":"2024-09-13T16:29:04.632329+0000","last_undegraded":"2024-09-13T16:29:04.632329+0000","last_fullsized":"2024-09-13T16:29:04.632329+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:53:09.169704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'12","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:06.643991+0000","last_change":"2024-09-13T16:28:38.846051+0000","last_active":"2024-09-13T16:29:06.643991+0000","last_peered":"2024-09-13T16:29:06.643991+0000","last_clean":"2024-09-13T16:29:06.643991+0000","last_became_active":"2024-09-13T16:28:38.845762+0000","last_became_peered":"2024-09-13T16:28:38.845762+0000","last_unstale":"2024-09-13T16:29:06.643991+0000","last_undegraded":"2024-09-13T16:29:06.643991+0000","last_fullsized":"2024-09-13T16:29:06.643991+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:33:47.048582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.630530+0000","last_change":"2024-09-13T16:28:39.210516+0000","last_active":"2024-09-13T16:29:04.630530+0000","last_peered":"2024-09-13T16:29:04.630530+0000","last_clean":"2024-09-13T16:29:04.630530+0000","last_became_active":"2024-09-13T16:28:39.210214+0000","last_became_peered":"2024-09-13T16:28:39.210214+0000","last_unstale":"2024-09-13T16:29:04.630530+0000","last_undegraded":"2024-09-13T16:29:04.630530+0000","last_fullsized":"2024-09-13T16:29:04.630530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:51:58.212685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.630396+0000","last_change":"2024-09-13T16:28:43.230530+0000","last_active":"2024-09-13T16:29:04.630396+0000","last_peered":"2024-09-13T16:29:04.630396+0000","last_clean":"2024-09-13T16:29:04.630396+0000","last_became_active":"2024-09-13T16:28:43.230154+0000","last_became_peered":"2024-09-13T16:28:43.230154+0000","last_unstale":"2024-09-13T16:29:04.630396+0000","last_undegraded":"2024-09-13T16:29:04.630396+0000","last_fullsized":"2024-09-13T16:29:04.630396+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:39:52.362177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.683814+0000","last_change":"2024-09-13T16:29:05.653460+0000","last_active":"2024-09-13T16:29:05.683814+0000","last_peered":"2024-09-13T16:29:05.683814+0000","last_clean":"2024-09-13T16:29:05.683814+0000","last_became_active":"2024-09-13T16:29:05.641836+0000","last_became_peered":"2024-09-13T16:29:05.641836+0000","last_unstale":"2024-09-13T16:29:05.683814+0000","last_undegraded":"2024-09-13T16:29:05.683814+0000","last_fullsized":"2024-09-13T16:29:05.683814+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:22:30.051033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.624846+0000","last_change":"2024-09-13T16:28:39.202381+0000","last_active":"2024-09-13T16:29:04.624846+0000","last_peered":"2024-09-13T16:29:04.624846+0000","last_clean":"2024-09-13T16:29:04.624846+0000","last_became_active":"2024-09-13T16:28:39.202023+0000","last_became_peered":"2024-09-13T16:28:39.202023+0000","last_unstale":"2024-09-13T16:29:04.624846+0000","last_undegraded":"2024-09-13T16:29:04.624846+0000","last_fullsized":"2024-09-13T16:29:04.624846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:55:46.445085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628398+0000","last_change":"2024-09-13T16:28:43.230623+0000","last_active":"2024-09-13T16:29:04.628398+0000","last_peered":"2024-09-13T16:29:04.628398+0000","last_clean":"2024-09-13T16:29:04.628398+0000","last_became_active":"2024-09-13T16:28:43.230240+0000","last_became_peered":"2024-09-13T16:28:43.230240+0000","last_unstale":"2024-09-13T16:29:04.628398+0000","last_undegraded":"2024-09-13T16:29:04.628398+0000","last_fullsized":"2024-09-13T16:29:04.628398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:04:07.723269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625334+0000","last_change":"2024-09-13T16:28:39.201896+0000","last_active":"2024-09-13T16:29:04.625334+0000","last_peered":"2024-09-13T16:29:04.625334+0000","last_clean":"2024-09-13T16:29:04.625334+0000","last_became_active":"2024-09-13T16:28:39.201570+0000","last_became_peered":"2024-09-13T16:28:39.201570+0000","last_unstale":"2024-09-13T16:29:04.625334+0000","last_undegraded":"2024-09-13T16:29:04.625334+0000","last_fullsized":"2024-09-13T16:29:04.625334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:07.532058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627167+0000","last_change":"2024-09-13T16:28:38.845998+0000","last_active":"2024-09-13T16:29:04.627167+0000","last_peered":"2024-09-13T16:29:04.627167+0000","last_clean":"2024-09-13T16:29:04.627167+0000","last_became_active":"2024-09-13T16:28:38.845451+0000","last_became_peered":"2024-09-13T16:28:38.845451+0000","last_unstale":"2024-09-13T16:29:04.627167+0000","last_undegraded":"2024-09-13T16:29:04.627167+0000","last_fullsized":"2024-09-13T16:29:04.627167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:15.696276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628820+0000","last_change":"2024-09-13T16:28:43.293598+0000","last_active":"2024-09-13T16:29:04.628820+0000","last_peered":"2024-09-13T16:29:04.628820+0000","last_clean":"2024-09-13T16:29:04.628820+0000","last_became_active":"2024-09-13T16:28:43.293045+0000","last_became_peered":"2024-09-13T16:28:43.293045+0000","last_unstale":"2024-09-13T16:29:04.628820+0000","last_undegraded":"2024-09-13T16:29:04.628820+0000","last_fullsized":"2024-09-13T16:29:04.628820+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:01.839626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.632056+0000","last_change":"2024-09-13T16:28:39.214302+0000","last_active":"2024-09-13T16:29:04.632056+0000","last_peered":"2024-09-13T16:29:04.632056+0000","last_clean":"2024-09-13T16:29:04.632056+0000","last_became_active":"2024-09-13T16:28:39.213672+0000","last_became_peered":"2024-09-13T16:28:39.213672+0000","last_unstale":"2024-09-13T16:29:04.632056+0000","last_undegraded":"2024-09-13T16:29:04.632056+0000","last_fullsized":"2024-09-13T16:29:04.632056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:03:25.440603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"91'2","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.631445+0000","last_change":"2024-09-13T16:28:38.840626+0000","last_active":"2024-09-13T16:29:05.631445+0000","last_peered":"2024-09-13T16:29:05.631445+0000","last_clean":"2024-09-13T16:29:05.631445+0000","last_became_active":"2024-09-13T16:28:38.840344+0000","last_became_peered":"2024-09-13T16:28:38.840344+0000","last_unstale":"2024-09-13T16:29:05.631445+0000","last_undegraded":"2024-09-13T16:29:05.631445+0000","last_fullsized":"2024-09-13T16:29:05.631445+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:03:10.850301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.633381+0000","last_change":"2024-09-13T16:28:43.229163+0000","last_active":"2024-09-13T16:29:05.633381+0000","last_peered":"2024-09-13T16:29:05.633381+0000","last_clean":"2024-09-13T16:29:05.633381+0000","last_became_active":"2024-09-13T16:28:43.228783+0000","last_became_peered":"2024-09-13T16:28:43.228783+0000","last_unstale":"2024-09-13T16:29:05.633381+0000","last_undegraded":"2024-09-13T16:29:05.633381+0000","last_fullsized":"2024-09-13T16:29:05.633381+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:48.183837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627877+0000","last_change":"2024-09-13T16:28:39.209247+0000","last_active":"2024-09-13T16:29:04.627877+0000","last_peered":"2024-09-13T16:29:04.627877+0000","last_clean":"2024-09-13T16:29:04.627877+0000","last_became_active":"2024-09-13T16:28:39.208921+0000","last_became_peered":"2024-09-13T16:28:39.208921+0000","last_unstale":"2024-09-13T16:29:04.627877+0000","last_undegraded":"2024-09-13T16:29:04.627877+0000","last_fullsized":"2024-09-13T16:29:04.627877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:32:41.383029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.641050+0000","last_change":"2024-09-13T16:28:38.845904+0000","last_active":"2024-09-13T16:29:04.641050+0000","last_peered":"2024-09-13T16:29:04.641050+0000","last_clean":"2024-09-13T16:29:04.641050+0000","last_became_active":"2024-09-13T16:28:38.845639+0000","last_became_peered":"2024-09-13T16:28:38.845639+0000","last_unstale":"2024-09-13T16:29:04.641050+0000","last_undegraded":"2024-09-13T16:29:04.641050+0000","last_fullsized":"2024-09-13T16:29:04.641050+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:25:34.148607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.640627+0000","last_change":"2024-09-13T16:29:05.640063+0000","last_active":"2024-09-13T16:29:05.640627+0000","last_peered":"2024-09-13T16:29:05.640627+0000","last_clean":"2024-09-13T16:29:05.640627+0000","last_became_active":"2024-09-13T16:29:05.639596+0000","last_became_peered":"2024-09-13T16:29:05.639596+0000","last_unstale":"2024-09-13T16:29:05.640627+0000","last_undegraded":"2024-09-13T16:29:05.640627+0000","last_fullsized":"2024-09-13T16:29:05.640627+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:41:38.351162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.626859+0000","last_change":"2024-09-13T16:28:39.212947+0000","last_active":"2024-09-13T16:29:04.626859+0000","last_peered":"2024-09-13T16:29:04.626859+0000","last_clean":"2024-09-13T16:29:04.626859+0000","last_became_active":"2024-09-13T16:28:39.212622+0000","last_became_peered":"2024-09-13T16:28:39.212622+0000","last_unstale":"2024-09-13T16:29:04.626859+0000","last_undegraded":"2024-09-13T16:29:04.626859+0000","last_fullsized":"2024-09-13T16:29:04.626859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:16.425154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.630087+0000","last_change":"2024-09-13T16:28:38.846173+0000","last_active":"2024-09-13T16:29:04.630087+0000","last_peered":"2024-09-13T16:29:04.630087+0000","last_clean":"2024-09-13T16:29:04.630087+0000","last_became_active":"2024-09-13T16:28:38.845873+0000","last_became_peered":"2024-09-13T16:28:38.845873+0000","last_unstale":"2024-09-13T16:29:04.630087+0000","last_undegraded":"2024-09-13T16:29:04.630087+0000","last_fullsized":"2024-09-13T16:29:04.630087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:19:55.817524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629006+0000","last_change":"2024-09-13T16:28:43.294347+0000","last_active":"2024-09-13T16:29:04.629006+0000","last_peered":"2024-09-13T16:29:04.629006+0000","last_clean":"2024-09-13T16:29:04.629006+0000","last_became_active":"2024-09-13T16:28:43.293902+0000","last_became_peered":"2024-09-13T16:28:43.293902+0000","last_unstale":"2024-09-13T16:29:04.629006+0000","last_undegraded":"2024-09-13T16:29:04.629006+0000","last_fullsized":"2024-09-13T16:29:04.629006+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:27:37.470163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.631951+0000","last_change":"2024-09-13T16:28:39.205817+0000","last_active":"2024-09-13T16:29:04.631951+0000","last_peered":"2024-09-13T16:29:04.631951+0000","last_clean":"2024-09-13T16:29:04.631951+0000","last_became_active":"2024-09-13T16:28:39.205534+0000","last_became_peered":"2024-09-13T16:28:39.205534+0000","last_unstale":"2024-09-13T16:29:04.631951+0000","last_undegraded":"2024-09-13T16:29:04.631951+0000","last_fullsized":"2024-09-13T16:29:04.631951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:09:16.294842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.633540+0000","last_change":"2024-09-13T16:28:38.847582+0000","last_active":"2024-09-13T16:29:05.633540+0000","last_peered":"2024-09-13T16:29:05.633540+0000","last_clean":"2024-09-13T16:29:05.633540+0000","last_became_active":"2024-09-13T16:28:38.847353+0000","last_became_peered":"2024-09-13T16:28:38.847353+0000","last_unstale":"2024-09-13T16:29:05.633540+0000","last_undegraded":"2024-09-13T16:29:05.633540+0000","last_fullsized":"2024-09-13T16:29:05.633540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:24.944088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.634008+0000","last_change":"2024-09-13T16:28:43.225202+0000","last_active":"2024-09-13T16:29:04.634008+0000","last_peered":"2024-09-13T16:29:04.634008+0000","last_clean":"2024-09-13T16:29:04.634008+0000","last_became_active":"2024-09-13T16:28:43.224802+0000","last_became_peered":"2024-09-13T16:28:43.224802+0000","last_unstale":"2024-09-13T16:29:04.634008+0000","last_undegraded":"2024-09-13T16:29:04.634008+0000","last_fullsized":"2024-09-13T16:29:04.634008+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:51:19.325859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.630738+0000","last_change":"2024-09-13T16:28:39.209630+0000","last_active":"2024-09-13T16:29:04.630738+0000","last_peered":"2024-09-13T16:29:04.630738+0000","last_clean":"2024-09-13T16:29:04.630738+0000","last_became_active":"2024-09-13T16:28:39.209303+0000","last_became_peered":"2024-09-13T16:28:39.209303+0000","last_unstale":"2024-09-13T16:29:04.630738+0000","last_undegraded":"2024-09-13T16:29:04.630738+0000","last_fullsized":"2024-09-13T16:29:04.630738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:01:31.822673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'8","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.630812+0000","last_change":"2024-09-13T16:28:38.837858+0000","last_active":"2024-09-13T16:29:05.630812+0000","last_peered":"2024-09-13T16:29:05.630812+0000","last_clean":"2024-09-13T16:29:05.630812+0000","last_became_active":"2024-09-13T16:28:38.837378+0000","last_became_peered":"2024-09-13T16:28:38.837378+0000","last_unstale":"2024-09-13T16:29:05.630812+0000","last_undegraded":"2024-09-13T16:29:05.630812+0000","last_fullsized":"2024-09-13T16:29:05.630812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:59:41.055902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629967+0000","last_change":"2024-09-13T16:28:43.294241+0000","last_active":"2024-09-13T16:29:04.629967+0000","last_peered":"2024-09-13T16:29:04.629967+0000","last_clean":"2024-09-13T16:29:04.629967+0000","last_became_active":"2024-09-13T16:28:43.293873+0000","last_became_peered":"2024-09-13T16:28:43.293873+0000","last_unstale":"2024-09-13T16:29:04.629967+0000","last_undegraded":"2024-09-13T16:29:04.629967+0000","last_fullsized":"2024-09-13T16:29:04.629967+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:34.018162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.622670+0000","last_change":"2024-09-13T16:28:39.218071+0000","last_active":"2024-09-13T16:29:04.622670+0000","last_peered":"2024-09-13T16:29:04.622670+0000","last_clean":"2024-09-13T16:29:04.622670+0000","last_became_active":"2024-09-13T16:28:39.217798+0000","last_became_peered":"2024-09-13T16:28:39.217798+0000","last_unstale":"2024-09-13T16:29:04.622670+0000","last_undegraded":"2024-09-13T16:29:04.622670+0000","last_fullsized":"2024-09-13T16:29:04.622670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:39:09.769384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.640833+0000","last_change":"2024-09-13T16:28:38.847238+0000","last_active":"2024-09-13T16:29:04.640833+0000","last_peered":"2024-09-13T16:29:04.640833+0000","last_clean":"2024-09-13T16:29:04.640833+0000","last_became_active":"2024-09-13T16:28:38.846741+0000","last_became_peered":"2024-09-13T16:28:38.846741+0000","last_unstale":"2024-09-13T16:29:04.640833+0000","last_undegraded":"2024-09-13T16:29:04.640833+0000","last_fullsized":"2024-09-13T16:29:04.640833+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:52:51.946158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628298+0000","last_change":"2024-09-13T16:28:43.228431+0000","last_active":"2024-09-13T16:29:04.628298+0000","last_peered":"2024-09-13T16:29:04.628298+0000","last_clean":"2024-09-13T16:29:04.628298+0000","last_became_active":"2024-09-13T16:28:43.228030+0000","last_became_peered":"2024-09-13T16:28:43.228030+0000","last_unstale":"2024-09-13T16:29:04.628298+0000","last_undegraded":"2024-09-13T16:29:04.628298+0000","last_fullsized":"2024-09-13T16:29:04.628298+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:14:50.573033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629589+0000","last_change":"2024-09-13T16:28:39.209485+0000","last_active":"2024-09-13T16:29:04.629589+0000","last_peered":"2024-09-13T16:29:04.629589+0000","last_clean":"2024-09-13T16:29:04.629589+0000","last_became_active":"2024-09-13T16:28:39.209157+0000","last_became_peered":"2024-09-13T16:28:39.209157+0000","last_unstale":"2024-09-13T16:29:04.629589+0000","last_undegraded":"2024-09-13T16:29:04.629589+0000","last_fullsized":"2024-09-13T16:29:04.629589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:12:19.398255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628085+0000","last_change":"2024-09-13T16:28:38.842551+0000","last_active":"2024-09-13T16:29:04.628085+0000","last_peered":"2024-09-13T16:29:04.628085+0000","last_clean":"2024-09-13T16:29:04.628085+0000","last_became_active":"2024-09-13T16:28:38.841741+0000","last_became_peered":"2024-09-13T16:28:38.841741+0000","last_unstale":"2024-09-13T16:29:04.628085+0000","last_undegraded":"2024-09-13T16:29:04.628085+0000","last_fullsized":"2024-09-13T16:29:04.628085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:35:25.131693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.626884+0000","last_change":"2024-09-13T16:28:43.294300+0000","last_active":"2024-09-13T16:29:04.626884+0000","last_peered":"2024-09-13T16:29:04.626884+0000","last_clean":"2024-09-13T16:29:04.626884+0000","last_became_active":"2024-09-13T16:28:43.293918+0000","last_became_peered":"2024-09-13T16:28:43.293918+0000","last_unstale":"2024-09-13T16:29:04.626884+0000","last_undegraded":"2024-09-13T16:29:04.626884+0000","last_fullsized":"2024-09-13T16:29:04.626884+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:19:21.959617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.631071+0000","last_change":"2024-09-13T16:28:39.212001+0000","last_active":"2024-09-13T16:29:04.631071+0000","last_peered":"2024-09-13T16:29:04.631071+0000","last_clean":"2024-09-13T16:29:04.631071+0000","last_became_active":"2024-09-13T16:28:39.211716+0000","last_became_peered":"2024-09-13T16:28:39.211716+0000","last_unstale":"2024-09-13T16:29:04.631071+0000","last_undegraded":"2024-09-13T16:29:04.631071+0000","last_fullsized":"2024-09-13T16:29:04.631071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:56.753677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:06.648061+0000","last_change":"2024-09-13T16:28:38.851229+0000","last_active":"2024-09-13T16:29:06.648061+0000","last_peered":"2024-09-13T16:29:06.648061+0000","last_clean":"2024-09-13T16:29:06.648061+0000","last_became_active":"2024-09-13T16:28:38.850937+0000","last_became_peered":"2024-09-13T16:28:38.850937+0000","last_unstale":"2024-09-13T16:29:06.648061+0000","last_undegraded":"2024-09-13T16:29:06.648061+0000","last_fullsized":"2024-09-13T16:29:06.648061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:19.460864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.622472+0000","last_change":"2024-09-13T16:28:43.224825+0000","last_active":"2024-09-13T16:29:04.622472+0000","last_peered":"2024-09-13T16:29:04.622472+0000","last_clean":"2024-09-13T16:29:04.622472+0000","last_became_active":"2024-09-13T16:28:43.223903+0000","last_became_peered":"2024-09-13T16:28:43.223903+0000","last_unstale":"2024-09-13T16:29:04.622472+0000","last_undegraded":"2024-09-13T16:29:04.622472+0000","last_fullsized":"2024-09-13T16:29:04.622472+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:08:46.850668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.639142+0000","last_change":"2024-09-13T16:29:05.636787+0000","last_active":"2024-09-13T16:29:05.639142+0000","last_peered":"2024-09-13T16:29:05.639142+0000","last_clean":"2024-09-13T16:29:05.639142+0000","last_became_active":"2024-09-13T16:29:05.636399+0000","last_became_peered":"2024-09-13T16:29:05.636399+0000","last_unstale":"2024-09-13T16:29:05.639142+0000","last_undegraded":"2024-09-13T16:29:05.639142+0000","last_fullsized":"2024-09-13T16:29:05.639142+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:38:25.545325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:08.640308+0000","last_change":"2024-09-13T16:28:38.845374+0000","last_active":"2024-09-13T16:29:08.640308+0000","last_peered":"2024-09-13T16:29:08.640308+0000","last_clean":"2024-09-13T16:29:08.640308+0000","last_became_active":"2024-09-13T16:28:38.845076+0000","last_became_peered":"2024-09-13T16:28:38.845076+0000","last_unstale":"2024-09-13T16:29:08.640308+0000","last_undegraded":"2024-09-13T16:29:08.640308+0000","last_fullsized":"2024-09-13T16:29:08.640308+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:37:56.225682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.622790+0000","last_change":"2024-09-13T16:28:43.293529+0000","last_active":"2024-09-13T16:29:04.622790+0000","last_peered":"2024-09-13T16:29:04.622790+0000","last_clean":"2024-09-13T16:29:04.622790+0000","last_became_active":"2024-09-13T16:28:43.293083+0000","last_became_peered":"2024-09-13T16:28:43.293083+0000","last_unstale":"2024-09-13T16:29:04.622790+0000","last_undegraded":"2024-09-13T16:29:04.622790+0000","last_fullsized":"2024-09-13T16:29:04.622790+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:05:55.572984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.622261+0000","last_change":"2024-09-13T16:28:39.217546+0000","last_active":"2024-09-13T16:29:04.622261+0000","last_peered":"2024-09-13T16:29:04.622261+0000","last_clean":"2024-09-13T16:29:04.622261+0000","last_became_active":"2024-09-13T16:28:39.217242+0000","last_became_peered":"2024-09-13T16:28:39.217242+0000","last_unstale":"2024-09-13T16:29:04.622261+0000","last_undegraded":"2024-09-13T16:29:04.622261+0000","last_fullsized":"2024-09-13T16:29:04.622261+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:23:18.183616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.641545+0000","last_change":"2024-09-13T16:28:38.846436+0000","last_active":"2024-09-13T16:29:04.641545+0000","last_peered":"2024-09-13T16:29:04.641545+0000","last_clean":"2024-09-13T16:29:04.641545+0000","last_became_active":"2024-09-13T16:28:38.845953+0000","last_became_peered":"2024-09-13T16:28:38.845953+0000","last_unstale":"2024-09-13T16:29:04.641545+0000","last_undegraded":"2024-09-13T16:29:04.641545+0000","last_fullsized":"2024-09-13T16:29:04.641545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:33.998462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.624750+0000","last_change":"2024-09-13T16:28:43.294504+0000","last_active":"2024-09-13T16:29:04.624750+0000","last_peered":"2024-09-13T16:29:04.624750+0000","last_clean":"2024-09-13T16:29:04.624750+0000","last_became_active":"2024-09-13T16:28:43.294063+0000","last_became_peered":"2024-09-13T16:28:43.294063+0000","last_unstale":"2024-09-13T16:29:04.624750+0000","last_undegraded":"2024-09-13T16:29:04.624750+0000","last_fullsized":"2024-09-13T16:29:04.624750+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:51:35.774467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.630536+0000","last_change":"2024-09-13T16:28:39.206491+0000","last_active":"2024-09-13T16:29:04.630536+0000","last_peered":"2024-09-13T16:29:04.630536+0000","last_clean":"2024-09-13T16:29:04.630536+0000","last_became_active":"2024-09-13T16:28:39.206187+0000","last_became_peered":"2024-09-13T16:28:39.206187+0000","last_unstale":"2024-09-13T16:29:04.630536+0000","last_undegraded":"2024-09-13T16:29:04.630536+0000","last_fullsized":"2024-09-13T16:29:04.630536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:20:28.457994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.635563+0000","last_change":"2024-09-13T16:28:38.836270+0000","last_active":"2024-09-13T16:29:05.635563+0000","last_peered":"2024-09-13T16:29:05.635563+0000","last_clean":"2024-09-13T16:29:05.635563+0000","last_became_active":"2024-09-13T16:28:38.835966+0000","last_became_peered":"2024-09-13T16:28:38.835966+0000","last_unstale":"2024-09-13T16:29:05.635563+0000","last_undegraded":"2024-09-13T16:29:05.635563+0000","last_fullsized":"2024-09-13T16:29:05.635563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:03:11.388654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":331,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625402+0000","last_change":"2024-09-13T16:27:34.264484+0000","last_active":"2024-09-13T16:29:04.625402+0000","last_peered":"2024-09-13T16:29:04.625402+0000","last_clean":"2024-09-13T16:29:04.625402+0000","last_became_active":"2024-09-13T16:27:34.263995+0000","last_became_peered":"2024-09-13T16:27:34.263995+0000","last_unstale":"2024-09-13T16:29:04.625402+0000","last_undegraded":"2024-09-13T16:29:04.625402+0000","last_fullsized":"2024-09-13T16:29:04.625402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:24:23.015578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:24:23.015578+0000","last_clean_scrub_stamp":"2024-09-13T16:24:23.015578+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:25:04.376312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.634973+0000","last_change":"2024-09-13T16:28:43.229347+0000","last_active":"2024-09-13T16:29:05.634973+0000","last_peered":"2024-09-13T16:29:05.634973+0000","last_clean":"2024-09-13T16:29:05.634973+0000","last_became_active":"2024-09-13T16:28:43.228981+0000","last_became_peered":"2024-09-13T16:28:43.228981+0000","last_unstale":"2024-09-13T16:29:05.634973+0000","last_undegraded":"2024-09-13T16:29:05.634973+0000","last_fullsized":"2024-09-13T16:29:05.634973+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:46:15.151295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625877+0000","last_change":"2024-09-13T16:28:39.203195+0000","last_active":"2024-09-13T16:29:04.625877+0000","last_peered":"2024-09-13T16:29:04.625877+0000","last_clean":"2024-09-13T16:29:04.625877+0000","last_became_active":"2024-09-13T16:28:39.202878+0000","last_became_peered":"2024-09-13T16:28:39.202878+0000","last_unstale":"2024-09-13T16:29:04.625877+0000","last_undegraded":"2024-09-13T16:29:04.625877+0000","last_fullsized":"2024-09-13T16:29:04.625877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:17.615531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:06.646697+0000","last_change":"2024-09-13T16:28:38.844856+0000","last_active":"2024-09-13T16:29:06.646697+0000","last_peered":"2024-09-13T16:29:06.646697+0000","last_clean":"2024-09-13T16:29:06.646697+0000","last_became_active":"2024-09-13T16:28:38.844578+0000","last_became_peered":"2024-09-13T16:28:38.844578+0000","last_unstale":"2024-09-13T16:29:06.646697+0000","last_undegraded":"2024-09-13T16:29:06.646697+0000","last_fullsized":"2024-09-13T16:29:06.646697+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:41:48.253174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.630271+0000","last_change":"2024-09-13T16:28:43.228472+0000","last_active":"2024-09-13T16:29:04.630271+0000","last_peered":"2024-09-13T16:29:04.630271+0000","last_clean":"2024-09-13T16:29:04.630271+0000","last_became_active":"2024-09-13T16:28:43.228056+0000","last_became_peered":"2024-09-13T16:28:43.228056+0000","last_unstale":"2024-09-13T16:29:04.630271+0000","last_undegraded":"2024-09-13T16:29:04.630271+0000","last_fullsized":"2024-09-13T16:29:04.630271+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:07:55.980691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625637+0000","last_change":"2024-09-13T16:28:39.203278+0000","last_active":"2024-09-13T16:29:04.625637+0000","last_peered":"2024-09-13T16:29:04.625637+0000","last_clean":"2024-09-13T16:29:04.625637+0000","last_became_active":"2024-09-13T16:28:39.202936+0000","last_became_peered":"2024-09-13T16:28:39.202936+0000","last_unstale":"2024-09-13T16:29:04.625637+0000","last_undegraded":"2024-09-13T16:29:04.625637+0000","last_fullsized":"2024-09-13T16:29:04.625637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:44.210612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.640824+0000","last_change":"2024-09-13T16:28:38.851085+0000","last_active":"2024-09-13T16:29:04.640824+0000","last_peered":"2024-09-13T16:29:04.640824+0000","last_clean":"2024-09-13T16:29:04.640824+0000","last_became_active":"2024-09-13T16:28:38.850785+0000","last_became_peered":"2024-09-13T16:28:38.850785+0000","last_unstale":"2024-09-13T16:29:04.640824+0000","last_undegraded":"2024-09-13T16:29:04.640824+0000","last_fullsized":"2024-09-13T16:29:04.640824+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:52.713431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.631346+0000","last_change":"2024-09-13T16:28:43.229980+0000","last_active":"2024-09-13T16:29:04.631346+0000","last_peered":"2024-09-13T16:29:04.631346+0000","last_clean":"2024-09-13T16:29:04.631346+0000","last_became_active":"2024-09-13T16:28:43.229647+0000","last_became_peered":"2024-09-13T16:28:43.229647+0000","last_unstale":"2024-09-13T16:29:04.631346+0000","last_undegraded":"2024-09-13T16:29:04.631346+0000","last_fullsized":"2024-09-13T16:29:04.631346+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:39:42.871613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625770+0000","last_change":"2024-09-13T16:28:39.204167+0000","last_active":"2024-09-13T16:29:04.625770+0000","last_peered":"2024-09-13T16:29:04.625770+0000","last_clean":"2024-09-13T16:29:04.625770+0000","last_became_active":"2024-09-13T16:28:39.203776+0000","last_became_peered":"2024-09-13T16:28:39.203776+0000","last_unstale":"2024-09-13T16:29:04.625770+0000","last_undegraded":"2024-09-13T16:29:04.625770+0000","last_fullsized":"2024-09-13T16:29:04.625770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:53:36.410365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.639989+0000","last_change":"2024-09-13T16:28:38.846331+0000","last_active":"2024-09-13T16:29:04.639989+0000","last_peered":"2024-09-13T16:29:04.639989+0000","last_clean":"2024-09-13T16:29:04.639989+0000","last_became_active":"2024-09-13T16:28:38.846012+0000","last_became_peered":"2024-09-13T16:28:38.846012+0000","last_unstale":"2024-09-13T16:29:04.639989+0000","last_undegraded":"2024-09-13T16:29:04.639989+0000","last_fullsized":"2024-09-13T16:29:04.639989+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:23:54.282736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.623954+0000","last_change":"2024-09-13T16:28:43.227886+0000","last_active":"2024-09-13T16:29:04.623954+0000","last_peered":"2024-09-13T16:29:04.623954+0000","last_clean":"2024-09-13T16:29:04.623954+0000","last_became_active":"2024-09-13T16:28:43.227346+0000","last_became_peered":"2024-09-13T16:28:43.227346+0000","last_unstale":"2024-09-13T16:29:04.623954+0000","last_undegraded":"2024-09-13T16:29:04.623954+0000","last_fullsized":"2024-09-13T16:29:04.623954+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:41:36.559064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629086+0000","last_change":"2024-09-13T16:28:39.205975+0000","last_active":"2024-09-13T16:29:04.629086+0000","last_peered":"2024-09-13T16:29:04.629086+0000","last_clean":"2024-09-13T16:29:04.629086+0000","last_became_active":"2024-09-13T16:28:39.205657+0000","last_became_peered":"2024-09-13T16:28:39.205657+0000","last_unstale":"2024-09-13T16:29:04.629086+0000","last_undegraded":"2024-09-13T16:29:04.629086+0000","last_fullsized":"2024-09-13T16:29:04.629086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:31:47.309482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629013+0000","last_change":"2024-09-13T16:28:38.841589+0000","last_active":"2024-09-13T16:29:04.629013+0000","last_peered":"2024-09-13T16:29:04.629013+0000","last_clean":"2024-09-13T16:29:04.629013+0000","last_became_active":"2024-09-13T16:28:38.841298+0000","last_became_peered":"2024-09-13T16:28:38.841298+0000","last_unstale":"2024-09-13T16:29:04.629013+0000","last_undegraded":"2024-09-13T16:29:04.629013+0000","last_fullsized":"2024-09-13T16:29:04.629013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:27:38.358320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.623986+0000","last_change":"2024-09-13T16:28:43.233790+0000","last_active":"2024-09-13T16:29:04.623986+0000","last_peered":"2024-09-13T16:29:04.623986+0000","last_clean":"2024-09-13T16:29:04.623986+0000","last_became_active":"2024-09-13T16:28:43.231963+0000","last_became_peered":"2024-09-13T16:28:43.231963+0000","last_unstale":"2024-09-13T16:29:04.623986+0000","last_undegraded":"2024-09-13T16:29:04.623986+0000","last_fullsized":"2024-09-13T16:29:04.623986+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:45:23.406395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.624233+0000","last_change":"2024-09-13T16:28:39.201476+0000","last_active":"2024-09-13T16:29:04.624233+0000","last_peered":"2024-09-13T16:29:04.624233+0000","last_clean":"2024-09-13T16:29:04.624233+0000","last_became_active":"2024-09-13T16:28:39.201168+0000","last_became_peered":"2024-09-13T16:28:39.201168+0000","last_unstale":"2024-09-13T16:29:04.624233+0000","last_undegraded":"2024-09-13T16:29:04.624233+0000","last_fullsized":"2024-09-13T16:29:04.624233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:26:43.570295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":62,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.652225+0000","last_change":"2024-09-13T16:29:05.652225+0000","last_active":"2024-09-13T16:29:05.652225+0000","last_peered":"2024-09-13T16:29:05.652225+0000","last_clean":"2024-09-13T16:29:05.652225+0000","last_became_active":"2024-09-13T16:29:05.639325+0000","last_became_peered":"2024-09-13T16:29:05.639325+0000","last_unstale":"2024-09-13T16:29:05.652225+0000","last_undegraded":"2024-09-13T16:29:05.652225+0000","last_fullsized":"2024-09-13T16:29:05.652225+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:18:30.083447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.622797+0000","last_change":"2024-09-13T16:28:43.228563+0000","last_active":"2024-09-13T16:29:04.622797+0000","last_peered":"2024-09-13T16:29:04.622797+0000","last_clean":"2024-09-13T16:29:04.622797+0000","last_became_active":"2024-09-13T16:28:43.228184+0000","last_became_peered":"2024-09-13T16:28:43.228184+0000","last_unstale":"2024-09-13T16:29:04.622797+0000","last_undegraded":"2024-09-13T16:29:04.622797+0000","last_fullsized":"2024-09-13T16:29:04.622797+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:05:33.368302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625221+0000","last_change":"2024-09-13T16:28:39.210750+0000","last_active":"2024-09-13T16:29:04.625221+0000","last_peered":"2024-09-13T16:29:04.625221+0000","last_clean":"2024-09-13T16:29:04.625221+0000","last_became_active":"2024-09-13T16:28:39.210401+0000","last_became_peered":"2024-09-13T16:28:39.210401+0000","last_unstale":"2024-09-13T16:29:04.625221+0000","last_undegraded":"2024-09-13T16:29:04.625221+0000","last_fullsized":"2024-09-13T16:29:04.625221+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:36.670420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:06.638428+0000","last_change":"2024-09-13T16:28:38.843573+0000","last_active":"2024-09-13T16:29:06.638428+0000","last_peered":"2024-09-13T16:29:06.638428+0000","last_clean":"2024-09-13T16:29:06.638428+0000","last_became_active":"2024-09-13T16:28:38.843245+0000","last_became_peered":"2024-09-13T16:28:38.843245+0000","last_unstale":"2024-09-13T16:29:06.638428+0000","last_undegraded":"2024-09-13T16:29:06.638428+0000","last_fullsized":"2024-09-13T16:29:06.638428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:45.983123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628374+0000","last_change":"2024-09-13T16:28:43.231002+0000","last_active":"2024-09-13T16:29:04.628374+0000","last_peered":"2024-09-13T16:29:04.628374+0000","last_clean":"2024-09-13T16:29:04.628374+0000","last_became_active":"2024-09-13T16:28:43.230237+0000","last_became_peered":"2024-09-13T16:28:43.230237+0000","last_unstale":"2024-09-13T16:29:04.628374+0000","last_undegraded":"2024-09-13T16:29:04.628374+0000","last_fullsized":"2024-09-13T16:29:04.628374+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:11:47.494738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625559+0000","last_change":"2024-09-13T16:28:39.201471+0000","last_active":"2024-09-13T16:29:04.625559+0000","last_peered":"2024-09-13T16:29:04.625559+0000","last_clean":"2024-09-13T16:29:04.625559+0000","last_became_active":"2024-09-13T16:28:39.201140+0000","last_became_peered":"2024-09-13T16:28:39.201140+0000","last_unstale":"2024-09-13T16:29:04.625559+0000","last_undegraded":"2024-09-13T16:29:04.625559+0000","last_fullsized":"2024-09-13T16:29:04.625559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:37:25.037490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.637966+0000","last_change":"2024-09-13T16:28:38.845703+0000","last_active":"2024-09-13T16:29:05.637966+0000","last_peered":"2024-09-13T16:29:05.637966+0000","last_clean":"2024-09-13T16:29:05.637966+0000","last_became_active":"2024-09-13T16:28:38.845354+0000","last_became_peered":"2024-09-13T16:28:38.845354+0000","last_unstale":"2024-09-13T16:29:05.637966+0000","last_undegraded":"2024-09-13T16:29:05.637966+0000","last_fullsized":"2024-09-13T16:29:05.637966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:18.208723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629807+0000","last_change":"2024-09-13T16:28:43.223924+0000","last_active":"2024-09-13T16:29:04.629807+0000","last_peered":"2024-09-13T16:29:04.629807+0000","last_clean":"2024-09-13T16:29:04.629807+0000","last_became_active":"2024-09-13T16:28:43.223535+0000","last_became_peered":"2024-09-13T16:28:43.223535+0000","last_unstale":"2024-09-13T16:29:04.629807+0000","last_undegraded":"2024-09-13T16:29:04.629807+0000","last_fullsized":"2024-09-13T16:29:04.629807+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:10.622436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.622546+0000","last_change":"2024-09-13T16:28:39.203943+0000","last_active":"2024-09-13T16:29:04.622546+0000","last_peered":"2024-09-13T16:29:04.622546+0000","last_clean":"2024-09-13T16:29:04.622546+0000","last_became_active":"2024-09-13T16:28:39.203657+0000","last_became_peered":"2024-09-13T16:28:39.203657+0000","last_unstale":"2024-09-13T16:29:04.622546+0000","last_undegraded":"2024-09-13T16:29:04.622546+0000","last_fullsized":"2024-09-13T16:29:04.622546+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:52:48.523939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.683795+0000","last_change":"2024-09-13T16:29:05.651015+0000","last_active":"2024-09-13T16:29:05.683795+0000","last_peered":"2024-09-13T16:29:05.683795+0000","last_clean":"2024-09-13T16:29:05.683795+0000","last_became_active":"2024-09-13T16:29:05.640006+0000","last_became_peered":"2024-09-13T16:29:05.640006+0000","last_unstale":"2024-09-13T16:29:05.683795+0000","last_undegraded":"2024-09-13T16:29:05.683795+0000","last_fullsized":"2024-09-13T16:29:05.683795+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:12:25.508262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.624674+0000","last_change":"2024-09-13T16:28:43.232634+0000","last_active":"2024-09-13T16:29:04.624674+0000","last_peered":"2024-09-13T16:29:04.624674+0000","last_clean":"2024-09-13T16:29:04.624674+0000","last_became_active":"2024-09-13T16:28:43.232107+0000","last_became_peered":"2024-09-13T16:28:43.232107+0000","last_unstale":"2024-09-13T16:29:04.624674+0000","last_undegraded":"2024-09-13T16:29:04.624674+0000","last_fullsized":"2024-09-13T16:29:04.624674+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:45.952933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627482+0000","last_change":"2024-09-13T16:28:39.212883+0000","last_active":"2024-09-13T16:29:04.627482+0000","last_peered":"2024-09-13T16:29:04.627482+0000","last_clean":"2024-09-13T16:29:04.627482+0000","last_became_active":"2024-09-13T16:28:39.212591+0000","last_became_peered":"2024-09-13T16:28:39.212591+0000","last_unstale":"2024-09-13T16:29:04.627482+0000","last_undegraded":"2024-09-13T16:29:04.627482+0000","last_fullsized":"2024-09-13T16:29:04.627482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:17:30.926940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'26","reported_seq":69,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:11.654391+0000","last_change":"2024-09-13T16:28:38.846564+0000","last_active":"2024-09-13T16:29:11.654391+0000","last_peered":"2024-09-13T16:29:11.654391+0000","last_clean":"2024-09-13T16:29:11.654391+0000","last_became_active":"2024-09-13T16:28:38.846165+0000","last_became_peered":"2024-09-13T16:28:38.846165+0000","last_unstale":"2024-09-13T16:29:11.654391+0000","last_undegraded":"2024-09-13T16:29:11.654391+0000","last_fullsized":"2024-09-13T16:29:11.654391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:33:06.124592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45625,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":25,"num_read_kb":57,"num_write":28,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627853+0000","last_change":"2024-09-13T16:28:43.231464+0000","last_active":"2024-09-13T16:29:04.627853+0000","last_peered":"2024-09-13T16:29:04.627853+0000","last_clean":"2024-09-13T16:29:04.627853+0000","last_became_active":"2024-09-13T16:28:43.231040+0000","last_became_peered":"2024-09-13T16:28:43.231040+0000","last_unstale":"2024-09-13T16:29:04.627853+0000","last_undegraded":"2024-09-13T16:29:04.627853+0000","last_fullsized":"2024-09-13T16:29:04.627853+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:11:21.847375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.639415+0000","last_change":"2024-09-13T16:29:05.639415+0000","last_active":"2024-09-13T16:29:05.639415+0000","last_peered":"2024-09-13T16:29:05.639415+0000","last_clean":"2024-09-13T16:29:05.639415+0000","last_became_active":"2024-09-13T16:29:05.639101+0000","last_became_peered":"2024-09-13T16:29:05.639101+0000","last_unstale":"2024-09-13T16:29:05.639415+0000","last_undegraded":"2024-09-13T16:29:05.639415+0000","last_fullsized":"2024-09-13T16:29:05.639415+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:32:42.630013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625089+0000","last_change":"2024-09-13T16:28:38.841637+0000","last_active":"2024-09-13T16:29:04.625089+0000","last_peered":"2024-09-13T16:29:04.625089+0000","last_clean":"2024-09-13T16:29:04.625089+0000","last_became_active":"2024-09-13T16:28:38.841359+0000","last_became_peered":"2024-09-13T16:28:38.841359+0000","last_unstale":"2024-09-13T16:29:04.625089+0000","last_undegraded":"2024-09-13T16:29:04.625089+0000","last_fullsized":"2024-09-13T16:29:04.625089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:47:05.284987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.638870+0000","last_change":"2024-09-13T16:29:05.638870+0000","last_active":"2024-09-13T16:29:05.638870+0000","last_peered":"2024-09-13T16:29:05.638870+0000","last_clean":"2024-09-13T16:29:05.638870+0000","last_became_active":"2024-09-13T16:29:05.638422+0000","last_became_peered":"2024-09-13T16:29:05.638422+0000","last_unstale":"2024-09-13T16:29:05.638870+0000","last_undegraded":"2024-09-13T16:29:05.638870+0000","last_fullsized":"2024-09-13T16:29:05.638870+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:41.263661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.631976+0000","last_change":"2024-09-13T16:28:39.203595+0000","last_active":"2024-09-13T16:29:04.631976+0000","last_peered":"2024-09-13T16:29:04.631976+0000","last_clean":"2024-09-13T16:29:04.631976+0000","last_became_active":"2024-09-13T16:28:39.203283+0000","last_became_peered":"2024-09-13T16:28:39.203283+0000","last_unstale":"2024-09-13T16:29:04.631976+0000","last_undegraded":"2024-09-13T16:29:04.631976+0000","last_fullsized":"2024-09-13T16:29:04.631976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:58:45.970761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.642267+0000","last_change":"2024-09-13T16:28:38.850203+0000","last_active":"2024-09-13T16:29:04.642267+0000","last_peered":"2024-09-13T16:29:04.642267+0000","last_clean":"2024-09-13T16:29:04.642267+0000","last_became_active":"2024-09-13T16:28:38.849963+0000","last_became_peered":"2024-09-13T16:28:38.849963+0000","last_unstale":"2024-09-13T16:29:04.642267+0000","last_undegraded":"2024-09-13T16:29:04.642267+0000","last_fullsized":"2024-09-13T16:29:04.642267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:15.849627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627470+0000","last_change":"2024-09-13T16:28:43.227679+0000","last_active":"2024-09-13T16:29:04.627470+0000","last_peered":"2024-09-13T16:29:04.627470+0000","last_clean":"2024-09-13T16:29:04.627470+0000","last_became_active":"2024-09-13T16:28:43.227269+0000","last_became_peered":"2024-09-13T16:28:43.227269+0000","last_unstale":"2024-09-13T16:29:04.627470+0000","last_undegraded":"2024-09-13T16:29:04.627470+0000","last_fullsized":"2024-09-13T16:29:04.627470+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:05:00.186305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.630707+0000","last_change":"2024-09-13T16:28:39.211401+0000","last_active":"2024-09-13T16:29:04.630707+0000","last_peered":"2024-09-13T16:29:04.630707+0000","last_clean":"2024-09-13T16:29:04.630707+0000","last_became_active":"2024-09-13T16:28:39.211133+0000","last_became_peered":"2024-09-13T16:28:39.211133+0000","last_unstale":"2024-09-13T16:29:04.630707+0000","last_undegraded":"2024-09-13T16:29:04.630707+0000","last_fullsized":"2024-09-13T16:29:04.630707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:36.468666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:11.649241+0000","last_change":"2024-09-13T16:28:38.845132+0000","last_active":"2024-09-13T16:29:11.649241+0000","last_peered":"2024-09-13T16:29:11.649241+0000","last_clean":"2024-09-13T16:29:11.649241+0000","last_became_active":"2024-09-13T16:28:38.844804+0000","last_became_peered":"2024-09-13T16:28:38.844804+0000","last_unstale":"2024-09-13T16:29:11.649241+0000","last_undegraded":"2024-09-13T16:29:11.649241+0000","last_fullsized":"2024-09-13T16:29:11.649241+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:15:08.643227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.630994+0000","last_change":"2024-09-13T16:28:43.230829+0000","last_active":"2024-09-13T16:29:04.630994+0000","last_peered":"2024-09-13T16:29:04.630994+0000","last_clean":"2024-09-13T16:29:04.630994+0000","last_became_active":"2024-09-13T16:28:43.230405+0000","last_became_peered":"2024-09-13T16:28:43.230405+0000","last_unstale":"2024-09-13T16:29:04.630994+0000","last_undegraded":"2024-09-13T16:29:04.630994+0000","last_fullsized":"2024-09-13T16:29:04.630994+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:04:56.083193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.631569+0000","last_change":"2024-09-13T16:28:43.226662+0000","last_active":"2024-09-13T16:29:04.631569+0000","last_peered":"2024-09-13T16:29:04.631569+0000","last_clean":"2024-09-13T16:29:04.631569+0000","last_became_active":"2024-09-13T16:28:43.226232+0000","last_became_peered":"2024-09-13T16:28:43.226232+0000","last_unstale":"2024-09-13T16:29:04.631569+0000","last_undegraded":"2024-09-13T16:29:04.631569+0000","last_fullsized":"2024-09-13T16:29:04.631569+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:03.568500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627636+0000","last_change":"2024-09-13T16:28:39.208909+0000","last_active":"2024-09-13T16:29:04.627636+0000","last_peered":"2024-09-13T16:29:04.627636+0000","last_clean":"2024-09-13T16:29:04.627636+0000","last_became_active":"2024-09-13T16:28:39.208572+0000","last_became_peered":"2024-09-13T16:28:39.208572+0000","last_unstale":"2024-09-13T16:29:04.627636+0000","last_undegraded":"2024-09-13T16:29:04.627636+0000","last_fullsized":"2024-09-13T16:29:04.627636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:01:40.173308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.637548+0000","last_change":"2024-09-13T16:28:38.844259+0000","last_active":"2024-09-13T16:29:05.637548+0000","last_peered":"2024-09-13T16:29:05.637548+0000","last_clean":"2024-09-13T16:29:05.637548+0000","last_became_active":"2024-09-13T16:28:38.843956+0000","last_became_peered":"2024-09-13T16:28:38.843956+0000","last_unstale":"2024-09-13T16:29:05.637548+0000","last_undegraded":"2024-09-13T16:29:05.637548+0000","last_fullsized":"2024-09-13T16:29:05.637548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:05.652852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629221+0000","last_change":"2024-09-13T16:28:43.229385+0000","last_active":"2024-09-13T16:29:04.629221+0000","last_peered":"2024-09-13T16:29:04.629221+0000","last_clean":"2024-09-13T16:29:04.629221+0000","last_became_active":"2024-09-13T16:28:43.229033+0000","last_became_peered":"2024-09-13T16:28:43.229033+0000","last_unstale":"2024-09-13T16:29:04.629221+0000","last_undegraded":"2024-09-13T16:29:04.629221+0000","last_fullsized":"2024-09-13T16:29:04.629221+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:46:41.611758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.630167+0000","last_change":"2024-09-13T16:28:39.208100+0000","last_active":"2024-09-13T16:29:04.630167+0000","last_peered":"2024-09-13T16:29:04.630167+0000","last_clean":"2024-09-13T16:29:04.630167+0000","last_became_active":"2024-09-13T16:28:39.207808+0000","last_became_peered":"2024-09-13T16:28:39.207808+0000","last_unstale":"2024-09-13T16:29:04.630167+0000","last_undegraded":"2024-09-13T16:29:04.630167+0000","last_fullsized":"2024-09-13T16:29:04.630167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:55.446568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.640584+0000","last_change":"2024-09-13T16:28:38.844689+0000","last_active":"2024-09-13T16:29:04.640584+0000","last_peered":"2024-09-13T16:29:04.640584+0000","last_clean":"2024-09-13T16:29:04.640584+0000","last_became_active":"2024-09-13T16:28:38.844326+0000","last_became_peered":"2024-09-13T16:28:38.844326+0000","last_unstale":"2024-09-13T16:29:04.640584+0000","last_undegraded":"2024-09-13T16:29:04.640584+0000","last_fullsized":"2024-09-13T16:29:04.640584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:02.616518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629530+0000","last_change":"2024-09-13T16:28:38.849233+0000","last_active":"2024-09-13T16:29:04.629530+0000","last_peered":"2024-09-13T16:29:04.629530+0000","last_clean":"2024-09-13T16:29:04.629530+0000","last_became_active":"2024-09-13T16:28:38.848911+0000","last_became_peered":"2024-09-13T16:28:38.848911+0000","last_unstale":"2024-09-13T16:29:04.629530+0000","last_undegraded":"2024-09-13T16:29:04.629530+0000","last_fullsized":"2024-09-13T16:29:04.629530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:02:35.951263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.624339+0000","last_change":"2024-09-13T16:28:39.203855+0000","last_active":"2024-09-13T16:29:04.624339+0000","last_peered":"2024-09-13T16:29:04.624339+0000","last_clean":"2024-09-13T16:29:04.624339+0000","last_became_active":"2024-09-13T16:28:39.203532+0000","last_became_peered":"2024-09-13T16:28:39.203532+0000","last_unstale":"2024-09-13T16:29:04.624339+0000","last_undegraded":"2024-09-13T16:29:04.624339+0000","last_fullsized":"2024-09-13T16:29:04.624339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:15.048930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.633730+0000","last_change":"2024-09-13T16:28:43.228554+0000","last_active":"2024-09-13T16:29:04.633730+0000","last_peered":"2024-09-13T16:29:04.633730+0000","last_clean":"2024-09-13T16:29:04.633730+0000","last_became_active":"2024-09-13T16:28:43.228204+0000","last_became_peered":"2024-09-13T16:28:43.228204+0000","last_unstale":"2024-09-13T16:29:04.633730+0000","last_undegraded":"2024-09-13T16:29:04.633730+0000","last_fullsized":"2024-09-13T16:29:04.633730+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:16:15.383932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:09.647140+0000","last_change":"2024-09-13T16:28:38.848425+0000","last_active":"2024-09-13T16:29:09.647140+0000","last_peered":"2024-09-13T16:29:09.647140+0000","last_clean":"2024-09-13T16:29:09.647140+0000","last_became_active":"2024-09-13T16:28:38.848136+0000","last_became_peered":"2024-09-13T16:28:38.848136+0000","last_unstale":"2024-09-13T16:29:09.647140+0000","last_undegraded":"2024-09-13T16:29:09.647140+0000","last_fullsized":"2024-09-13T16:29:09.647140+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:47:03.904315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627249+0000","last_change":"2024-09-13T16:28:39.210565+0000","last_active":"2024-09-13T16:29:04.627249+0000","last_peered":"2024-09-13T16:29:04.627249+0000","last_clean":"2024-09-13T16:29:04.627249+0000","last_became_active":"2024-09-13T16:28:39.210272+0000","last_became_peered":"2024-09-13T16:28:39.210272+0000","last_unstale":"2024-09-13T16:29:04.627249+0000","last_undegraded":"2024-09-13T16:29:04.627249+0000","last_fullsized":"2024-09-13T16:29:04.627249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:12.746087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.634077+0000","last_change":"2024-09-13T16:28:43.231623+0000","last_active":"2024-09-13T16:29:04.634077+0000","last_peered":"2024-09-13T16:29:04.634077+0000","last_clean":"2024-09-13T16:29:04.634077+0000","last_became_active":"2024-09-13T16:28:43.231255+0000","last_became_peered":"2024-09-13T16:28:43.231255+0000","last_unstale":"2024-09-13T16:29:04.634077+0000","last_undegraded":"2024-09-13T16:29:04.634077+0000","last_fullsized":"2024-09-13T16:29:04.634077+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:29.755772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627783+0000","last_change":"2024-09-13T16:28:39.208438+0000","last_active":"2024-09-13T16:29:04.627783+0000","last_peered":"2024-09-13T16:29:04.627783+0000","last_clean":"2024-09-13T16:29:04.627783+0000","last_became_active":"2024-09-13T16:28:39.208156+0000","last_became_peered":"2024-09-13T16:28:39.208156+0000","last_unstale":"2024-09-13T16:29:04.627783+0000","last_undegraded":"2024-09-13T16:29:04.627783+0000","last_fullsized":"2024-09-13T16:29:04.627783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:53:46.923166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.633806+0000","last_change":"2024-09-13T16:28:39.208529+0000","last_active":"2024-09-13T16:29:04.633806+0000","last_peered":"2024-09-13T16:29:04.633806+0000","last_clean":"2024-09-13T16:29:04.633806+0000","last_became_active":"2024-09-13T16:28:39.208202+0000","last_became_peered":"2024-09-13T16:28:39.208202+0000","last_unstale":"2024-09-13T16:29:04.633806+0000","last_undegraded":"2024-09-13T16:29:04.633806+0000","last_fullsized":"2024-09-13T16:29:04.633806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:17:11.357410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.632611+0000","last_change":"2024-09-13T16:28:39.212958+0000","last_active":"2024-09-13T16:29:04.632611+0000","last_peered":"2024-09-13T16:29:04.632611+0000","last_clean":"2024-09-13T16:29:04.632611+0000","last_became_active":"2024-09-13T16:28:39.212646+0000","last_became_peered":"2024-09-13T16:28:39.212646+0000","last_unstale":"2024-09-13T16:29:04.632611+0000","last_undegraded":"2024-09-13T16:29:04.632611+0000","last_fullsized":"2024-09-13T16:29:04.632611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:31:04.047778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.631429+0000","last_change":"2024-09-13T16:28:39.200449+0000","last_active":"2024-09-13T16:29:04.631429+0000","last_peered":"2024-09-13T16:29:04.631429+0000","last_clean":"2024-09-13T16:29:04.631429+0000","last_became_active":"2024-09-13T16:28:39.200133+0000","last_became_peered":"2024-09-13T16:28:39.200133+0000","last_unstale":"2024-09-13T16:29:04.631429+0000","last_undegraded":"2024-09-13T16:29:04.631429+0000","last_fullsized":"2024-09-13T16:29:04.631429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:18:06.480691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.622597+0000","last_change":"2024-09-13T16:28:39.200274+0000","last_active":"2024-09-13T16:29:04.622597+0000","last_peered":"2024-09-13T16:29:04.622597+0000","last_clean":"2024-09-13T16:29:04.622597+0000","last_became_active":"2024-09-13T16:28:39.199951+0000","last_became_peered":"2024-09-13T16:28:39.199951+0000","last_unstale":"2024-09-13T16:29:04.622597+0000","last_undegraded":"2024-09-13T16:29:04.622597+0000","last_fullsized":"2024-09-13T16:29:04.622597+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:13:31.199857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.630000+0000","last_change":"2024-09-13T16:28:39.213591+0000","last_active":"2024-09-13T16:29:04.630000+0000","last_peered":"2024-09-13T16:29:04.630000+0000","last_clean":"2024-09-13T16:29:04.630000+0000","last_became_active":"2024-09-13T16:28:39.213324+0000","last_became_peered":"2024-09-13T16:28:39.213324+0000","last_unstale":"2024-09-13T16:29:04.630000+0000","last_undegraded":"2024-09-13T16:29:04.630000+0000","last_fullsized":"2024-09-13T16:29:04.630000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:54:03.172919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.633921+0000","last_change":"2024-09-13T16:28:39.217442+0000","last_active":"2024-09-13T16:29:04.633921+0000","last_peered":"2024-09-13T16:29:04.633921+0000","last_clean":"2024-09-13T16:29:04.633921+0000","last_became_active":"2024-09-13T16:28:39.217151+0000","last_became_peered":"2024-09-13T16:28:39.217151+0000","last_unstale":"2024-09-13T16:29:04.633921+0000","last_undegraded":"2024-09-13T16:29:04.633921+0000","last_fullsized":"2024-09-13T16:29:04.633921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:17.132278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.630659+0000","last_change":"2024-09-13T16:28:39.203938+0000","last_active":"2024-09-13T16:29:04.630659+0000","last_peered":"2024-09-13T16:29:04.630659+0000","last_clean":"2024-09-13T16:29:04.630659+0000","last_became_active":"2024-09-13T16:28:39.203655+0000","last_became_peered":"2024-09-13T16:28:39.203655+0000","last_unstale":"2024-09-13T16:29:04.630659+0000","last_undegraded":"2024-09-13T16:29:04.630659+0000","last_fullsized":"2024-09-13T16:29:04.630659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:03:25.415256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.624165+0000","last_change":"2024-09-13T16:28:39.209592+0000","last_active":"2024-09-13T16:29:04.624165+0000","last_peered":"2024-09-13T16:29:04.624165+0000","last_clean":"2024-09-13T16:29:04.624165+0000","last_became_active":"2024-09-13T16:28:39.209299+0000","last_became_peered":"2024-09-13T16:28:39.209299+0000","last_unstale":"2024-09-13T16:29:04.624165+0000","last_undegraded":"2024-09-13T16:29:04.624165+0000","last_fullsized":"2024-09-13T16:29:04.624165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:04.053405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.630891+0000","last_change":"2024-09-13T16:28:39.208437+0000","last_active":"2024-09-13T16:29:04.630891+0000","last_peered":"2024-09-13T16:29:04.630891+0000","last_clean":"2024-09-13T16:29:04.630891+0000","last_became_active":"2024-09-13T16:28:39.208136+0000","last_became_peered":"2024-09-13T16:28:39.208136+0000","last_unstale":"2024-09-13T16:29:04.630891+0000","last_undegraded":"2024-09-13T16:29:04.630891+0000","last_fullsized":"2024-09-13T16:29:04.630891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:21.304037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.630611+0000","last_change":"2024-09-13T16:28:39.203970+0000","last_active":"2024-09-13T16:29:04.630611+0000","last_peered":"2024-09-13T16:29:04.630611+0000","last_clean":"2024-09-13T16:29:04.630611+0000","last_became_active":"2024-09-13T16:28:39.203559+0000","last_became_peered":"2024-09-13T16:28:39.203559+0000","last_unstale":"2024-09-13T16:29:04.630611+0000","last_undegraded":"2024-09-13T16:29:04.630611+0000","last_fullsized":"2024-09-13T16:29:04.630611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:26:25.158393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.622556+0000","last_change":"2024-09-13T16:28:39.205584+0000","last_active":"2024-09-13T16:29:04.622556+0000","last_peered":"2024-09-13T16:29:04.622556+0000","last_clean":"2024-09-13T16:29:04.622556+0000","last_became_active":"2024-09-13T16:28:39.205281+0000","last_became_peered":"2024-09-13T16:28:39.205281+0000","last_unstale":"2024-09-13T16:29:04.622556+0000","last_undegraded":"2024-09-13T16:29:04.622556+0000","last_fullsized":"2024-09-13T16:29:04.622556+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:58:10.109647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629903+0000","last_change":"2024-09-13T16:28:39.214479+0000","last_active":"2024-09-13T16:29:04.629903+0000","last_peered":"2024-09-13T16:29:04.629903+0000","last_clean":"2024-09-13T16:29:04.629903+0000","last_became_active":"2024-09-13T16:28:39.213984+0000","last_became_peered":"2024-09-13T16:28:39.213984+0000","last_unstale":"2024-09-13T16:29:04.629903+0000","last_undegraded":"2024-09-13T16:29:04.629903+0000","last_fullsized":"2024-09-13T16:29:04.629903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:53:43.137726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.634317+0000","last_change":"2024-09-13T16:28:39.209595+0000","last_active":"2024-09-13T16:29:04.634317+0000","last_peered":"2024-09-13T16:29:04.634317+0000","last_clean":"2024-09-13T16:29:04.634317+0000","last_became_active":"2024-09-13T16:28:39.209282+0000","last_became_peered":"2024-09-13T16:28:39.209282+0000","last_unstale":"2024-09-13T16:29:04.634317+0000","last_undegraded":"2024-09-13T16:29:04.634317+0000","last_fullsized":"2024-09-13T16:29:04.634317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:26:41.452462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.628010+0000","last_change":"2024-09-13T16:28:39.210610+0000","last_active":"2024-09-13T16:29:04.628010+0000","last_peered":"2024-09-13T16:29:04.628010+0000","last_clean":"2024-09-13T16:29:04.628010+0000","last_became_active":"2024-09-13T16:28:39.210296+0000","last_became_peered":"2024-09-13T16:28:39.210296+0000","last_unstale":"2024-09-13T16:29:04.628010+0000","last_undegraded":"2024-09-13T16:29:04.628010+0000","last_fullsized":"2024-09-13T16:29:04.628010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:06.385733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629594+0000","last_change":"2024-09-13T16:28:39.207609+0000","last_active":"2024-09-13T16:29:04.629594+0000","last_peered":"2024-09-13T16:29:04.629594+0000","last_clean":"2024-09-13T16:29:04.629594+0000","last_became_active":"2024-09-13T16:28:39.206893+0000","last_became_peered":"2024-09-13T16:28:39.206893+0000","last_unstale":"2024-09-13T16:29:04.629594+0000","last_undegraded":"2024-09-13T16:29:04.629594+0000","last_fullsized":"2024-09-13T16:29:04.629594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:51.582666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629913+0000","last_change":"2024-09-13T16:28:39.201856+0000","last_active":"2024-09-13T16:29:04.629913+0000","last_peered":"2024-09-13T16:29:04.629913+0000","last_clean":"2024-09-13T16:29:04.629913+0000","last_became_active":"2024-09-13T16:28:39.201511+0000","last_became_peered":"2024-09-13T16:28:39.201511+0000","last_unstale":"2024-09-13T16:29:04.629913+0000","last_undegraded":"2024-09-13T16:29:04.629913+0000","last_fullsized":"2024-09-13T16:29:04.629913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:05:19.722354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.630897+0000","last_change":"2024-09-13T16:28:39.216106+0000","last_active":"2024-09-13T16:29:04.630897+0000","last_peered":"2024-09-13T16:29:04.630897+0000","last_clean":"2024-09-13T16:29:04.630897+0000","last_became_active":"2024-09-13T16:28:39.215432+0000","last_became_peered":"2024-09-13T16:28:39.215432+0000","last_unstale":"2024-09-13T16:29:04.630897+0000","last_undegraded":"2024-09-13T16:29:04.630897+0000","last_fullsized":"2024-09-13T16:29:04.630897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:21:18.439891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.631100+0000","last_change":"2024-09-13T16:28:39.210578+0000","last_active":"2024-09-13T16:29:04.631100+0000","last_peered":"2024-09-13T16:29:04.631100+0000","last_clean":"2024-09-13T16:29:04.631100+0000","last_became_active":"2024-09-13T16:28:39.210269+0000","last_became_peered":"2024-09-13T16:28:39.210269+0000","last_unstale":"2024-09-13T16:29:04.631100+0000","last_undegraded":"2024-09-13T16:29:04.631100+0000","last_fullsized":"2024-09-13T16:29:04.631100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:27:27.078880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.622497+0000","last_change":"2024-09-13T16:28:39.213409+0000","last_active":"2024-09-13T16:29:04.622497+0000","last_peered":"2024-09-13T16:29:04.622497+0000","last_clean":"2024-09-13T16:29:04.622497+0000","last_became_active":"2024-09-13T16:28:39.213077+0000","last_became_peered":"2024-09-13T16:28:39.213077+0000","last_unstale":"2024-09-13T16:29:04.622497+0000","last_undegraded":"2024-09-13T16:29:04.622497+0000","last_fullsized":"2024-09-13T16:29:04.622497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:32:24.747516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.632419+0000","last_change":"2024-09-13T16:28:39.215216+0000","last_active":"2024-09-13T16:29:04.632419+0000","last_peered":"2024-09-13T16:29:04.632419+0000","last_clean":"2024-09-13T16:29:04.632419+0000","last_became_active":"2024-09-13T16:28:39.214920+0000","last_became_peered":"2024-09-13T16:28:39.214920+0000","last_unstale":"2024-09-13T16:29:04.632419+0000","last_undegraded":"2024-09-13T16:29:04.632419+0000","last_fullsized":"2024-09-13T16:29:04.632419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:25:38.132833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625826+0000","last_change":"2024-09-13T16:28:39.202506+0000","last_active":"2024-09-13T16:29:04.625826+0000","last_peered":"2024-09-13T16:29:04.625826+0000","last_clean":"2024-09-13T16:29:04.625826+0000","last_became_active":"2024-09-13T16:28:39.202224+0000","last_became_peered":"2024-09-13T16:28:39.202224+0000","last_unstale":"2024-09-13T16:29:04.625826+0000","last_undegraded":"2024-09-13T16:29:04.625826+0000","last_fullsized":"2024-09-13T16:29:04.625826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:00:37.208442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.640477+0000","last_change":"2024-09-13T16:29:05.640477+0000","last_active":"2024-09-13T16:29:05.640477+0000","last_peered":"2024-09-13T16:29:05.640477+0000","last_clean":"2024-09-13T16:29:05.640477+0000","last_became_active":"2024-09-13T16:29:05.640014+0000","last_became_peered":"2024-09-13T16:29:05.640014+0000","last_unstale":"2024-09-13T16:29:05.640477+0000","last_undegraded":"2024-09-13T16:29:05.640477+0000","last_fullsized":"2024-09-13T16:29:05.640477+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:47:45.202113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.631206+0000","last_change":"2024-09-13T16:28:39.200217+0000","last_active":"2024-09-13T16:29:04.631206+0000","last_peered":"2024-09-13T16:29:04.631206+0000","last_clean":"2024-09-13T16:29:04.631206+0000","last_became_active":"2024-09-13T16:28:39.199946+0000","last_became_peered":"2024-09-13T16:28:39.199946+0000","last_unstale":"2024-09-13T16:29:04.631206+0000","last_undegraded":"2024-09-13T16:29:04.631206+0000","last_fullsized":"2024-09-13T16:29:04.631206+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:20:02.906225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.630140+0000","last_change":"2024-09-13T16:28:39.215013+0000","last_active":"2024-09-13T16:29:04.630140+0000","last_peered":"2024-09-13T16:29:04.630140+0000","last_clean":"2024-09-13T16:29:04.630140+0000","last_became_active":"2024-09-13T16:28:39.214731+0000","last_became_peered":"2024-09-13T16:28:39.214731+0000","last_unstale":"2024-09-13T16:29:04.630140+0000","last_undegraded":"2024-09-13T16:29:04.630140+0000","last_fullsized":"2024-09-13T16:29:04.630140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:35:01.836001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.625268+0000","last_change":"2024-09-13T16:28:39.200300+0000","last_active":"2024-09-13T16:29:04.625268+0000","last_peered":"2024-09-13T16:29:04.625268+0000","last_clean":"2024-09-13T16:29:04.625268+0000","last_became_active":"2024-09-13T16:28:39.200006+0000","last_became_peered":"2024-09-13T16:28:39.200006+0000","last_unstale":"2024-09-13T16:29:04.625268+0000","last_undegraded":"2024-09-13T16:29:04.625268+0000","last_fullsized":"2024-09-13T16:29:04.625268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:35:14.783905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629430+0000","last_change":"2024-09-13T16:28:39.213552+0000","last_active":"2024-09-13T16:29:04.629430+0000","last_peered":"2024-09-13T16:29:04.629430+0000","last_clean":"2024-09-13T16:29:04.629430+0000","last_became_active":"2024-09-13T16:28:39.213287+0000","last_became_peered":"2024-09-13T16:28:39.213287+0000","last_unstale":"2024-09-13T16:29:04.629430+0000","last_undegraded":"2024-09-13T16:29:04.629430+0000","last_fullsized":"2024-09-13T16:29:04.629430+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:30.588770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.627025+0000","last_change":"2024-09-13T16:28:39.211126+0000","last_active":"2024-09-13T16:29:04.627025+0000","last_peered":"2024-09-13T16:29:04.627025+0000","last_clean":"2024-09-13T16:29:04.627025+0000","last_became_active":"2024-09-13T16:28:39.210810+0000","last_became_peered":"2024-09-13T16:28:39.210810+0000","last_unstale":"2024-09-13T16:29:04.627025+0000","last_undegraded":"2024-09-13T16:29:04.627025+0000","last_fullsized":"2024-09-13T16:29:04.627025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:52:45.249309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T16:29:05.639070+0000","last_change":"2024-09-13T16:29:05.636535+0000","last_active":"2024-09-13T16:29:05.639070+0000","last_peered":"2024-09-13T16:29:05.639070+0000","last_clean":"2024-09-13T16:29:05.639070+0000","last_became_active":"2024-09-13T16:29:05.636161+0000","last_became_peered":"2024-09-13T16:29:05.636161+0000","last_unstale":"2024-09-13T16:29:05.639070+0000","last_undegraded":"2024-09-13T16:29:05.639070+0000","last_fullsized":"2024-09-13T16:29:05.639070+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:09.606922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.631719+0000","last_change":"2024-09-13T16:28:39.202222+0000","last_active":"2024-09-13T16:29:04.631719+0000","last_peered":"2024-09-13T16:29:04.631719+0000","last_clean":"2024-09-13T16:29:04.631719+0000","last_became_active":"2024-09-13T16:28:39.201897+0000","last_became_peered":"2024-09-13T16:28:39.201897+0000","last_unstale":"2024-09-13T16:29:04.631719+0000","last_undegraded":"2024-09-13T16:29:04.631719+0000","last_fullsized":"2024-09-13T16:29:04.631719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:41:09.087185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.631804+0000","last_change":"2024-09-13T16:28:39.211164+0000","last_active":"2024-09-13T16:29:04.631804+0000","last_peered":"2024-09-13T16:29:04.631804+0000","last_clean":"2024-09-13T16:29:04.631804+0000","last_became_active":"2024-09-13T16:28:39.210826+0000","last_became_peered":"2024-09-13T16:28:39.210826+0000","last_unstale":"2024-09-13T16:29:04.631804+0000","last_undegraded":"2024-09-13T16:29:04.631804+0000","last_fullsized":"2024-09-13T16:29:04.631804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:40:30.663867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T16:29:04.629508+0000","last_change":"2024-09-13T16:28:39.207991+0000","last_active":"2024-09-13T16:29:04.629508+0000","last_peered":"2024-09-13T16:29:04.629508+0000","last_clean":"2024-09-13T16:29:04.629508+0000","last_became_active":"2024-09-13T16:28:39.207332+0000","last_became_peered":"2024-09-13T16:28:39.207332+0000","last_unstale":"2024-09-13T16:29:04.629508+0000","last_undegraded":"2024-09-13T16:29:04.629508+0000","last_fullsized":"2024-09-13T16:29:04.629508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:30:54.271361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":53031,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":101,"num_read_kb":117,"num_write":225,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":762,"num_keys_recovered":10,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":495616,"data_stored":136563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":198,"ondisk_log_size":198,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250749,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383697,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":501231,"data_compressed":17719,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.54100000000000004}]},{"osd":1,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.51500000000000001}]},{"osd":2,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.55100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":4,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.753},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.65300000000000002}]},{"osd":5,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.61299999999999999}]},{"osd":6,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.627},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.58199999999999996}]},{"osd":7,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.56699999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":9,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.84899999999999998}]},{"osd":10,"last update":"Fri Sep 13 16:28:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.70199999999999996}]}]},{"osd":10,"up_from":72,"seq":309237645335,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1130496,"data_stored":1133083,"data_compressed":54008,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":1,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.315}]},{"osd":2,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.67900000000000005}]},{"osd":3,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":7.2320000000000002,"5min":7.2320000000000002,"15min":7.2320000000000002},"last":0.495}]},{"osd":4,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.437}]},{"osd":5,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.71799999999999997}]},{"osd":6,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.66600000000000004}]},{"osd":7,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.53000000000000003}]},{"osd":8,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":7.2569999999999997,"5min":7.2569999999999997,"15min":7.2569999999999997},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.59599999999999997}]},{"osd":9,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.74099999999999999}]},{"osd":11,"last update":"Fri Sep 13 16:28:58 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.70299999999999996},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":7.2199999999999998,"5min":7.2199999999999998,"15min":7.2199999999999998},"last":0.65100000000000002}]}]},{"osd":9,"up_from":67,"seq":287762808859,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":504655,"data_compressed":17719,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:29:09 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.41899999999999998,"15min":0.42699999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.52600000000000002,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.47599999999999998,"15min":0.48799999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.38300000000000001}]},{"osd":1,"last update":"Fri Sep 13 16:29:09 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58799999999999997,"15min":0.58899999999999997},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88300000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.442},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.59299999999999997,"15min":0.59999999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76400000000000001,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.5}]},{"osd":2,"last update":"Fri Sep 13 16:29:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56999999999999995,"15min":0.57099999999999995},"min":{"1min":0.40899999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.42999999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.53200000000000003}]},{"osd":3,"last update":"Fri Sep 13 16:29:09 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.47699999999999998,"15min":0.48499999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.63200000000000001,"5min":0.754,"15min":0.754},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.436,"5min":0.53100000000000003,"15min":0.54700000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67100000000000004,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.36899999999999999}]},{"osd":4,"last update":"Fri Sep 13 16:29:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.623,"15min":0.628},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.751,"5min":0.997,"15min":0.997},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.628,"5min":0.65300000000000002,"15min":0.65700000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.81299999999999994,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.78500000000000003}]},{"osd":5,"last update":"Fri Sep 13 16:29:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.61199999999999999,"15min":0.61299999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86699999999999999,"5min":0.877,"15min":0.877},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.86299999999999999,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.55900000000000005}]},{"osd":6,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.45700000000000002}]},{"osd":7,"last update":"Fri Sep 13 16:29:09 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.73799999999999999,"15min":0.746},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.94699999999999995,"5min":1.107,"15min":1.107},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.74299999999999999,"15min":0.75700000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.86199999999999999,"5min":1.139,"15min":1.139},"last":0.59299999999999997}]},{"osd":8,"last update":"Fri Sep 13 16:29:09 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.72699999999999998,"15min":0.73799999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.88700000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.73999999999999999,"15min":0.746},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.995,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.81599999999999995}]},{"osd":10,"last update":"Fri Sep 13 16:28:33 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.755},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.627}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]}]},{"osd":8,"up_from":59,"seq":253403070496,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":1011712,"data_stored":499683,"data_compressed":17719,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.50600000000000001,"15min":0.50800000000000001},"min":{"1min":0.28100000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.70999999999999996,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51700000000000002,"15min":0.51200000000000001},"min":{"1min":0.32800000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.41299999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.40400000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82399999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.68200000000000005}]},{"osd":2,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.41699999999999998,"15min":0.41199999999999998},"min":{"1min":0.28299999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.497}]},{"osd":3,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.78700000000000003,"5min":0.871,"15min":0.871},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.51000000000000001}]},{"osd":4,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56799999999999995,"15min":0.56399999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.70999999999999996}]},{"osd":5,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53800000000000003,"15min":0.54300000000000004},"min":{"1min":0.29699999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.53600000000000003}]},{"osd":6,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.628,"15min":0.63},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.65900000000000003,"15min":0.66300000000000003},"min":{"1min":0.38300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.56899999999999995}]},{"osd":7,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.66600000000000004,"15min":0.67000000000000004},"min":{"1min":0.47899999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.97699999999999998,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.627},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.65800000000000003,"15min":0.66200000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.92200000000000004,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.69499999999999995}]},{"osd":9,"last update":"Fri Sep 13 16:28:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.76200000000000001}]},{"osd":10,"last update":"Fri Sep 13 16:28:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.82699999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]}]},{"osd":7,"up_from":51,"seq":219043332132,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":1093979,"data_compressed":54008,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.39600000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.41199999999999998}]},{"osd":1,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.39200000000000002,"15min":0.38900000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.312},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.68600000000000005,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.68999999999999995}]},{"osd":2,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.54400000000000004,"15min":0.54900000000000004},"min":{"1min":0.40699999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.66700000000000004,"5min":0.748,"15min":0.748},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54700000000000004,"15min":0.54800000000000004},"min":{"1min":0.40000000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.60299999999999998}]},{"osd":3,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.42499999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.441},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56799999999999995,"15min":0.56399999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.57499999999999996}]},{"osd":4,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45200000000000001,"15min":0.45100000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.45800000000000002,"15min":0.46300000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.32700000000000001}]},{"osd":5,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57699999999999996,"15min":0.57799999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.496},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.44600000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.54200000000000004}]},{"osd":6,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60899999999999999,"15min":0.60999999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.41699999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.373}]},{"osd":8,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.627,"15min":0.627},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.624},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.66100000000000003,"15min":0.66500000000000004},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.65800000000000003}]},{"osd":9,"last update":"Fri Sep 13 16:28:08 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.64200000000000002}]},{"osd":10,"last update":"Fri Sep 13 16:28:33 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.67600000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]}]},{"osd":6,"up_from":46,"seq":197568495658,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":1003520,"data_stored":496407,"data_compressed":17719,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.38300000000000001,"15min":0.372},"min":{"1min":0.26900000000000002,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.39700000000000002,"15min":0.39000000000000001},"min":{"1min":0.28000000000000003,"5min":0.217,"15min":0.217},"max":{"1min":0.61599999999999999,"5min":0.625,"15min":0.625},"last":0.36599999999999999}]},{"osd":1,"last update":"Fri Sep 13 16:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53400000000000003,"15min":0.51200000000000001},"min":{"1min":0.373,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.753,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56100000000000005,"15min":0.54500000000000004},"min":{"1min":0.39700000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.77700000000000002,"5min":0.873,"15min":0.873},"last":0.44}]},{"osd":2,"last update":"Fri Sep 13 16:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55300000000000005,"15min":0.54900000000000004},"min":{"1min":0.35199999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83499999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59699999999999998,"15min":0.58799999999999997},"min":{"1min":0.36899999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81399999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.51000000000000001}]},{"osd":3,"last update":"Fri Sep 13 16:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.73699999999999999,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.499,"15min":0.48899999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.58599999999999997}]},{"osd":4,"last update":"Fri Sep 13 16:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.30299999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.73599999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.42299999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.92300000000000004,"5min":1.079,"15min":1.079},"last":0.55800000000000005}]},{"osd":5,"last update":"Fri Sep 13 16:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61799999999999999,"15min":0.60599999999999998},"min":{"1min":0.47199999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96599999999999997,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.45800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94999999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.45400000000000001}]},{"osd":7,"last update":"Fri Sep 13 16:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63800000000000001,"15min":0.63200000000000001},"min":{"1min":0.40600000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.02,"5min":1.048,"15min":1.048},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.66300000000000003,"15min":0.65200000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.60199999999999998}]},{"osd":8,"last update":"Fri Sep 13 16:28:43 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.69299999999999995,"15min":0.68899999999999995},"min":{"1min":0.505,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72699999999999998,"15min":0.72799999999999998},"min":{"1min":0.46600000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.66100000000000003}]},{"osd":9,"last update":"Fri Sep 13 16:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.64200000000000002}]},{"osd":10,"last update":"Fri Sep 13 16:28:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.622},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.69099999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]}]},{"osd":5,"up_from":38,"seq":163208757295,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1081344,"data_stored":1092638,"data_compressed":54008,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.52200000000000002,"15min":0.51600000000000001},"min":{"1min":0.377,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76700000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54000000000000004,"15min":0.53500000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.80700000000000005,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.56799999999999995}]},{"osd":1,"last update":"Fri Sep 13 16:28:40 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.48699999999999999,"15min":0.47699999999999998},"min":{"1min":0.308,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.51900000000000002,"15min":0.505},"min":{"1min":0.34399999999999997,"5min":0.245,"15min":0.245},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.48299999999999998}]},{"osd":2,"last update":"Fri Sep 13 16:28:40 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.375,"15min":0.35999999999999999},"min":{"1min":0.254,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.65000000000000002,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.379,"15min":0.36699999999999999},"min":{"1min":0.23100000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.42399999999999999}]},{"osd":3,"last update":"Fri Sep 13 16:28:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.80000000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.63300000000000001}]},{"osd":4,"last update":"Fri Sep 13 16:28:40 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52400000000000002,"15min":0.51100000000000001},"min":{"1min":0.41799999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53400000000000003,"15min":0.52800000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.376}]},{"osd":6,"last update":"Fri Sep 13 16:29:03 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.45800000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.434,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.78900000000000003,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.58399999999999996}]},{"osd":7,"last update":"Fri Sep 13 16:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.60799999999999998,"15min":0.60499999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.502},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59999999999999998,"15min":0.59699999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.39800000000000002}]},{"osd":8,"last update":"Fri Sep 13 16:28:46 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52800000000000002,"15min":0.53300000000000003},"min":{"1min":0.34499999999999997,"5min":0.318,"15min":0.318},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.56200000000000006,"15min":0.56899999999999995},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.46500000000000002}]},{"osd":9,"last update":"Fri Sep 13 16:29:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65400000000000003,"15min":0.64800000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65900000000000003,"15min":0.65500000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.65900000000000003}]},{"osd":10,"last update":"Fri Sep 13 16:28:31 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.51900000000000002}]}]},{"osd":4,"up_from":33,"seq":141733920819,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":496497,"data_compressed":17719,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.48699999999999999,"15min":0.46300000000000002},"min":{"1min":0.32200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.751,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.504,"15min":0.48199999999999998},"min":{"1min":0.41399999999999998,"5min":0.25,"15min":0.25},"max":{"1min":0.80800000000000005,"5min":0.874,"15min":0.874},"last":0.60499999999999998}]},{"osd":1,"last update":"Fri Sep 13 16:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.34699999999999998,"15min":0.33100000000000002},"min":{"1min":0.24099999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.34300000000000003,"15min":0.33200000000000002},"min":{"1min":0.27800000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.372}]},{"osd":2,"last update":"Fri Sep 13 16:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.495,"15min":0.48599999999999999},"min":{"1min":0.37,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71599999999999997,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.47399999999999998,"15min":0.45500000000000002},"min":{"1min":0.34399999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67200000000000004,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.59299999999999997}]},{"osd":3,"last update":"Fri Sep 13 16:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.504,"15min":0.48599999999999999},"min":{"1min":0.38700000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.505},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.52100000000000002,"15min":0.496},"min":{"1min":0.39900000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.79300000000000004,"5min":0.875,"15min":0.875},"last":0.56799999999999995}]},{"osd":5,"last update":"Fri Sep 13 16:28:38 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.33800000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54500000000000004,"15min":0.53100000000000003},"min":{"1min":0.39000000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80700000000000005,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Sep 13 16:28:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.54900000000000004,"15min":0.53500000000000003},"min":{"1min":0.47599999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.73299999999999998,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57699999999999996,"15min":0.56599999999999995},"min":{"1min":0.41699999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80400000000000005,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.63}]},{"osd":7,"last update":"Fri Sep 13 16:28:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.495,"15min":0.48099999999999998},"min":{"1min":0.33600000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.85099999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.49399999999999999,"15min":0.48999999999999999},"min":{"1min":0.34999999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.81799999999999995,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.621}]},{"osd":8,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.44,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86899999999999999,"5min":0.872,"15min":0.872},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.61499999999999999,"15min":0.60999999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.872,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.64500000000000002}]},{"osd":9,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.72599999999999998}]},{"osd":10,"last update":"Fri Sep 13 16:28:32 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.65400000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":1011712,"data_stored":495440,"data_compressed":17719,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:56 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.34899999999999998,"15min":0.32500000000000001},"min":{"1min":0.252,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.63,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.35699999999999998,"15min":0.32900000000000001},"min":{"1min":0.27600000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52300000000000002,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.38500000000000001}]},{"osd":1,"last update":"Fri Sep 13 16:28:56 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.54600000000000004,"15min":0.48199999999999998},"min":{"1min":0.48899999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.84299999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53500000000000003,"15min":0.48699999999999999},"min":{"1min":0.42399999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.77000000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.505}]},{"osd":2,"last update":"Fri Sep 13 16:28:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52200000000000002,"15min":0.51400000000000001},"min":{"1min":0.375,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79100000000000004,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55300000000000005,"15min":0.52600000000000002},"min":{"1min":0.38600000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.94599999999999995,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Sep 13 16:28:09 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.51500000000000001,"15min":0.49099999999999999},"min":{"1min":0.40400000000000003,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.56699999999999995,"15min":0.54000000000000004},"min":{"1min":0.44700000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.60099999999999998}]},{"osd":5,"last update":"Fri Sep 13 16:28:36 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58299999999999996,"15min":0.57199999999999995},"min":{"1min":0.433,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.86299999999999999,"5min":1.038,"15min":1.038},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59699999999999998,"15min":0.59599999999999997},"min":{"1min":0.45900000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.73799999999999999,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.71299999999999997}]},{"osd":6,"last update":"Fri Sep 13 16:29:01 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.505,"15min":0.496},"min":{"1min":0.33600000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.48599999999999999,"15min":0.47999999999999998},"min":{"1min":0.34599999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74099999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.57399999999999995}]},{"osd":7,"last update":"Fri Sep 13 16:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.49399999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.60699999999999998,"15min":0.59299999999999997},"min":{"1min":0.52100000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.67500000000000004}]},{"osd":8,"last update":"Fri Sep 13 16:27:45 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.71499999999999997}]},{"osd":9,"last update":"Fri Sep 13 16:28:08 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.62}]},{"osd":10,"last update":"Fri Sep 13 16:28:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.74199999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":1032192,"data_stored":520251,"data_compressed":17719,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.60399999999999998,"15min":0.498},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.90900000000000003,"5min":16.027999999999999,"15min":16.027999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.47299999999999998,"15min":0.46500000000000002},"min":{"1min":0.309,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.69599999999999995}]},{"osd":1,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.48399999999999999,"15min":0.46500000000000002},"min":{"1min":0.372,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.51500000000000001,"15min":0.49299999999999999},"min":{"1min":0.44500000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.83399999999999996}]},{"osd":3,"last update":"Fri Sep 13 16:28:52 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.54000000000000004,"15min":0.51400000000000001},"min":{"1min":0.43099999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54000000000000004,"15min":0.51500000000000001},"min":{"1min":0.36799999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.59099999999999997}]},{"osd":4,"last update":"Fri Sep 13 16:28:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":0.623,"15min":0.55500000000000005},"min":{"1min":0.36399999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":9.6790000000000003,"5min":9.6790000000000003,"15min":9.6790000000000003},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.54500000000000004,"15min":0.53600000000000003},"min":{"1min":0.34399999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.75900000000000001}]},{"osd":5,"last update":"Fri Sep 13 16:28:39 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.40999999999999998,"15min":0.40200000000000002},"min":{"1min":0.33700000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.432},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.40300000000000002,"15min":0.39200000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.46100000000000002}]},{"osd":6,"last update":"Fri Sep 13 16:29:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.57699999999999996,"15min":0.55200000000000005},"min":{"1min":0.437,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.60599999999999998,"15min":0.58399999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.79400000000000004}]},{"osd":7,"last update":"Fri Sep 13 16:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.379,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59099999999999997,"15min":0.58299999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.748}]},{"osd":8,"last update":"Fri Sep 13 16:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.52600000000000002,"15min":0.51800000000000002},"min":{"1min":0.33500000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.501},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.51700000000000002,"15min":0.50800000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.55100000000000005}]},{"osd":9,"last update":"Fri Sep 13 16:29:06 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.67900000000000005,"15min":0.66900000000000004},"min":{"1min":0.45100000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.016,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.68100000000000005,"15min":0.67100000000000004},"min":{"1min":0.50600000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.85299999999999998}]},{"osd":10,"last update":"Fri Sep 13 16:28:34 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.89400000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":543734,"data_compressed":17719,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:09 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.47999999999999998,"15min":0.47799999999999998},"min":{"1min":0.34100000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.497},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.48399999999999999,"15min":0.48099999999999998},"min":{"1min":0.38100000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.435}]},{"osd":2,"last update":"Fri Sep 13 16:28:27 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.67100000000000004,"15min":0.54100000000000004},"min":{"1min":0.34100000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.81200000000000006,"5min":17.655000000000001,"15min":17.655000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.49099999999999999,"15min":0.47899999999999998},"min":{"1min":0.29299999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.76200000000000001}]},{"osd":3,"last update":"Fri Sep 13 16:28:51 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52500000000000002,"15min":0.50700000000000001},"min":{"1min":0.379,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72599999999999998,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.55800000000000005,"15min":0.52200000000000002},"min":{"1min":0.42199999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.73899999999999999}]},{"osd":4,"last update":"Fri Sep 13 16:28:13 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.40400000000000003,"15min":0.38},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.433,"5min":0.40899999999999997,"15min":0.39100000000000001},"min":{"1min":0.28299999999999997,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.57899999999999996}]},{"osd":5,"last update":"Fri Sep 13 16:28:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":1.0609999999999999,"15min":1.2829999999999999},"min":{"1min":0.38400000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.82499999999999996,"5min":17.640000000000001,"15min":17.640000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54600000000000004,"15min":0.54300000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.72499999999999998}]},{"osd":6,"last update":"Fri Sep 13 16:29:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59299999999999997,"15min":0.59399999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.72799999999999998,"5min":0.872,"15min":0.872},"last":0.60899999999999999}]},{"osd":7,"last update":"Fri Sep 13 16:28:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.372,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.78900000000000003}]},{"osd":8,"last update":"Fri Sep 13 16:28:43 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.60399999999999998,"15min":0.61099999999999999},"min":{"1min":0.40699999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.45500000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.83499999999999996}]},{"osd":9,"last update":"Fri Sep 13 16:28:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.751}]},{"osd":10,"last update":"Fri Sep 13 16:28:34 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.82199999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":497471,"data_compressed":17719,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 16:28:15 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.46999999999999997,"15min":0.46899999999999997},"min":{"1min":0.37,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.755,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.50700000000000001,"15min":0.495},"min":{"1min":0.41199999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81000000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri Sep 13 16:28:33 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.55000000000000004,"15min":0.52600000000000002},"min":{"1min":0.42699999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73799999999999999,"5min":1.111,"15min":1.111},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.54800000000000004,"15min":0.56100000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.621,"5min":1.698,"15min":1.698},"last":0.73499999999999999}]},{"osd":3,"last update":"Fri Sep 13 16:27:47 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.438,"15min":0.42399999999999999},"min":{"1min":0.38200000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.42299999999999999,"15min":0.39800000000000002},"min":{"1min":0.22500000000000001,"5min":0.217,"15min":0.217},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.65400000000000003}]},{"osd":4,"last update":"Fri Sep 13 16:28:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.48199999999999998,"15min":0.45400000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.51500000000000001,"15min":0.48399999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.77500000000000002}]},{"osd":5,"last update":"Fri Sep 13 16:28:33 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.55300000000000005,"15min":0.54100000000000004},"min":{"1min":0.44,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56799999999999995,"15min":0.55600000000000005},"min":{"1min":0.46899999999999997,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77800000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.80300000000000005}]},{"osd":6,"last update":"Fri Sep 13 16:28:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.504,"15min":0.48199999999999998},"min":{"1min":0.29899999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.66500000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55300000000000005,"15min":0.52100000000000002},"min":{"1min":0.34399999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.74299999999999999,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.41699999999999998}]},{"osd":7,"last update":"Fri Sep 13 16:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.68300000000000005,"15min":0.67500000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96999999999999997,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.63},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.63700000000000001,"15min":0.63100000000000001},"min":{"1min":0.43099999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.67800000000000005}]},{"osd":8,"last update":"Fri Sep 13 16:28:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1,"5min":1,"15min":1},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69899999999999995,"15min":0.70299999999999996},"min":{"1min":0.55000000000000004,"5min":0.497,"15min":0.497},"max":{"1min":0.81799999999999995,"5min":1.004,"15min":1.004},"last":0.86299999999999999}]},{"osd":9,"last update":"Fri Sep 13 16:28:10 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.71799999999999997}]},{"osd":10,"last update":"Fri Sep 13 16:28:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.83299999999999996}]},{"osd":11,"last update":"Fri Sep 13 16:28:58 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.92700000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":48393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":25183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":47705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T16:29:13.751 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-09-13T16:29:13.752 INFO:tasks.cephadm:Setting up client nodes... 2024-09-13T16:29:13.752 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-13T16:29:13.990 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:29:14.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:14 smithi111 ceph-mon[34600]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T16:29:14.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:14 smithi111 ceph-mon[34600]: pgmap v296: 129 pgs: 129 active+clean; 629 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 KiB/s wr, 28 op/s; 76 B/s, 1 keys/s, 1 objects/s recovering 2024-09-13T16:29:15.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:14 smithi107 ceph-mon[34139]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T16:29:15.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:14 smithi107 ceph-mon[34139]: pgmap v296: 129 pgs: 129 active+clean; 629 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 KiB/s wr, 28 op/s; 76 B/s, 1 keys/s, 1 objects/s recovering 2024-09-13T16:29:15.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:14 smithi016 ceph-mon[31187]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T16:29:15.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:14 smithi016 ceph-mon[31187]: pgmap v296: 129 pgs: 129 active+clean; 629 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 KiB/s wr, 28 op/s; 76 B/s, 1 keys/s, 1 objects/s recovering 2024-09-13T16:29:15.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.738+0000 7f741356f640 1 -- 172.21.15.16:0/1738635549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f740c0744e0 msgr2=0x7f740c072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:15.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.738+0000 7f741356f640 1 --2- 172.21.15.16:0/1738635549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f740c0744e0 0x7f740c072940 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7408009900 tx=0x7f740802a410 comp rx=0 tx=0).stop 2024-09-13T16:29:15.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.739+0000 7f741356f640 1 -- 172.21.15.16:0/1738635549 shutdown_connections 2024-09-13T16:29:15.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.739+0000 7f741356f640 1 --2- 172.21.15.16:0/1738635549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f740c107c80 0x7f740c10a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:15.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.739+0000 7f741356f640 1 --2- 172.21.15.16:0/1738635549 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f740c072f10 0x7f740c073370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:15.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.739+0000 7f741356f640 1 --2- 172.21.15.16:0/1738635549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f740c0744e0 0x7f740c072940 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:15.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.739+0000 7f741356f640 1 -- 172.21.15.16:0/1738635549 >> 172.21.15.16:0/1738635549 conn(0x7f740c0fb5d0 msgr2=0x7f740c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:15.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.739+0000 7f741356f640 1 -- 172.21.15.16:0/1738635549 shutdown_connections 2024-09-13T16:29:15.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.739+0000 7f741356f640 1 -- 172.21.15.16:0/1738635549 wait complete. 2024-09-13T16:29:15.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.739+0000 7f741356f640 1 Processor -- start 2024-09-13T16:29:15.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.739+0000 7f741356f640 1 -- start start 2024-09-13T16:29:15.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.739+0000 7f741356f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f740c072f10 0x7f740c197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:15.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.739+0000 7f741356f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f740c0744e0 0x7f740c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:15.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.740+0000 7f741356f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f740c107c80 0x7f740c19f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:15.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.740+0000 7f741356f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c19f9c0 con 0x7f740c072f10 2024-09-13T16:29:15.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.740+0000 7f741356f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c19fb30 con 0x7f740c0744e0 2024-09-13T16:29:15.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.740+0000 7f741356f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c19fcf0 con 0x7f740c107c80 2024-09-13T16:29:15.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.740+0000 7f741130b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f740c072f10 0x7f740c197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:15.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.740+0000 7f741130b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f740c072f10 0x7f740c197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49216/0 (socket says 172.21.15.16:49216) 2024-09-13T16:29:15.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.740+0000 7f741130b640 1 -- 172.21.15.16:0/3968128122 learned_addr learned my addr 172.21.15.16:0/3968128122 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:15.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.740+0000 7f7410b0a640 1 --2- 172.21.15.16:0/3968128122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f740c0744e0 0x7f740c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:15.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.740+0000 7f7411b0c640 1 --2- 172.21.15.16:0/3968128122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f740c107c80 0x7f740c19f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:15.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.740+0000 7f741130b640 1 -- 172.21.15.16:0/3968128122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f740c107c80 msgr2=0x7f740c19f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:15.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.740+0000 7f741130b640 1 --2- 172.21.15.16:0/3968128122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f740c107c80 0x7f740c19f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:15.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.740+0000 7f741130b640 1 -- 172.21.15.16:0/3968128122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f740c0744e0 msgr2=0x7f740c198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:15.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.740+0000 7f741130b640 1 --2- 172.21.15.16:0/3968128122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f740c0744e0 0x7f740c198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:15.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.740+0000 7f741130b640 1 -- 172.21.15.16:0/3968128122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7408009560 con 0x7f740c072f10 2024-09-13T16:29:15.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.740+0000 7f7410b0a640 1 --2- 172.21.15.16:0/3968128122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f740c0744e0 0x7f740c198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:29:15.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.740+0000 7f7411b0c640 1 --2- 172.21.15.16:0/3968128122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f740c107c80 0x7f740c19f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:29:15.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.740+0000 7f741130b640 1 --2- 172.21.15.16:0/3968128122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f740c072f10 0x7f740c197f10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7408005ab0 tx=0x7f74080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:15.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.740+0000 7f74027fc640 1 -- 172.21.15.16:0/3968128122 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7408031d70 con 0x7f740c072f10 2024-09-13T16:29:15.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.740+0000 7f74027fc640 1 -- 172.21.15.16:0/3968128122 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74080044e0 con 0x7f740c072f10 2024-09-13T16:29:15.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.740+0000 7f741356f640 1 -- 172.21.15.16:0/3968128122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f740c19ff70 con 0x7f740c072f10 2024-09-13T16:29:15.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.740+0000 7f74027fc640 1 -- 172.21.15.16:0/3968128122 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7408003b00 con 0x7f740c072f10 2024-09-13T16:29:15.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.741+0000 7f741356f640 1 -- 172.21.15.16:0/3968128122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f740c1a0320 con 0x7f740c072f10 2024-09-13T16:29:15.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.741+0000 7f741356f640 1 -- 172.21.15.16:0/3968128122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f740c106a00 con 0x7f740c072f10 2024-09-13T16:29:15.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.743+0000 7f74027fc640 1 -- 172.21.15.16:0/3968128122 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f7408004c50 con 0x7f740c072f10 2024-09-13T16:29:15.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.743+0000 7f74027fc640 1 --2- 172.21.15.16:0/3968128122 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f73f80a1140 0x7f73f80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:15.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.743+0000 7f7410b0a640 1 --2- 172.21.15.16:0/3968128122 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f73f80a1140 0x7f73f80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:15.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.743+0000 7f74027fc640 1 -- 172.21.15.16:0/3968128122 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f74080edea0 con 0x7f740c072f10 2024-09-13T16:29:15.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.744+0000 7f7410b0a640 1 --2- 172.21.15.16:0/3968128122 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f73f80a1140 0x7f73f80a3600 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f73fc0058c0 tx=0x7f73fc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:15.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.748+0000 7f74027fc640 1 -- 172.21.15.16:0/3968128122 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f74080b71c0 con 0x7f740c072f10 2024-09-13T16:29:15.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.983+0000 7f741356f640 1 -- 172.21.15.16:0/3968128122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f740c10d770 con 0x7f740c072f10 2024-09-13T16:29:15.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.987+0000 7f74027fc640 1 -- 172.21.15.16:0/3968128122 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f74080400f0 con 0x7f740c072f10 2024-09-13T16:29:15.988 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-13T16:29:15.988 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBbaORmNUCrOhAAG+b27dg/r8vkORT92rztfw== 2024-09-13T16:29:15.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.989+0000 7f741356f640 1 -- 172.21.15.16:0/3968128122 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f73f80a1140 msgr2=0x7f73f80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:15.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.989+0000 7f741356f640 1 --2- 172.21.15.16:0/3968128122 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f73f80a1140 0x7f73f80a3600 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f73fc0058c0 tx=0x7f73fc005850 comp rx=0 tx=0).stop 2024-09-13T16:29:15.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.989+0000 7f741356f640 1 -- 172.21.15.16:0/3968128122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f740c072f10 msgr2=0x7f740c197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:15.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.989+0000 7f741356f640 1 --2- 172.21.15.16:0/3968128122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f740c072f10 0x7f740c197f10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7408005ab0 tx=0x7f74080042f0 comp rx=0 tx=0).stop 2024-09-13T16:29:15.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.989+0000 7f741356f640 1 -- 172.21.15.16:0/3968128122 shutdown_connections 2024-09-13T16:29:15.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.989+0000 7f741356f640 1 --2- 172.21.15.16:0/3968128122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f740c107c80 0x7f740c19f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:15.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.989+0000 7f741356f640 1 --2- 172.21.15.16:0/3968128122 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f73f80a1140 0x7f73f80a3600 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:15.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.989+0000 7f741356f640 1 --2- 172.21.15.16:0/3968128122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f740c0744e0 0x7f740c198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:15.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.989+0000 7f741356f640 1 --2- 172.21.15.16:0/3968128122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f740c072f10 0x7f740c197f10 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:15.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.989+0000 7f741356f640 1 -- 172.21.15.16:0/3968128122 >> 172.21.15.16:0/3968128122 conn(0x7f740c0fb5d0 msgr2=0x7f740c103bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:15.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.990+0000 7f741356f640 1 -- 172.21.15.16:0/3968128122 shutdown_connections 2024-09-13T16:29:15.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:15.990+0000 7f741356f640 1 -- 172.21.15.16:0/3968128122 wait complete. 2024-09-13T16:29:16.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:16 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3968128122' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-13T16:29:16.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:16 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3968128122' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-13T16:29:16.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:16 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3968128122' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-13T16:29:16.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:16 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3968128122' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-13T16:29:16.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:16 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3968128122' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-13T16:29:16.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:16 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3968128122' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-13T16:29:16.691 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-13T16:29:16.691 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-13T16:29:16.692 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-13T16:29:16.734 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-13T16:29:16.735 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-13T16:29:16.735 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-13T16:29:16.981 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:29:17.345 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:17 smithi016 ceph-mon[31187]: pgmap v297: 129 pgs: 129 active+clean; 629 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 KiB/s wr, 23 op/s; 69 B/s, 0 keys/s, 1 objects/s recovering 2024-09-13T16:29:17.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:17 smithi107 ceph-mon[34139]: pgmap v297: 129 pgs: 129 active+clean; 629 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 KiB/s wr, 23 op/s; 69 B/s, 0 keys/s, 1 objects/s recovering 2024-09-13T16:29:17.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:17 smithi111 ceph-mon[34600]: pgmap v297: 129 pgs: 129 active+clean; 629 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 KiB/s wr, 23 op/s; 69 B/s, 0 keys/s, 1 objects/s recovering 2024-09-13T16:29:18.668 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:18 smithi016 ceph-mon[31187]: pgmap v298: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.2 KiB/s wr, 25 op/s; 57 B/s, 1 objects/s recovering 2024-09-13T16:29:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.667+0000 7fe611c15640 1 -- 172.21.15.16:0/1331555546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe60c10a390 msgr2=0x7fe60c10c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.667+0000 7fe611c15640 1 --2- 172.21.15.16:0/1331555546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe60c10a390 0x7fe60c10c800 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe604009ca0 tx=0x7fe60402a950 comp rx=0 tx=0).stop 2024-09-13T16:29:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.667+0000 7fe611c15640 1 -- 172.21.15.16:0/1331555546 shutdown_connections 2024-09-13T16:29:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.667+0000 7fe611c15640 1 --2- 172.21.15.16:0/1331555546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe60c10a390 0x7fe60c10c800 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.667+0000 7fe611c15640 1 --2- 172.21.15.16:0/1331555546 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe60c1018d0 0x7fe60c109dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.667+0000 7fe611c15640 1 --2- 172.21.15.16:0/1331555546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe60c100fb0 0x7fe60c101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.667+0000 7fe611c15640 1 -- 172.21.15.16:0/1331555546 >> 172.21.15.16:0/1331555546 conn(0x7fe60c0fa9b0 msgr2=0x7fe60c0fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.667+0000 7fe611c15640 1 -- 172.21.15.16:0/1331555546 shutdown_connections 2024-09-13T16:29:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.667+0000 7fe611c15640 1 -- 172.21.15.16:0/1331555546 wait complete. 2024-09-13T16:29:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.668+0000 7fe611c15640 1 Processor -- start 2024-09-13T16:29:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.668+0000 7fe611c15640 1 -- start start 2024-09-13T16:29:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.668+0000 7fe611c15640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe60c100fb0 0x7fe60c196120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.669+0000 7fe611c15640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe60c1018d0 0x7fe60c196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.669+0000 7fe611c15640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe60c10a390 0x7fe60c19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.669+0000 7fe611c15640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60c19b060 con 0x7fe60c100fb0 2024-09-13T16:29:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.669+0000 7fe611c15640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60c19b1d0 con 0x7fe60c10a390 2024-09-13T16:29:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.669+0000 7fe611c15640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60c19b340 con 0x7fe60c1018d0 2024-09-13T16:29:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.669+0000 7fe60b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe60c100fb0 0x7fe60c196120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.669+0000 7fe60b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe60c100fb0 0x7fe60c196120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49228/0 (socket says 172.21.15.16:49228) 2024-09-13T16:29:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.669+0000 7fe60b7fe640 1 -- 172.21.15.16:0/1088063807 learned_addr learned my addr 172.21.15.16:0/1088063807 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.669+0000 7fe60affd640 1 --2- 172.21.15.16:0/1088063807 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe60c1018d0 0x7fe60c196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.669+0000 7fe60bfff640 1 --2- 172.21.15.16:0/1088063807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe60c10a390 0x7fe60c19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.669+0000 7fe60b7fe640 1 -- 172.21.15.16:0/1088063807 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe60c1018d0 msgr2=0x7fe60c196660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.669+0000 7fe60b7fe640 1 --2- 172.21.15.16:0/1088063807 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe60c1018d0 0x7fe60c196660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.669+0000 7fe60b7fe640 1 -- 172.21.15.16:0/1088063807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe60c10a390 msgr2=0x7fe60c19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.669+0000 7fe60b7fe640 1 --2- 172.21.15.16:0/1088063807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe60c10a390 0x7fe60c19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.669+0000 7fe60b7fe640 1 -- 172.21.15.16:0/1088063807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe604009900 con 0x7fe60c100fb0 2024-09-13T16:29:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.669+0000 7fe60affd640 1 --2- 172.21.15.16:0/1088063807 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe60c1018d0 0x7fe60c196660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:29:18.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.669+0000 7fe60b7fe640 1 --2- 172.21.15.16:0/1088063807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe60c100fb0 0x7fe60c196120 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc00e9f0 tx=0x7fe5fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:18.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.669+0000 7fe608ff9640 1 -- 172.21.15.16:0/1088063807 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc009700 con 0x7fe60c100fb0 2024-09-13T16:29:18.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.670+0000 7fe608ff9640 1 -- 172.21.15.16:0/1088063807 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe5fc00cd80 con 0x7fe60c100fb0 2024-09-13T16:29:18.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.670+0000 7fe611c15640 1 -- 172.21.15.16:0/1088063807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe60c19b620 con 0x7fe60c100fb0 2024-09-13T16:29:18.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.670+0000 7fe608ff9640 1 -- 172.21.15.16:0/1088063807 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc0187e0 con 0x7fe60c100fb0 2024-09-13T16:29:18.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.670+0000 7fe611c15640 1 -- 172.21.15.16:0/1088063807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe60c1a2ee0 con 0x7fe60c100fb0 2024-09-13T16:29:18.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.671+0000 7fe611c15640 1 -- 172.21.15.16:0/1088063807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe60c068760 con 0x7fe60c100fb0 2024-09-13T16:29:18.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.672+0000 7fe608ff9640 1 -- 172.21.15.16:0/1088063807 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fe5fc010430 con 0x7fe60c100fb0 2024-09-13T16:29:18.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.673+0000 7fe608ff9640 1 --2- 172.21.15.16:0/1088063807 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe5f40a1110 0x7fe5f40a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:18.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.673+0000 7fe60affd640 1 --2- 172.21.15.16:0/1088063807 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe5f40a1110 0x7fe5f40a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:18.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.673+0000 7fe608ff9640 1 -- 172.21.15.16:0/1088063807 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe5fc014070 con 0x7fe60c100fb0 2024-09-13T16:29:18.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.674+0000 7fe60affd640 1 --2- 172.21.15.16:0/1088063807 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe5f40a1110 0x7fe5f40a35d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe60c1976d0 tx=0x7fe600005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:18.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.677+0000 7fe608ff9640 1 -- 172.21.15.16:0/1088063807 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fe5fc052e20 con 0x7fe60c100fb0 2024-09-13T16:29:18.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.897+0000 7fe611c15640 1 -- 172.21.15.16:0/1088063807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fe60c101390 con 0x7fe60c100fb0 2024-09-13T16:29:18.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.901+0000 7fe608ff9640 1 -- 172.21.15.16:0/1088063807 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260286 (secure 0 0 0) 0x7fe5fc096460 con 0x7fe60c100fb0 2024-09-13T16:29:18.905 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:29:18.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.904+0000 7fe611c15640 1 -- 172.21.15.16:0/1088063807 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe5f40a1110 msgr2=0x7fe5f40a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.904+0000 7fe611c15640 1 --2- 172.21.15.16:0/1088063807 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe5f40a1110 0x7fe5f40a35d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe60c1976d0 tx=0x7fe600005f90 comp rx=0 tx=0).stop 2024-09-13T16:29:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.904+0000 7fe611c15640 1 -- 172.21.15.16:0/1088063807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe60c100fb0 msgr2=0x7fe60c196120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.904+0000 7fe611c15640 1 --2- 172.21.15.16:0/1088063807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe60c100fb0 0x7fe60c196120 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc00e9f0 tx=0x7fe5fc00eeb0 comp rx=0 tx=0).stop 2024-09-13T16:29:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.904+0000 7fe611c15640 1 -- 172.21.15.16:0/1088063807 shutdown_connections 2024-09-13T16:29:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.904+0000 7fe611c15640 1 --2- 172.21.15.16:0/1088063807 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe5f40a1110 0x7fe5f40a35d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.904+0000 7fe611c15640 1 --2- 172.21.15.16:0/1088063807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe60c10a390 0x7fe60c19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.904+0000 7fe611c15640 1 --2- 172.21.15.16:0/1088063807 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe60c1018d0 0x7fe60c196660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.904+0000 7fe611c15640 1 --2- 172.21.15.16:0/1088063807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe60c100fb0 0x7fe60c196120 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.904+0000 7fe611c15640 1 -- 172.21.15.16:0/1088063807 >> 172.21.15.16:0/1088063807 conn(0x7fe60c0fa9b0 msgr2=0x7fe60c0fc380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.904+0000 7fe611c15640 1 -- 172.21.15.16:0/1088063807 shutdown_connections 2024-09-13T16:29:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:18.904+0000 7fe611c15640 1 -- 172.21.15.16:0/1088063807 wait complete. 2024-09-13T16:29:18.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:18 smithi111 ceph-mon[34600]: pgmap v298: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.2 KiB/s wr, 25 op/s; 57 B/s, 1 objects/s recovering 2024-09-13T16:29:19.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:18 smithi107 ceph-mon[34139]: pgmap v298: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.2 KiB/s wr, 25 op/s; 57 B/s, 1 objects/s recovering 2024-09-13T16:29:19.701 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3739009683},{"type":"v1","addr":"172.21.15.16:6801","nonce":3739009683}]},"active_addr":"172.21.15.16:6801/3739009683","active_change":"2024-09-13T16:22:04.485388+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.16:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":3937301827}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":2872798669}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":3459043361}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":2780553042}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":2990598251}]}]} 2024-09-13T16:29:19.707 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-13T16:29:19.707 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-13T16:29:19.707 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-13T16:29:19.915 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:29:19.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:19 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1088063807' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-13T16:29:20.036 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:19 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1088063807' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-13T16:29:20.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:19 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1088063807' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-13T16:29:20.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:20 smithi111 ceph-mon[34600]: pgmap v299: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.5 KiB/s wr, 11 op/s 2024-09-13T16:29:21.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:20 smithi016 ceph-mon[31187]: pgmap v299: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.5 KiB/s wr, 11 op/s 2024-09-13T16:29:21.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:20 smithi107 ceph-mon[34139]: pgmap v299: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.5 KiB/s wr, 11 op/s 2024-09-13T16:29:21.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.909+0000 7fa9f9697640 1 -- 172.21.15.16:0/2785388665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4105110 msgr2=0x7fa9f410cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:21.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.909+0000 7fa9f9697640 1 --2- 172.21.15.16:0/2785388665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4105110 0x7fa9f410cc90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec009cd0 tx=0x7fa9ec02a8e0 comp rx=0 tx=0).stop 2024-09-13T16:29:21.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.909+0000 7fa9f9697640 1 -- 172.21.15.16:0/2785388665 shutdown_connections 2024-09-13T16:29:21.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.909+0000 7fa9f9697640 1 --2- 172.21.15.16:0/2785388665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4105110 0x7fa9f410cc90 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:21.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.909+0000 7fa9f9697640 1 --2- 172.21.15.16:0/2785388665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9f4100990 0x7fa9f41049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:21.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.909+0000 7fa9f9697640 1 --2- 172.21.15.16:0/2785388665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9f40fffe0 0x7fa9f41003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:21.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.909+0000 7fa9f9697640 1 -- 172.21.15.16:0/2785388665 >> 172.21.15.16:0/2785388665 conn(0x7fa9f40fbae0 msgr2=0x7fa9f40fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:21.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.909+0000 7fa9f9697640 1 -- 172.21.15.16:0/2785388665 shutdown_connections 2024-09-13T16:29:21.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.909+0000 7fa9f9697640 1 -- 172.21.15.16:0/2785388665 wait complete. 2024-09-13T16:29:21.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.910+0000 7fa9f9697640 1 Processor -- start 2024-09-13T16:29:21.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.910+0000 7fa9f9697640 1 -- start start 2024-09-13T16:29:21.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.910+0000 7fa9f9697640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9f40fffe0 0x7fa9f4196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:21.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.910+0000 7fa9f9697640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4100990 0x7fa9f4196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:21.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.910+0000 7fa9f9697640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9f4105110 0x7fa9f419ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:21.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.910+0000 7fa9f9697640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f419b280 con 0x7fa9f4100990 2024-09-13T16:29:21.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.910+0000 7fa9f9697640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f419b3f0 con 0x7fa9f40fffe0 2024-09-13T16:29:21.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.910+0000 7fa9f9697640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f419b560 con 0x7fa9f4105110 2024-09-13T16:29:21.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.910+0000 7fa9ea7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4100990 0x7fa9f4196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:21.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.910+0000 7fa9ea7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4100990 0x7fa9f4196880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49246/0 (socket says 172.21.15.16:49246) 2024-09-13T16:29:21.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.910+0000 7fa9ea7fc640 1 -- 172.21.15.16:0/787012427 learned_addr learned my addr 172.21.15.16:0/787012427 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:21.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.911+0000 7fa9f37fe640 1 --2- 172.21.15.16:0/787012427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9f4105110 0x7fa9f419ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:21.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.911+0000 7fa9f2ffd640 1 --2- 172.21.15.16:0/787012427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9f40fffe0 0x7fa9f4196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:21.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.911+0000 7fa9ea7fc640 1 -- 172.21.15.16:0/787012427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9f4105110 msgr2=0x7fa9f419ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:21.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.911+0000 7fa9ea7fc640 1 --2- 172.21.15.16:0/787012427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9f4105110 0x7fa9f419ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:21.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.911+0000 7fa9ea7fc640 1 -- 172.21.15.16:0/787012427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9f40fffe0 msgr2=0x7fa9f4196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:21.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.911+0000 7fa9ea7fc640 1 --2- 172.21.15.16:0/787012427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9f40fffe0 0x7fa9f4196340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:21.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.911+0000 7fa9ea7fc640 1 -- 172.21.15.16:0/787012427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ec009900 con 0x7fa9f4100990 2024-09-13T16:29:21.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.911+0000 7fa9ea7fc640 1 --2- 172.21.15.16:0/787012427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4100990 0x7fa9f4196880 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00b4a0 tx=0x7fa9dc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:21.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.911+0000 7fa9f0ff9640 1 -- 172.21.15.16:0/787012427 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc010610 con 0x7fa9f4100990 2024-09-13T16:29:21.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.911+0000 7fa9f0ff9640 1 -- 172.21.15.16:0/787012427 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa9dc010c00 con 0x7fa9f4100990 2024-09-13T16:29:21.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.911+0000 7fa9f0ff9640 1 -- 172.21.15.16:0/787012427 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc017ca0 con 0x7fa9f4100990 2024-09-13T16:29:21.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.911+0000 7fa9f9697640 1 -- 172.21.15.16:0/787012427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f419b840 con 0x7fa9f4100990 2024-09-13T16:29:21.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.911+0000 7fa9f9697640 1 -- 172.21.15.16:0/787012427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f41a3100 con 0x7fa9f4100990 2024-09-13T16:29:21.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.912+0000 7fa9f9697640 1 -- 172.21.15.16:0/787012427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f41014c0 con 0x7fa9f4100990 2024-09-13T16:29:21.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.913+0000 7fa9f0ff9640 1 -- 172.21.15.16:0/787012427 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fa9dc010770 con 0x7fa9f4100990 2024-09-13T16:29:21.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.914+0000 7fa9f0ff9640 1 --2- 172.21.15.16:0/787012427 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa9e00a1190 0x7fa9e00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:21.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.914+0000 7fa9f0ff9640 1 -- 172.21.15.16:0/787012427 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa9dc0cc640 con 0x7fa9f4100990 2024-09-13T16:29:21.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.914+0000 7fa9f2ffd640 1 --2- 172.21.15.16:0/787012427 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa9e00a1190 0x7fa9e00a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:21.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.914+0000 7fa9f2ffd640 1 --2- 172.21.15.16:0/787012427 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa9e00a1190 0x7fa9e00a3650 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa9e400a810 tx=0x7fa9e4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:21.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:21.918+0000 7fa9f0ff9640 1 -- 172.21.15.16:0/787012427 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa9dc095a10 con 0x7fa9f4100990 2024-09-13T16:29:22.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:22.091+0000 7fa9f9697640 1 -- 172.21.15.16:0/787012427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9f4100440 con 0x7fa9f4100990 2024-09-13T16:29:22.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:22.093+0000 7fa9f0ff9640 1 -- 172.21.15.16:0/787012427 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24499 (secure 0 0 0) 0x7fa9dc099030 con 0x7fa9f4100990 2024-09-13T16:29:22.094 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:29:22.094 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":91,"fsid":"0d241f6a-71ec-11ef-bceb-c7b262605968","created":"2024-09-13T16:21:17.971738+0000","modified":"2024-09-13T16:29:05.626781+0000","last_up_change":"2024-09-13T16:27:54.673413+0000","last_in_change":"2024-09-13T16:27:35.718390+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:24:22.981019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:28:37.746027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:28:38.602731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:28:41.633322+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e11ef8f-08d6-4e74-afd6-f49f4bd40032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6803","nonce":2079394186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6805","nonce":2079394186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6809","nonce":2079394186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6807","nonce":2079394186}]},"public_addr":"172.21.15.16:6803/2079394186","cluster_addr":"172.21.15.16:6805/2079394186","heartbeat_back_addr":"172.21.15.16:6809/2079394186","heartbeat_front_addr":"172.21.15.16:6807/2079394186","state":["exists","up"]},{"osd":1,"uuid":"f2cd3694-9c9e-4da7-976e-f80a8df120f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6801","nonce":4021360958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6803","nonce":4021360958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6807","nonce":4021360958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6805","nonce":4021360958}]},"public_addr":"172.21.15.107:6801/4021360958","cluster_addr":"172.21.15.107:6803/4021360958","heartbeat_back_addr":"172.21.15.107:6807/4021360958","heartbeat_front_addr":"172.21.15.107:6805/4021360958","state":["exists","up"]},{"osd":2,"uuid":"a5bbf2eb-6599-44d5-ad96-68bb89ee19f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6801","nonce":3227179825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6803","nonce":3227179825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6807","nonce":3227179825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6805","nonce":3227179825}]},"public_addr":"172.21.15.111:6801/3227179825","cluster_addr":"172.21.15.111:6803/3227179825","heartbeat_back_addr":"172.21.15.111:6807/3227179825","heartbeat_front_addr":"172.21.15.111:6805/3227179825","state":["exists","up"]},{"osd":3,"uuid":"d6295ffe-11bc-4964-a250-9fecca870b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6811","nonce":2390375937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6813","nonce":2390375937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6817","nonce":2390375937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6815","nonce":2390375937}]},"public_addr":"172.21.15.16:6811/2390375937","cluster_addr":"172.21.15.16:6813/2390375937","heartbeat_back_addr":"172.21.15.16:6817/2390375937","heartbeat_front_addr":"172.21.15.16:6815/2390375937","state":["exists","up"]},{"osd":4,"uuid":"0a8446e0-fc86-4646-beb6-a7b9d433093d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6809","nonce":1936582048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6811","nonce":1936582048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6815","nonce":1936582048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6813","nonce":1936582048}]},"public_addr":"172.21.15.107:6809/1936582048","cluster_addr":"172.21.15.107:6811/1936582048","heartbeat_back_addr":"172.21.15.107:6815/1936582048","heartbeat_front_addr":"172.21.15.107:6813/1936582048","state":["exists","up"]},{"osd":5,"uuid":"2c97a446-357e-4975-9e51-9c95fe84a663","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6809","nonce":483858448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6811","nonce":483858448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6815","nonce":483858448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6813","nonce":483858448}]},"public_addr":"172.21.15.111:6809/483858448","cluster_addr":"172.21.15.111:6811/483858448","heartbeat_back_addr":"172.21.15.111:6815/483858448","heartbeat_front_addr":"172.21.15.111:6813/483858448","state":["exists","up"]},{"osd":6,"uuid":"890239e6-e6fb-4f9c-b973-05cc66d199bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6819","nonce":1330214141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6821","nonce":1330214141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6825","nonce":1330214141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6823","nonce":1330214141}]},"public_addr":"172.21.15.16:6819/1330214141","cluster_addr":"172.21.15.16:6821/1330214141","heartbeat_back_addr":"172.21.15.16:6825/1330214141","heartbeat_front_addr":"172.21.15.16:6823/1330214141","state":["exists","up"]},{"osd":7,"uuid":"ee826a9c-f441-4903-9b1b-66b627f7b712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6817","nonce":1133876754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6819","nonce":1133876754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6823","nonce":1133876754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6821","nonce":1133876754}]},"public_addr":"172.21.15.107:6817/1133876754","cluster_addr":"172.21.15.107:6819/1133876754","heartbeat_back_addr":"172.21.15.107:6823/1133876754","heartbeat_front_addr":"172.21.15.107:6821/1133876754","state":["exists","up"]},{"osd":8,"uuid":"e19773f2-dec8-44e0-9d33-0252e07d6415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6817","nonce":837548190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6819","nonce":837548190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6823","nonce":837548190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6821","nonce":837548190}]},"public_addr":"172.21.15.111:6817/837548190","cluster_addr":"172.21.15.111:6819/837548190","heartbeat_back_addr":"172.21.15.111:6823/837548190","heartbeat_front_addr":"172.21.15.111:6821/837548190","state":["exists","up"]},{"osd":9,"uuid":"bae4d6db-8b90-4d02-8e83-7d8f2a6f8fdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6827","nonce":4175383686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6829","nonce":4175383686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6833","nonce":4175383686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6831","nonce":4175383686}]},"public_addr":"172.21.15.16:6827/4175383686","cluster_addr":"172.21.15.16:6829/4175383686","heartbeat_back_addr":"172.21.15.16:6833/4175383686","heartbeat_front_addr":"172.21.15.16:6831/4175383686","state":["exists","up"]},{"osd":10,"uuid":"fd0e7740-76c8-4c99-a349-7077b4905143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6825","nonce":2659550426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6827","nonce":2659550426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6831","nonce":2659550426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6829","nonce":2659550426}]},"public_addr":"172.21.15.107:6825/2659550426","cluster_addr":"172.21.15.107:6827/2659550426","heartbeat_back_addr":"172.21.15.107:6831/2659550426","heartbeat_front_addr":"172.21.15.107:6829/2659550426","state":["exists","up"]},{"osd":11,"uuid":"94a1751d-b00c-4e23-815d-3a8bc3dea146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6825","nonce":1633548683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6827","nonce":1633548683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6831","nonce":1633548683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6829","nonce":1633548683}]},"public_addr":"172.21.15.111:6825/1633548683","cluster_addr":"172.21.15.111:6827/1633548683","heartbeat_back_addr":"172.21.15.111:6831/1633548683","heartbeat_front_addr":"172.21.15.111:6829/1633548683","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:36.359852+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:58.014202+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:19.753672+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:43.404817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:06.001677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:29.361587+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:52.554847+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:15.957553+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:39.581093+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:04.911389+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:28.971280+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:53.473561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/2098378667":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/3536229016":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6801/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/2684687547":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6800/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/368410612":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/12158160":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6800/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6801/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/3754423379":"2024-09-14T16:21:40.126459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:29:22.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:22.095+0000 7fa9f9697640 1 -- 172.21.15.16:0/787012427 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa9e00a1190 msgr2=0x7fa9e00a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:22.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:22.095+0000 7fa9f9697640 1 --2- 172.21.15.16:0/787012427 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa9e00a1190 0x7fa9e00a3650 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa9e400a810 tx=0x7fa9e4008040 comp rx=0 tx=0).stop 2024-09-13T16:29:22.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:22.095+0000 7fa9f9697640 1 -- 172.21.15.16:0/787012427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4100990 msgr2=0x7fa9f4196880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:22.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:22.095+0000 7fa9f9697640 1 --2- 172.21.15.16:0/787012427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4100990 0x7fa9f4196880 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00b4a0 tx=0x7fa9dc00b960 comp rx=0 tx=0).stop 2024-09-13T16:29:22.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:22.095+0000 7fa9f9697640 1 -- 172.21.15.16:0/787012427 shutdown_connections 2024-09-13T16:29:22.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:22.095+0000 7fa9f9697640 1 --2- 172.21.15.16:0/787012427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9f4105110 0x7fa9f419ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:22.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:22.095+0000 7fa9f9697640 1 --2- 172.21.15.16:0/787012427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4100990 0x7fa9f4196880 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:22.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:22.095+0000 7fa9f9697640 1 --2- 172.21.15.16:0/787012427 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa9e00a1190 0x7fa9e00a3650 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:22.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:22.095+0000 7fa9f9697640 1 --2- 172.21.15.16:0/787012427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9f40fffe0 0x7fa9f4196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:22.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:22.095+0000 7fa9f9697640 1 -- 172.21.15.16:0/787012427 >> 172.21.15.16:0/787012427 conn(0x7fa9f40fbae0 msgr2=0x7fa9f40fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:22.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:22.095+0000 7fa9f9697640 1 -- 172.21.15.16:0/787012427 shutdown_connections 2024-09-13T16:29:22.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:22.095+0000 7fa9f9697640 1 -- 172.21.15.16:0/787012427 wait complete. 2024-09-13T16:29:22.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:22 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/787012427' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:29:22.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:22 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/787012427' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:29:22.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:22 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/787012427' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:29:22.882 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-13T16:29:22.882 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-13T16:29:23.092 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:29:23.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:23 smithi111 ceph-mon[34600]: pgmap v300: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-09-13T16:29:23.452 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:23 smithi016 ceph-mon[31187]: pgmap v300: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-09-13T16:29:23.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:23 smithi107 ceph-mon[34139]: pgmap v300: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-09-13T16:29:24.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:24 smithi016 ceph-mon[31187]: pgmap v301: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-13T16:29:24.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:24 smithi111 ceph-mon[34600]: pgmap v301: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-13T16:29:24.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.910+0000 7fd40e8d8640 1 -- 172.21.15.16:0/1919306469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd408100990 msgr2=0x7fd4081049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:24.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.910+0000 7fd40e8d8640 1 --2- 172.21.15.16:0/1919306469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd408100990 0x7fd4081049e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc009930 tx=0x7fd3fc02a420 comp rx=0 tx=0).stop 2024-09-13T16:29:24.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.910+0000 7fd40e8d8640 1 -- 172.21.15.16:0/1919306469 shutdown_connections 2024-09-13T16:29:24.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.910+0000 7fd40e8d8640 1 --2- 172.21.15.16:0/1919306469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd408105110 0x7fd40810cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:24.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.910+0000 7fd40e8d8640 1 --2- 172.21.15.16:0/1919306469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd408100990 0x7fd4081049e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:24.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.910+0000 7fd40e8d8640 1 --2- 172.21.15.16:0/1919306469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4080fffe0 0x7fd4081003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:24.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.910+0000 7fd40e8d8640 1 -- 172.21.15.16:0/1919306469 >> 172.21.15.16:0/1919306469 conn(0x7fd4080fbae0 msgr2=0x7fd4080fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:24.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.911+0000 7fd40e8d8640 1 -- 172.21.15.16:0/1919306469 shutdown_connections 2024-09-13T16:29:24.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.911+0000 7fd40e8d8640 1 -- 172.21.15.16:0/1919306469 wait complete. 2024-09-13T16:29:24.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.911+0000 7fd40e8d8640 1 Processor -- start 2024-09-13T16:29:24.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.911+0000 7fd40e8d8640 1 -- start start 2024-09-13T16:29:24.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.912+0000 7fd40e8d8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4080fffe0 0x7fd408196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:24.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.912+0000 7fd40e8d8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd408100990 0x7fd408196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:24.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.912+0000 7fd40e8d8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd408105110 0x7fd40819ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:24.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.912+0000 7fd40e8d8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40819b260 con 0x7fd408105110 2024-09-13T16:29:24.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.912+0000 7fd40e8d8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40819b3d0 con 0x7fd4080fffe0 2024-09-13T16:29:24.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.912+0000 7fd40e8d8640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40819b540 con 0x7fd408100990 2024-09-13T16:29:24.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.912+0000 7fd40ce75640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd408105110 0x7fd40819ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:24.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.912+0000 7fd40ce75640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd408105110 0x7fd40819ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40864/0 (socket says 172.21.15.16:40864) 2024-09-13T16:29:24.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.912+0000 7fd40ce75640 1 -- 172.21.15.16:0/3458563865 learned_addr learned my addr 172.21.15.16:0/3458563865 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:24.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.912+0000 7fd407fff640 1 --2- 172.21.15.16:0/3458563865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4080fffe0 0x7fd408196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:24.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.912+0000 7fd4077fe640 1 --2- 172.21.15.16:0/3458563865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd408100990 0x7fd408196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:24.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.912+0000 7fd40ce75640 1 -- 172.21.15.16:0/3458563865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd408100990 msgr2=0x7fd408196860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:24.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.912+0000 7fd40ce75640 1 --2- 172.21.15.16:0/3458563865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd408100990 0x7fd408196860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:24.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.912+0000 7fd40ce75640 1 -- 172.21.15.16:0/3458563865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4080fffe0 msgr2=0x7fd408196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:24.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.912+0000 7fd40ce75640 1 --2- 172.21.15.16:0/3458563865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4080fffe0 0x7fd408196320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:24.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.912+0000 7fd40ce75640 1 -- 172.21.15.16:0/3458563865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3fc009560 con 0x7fd408105110 2024-09-13T16:29:24.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.912+0000 7fd40ce75640 1 --2- 172.21.15.16:0/3458563865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd408105110 0x7fd40819ac60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd40000eaa0 tx=0x7fd40000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:24.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.913+0000 7fd4057fa640 1 -- 172.21.15.16:0/3458563865 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40000cdb0 con 0x7fd408105110 2024-09-13T16:29:24.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.913+0000 7fd4057fa640 1 -- 172.21.15.16:0/3458563865 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd400005020 con 0x7fd408105110 2024-09-13T16:29:24.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.913+0000 7fd4057fa640 1 -- 172.21.15.16:0/3458563865 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4000116d0 con 0x7fd408105110 2024-09-13T16:29:24.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.913+0000 7fd40e8d8640 1 -- 172.21.15.16:0/3458563865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd40819b820 con 0x7fd408105110 2024-09-13T16:29:24.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.913+0000 7fd40e8d8640 1 -- 172.21.15.16:0/3458563865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd40819bad0 con 0x7fd408105110 2024-09-13T16:29:24.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.914+0000 7fd40e8d8640 1 -- 172.21.15.16:0/3458563865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4081014c0 con 0x7fd408105110 2024-09-13T16:29:24.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.915+0000 7fd4057fa640 1 -- 172.21.15.16:0/3458563865 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fd400005190 con 0x7fd408105110 2024-09-13T16:29:24.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.916+0000 7fd4057fa640 1 --2- 172.21.15.16:0/3458563865 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd3f00a1190 0x7fd3f00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:24.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.916+0000 7fd407fff640 1 --2- 172.21.15.16:0/3458563865 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd3f00a1190 0x7fd3f00a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:24.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.916+0000 7fd4057fa640 1 -- 172.21.15.16:0/3458563865 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd4000ccc70 con 0x7fd408105110 2024-09-13T16:29:24.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.916+0000 7fd407fff640 1 --2- 172.21.15.16:0/3458563865 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd3f00a1190 0x7fd3f00a3650 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8005fd0 tx=0x7fd3f800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:24.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:24.920+0000 7fd4057fa640 1 -- 172.21.15.16:0/3458563865 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd400096040 con 0x7fd408105110 2024-09-13T16:29:25.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:25.092+0000 7fd40e8d8640 1 -- 172.21.15.16:0/3458563865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd40819bad0 con 0x7fd408105110 2024-09-13T16:29:25.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:25.093+0000 7fd4057fa640 1 -- 172.21.15.16:0/3458563865 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24499 (secure 0 0 0) 0x7fd40819bad0 con 0x7fd408105110 2024-09-13T16:29:25.094 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:29:25.095 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":91,"fsid":"0d241f6a-71ec-11ef-bceb-c7b262605968","created":"2024-09-13T16:21:17.971738+0000","modified":"2024-09-13T16:29:05.626781+0000","last_up_change":"2024-09-13T16:27:54.673413+0000","last_in_change":"2024-09-13T16:27:35.718390+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:24:22.981019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:28:37.746027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:28:38.602731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:28:41.633322+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e11ef8f-08d6-4e74-afd6-f49f4bd40032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6803","nonce":2079394186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6805","nonce":2079394186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6809","nonce":2079394186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6807","nonce":2079394186}]},"public_addr":"172.21.15.16:6803/2079394186","cluster_addr":"172.21.15.16:6805/2079394186","heartbeat_back_addr":"172.21.15.16:6809/2079394186","heartbeat_front_addr":"172.21.15.16:6807/2079394186","state":["exists","up"]},{"osd":1,"uuid":"f2cd3694-9c9e-4da7-976e-f80a8df120f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6801","nonce":4021360958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6803","nonce":4021360958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6807","nonce":4021360958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6805","nonce":4021360958}]},"public_addr":"172.21.15.107:6801/4021360958","cluster_addr":"172.21.15.107:6803/4021360958","heartbeat_back_addr":"172.21.15.107:6807/4021360958","heartbeat_front_addr":"172.21.15.107:6805/4021360958","state":["exists","up"]},{"osd":2,"uuid":"a5bbf2eb-6599-44d5-ad96-68bb89ee19f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6801","nonce":3227179825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6803","nonce":3227179825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6807","nonce":3227179825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6805","nonce":3227179825}]},"public_addr":"172.21.15.111:6801/3227179825","cluster_addr":"172.21.15.111:6803/3227179825","heartbeat_back_addr":"172.21.15.111:6807/3227179825","heartbeat_front_addr":"172.21.15.111:6805/3227179825","state":["exists","up"]},{"osd":3,"uuid":"d6295ffe-11bc-4964-a250-9fecca870b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6811","nonce":2390375937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6813","nonce":2390375937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6817","nonce":2390375937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6815","nonce":2390375937}]},"public_addr":"172.21.15.16:6811/2390375937","cluster_addr":"172.21.15.16:6813/2390375937","heartbeat_back_addr":"172.21.15.16:6817/2390375937","heartbeat_front_addr":"172.21.15.16:6815/2390375937","state":["exists","up"]},{"osd":4,"uuid":"0a8446e0-fc86-4646-beb6-a7b9d433093d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6809","nonce":1936582048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6811","nonce":1936582048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6815","nonce":1936582048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6813","nonce":1936582048}]},"public_addr":"172.21.15.107:6809/1936582048","cluster_addr":"172.21.15.107:6811/1936582048","heartbeat_back_addr":"172.21.15.107:6815/1936582048","heartbeat_front_addr":"172.21.15.107:6813/1936582048","state":["exists","up"]},{"osd":5,"uuid":"2c97a446-357e-4975-9e51-9c95fe84a663","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6809","nonce":483858448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6811","nonce":483858448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6815","nonce":483858448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6813","nonce":483858448}]},"public_addr":"172.21.15.111:6809/483858448","cluster_addr":"172.21.15.111:6811/483858448","heartbeat_back_addr":"172.21.15.111:6815/483858448","heartbeat_front_addr":"172.21.15.111:6813/483858448","state":["exists","up"]},{"osd":6,"uuid":"890239e6-e6fb-4f9c-b973-05cc66d199bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6819","nonce":1330214141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6821","nonce":1330214141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6825","nonce":1330214141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6823","nonce":1330214141}]},"public_addr":"172.21.15.16:6819/1330214141","cluster_addr":"172.21.15.16:6821/1330214141","heartbeat_back_addr":"172.21.15.16:6825/1330214141","heartbeat_front_addr":"172.21.15.16:6823/1330214141","state":["exists","up"]},{"osd":7,"uuid":"ee826a9c-f441-4903-9b1b-66b627f7b712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6817","nonce":1133876754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6819","nonce":1133876754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6823","nonce":1133876754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6821","nonce":1133876754}]},"public_addr":"172.21.15.107:6817/1133876754","cluster_addr":"172.21.15.107:6819/1133876754","heartbeat_back_addr":"172.21.15.107:6823/1133876754","heartbeat_front_addr":"172.21.15.107:6821/1133876754","state":["exists","up"]},{"osd":8,"uuid":"e19773f2-dec8-44e0-9d33-0252e07d6415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6817","nonce":837548190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6819","nonce":837548190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6823","nonce":837548190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6821","nonce":837548190}]},"public_addr":"172.21.15.111:6817/837548190","cluster_addr":"172.21.15.111:6819/837548190","heartbeat_back_addr":"172.21.15.111:6823/837548190","heartbeat_front_addr":"172.21.15.111:6821/837548190","state":["exists","up"]},{"osd":9,"uuid":"bae4d6db-8b90-4d02-8e83-7d8f2a6f8fdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6827","nonce":4175383686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6829","nonce":4175383686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6833","nonce":4175383686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6831","nonce":4175383686}]},"public_addr":"172.21.15.16:6827/4175383686","cluster_addr":"172.21.15.16:6829/4175383686","heartbeat_back_addr":"172.21.15.16:6833/4175383686","heartbeat_front_addr":"172.21.15.16:6831/4175383686","state":["exists","up"]},{"osd":10,"uuid":"fd0e7740-76c8-4c99-a349-7077b4905143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6825","nonce":2659550426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6827","nonce":2659550426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6831","nonce":2659550426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6829","nonce":2659550426}]},"public_addr":"172.21.15.107:6825/2659550426","cluster_addr":"172.21.15.107:6827/2659550426","heartbeat_back_addr":"172.21.15.107:6831/2659550426","heartbeat_front_addr":"172.21.15.107:6829/2659550426","state":["exists","up"]},{"osd":11,"uuid":"94a1751d-b00c-4e23-815d-3a8bc3dea146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6825","nonce":1633548683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6827","nonce":1633548683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6831","nonce":1633548683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6829","nonce":1633548683}]},"public_addr":"172.21.15.111:6825/1633548683","cluster_addr":"172.21.15.111:6827/1633548683","heartbeat_back_addr":"172.21.15.111:6831/1633548683","heartbeat_front_addr":"172.21.15.111:6829/1633548683","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:36.359852+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:58.014202+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:19.753672+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:43.404817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:06.001677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:29.361587+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:52.554847+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:15.957553+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:39.581093+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:04.911389+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:28.971280+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:53.473561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/2098378667":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/3536229016":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6801/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/2684687547":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6800/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/368410612":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/12158160":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6800/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6801/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/3754423379":"2024-09-14T16:21:40.126459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:29:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:25.095+0000 7fd40e8d8640 1 -- 172.21.15.16:0/3458563865 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd3f00a1190 msgr2=0x7fd3f00a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:25.095+0000 7fd40e8d8640 1 --2- 172.21.15.16:0/3458563865 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd3f00a1190 0x7fd3f00a3650 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8005fd0 tx=0x7fd3f800b390 comp rx=0 tx=0).stop 2024-09-13T16:29:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:25.095+0000 7fd40e8d8640 1 -- 172.21.15.16:0/3458563865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd408105110 msgr2=0x7fd40819ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:25.095+0000 7fd40e8d8640 1 --2- 172.21.15.16:0/3458563865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd408105110 0x7fd40819ac60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd40000eaa0 tx=0x7fd40000ef60 comp rx=0 tx=0).stop 2024-09-13T16:29:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:25.095+0000 7fd40e8d8640 1 -- 172.21.15.16:0/3458563865 shutdown_connections 2024-09-13T16:29:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:25.095+0000 7fd40e8d8640 1 --2- 172.21.15.16:0/3458563865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd408105110 0x7fd40819ac60 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:25.095+0000 7fd40e8d8640 1 --2- 172.21.15.16:0/3458563865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd408100990 0x7fd408196860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:25.095+0000 7fd40e8d8640 1 --2- 172.21.15.16:0/3458563865 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd3f00a1190 0x7fd3f00a3650 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:25.095+0000 7fd40e8d8640 1 --2- 172.21.15.16:0/3458563865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4080fffe0 0x7fd408196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:25.095+0000 7fd40e8d8640 1 -- 172.21.15.16:0/3458563865 >> 172.21.15.16:0/3458563865 conn(0x7fd4080fbae0 msgr2=0x7fd4080fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:25.095+0000 7fd40e8d8640 1 -- 172.21.15.16:0/3458563865 shutdown_connections 2024-09-13T16:29:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:25.095+0000 7fd40e8d8640 1 -- 172.21.15.16:0/3458563865 wait complete. 2024-09-13T16:29:25.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:24 smithi107 ceph-mon[34139]: pgmap v301: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-13T16:29:25.810 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-13T16:29:25.811 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-13T16:29:25.812 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-13T16:29:25.812 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-13T16:29:25.813 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-13T16:29:25.814 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-13T16:29:25.814 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-13T16:29:25.815 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-13T16:29:25.886 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-13T16:29:25.886 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-13T16:29:25.887 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-13T16:29:25.887 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-13T16:29:25.930 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:25 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3458563865' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:29:25.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:25 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3458563865' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:29:26.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:25 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3458563865' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:29:26.424 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:29:26.543 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:29:26.544 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:29:26.545 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:29:26.546 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:29:26.546 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:29:26.551 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:29:26.552 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:29:26.552 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:29:26.555 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:29:26.560 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:29:26.561 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:29:26.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:26 smithi016 ceph-mon[31187]: pgmap v302: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-13T16:29:26.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:26 smithi111 ceph-mon[34600]: pgmap v302: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-13T16:29:27.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:26 smithi107 ceph-mon[34139]: pgmap v302: 129 pgs: 129 active+clean; 641 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-13T16:29:28.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:28 smithi111 ceph-mon[34600]: pgmap v303: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-09-13T16:29:28.967 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:28 smithi016 ceph-mon[31187]: pgmap v303: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-09-13T16:29:29.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:28 smithi107 ceph-mon[34139]: pgmap v303: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-09-13T16:29:30.706 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:30 smithi016 ceph-mon[31187]: pgmap v304: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-13T16:29:30.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:30 smithi111 ceph-mon[34600]: pgmap v304: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-13T16:29:31.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:30 smithi107 ceph-mon[34139]: pgmap v304: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-13T16:29:32.934 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:32 smithi016 ceph-mon[31187]: pgmap v305: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-13T16:29:32.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:32 smithi111 ceph-mon[34600]: pgmap v305: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-13T16:29:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.017+0000 7f4baa705640 1 -- 172.21.15.16:0/15492823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba40ff540 msgr2=0x7f4ba4109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.017+0000 7f4baa705640 1 --2- 172.21.15.16:0/15492823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba40ff540 0x7f4ba4109cb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c009900 tx=0x7f4b8c02a410 comp rx=0 tx=0).stop 2024-09-13T16:29:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.017+0000 7f4baa705640 1 -- 172.21.15.16:0/15492823 shutdown_connections 2024-09-13T16:29:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.017+0000 7f4baa705640 1 --2- 172.21.15.16:0/15492823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ba410a3a0 0x7f4ba410c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.017+0000 7f4baa705640 1 --2- 172.21.15.16:0/15492823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba40ff540 0x7f4ba4109cb0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.017+0000 7f4baa705640 1 --2- 172.21.15.16:0/15492823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ba40feb90 0x7f4ba40fef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.017+0000 7f4baa705640 1 -- 172.21.15.16:0/15492823 >> 172.21.15.16:0/15492823 conn(0x7f4ba4074130 msgr2=0x7f4ba4074540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.018+0000 7f4baa705640 1 -- 172.21.15.16:0/15492823 shutdown_connections 2024-09-13T16:29:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.018+0000 7f4baa705640 1 -- 172.21.15.16:0/15492823 wait complete. 2024-09-13T16:29:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.018+0000 7f4baa705640 1 Processor -- start 2024-09-13T16:29:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.018+0000 7f4baa705640 1 -- start start 2024-09-13T16:29:33.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.018+0000 7f4baa705640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ba40feb90 0x7f4ba40ffb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:33.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.018+0000 7f4baa705640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba40ff540 0x7f4ba4100050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:33.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.018+0000 7f4baa705640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ba410a3a0 0x7f4ba4100590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:33.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.019+0000 7f4baa705640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba41a4580 con 0x7f4ba40ff540 2024-09-13T16:29:33.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.019+0000 7f4baa705640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba41a46f0 con 0x7f4ba40feb90 2024-09-13T16:29:33.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.019+0000 7f4baa705640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba41a48b0 con 0x7f4ba410a3a0 2024-09-13T16:29:33.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.019+0000 7f4ba37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba40ff540 0x7f4ba4100050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:33.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.019+0000 7f4ba37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba40ff540 0x7f4ba4100050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40882/0 (socket says 172.21.15.16:40882) 2024-09-13T16:29:33.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.019+0000 7f4ba37fe640 1 -- 172.21.15.16:0/2154817265 learned_addr learned my addr 172.21.15.16:0/2154817265 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:33.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.019+0000 7f4ba3fff640 1 --2- 172.21.15.16:0/2154817265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ba40feb90 0x7f4ba40ffb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:33.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.019+0000 7f4ba8ca2640 1 --2- 172.21.15.16:0/2154817265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ba410a3a0 0x7f4ba4100590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:33.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.019+0000 7f4ba37fe640 1 -- 172.21.15.16:0/2154817265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ba410a3a0 msgr2=0x7f4ba4100590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:33.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.019+0000 7f4ba37fe640 1 --2- 172.21.15.16:0/2154817265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ba410a3a0 0x7f4ba4100590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:33.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.019+0000 7f4ba37fe640 1 -- 172.21.15.16:0/2154817265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ba40feb90 msgr2=0x7f4ba40ffb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:33.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.019+0000 7f4ba37fe640 1 --2- 172.21.15.16:0/2154817265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ba40feb90 0x7f4ba40ffb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:33.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.019+0000 7f4ba37fe640 1 -- 172.21.15.16:0/2154817265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8c009560 con 0x7f4ba40ff540 2024-09-13T16:29:33.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.019+0000 7f4ba37fe640 1 --2- 172.21.15.16:0/2154817265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba40ff540 0x7f4ba4100050 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c02a930 tx=0x7f4b8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:33.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.019+0000 7f4ba17fa640 1 -- 172.21.15.16:0/2154817265 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c004cf0 con 0x7f4ba40ff540 2024-09-13T16:29:33.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.019+0000 7f4ba17fa640 1 -- 172.21.15.16:0/2154817265 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b8c003a80 con 0x7f4ba40ff540 2024-09-13T16:29:33.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.019+0000 7f4baa705640 1 -- 172.21.15.16:0/2154817265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba41a4b30 con 0x7f4ba40ff540 2024-09-13T16:29:33.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.020+0000 7f4ba17fa640 1 -- 172.21.15.16:0/2154817265 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c03ca70 con 0x7f4ba40ff540 2024-09-13T16:29:33.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.020+0000 7f4baa705640 1 -- 172.21.15.16:0/2154817265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba41a5020 con 0x7f4ba40ff540 2024-09-13T16:29:33.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.020+0000 7f4baa705640 1 -- 172.21.15.16:0/2154817265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4ba410ff80 con 0x7f4ba40ff540 2024-09-13T16:29:33.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.022+0000 7f4ba17fa640 1 -- 172.21.15.16:0/2154817265 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f4b8c003bf0 con 0x7f4ba40ff540 2024-09-13T16:29:33.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.022+0000 7f4ba17fa640 1 --2- 172.21.15.16:0/2154817265 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f4b900a1190 0x7f4b900a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:33.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.022+0000 7f4ba17fa640 1 -- 172.21.15.16:0/2154817265 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4b8c0edfe0 con 0x7f4ba40ff540 2024-09-13T16:29:33.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.022+0000 7f4ba3fff640 1 --2- 172.21.15.16:0/2154817265 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f4b900a1190 0x7f4b900a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:33.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.023+0000 7f4ba17fa640 1 --2- 172.21.15.16:0/2154817265 >> [v2:172.21.15.111:6800/3227179825,v1:172.21.15.111:6801/3227179825] conn(0x7f4b900ac220 0x7f4b900ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:33.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.023+0000 7f4ba17fa640 1 -- 172.21.15.16:0/2154817265 --> [v2:172.21.15.111:6800/3227179825,v1:172.21.15.111:6801/3227179825] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4b900aed90 con 0x7f4b900ac220 2024-09-13T16:29:33.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.023+0000 7f4ba17fa640 1 -- 172.21.15.16:0/2154817265 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b8c0ee3f0 con 0x7f4ba40ff540 2024-09-13T16:29:33.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.023+0000 7f4ba3fff640 1 --2- 172.21.15.16:0/2154817265 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f4b900a1190 0x7f4b900a3650 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4b94005fd0 tx=0x7f4b9400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:33.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.023+0000 7f4ba8ca2640 1 --2- 172.21.15.16:0/2154817265 >> [v2:172.21.15.111:6800/3227179825,v1:172.21.15.111:6801/3227179825] conn(0x7f4b900ac220 0x7f4b900ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:33.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.023+0000 7f4ba8ca2640 1 --2- 172.21.15.16:0/2154817265 >> [v2:172.21.15.111:6800/3227179825,v1:172.21.15.111:6801/3227179825] conn(0x7f4b900ac220 0x7f4b900ae6c0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:33.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.024+0000 7f4ba17fa640 1 -- 172.21.15.16:0/2154817265 <== osd.2 v2:172.21.15.111:6800/3227179825 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f4b900aed90 con 0x7f4b900ac220 2024-09-13T16:29:33.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.037+0000 7f4baa705640 1 -- 172.21.15.16:0/2154817265 --> [v2:172.21.15.111:6800/3227179825,v1:172.21.15.111:6801/3227179825] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4ba4060b70 con 0x7f4b900ac220 2024-09-13T16:29:33.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.038+0000 7f4ba17fa640 1 -- 172.21.15.16:0/2154817265 <== osd.2 v2:172.21.15.111:6800/3227179825 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4ba4060b70 con 0x7f4b900ac220 2024-09-13T16:29:33.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.039+0000 7f4baa705640 1 -- 172.21.15.16:0/2154817265 >> [v2:172.21.15.111:6800/3227179825,v1:172.21.15.111:6801/3227179825] conn(0x7f4b900ac220 msgr2=0x7f4b900ae6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:33.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.039+0000 7f4baa705640 1 --2- 172.21.15.16:0/2154817265 >> [v2:172.21.15.111:6800/3227179825,v1:172.21.15.111:6801/3227179825] conn(0x7f4b900ac220 0x7f4b900ae6c0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:33.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.039+0000 7f4baa705640 1 -- 172.21.15.16:0/2154817265 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f4b900a1190 msgr2=0x7f4b900a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:33.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.039+0000 7f4baa705640 1 --2- 172.21.15.16:0/2154817265 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f4b900a1190 0x7f4b900a3650 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4b94005fd0 tx=0x7f4b9400b390 comp rx=0 tx=0).stop 2024-09-13T16:29:33.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.039+0000 7f4baa705640 1 -- 172.21.15.16:0/2154817265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba40ff540 msgr2=0x7f4ba4100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:33.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.039+0000 7f4baa705640 1 --2- 172.21.15.16:0/2154817265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba40ff540 0x7f4ba4100050 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c02a930 tx=0x7f4b8c0046c0 comp rx=0 tx=0).stop 2024-09-13T16:29:33.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.039+0000 7f4ba8ca2640 1 -- 172.21.15.16:0/2154817265 reap_dead start 2024-09-13T16:29:33.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.039+0000 7f4baa705640 1 -- 172.21.15.16:0/2154817265 shutdown_connections 2024-09-13T16:29:33.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.039+0000 7f4baa705640 1 -- 172.21.15.16:0/2154817265 >> 172.21.15.16:0/2154817265 conn(0x7f4ba4074130 msgr2=0x7f4ba40fbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:33.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.039+0000 7f4baa705640 1 -- 172.21.15.16:0/2154817265 shutdown_connections 2024-09-13T16:29:33.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:33.039+0000 7f4baa705640 1 -- 172.21.15.16:0/2154817265 wait complete. 2024-09-13T16:29:33.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:32 smithi107 ceph-mon[34139]: pgmap v305: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-13T16:29:34.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:34 smithi016 ceph-mon[31187]: pgmap v306: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-13T16:29:34.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:34 smithi111 ceph-mon[34600]: pgmap v306: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-13T16:29:35.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:34 smithi107 ceph-mon[34139]: pgmap v306: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-13T16:29:36.842 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:36 smithi016 ceph-mon[31187]: pgmap v307: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-13T16:29:36.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:36 smithi111 ceph-mon[34600]: pgmap v307: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-13T16:29:37.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:36 smithi107 ceph-mon[34139]: pgmap v307: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-13T16:29:38.877 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:38 smithi016 ceph-mon[31187]: pgmap v308: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-13T16:29:38.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:38 smithi111 ceph-mon[34600]: pgmap v308: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-13T16:29:39.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:38 smithi107 ceph-mon[34139]: pgmap v308: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-13T16:29:39.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:39 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:29:39.927 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:39 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:29:39.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:39 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:29:40.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.400+0000 7fa45be1b640 1 -- 172.21.15.16:0/2734638503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa454100f90 msgr2=0x7fa454101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:40.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.400+0000 7fa45be1b640 1 --2- 172.21.15.16:0/2734638503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa454100f90 0x7fa454101370 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa444009ca0 tx=0x7fa44402a950 comp rx=0 tx=0).stop 2024-09-13T16:29:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.400+0000 7fa45be1b640 1 -- 172.21.15.16:0/2734638503 shutdown_connections 2024-09-13T16:29:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.400+0000 7fa45be1b640 1 --2- 172.21.15.16:0/2734638503 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa45410a370 0x7fa45410c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.400+0000 7fa45be1b640 1 --2- 172.21.15.16:0/2734638503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4541018b0 0x7fa454109da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.400+0000 7fa45be1b640 1 --2- 172.21.15.16:0/2734638503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa454100f90 0x7fa454101370 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.400+0000 7fa45be1b640 1 -- 172.21.15.16:0/2734638503 >> 172.21.15.16:0/2734638503 conn(0x7fa4540fa9b0 msgr2=0x7fa4540fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.400+0000 7fa45be1b640 1 -- 172.21.15.16:0/2734638503 shutdown_connections 2024-09-13T16:29:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.400+0000 7fa45be1b640 1 -- 172.21.15.16:0/2734638503 wait complete. 2024-09-13T16:29:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.401+0000 7fa45be1b640 1 Processor -- start 2024-09-13T16:29:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.401+0000 7fa45be1b640 1 -- start start 2024-09-13T16:29:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.401+0000 7fa45be1b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa454100f90 0x7fa454196230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.401+0000 7fa45be1b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4541018b0 0x7fa454196770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.401+0000 7fa45be1b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa45410a370 0x7fa45419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.401+0000 7fa45be1b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45419b120 con 0x7fa4541018b0 2024-09-13T16:29:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.401+0000 7fa45be1b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45419b290 con 0x7fa45410a370 2024-09-13T16:29:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.401+0000 7fa45be1b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45419b400 con 0x7fa454100f90 2024-09-13T16:29:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.401+0000 7fa4593b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4541018b0 0x7fa454196770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.401+0000 7fa4593b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4541018b0 0x7fa454196770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42884/0 (socket says 172.21.15.16:42884) 2024-09-13T16:29:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.401+0000 7fa459bb7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa454100f90 0x7fa454196230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.401+0000 7fa4593b6640 1 -- 172.21.15.16:0/3366790135 learned_addr learned my addr 172.21.15.16:0/3366790135 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:40.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.401+0000 7fa45a3b8640 1 --2- 172.21.15.16:0/3366790135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa45410a370 0x7fa45419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:40.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.401+0000 7fa4593b6640 1 -- 172.21.15.16:0/3366790135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa454100f90 msgr2=0x7fa454196230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:40.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.401+0000 7fa4593b6640 1 --2- 172.21.15.16:0/3366790135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa454100f90 0x7fa454196230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:40.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.401+0000 7fa4593b6640 1 -- 172.21.15.16:0/3366790135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa45410a370 msgr2=0x7fa45419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:40.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.401+0000 7fa4593b6640 1 --2- 172.21.15.16:0/3366790135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa45410a370 0x7fa45419ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:40.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.401+0000 7fa4593b6640 1 -- 172.21.15.16:0/3366790135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa444009900 con 0x7fa4541018b0 2024-09-13T16:29:40.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.402+0000 7fa4593b6640 1 --2- 172.21.15.16:0/3366790135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4541018b0 0x7fa454196770 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa44800d950 tx=0x7fa44800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:40.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.402+0000 7fa452ffd640 1 -- 172.21.15.16:0/3366790135 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa448009c30 con 0x7fa4541018b0 2024-09-13T16:29:40.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.402+0000 7fa452ffd640 1 -- 172.21.15.16:0/3366790135 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa448009d90 con 0x7fa4541018b0 2024-09-13T16:29:40.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.402+0000 7fa45be1b640 1 -- 172.21.15.16:0/3366790135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa45419b6e0 con 0x7fa4541018b0 2024-09-13T16:29:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.402+0000 7fa452ffd640 1 -- 172.21.15.16:0/3366790135 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44800f620 con 0x7fa4541018b0 2024-09-13T16:29:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.402+0000 7fa45be1b640 1 -- 172.21.15.16:0/3366790135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4541a2fa0 con 0x7fa4541018b0 2024-09-13T16:29:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.403+0000 7fa45be1b640 1 -- 172.21.15.16:0/3366790135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa45410fee0 con 0x7fa4541018b0 2024-09-13T16:29:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.404+0000 7fa452ffd640 1 -- 172.21.15.16:0/3366790135 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fa4480109f0 con 0x7fa4541018b0 2024-09-13T16:29:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.404+0000 7fa452ffd640 1 --2- 172.21.15.16:0/3366790135 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa43c0a1190 0x7fa43c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.405+0000 7fa459bb7640 1 --2- 172.21.15.16:0/3366790135 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa43c0a1190 0x7fa43c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.405+0000 7fa452ffd640 1 -- 172.21.15.16:0/3366790135 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa4480ccd70 con 0x7fa4541018b0 2024-09-13T16:29:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.405+0000 7fa452ffd640 1 --2- 172.21.15.16:0/3366790135 >> [v2:172.21.15.16:6802/2079394186,v1:172.21.15.16:6803/2079394186] conn(0x7fa43c0ac220 0x7fa43c0ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.405+0000 7fa452ffd640 1 -- 172.21.15.16:0/3366790135 --> [v2:172.21.15.16:6802/2079394186,v1:172.21.15.16:6803/2079394186] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa43c0aed90 con 0x7fa43c0ac220 2024-09-13T16:29:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.405+0000 7fa452ffd640 1 -- 172.21.15.16:0/3366790135 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4480cd0e0 con 0x7fa4541018b0 2024-09-13T16:29:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.405+0000 7fa45a3b8640 1 --2- 172.21.15.16:0/3366790135 >> [v2:172.21.15.16:6802/2079394186,v1:172.21.15.16:6803/2079394186] conn(0x7fa43c0ac220 0x7fa43c0ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.405+0000 7fa459bb7640 1 --2- 172.21.15.16:0/3366790135 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa43c0a1190 0x7fa43c0a3650 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa44402af40 tx=0x7fa44403d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:40.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.405+0000 7fa45a3b8640 1 --2- 172.21.15.16:0/3366790135 >> [v2:172.21.15.16:6802/2079394186,v1:172.21.15.16:6803/2079394186] conn(0x7fa43c0ac220 0x7fa43c0ae6c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:40.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.406+0000 7fa452ffd640 1 -- 172.21.15.16:0/3366790135 <== osd.0 v2:172.21.15.16:6802/2079394186 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fa43c0aed90 con 0x7fa43c0ac220 2024-09-13T16:29:40.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.420+0000 7fa45be1b640 1 -- 172.21.15.16:0/3366790135 --> [v2:172.21.15.16:6802/2079394186,v1:172.21.15.16:6803/2079394186] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa454060b30 con 0x7fa43c0ac220 2024-09-13T16:29:40.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.420+0000 7fa452ffd640 1 -- 172.21.15.16:0/3366790135 <== osd.0 v2:172.21.15.16:6802/2079394186 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa454060b30 con 0x7fa43c0ac220 2024-09-13T16:29:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.420+0000 7fa45be1b640 1 -- 172.21.15.16:0/3366790135 >> [v2:172.21.15.16:6802/2079394186,v1:172.21.15.16:6803/2079394186] conn(0x7fa43c0ac220 msgr2=0x7fa43c0ae6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.420+0000 7fa45be1b640 1 --2- 172.21.15.16:0/3366790135 >> [v2:172.21.15.16:6802/2079394186,v1:172.21.15.16:6803/2079394186] conn(0x7fa43c0ac220 0x7fa43c0ae6c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.420+0000 7fa45be1b640 1 -- 172.21.15.16:0/3366790135 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa43c0a1190 msgr2=0x7fa43c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.420+0000 7fa45be1b640 1 --2- 172.21.15.16:0/3366790135 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa43c0a1190 0x7fa43c0a3650 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa44402af40 tx=0x7fa44403d040 comp rx=0 tx=0).stop 2024-09-13T16:29:40.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.420+0000 7fa45be1b640 1 -- 172.21.15.16:0/3366790135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4541018b0 msgr2=0x7fa454196770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:40.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.420+0000 7fa45be1b640 1 --2- 172.21.15.16:0/3366790135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4541018b0 0x7fa454196770 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa44800d950 tx=0x7fa44800de10 comp rx=0 tx=0).stop 2024-09-13T16:29:40.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.421+0000 7fa45a3b8640 1 -- 172.21.15.16:0/3366790135 reap_dead start 2024-09-13T16:29:40.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.421+0000 7fa45be1b640 1 -- 172.21.15.16:0/3366790135 shutdown_connections 2024-09-13T16:29:40.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.421+0000 7fa45be1b640 1 -- 172.21.15.16:0/3366790135 >> 172.21.15.16:0/3366790135 conn(0x7fa4540fa9b0 msgr2=0x7fa4540fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:40.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.421+0000 7fa45be1b640 1 -- 172.21.15.16:0/3366790135 shutdown_connections 2024-09-13T16:29:40.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:40.421+0000 7fa45be1b640 1 -- 172.21.15.16:0/3366790135 wait complete. 2024-09-13T16:29:41.183 INFO:teuthology.orchestra.run.smithi016.stdout:34359738444 2024-09-13T16:29:41.183 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-13T16:29:41.384 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:41 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:29:41.384 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:41 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:29:41.384 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:41 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:29:41.384 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:41 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:29:41.384 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:41 smithi016 ceph-mon[31187]: pgmap v309: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-13T16:29:41.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.354+0000 7fbf08f36640 1 -- 172.21.15.16:0/3149524513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf04100f30 msgr2=0x7fbf04101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:41.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.354+0000 7fbf08f36640 1 --2- 172.21.15.16:0/3149524513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf04100f30 0x7fbf04101310 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbefc009ca0 tx=0x7fbefc02a950 comp rx=0 tx=0).stop 2024-09-13T16:29:41.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.354+0000 7fbf08f36640 1 -- 172.21.15.16:0/3149524513 shutdown_connections 2024-09-13T16:29:41.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.354+0000 7fbf08f36640 1 --2- 172.21.15.16:0/3149524513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf0410a360 0x7fbf0410c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:41.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.354+0000 7fbf08f36640 1 --2- 172.21.15.16:0/3149524513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf04101850 0x7fbf04109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:41.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.354+0000 7fbf08f36640 1 --2- 172.21.15.16:0/3149524513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf04100f30 0x7fbf04101310 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:41.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.354+0000 7fbf08f36640 1 -- 172.21.15.16:0/3149524513 >> 172.21.15.16:0/3149524513 conn(0x7fbf040fa9b0 msgr2=0x7fbf040fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:41.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.354+0000 7fbf08f36640 1 -- 172.21.15.16:0/3149524513 shutdown_connections 2024-09-13T16:29:41.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.354+0000 7fbf08f36640 1 -- 172.21.15.16:0/3149524513 wait complete. 2024-09-13T16:29:41.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.354+0000 7fbf08f36640 1 Processor -- start 2024-09-13T16:29:41.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.354+0000 7fbf08f36640 1 -- start start 2024-09-13T16:29:41.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.355+0000 7fbf08f36640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf04100f30 0x7fbf041961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:41.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.355+0000 7fbf08f36640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf04101850 0x7fbf04196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:41.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.355+0000 7fbf08f36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf0410a360 0x7fbf0419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:41.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.355+0000 7fbf08f36640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0419b100 con 0x7fbf0410a360 2024-09-13T16:29:41.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.355+0000 7fbf08f36640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0419b270 con 0x7fbf04101850 2024-09-13T16:29:41.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.355+0000 7fbf08f36640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0419b3e0 con 0x7fbf04100f30 2024-09-13T16:29:41.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.355+0000 7fbf02d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf0410a360 0x7fbf0419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.355+0000 7fbf02d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf0410a360 0x7fbf0419ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42900/0 (socket says 172.21.15.16:42900) 2024-09-13T16:29:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.355+0000 7fbf02d9d640 1 -- 172.21.15.16:0/2447496025 learned_addr learned my addr 172.21.15.16:0/2447496025 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.355+0000 7fbf0259c640 1 --2- 172.21.15.16:0/2447496025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf04100f30 0x7fbf041961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.355+0000 7fbf01d9b640 1 --2- 172.21.15.16:0/2447496025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf04101850 0x7fbf04196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.355+0000 7fbf02d9d640 1 -- 172.21.15.16:0/2447496025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf04100f30 msgr2=0x7fbf041961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.355+0000 7fbf02d9d640 1 --2- 172.21.15.16:0/2447496025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf04100f30 0x7fbf041961c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.355+0000 7fbf02d9d640 1 -- 172.21.15.16:0/2447496025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf04101850 msgr2=0x7fbf04196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.355+0000 7fbf02d9d640 1 --2- 172.21.15.16:0/2447496025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf04101850 0x7fbf04196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.355+0000 7fbf02d9d640 1 -- 172.21.15.16:0/2447496025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbefc009900 con 0x7fbf0410a360 2024-09-13T16:29:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.355+0000 7fbf01d9b640 1 --2- 172.21.15.16:0/2447496025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf04101850 0x7fbf04196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:29:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.355+0000 7fbf02d9d640 1 --2- 172.21.15.16:0/2447496025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf0410a360 0x7fbf0419ab00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbef400d750 tx=0x7fbef400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.356+0000 7fbefb7fe640 1 -- 172.21.15.16:0/2447496025 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef4009bd0 con 0x7fbf0410a360 2024-09-13T16:29:41.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.356+0000 7fbf08f36640 1 -- 172.21.15.16:0/2447496025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf0419b6c0 con 0x7fbf0410a360 2024-09-13T16:29:41.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.356+0000 7fbefb7fe640 1 -- 172.21.15.16:0/2447496025 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbef400de60 con 0x7fbf0410a360 2024-09-13T16:29:41.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.356+0000 7fbefb7fe640 1 -- 172.21.15.16:0/2447496025 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef4004cf0 con 0x7fbf0410a360 2024-09-13T16:29:41.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.356+0000 7fbf08f36640 1 -- 172.21.15.16:0/2447496025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf041a2f80 con 0x7fbf0410a360 2024-09-13T16:29:41.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.356+0000 7fbf08f36640 1 -- 172.21.15.16:0/2447496025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbf040687e0 con 0x7fbf0410a360 2024-09-13T16:29:41.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.358+0000 7fbefb7fe640 1 -- 172.21.15.16:0/2447496025 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fbef4004e50 con 0x7fbf0410a360 2024-09-13T16:29:41.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.358+0000 7fbefb7fe640 1 --2- 172.21.15.16:0/2447496025 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fbef00a1110 0x7fbef00a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:41.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.359+0000 7fbf0259c640 1 --2- 172.21.15.16:0/2447496025 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fbef00a1110 0x7fbef00a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:41.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.359+0000 7fbefb7fe640 1 -- 172.21.15.16:0/2447496025 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbef40ccaa0 con 0x7fbf0410a360 2024-09-13T16:29:41.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.359+0000 7fbefb7fe640 1 --2- 172.21.15.16:0/2447496025 >> [v2:172.21.15.16:6810/2390375937,v1:172.21.15.16:6811/2390375937] conn(0x7fbef00ac1a0 0x7fbef00ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:41.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.359+0000 7fbefb7fe640 1 -- 172.21.15.16:0/2447496025 --> [v2:172.21.15.16:6810/2390375937,v1:172.21.15.16:6811/2390375937] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbef00aed10 con 0x7fbef00ac1a0 2024-09-13T16:29:41.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.359+0000 7fbefb7fe640 1 -- 172.21.15.16:0/2447496025 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbef40cceb0 con 0x7fbf0410a360 2024-09-13T16:29:41.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.359+0000 7fbf01d9b640 1 --2- 172.21.15.16:0/2447496025 >> [v2:172.21.15.16:6810/2390375937,v1:172.21.15.16:6811/2390375937] conn(0x7fbef00ac1a0 0x7fbef00ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:41.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.359+0000 7fbf0259c640 1 --2- 172.21.15.16:0/2447496025 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fbef00a1110 0x7fbef00a35d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbefc02af40 tx=0x7fbefc03d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:41.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.359+0000 7fbf01d9b640 1 --2- 172.21.15.16:0/2447496025 >> [v2:172.21.15.16:6810/2390375937,v1:172.21.15.16:6811/2390375937] conn(0x7fbef00ac1a0 0x7fbef00ae640 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:41.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.360+0000 7fbefb7fe640 1 -- 172.21.15.16:0/2447496025 <== osd.3 v2:172.21.15.16:6810/2390375937 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fbef00aed10 con 0x7fbef00ac1a0 2024-09-13T16:29:41.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.373+0000 7fbf08f36640 1 -- 172.21.15.16:0/2447496025 --> [v2:172.21.15.16:6810/2390375937,v1:172.21.15.16:6811/2390375937] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbf0410ff70 con 0x7fbef00ac1a0 2024-09-13T16:29:41.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.374+0000 7fbefb7fe640 1 -- 172.21.15.16:0/2447496025 <== osd.3 v2:172.21.15.16:6810/2390375937 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbeec0086c0 con 0x7fbef00ac1a0 2024-09-13T16:29:41.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.374+0000 7fbf08f36640 1 -- 172.21.15.16:0/2447496025 >> [v2:172.21.15.16:6810/2390375937,v1:172.21.15.16:6811/2390375937] conn(0x7fbef00ac1a0 msgr2=0x7fbef00ae640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:41.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.374+0000 7fbf08f36640 1 --2- 172.21.15.16:0/2447496025 >> [v2:172.21.15.16:6810/2390375937,v1:172.21.15.16:6811/2390375937] conn(0x7fbef00ac1a0 0x7fbef00ae640 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:41.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.374+0000 7fbf08f36640 1 -- 172.21.15.16:0/2447496025 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fbef00a1110 msgr2=0x7fbef00a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:41.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.374+0000 7fbf08f36640 1 --2- 172.21.15.16:0/2447496025 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fbef00a1110 0x7fbef00a35d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbefc02af40 tx=0x7fbefc03d040 comp rx=0 tx=0).stop 2024-09-13T16:29:41.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.374+0000 7fbf08f36640 1 -- 172.21.15.16:0/2447496025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf0410a360 msgr2=0x7fbf0419ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:41.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.374+0000 7fbf08f36640 1 --2- 172.21.15.16:0/2447496025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf0410a360 0x7fbf0419ab00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbef400d750 tx=0x7fbef400dc10 comp rx=0 tx=0).stop 2024-09-13T16:29:41.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.374+0000 7fbf02d9d640 1 -- 172.21.15.16:0/2447496025 reap_dead start 2024-09-13T16:29:41.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.374+0000 7fbf08f36640 1 -- 172.21.15.16:0/2447496025 shutdown_connections 2024-09-13T16:29:41.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.374+0000 7fbf08f36640 1 -- 172.21.15.16:0/2447496025 >> 172.21.15.16:0/2447496025 conn(0x7fbf040fa9b0 msgr2=0x7fbf040fc310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:41.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.374+0000 7fbf08f36640 1 -- 172.21.15.16:0/2447496025 shutdown_connections 2024-09-13T16:29:41.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.374+0000 7fbf08f36640 1 -- 172.21.15.16:0/2447496025 wait complete. 2024-09-13T16:29:41.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:41 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:29:41.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:41 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:29:41.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:41 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:29:41.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:41 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:29:41.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:41 smithi111 ceph-mon[34600]: pgmap v309: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-13T16:29:41.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:41 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:29:41.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:41 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:29:41.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:41 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:29:41.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:41 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:29:41.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:41 smithi107 ceph-mon[34139]: pgmap v309: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-13T16:29:41.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.812+0000 7f89efdf0640 1 -- 172.21.15.16:0/3886420265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89e80ff670 msgr2=0x7f89e8103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:41.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.812+0000 7f89efdf0640 1 --2- 172.21.15.16:0/3886420265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89e80ff670 0x7f89e8103b40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f89d8009900 tx=0x7f89d802a470 comp rx=0 tx=0).stop 2024-09-13T16:29:41.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.812+0000 7f89efdf0640 1 -- 172.21.15.16:0/3886420265 shutdown_connections 2024-09-13T16:29:41.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.812+0000 7f89efdf0640 1 --2- 172.21.15.16:0/3886420265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89e8104160 0x7f89e81045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:41.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.812+0000 7f89efdf0640 1 --2- 172.21.15.16:0/3886420265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89e80ff670 0x7f89e8103b40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:41.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.812+0000 7f89efdf0640 1 --2- 172.21.15.16:0/3886420265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89e80fed50 0x7f89e80ff130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:41.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.812+0000 7f89efdf0640 1 -- 172.21.15.16:0/3886420265 >> 172.21.15.16:0/3886420265 conn(0x7f89e80fa9a0 msgr2=0x7f89e80fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.812+0000 7f89efdf0640 1 -- 172.21.15.16:0/3886420265 shutdown_connections 2024-09-13T16:29:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.812+0000 7f89efdf0640 1 -- 172.21.15.16:0/3886420265 wait complete. 2024-09-13T16:29:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.813+0000 7f89efdf0640 1 Processor -- start 2024-09-13T16:29:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.813+0000 7f89efdf0640 1 -- start start 2024-09-13T16:29:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.813+0000 7f89efdf0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89e80fed50 0x7f89e81a2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.813+0000 7f89efdf0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89e80ff670 0x7f89e81a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.813+0000 7f89efdf0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89e8104160 0x7f89e819c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.813+0000 7f89efdf0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e819ccc0 con 0x7f89e8104160 2024-09-13T16:29:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.813+0000 7f89efdf0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e819ce30 con 0x7f89e80ff670 2024-09-13T16:29:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.813+0000 7f89efdf0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e819cff0 con 0x7f89e80fed50 2024-09-13T16:29:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.814+0000 7f89ee38d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89e8104160 0x7f89e819c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.814+0000 7f89ee38d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89e8104160 0x7f89e819c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42914/0 (socket says 172.21.15.16:42914) 2024-09-13T16:29:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.814+0000 7f89ee38d640 1 -- 172.21.15.16:0/545064529 learned_addr learned my addr 172.21.15.16:0/545064529 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.814+0000 7f89ed38b640 1 --2- 172.21.15.16:0/545064529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89e80ff670 0x7f89e81a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.814+0000 7f89edb8c640 1 --2- 172.21.15.16:0/545064529 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89e80fed50 0x7f89e81a2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.814+0000 7f89ee38d640 1 -- 172.21.15.16:0/545064529 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89e80fed50 msgr2=0x7f89e81a2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.814+0000 7f89ee38d640 1 --2- 172.21.15.16:0/545064529 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89e80fed50 0x7f89e81a2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.814+0000 7f89ee38d640 1 -- 172.21.15.16:0/545064529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89e80ff670 msgr2=0x7f89e81a2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.814+0000 7f89ee38d640 1 --2- 172.21.15.16:0/545064529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89e80ff670 0x7f89e81a2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.814+0000 7f89ee38d640 1 -- 172.21.15.16:0/545064529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d8009560 con 0x7f89e8104160 2024-09-13T16:29:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.814+0000 7f89ed38b640 1 --2- 172.21.15.16:0/545064529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89e80ff670 0x7f89e81a2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:29:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.814+0000 7f89ee38d640 1 --2- 172.21.15.16:0/545064529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89e8104160 0x7f89e819c670 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f89e000cad0 tx=0x7f89e000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.814+0000 7f89deffd640 1 -- 172.21.15.16:0/545064529 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e000ae20 con 0x7f89e8104160 2024-09-13T16:29:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.814+0000 7f89deffd640 1 -- 172.21.15.16:0/545064529 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f89e0005020 con 0x7f89e8104160 2024-09-13T16:29:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.814+0000 7f89efdf0640 1 -- 172.21.15.16:0/545064529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e819d2d0 con 0x7f89e8104160 2024-09-13T16:29:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.814+0000 7f89deffd640 1 -- 172.21.15.16:0/545064529 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e000f6b0 con 0x7f89e8104160 2024-09-13T16:29:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.814+0000 7f89efdf0640 1 -- 172.21.15.16:0/545064529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e81a9520 con 0x7f89e8104160 2024-09-13T16:29:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.815+0000 7f89efdf0640 1 -- 172.21.15.16:0/545064529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f89e8116550 con 0x7f89e8104160 2024-09-13T16:29:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.816+0000 7f89deffd640 1 -- 172.21.15.16:0/545064529 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f89e0005190 con 0x7f89e8104160 2024-09-13T16:29:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.817+0000 7f89deffd640 1 --2- 172.21.15.16:0/545064529 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f89d40a1110 0x7f89d40a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.817+0000 7f89edb8c640 1 --2- 172.21.15.16:0/545064529 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f89d40a1110 0x7f89d40a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.817+0000 7f89deffd640 1 -- 172.21.15.16:0/545064529 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f89e00cd1e0 con 0x7f89e8104160 2024-09-13T16:29:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.817+0000 7f89deffd640 1 --2- 172.21.15.16:0/545064529 >> [v2:172.21.15.16:6826/4175383686,v1:172.21.15.16:6827/4175383686] conn(0x7f89d40ac1a0 0x7f89d40ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.817+0000 7f89deffd640 1 -- 172.21.15.16:0/545064529 --> [v2:172.21.15.16:6826/4175383686,v1:172.21.15.16:6827/4175383686] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f89d40aed10 con 0x7f89d40ac1a0 2024-09-13T16:29:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.817+0000 7f89deffd640 1 -- 172.21.15.16:0/545064529 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89e00cd5f0 con 0x7f89e8104160 2024-09-13T16:29:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.818+0000 7f89ed38b640 1 --2- 172.21.15.16:0/545064529 >> [v2:172.21.15.16:6826/4175383686,v1:172.21.15.16:6827/4175383686] conn(0x7f89d40ac1a0 0x7f89d40ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.818+0000 7f89edb8c640 1 --2- 172.21.15.16:0/545064529 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f89d40a1110 0x7f89d40a35d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f89e4009b60 tx=0x7f89e4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.818+0000 7f89ed38b640 1 --2- 172.21.15.16:0/545064529 >> [v2:172.21.15.16:6826/4175383686,v1:172.21.15.16:6827/4175383686] conn(0x7f89d40ac1a0 0x7f89d40ae640 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.819+0000 7f89deffd640 1 -- 172.21.15.16:0/545064529 <== osd.9 v2:172.21.15.16:6826/4175383686 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f89d40aed10 con 0x7f89d40ac1a0 2024-09-13T16:29:41.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.834+0000 7f89efdf0640 1 -- 172.21.15.16:0/545064529 --> [v2:172.21.15.16:6826/4175383686,v1:172.21.15.16:6827/4175383686] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f89e8069000 con 0x7f89d40ac1a0 2024-09-13T16:29:41.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.834+0000 7f89deffd640 1 -- 172.21.15.16:0/545064529 <== osd.9 v2:172.21.15.16:6826/4175383686 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f89e8069000 con 0x7f89d40ac1a0 2024-09-13T16:29:41.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.834+0000 7f89efdf0640 1 -- 172.21.15.16:0/545064529 >> [v2:172.21.15.16:6826/4175383686,v1:172.21.15.16:6827/4175383686] conn(0x7f89d40ac1a0 msgr2=0x7f89d40ae640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:41.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.834+0000 7f89efdf0640 1 --2- 172.21.15.16:0/545064529 >> [v2:172.21.15.16:6826/4175383686,v1:172.21.15.16:6827/4175383686] conn(0x7f89d40ac1a0 0x7f89d40ae640 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:41.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.834+0000 7f89efdf0640 1 -- 172.21.15.16:0/545064529 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f89d40a1110 msgr2=0x7f89d40a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:41.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.834+0000 7f89efdf0640 1 --2- 172.21.15.16:0/545064529 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f89d40a1110 0x7f89d40a35d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f89e4009b60 tx=0x7f89e4005f90 comp rx=0 tx=0).stop 2024-09-13T16:29:41.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.834+0000 7f89efdf0640 1 -- 172.21.15.16:0/545064529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89e8104160 msgr2=0x7f89e819c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:41.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.834+0000 7f89efdf0640 1 --2- 172.21.15.16:0/545064529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89e8104160 0x7f89e819c670 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f89e000cad0 tx=0x7f89e000cf90 comp rx=0 tx=0).stop 2024-09-13T16:29:41.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.834+0000 7f89ee38d640 1 -- 172.21.15.16:0/545064529 reap_dead start 2024-09-13T16:29:41.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.835+0000 7f89efdf0640 1 -- 172.21.15.16:0/545064529 shutdown_connections 2024-09-13T16:29:41.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.835+0000 7f89efdf0640 1 -- 172.21.15.16:0/545064529 >> 172.21.15.16:0/545064529 conn(0x7f89e80fa9a0 msgr2=0x7f89e810d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:41.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.835+0000 7f89efdf0640 1 -- 172.21.15.16:0/545064529 shutdown_connections 2024-09-13T16:29:41.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:41.835+0000 7f89efdf0640 1 -- 172.21.15.16:0/545064529 wait complete. 2024-09-13T16:29:42.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.417+0000 7fc1f5735640 1 -- 172.21.15.16:0/894035110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1f0100990 msgr2=0x7fc1f01049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.417+0000 7fc1f5735640 1 --2- 172.21.15.16:0/894035110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1f0100990 0x7fc1f01049e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8009930 tx=0x7fc1d802a420 comp rx=0 tx=0).stop 2024-09-13T16:29:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.417+0000 7fc1f5735640 1 -- 172.21.15.16:0/894035110 shutdown_connections 2024-09-13T16:29:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.417+0000 7fc1f5735640 1 --2- 172.21.15.16:0/894035110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1f0105110 0x7fc1f010cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.417+0000 7fc1f5735640 1 --2- 172.21.15.16:0/894035110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1f0100990 0x7fc1f01049e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.417+0000 7fc1f5735640 1 --2- 172.21.15.16:0/894035110 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1f00fffe0 0x7fc1f01003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.417+0000 7fc1f5735640 1 -- 172.21.15.16:0/894035110 >> 172.21.15.16:0/894035110 conn(0x7fc1f00fbae0 msgr2=0x7fc1f00fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.417+0000 7fc1f5735640 1 -- 172.21.15.16:0/894035110 shutdown_connections 2024-09-13T16:29:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.418+0000 7fc1f5735640 1 -- 172.21.15.16:0/894035110 wait complete. 2024-09-13T16:29:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.418+0000 7fc1f5735640 1 Processor -- start 2024-09-13T16:29:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.418+0000 7fc1f5735640 1 -- start start 2024-09-13T16:29:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.419+0000 7fc1f5735640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1f00fffe0 0x7fc1f00727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.419+0000 7fc1f5735640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1f0100990 0x7fc1f0072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.419+0000 7fc1f5735640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1f0105110 0x7fc1f0073230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.419+0000 7fc1f5735640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f01a4510 con 0x7fc1f0100990 2024-09-13T16:29:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.419+0000 7fc1f5735640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f01a4680 con 0x7fc1f00fffe0 2024-09-13T16:29:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.419+0000 7fc1f5735640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f01a4840 con 0x7fc1f0105110 2024-09-13T16:29:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.419+0000 7fc1ee7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1f0100990 0x7fc1f0072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.419+0000 7fc1ee7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1f0100990 0x7fc1f0072cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42934/0 (socket says 172.21.15.16:42934) 2024-09-13T16:29:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.419+0000 7fc1ee7fc640 1 -- 172.21.15.16:0/4238276435 learned_addr learned my addr 172.21.15.16:0/4238276435 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.419+0000 7fc1eeffd640 1 --2- 172.21.15.16:0/4238276435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1f00fffe0 0x7fc1f00727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.419+0000 7fc1ef7fe640 1 --2- 172.21.15.16:0/4238276435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1f0105110 0x7fc1f0073230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.419+0000 7fc1ee7fc640 1 -- 172.21.15.16:0/4238276435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1f0105110 msgr2=0x7fc1f0073230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.419+0000 7fc1ee7fc640 1 --2- 172.21.15.16:0/4238276435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1f0105110 0x7fc1f0073230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.419+0000 7fc1ee7fc640 1 -- 172.21.15.16:0/4238276435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1f00fffe0 msgr2=0x7fc1f00727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.419+0000 7fc1ee7fc640 1 --2- 172.21.15.16:0/4238276435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1f00fffe0 0x7fc1f00727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.419+0000 7fc1ee7fc640 1 -- 172.21.15.16:0/4238276435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d8009560 con 0x7fc1f0100990 2024-09-13T16:29:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.419+0000 7fc1ee7fc640 1 --2- 172.21.15.16:0/4238276435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1f0100990 0x7fc1f0072cf0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc1d802aa10 tx=0x7fc1d8032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.419+0000 7fc1e7fff640 1 -- 172.21.15.16:0/4238276435 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d8005210 con 0x7fc1f0100990 2024-09-13T16:29:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.419+0000 7fc1e7fff640 1 -- 172.21.15.16:0/4238276435 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1d8005370 con 0x7fc1f0100990 2024-09-13T16:29:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.419+0000 7fc1f5735640 1 -- 172.21.15.16:0/4238276435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f01a4ac0 con 0x7fc1f0100990 2024-09-13T16:29:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.419+0000 7fc1e7fff640 1 -- 172.21.15.16:0/4238276435 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d80038e0 con 0x7fc1f0100990 2024-09-13T16:29:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.420+0000 7fc1f5735640 1 -- 172.21.15.16:0/4238276435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f01a4f60 con 0x7fc1f0100990 2024-09-13T16:29:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.420+0000 7fc1f5735640 1 -- 172.21.15.16:0/4238276435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc1f010ff50 con 0x7fc1f0100990 2024-09-13T16:29:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.422+0000 7fc1e7fff640 1 -- 172.21.15.16:0/4238276435 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc1d8003a40 con 0x7fc1f0100990 2024-09-13T16:29:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.422+0000 7fc1e7fff640 1 --2- 172.21.15.16:0/4238276435 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc1dc0a1110 0x7fc1dc0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.422+0000 7fc1e7fff640 1 -- 172.21.15.16:0/4238276435 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc1d80eded0 con 0x7fc1f0100990 2024-09-13T16:29:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.422+0000 7fc1eeffd640 1 --2- 172.21.15.16:0/4238276435 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc1dc0a1110 0x7fc1dc0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:42.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.423+0000 7fc1e7fff640 1 --2- 172.21.15.16:0/4238276435 >> [v2:172.21.15.107:6808/1936582048,v1:172.21.15.107:6809/1936582048] conn(0x7fc1dc0ac1a0 0x7fc1dc0ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:42.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.423+0000 7fc1e7fff640 1 -- 172.21.15.16:0/4238276435 --> [v2:172.21.15.107:6808/1936582048,v1:172.21.15.107:6809/1936582048] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc1dc0aed10 con 0x7fc1dc0ac1a0 2024-09-13T16:29:42.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.423+0000 7fc1e7fff640 1 -- 172.21.15.16:0/4238276435 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1d80ee2e0 con 0x7fc1f0100990 2024-09-13T16:29:42.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.423+0000 7fc1eeffd640 1 --2- 172.21.15.16:0/4238276435 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc1dc0a1110 0x7fc1dc0a35d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8007ec0 tx=0x7fc1e80075b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:42.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.423+0000 7fc1ef7fe640 1 --2- 172.21.15.16:0/4238276435 >> [v2:172.21.15.107:6808/1936582048,v1:172.21.15.107:6809/1936582048] conn(0x7fc1dc0ac1a0 0x7fc1dc0ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:42.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.423+0000 7fc1ef7fe640 1 --2- 172.21.15.16:0/4238276435 >> [v2:172.21.15.107:6808/1936582048,v1:172.21.15.107:6809/1936582048] conn(0x7fc1dc0ac1a0 0x7fc1dc0ae640 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:42.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.424+0000 7fc1e7fff640 1 -- 172.21.15.16:0/4238276435 <== osd.4 v2:172.21.15.107:6808/1936582048 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fc1dc0aed10 con 0x7fc1dc0ac1a0 2024-09-13T16:29:42.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.438+0000 7fc1f5735640 1 -- 172.21.15.16:0/4238276435 --> [v2:172.21.15.107:6808/1936582048,v1:172.21.15.107:6809/1936582048] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc1f0060bb0 con 0x7fc1dc0ac1a0 2024-09-13T16:29:42.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.438+0000 7fc1e7fff640 1 -- 172.21.15.16:0/4238276435 <== osd.4 v2:172.21.15.107:6808/1936582048 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc1f0060bb0 con 0x7fc1dc0ac1a0 2024-09-13T16:29:42.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.439+0000 7fc1f5735640 1 -- 172.21.15.16:0/4238276435 >> [v2:172.21.15.107:6808/1936582048,v1:172.21.15.107:6809/1936582048] conn(0x7fc1dc0ac1a0 msgr2=0x7fc1dc0ae640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:42.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.439+0000 7fc1f5735640 1 --2- 172.21.15.16:0/4238276435 >> [v2:172.21.15.107:6808/1936582048,v1:172.21.15.107:6809/1936582048] conn(0x7fc1dc0ac1a0 0x7fc1dc0ae640 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:42.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.439+0000 7fc1f5735640 1 -- 172.21.15.16:0/4238276435 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc1dc0a1110 msgr2=0x7fc1dc0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:42.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.439+0000 7fc1f5735640 1 --2- 172.21.15.16:0/4238276435 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc1dc0a1110 0x7fc1dc0a35d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8007ec0 tx=0x7fc1e80075b0 comp rx=0 tx=0).stop 2024-09-13T16:29:42.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.439+0000 7fc1f5735640 1 -- 172.21.15.16:0/4238276435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1f0100990 msgr2=0x7fc1f0072cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:42.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.439+0000 7fc1f5735640 1 --2- 172.21.15.16:0/4238276435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1f0100990 0x7fc1f0072cf0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc1d802aa10 tx=0x7fc1d8032c20 comp rx=0 tx=0).stop 2024-09-13T16:29:42.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.439+0000 7fc1ef7fe640 1 -- 172.21.15.16:0/4238276435 reap_dead start 2024-09-13T16:29:42.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.439+0000 7fc1f5735640 1 -- 172.21.15.16:0/4238276435 shutdown_connections 2024-09-13T16:29:42.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.439+0000 7fc1f5735640 1 -- 172.21.15.16:0/4238276435 >> 172.21.15.16:0/4238276435 conn(0x7fc1f00fbae0 msgr2=0x7fc1f00fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:42.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.439+0000 7fc1f5735640 1 -- 172.21.15.16:0/4238276435 shutdown_connections 2024-09-13T16:29:42.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.439+0000 7fc1f5735640 1 -- 172.21.15.16:0/4238276435 wait complete. 2024-09-13T16:29:42.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:42 smithi111 ceph-mon[34600]: pgmap v310: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-13T16:29:42.954 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:42 smithi016 ceph-mon[31187]: pgmap v310: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-13T16:29:43.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:42 smithi107 ceph-mon[34139]: pgmap v310: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-13T16:29:44.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:44 smithi016 ceph-mon[31187]: pgmap v311: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-13T16:29:44.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:44 smithi111 ceph-mon[34600]: pgmap v311: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-13T16:29:45.016 INFO:teuthology.orchestra.run.smithi016.stdout:107374182461 2024-09-13T16:29:45.016 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-13T16:29:45.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:44 smithi107 ceph-mon[34139]: pgmap v311: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-13T16:29:45.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.297+0000 7f201728d640 1 -- 172.21.15.16:0/4063723797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20100ff8b0 msgr2=0x7f2010103900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:45.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.297+0000 7f201728d640 1 --2- 172.21.15.16:0/4063723797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20100ff8b0 0x7f2010103900 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2000009930 tx=0x7f200002a420 comp rx=0 tx=0).stop 2024-09-13T16:29:45.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.297+0000 7f201728d640 1 -- 172.21.15.16:0/4063723797 shutdown_connections 2024-09-13T16:29:45.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.297+0000 7f201728d640 1 --2- 172.21.15.16:0/4063723797 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2010104030 0x7f201010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:45.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.297+0000 7f201728d640 1 --2- 172.21.15.16:0/4063723797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20100ff8b0 0x7f2010103900 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:45.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.297+0000 7f201728d640 1 --2- 172.21.15.16:0/4063723797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20100fef00 0x7f20100ff2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:45.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.297+0000 7f201728d640 1 -- 172.21.15.16:0/4063723797 >> 172.21.15.16:0/4063723797 conn(0x7f20100fa9a0 msgr2=0x7f20100fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:45.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.297+0000 7f201728d640 1 -- 172.21.15.16:0/4063723797 shutdown_connections 2024-09-13T16:29:45.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.297+0000 7f201728d640 1 -- 172.21.15.16:0/4063723797 wait complete. 2024-09-13T16:29:45.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.298+0000 7f201728d640 1 Processor -- start 2024-09-13T16:29:45.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.298+0000 7f201728d640 1 -- start start 2024-09-13T16:29:45.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.298+0000 7f201728d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20100fef00 0x7f2010197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:45.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.298+0000 7f201728d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20100ff8b0 0x7f2010197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:45.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.298+0000 7f201728d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2010104030 0x7f20101a2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:45.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.298+0000 7f201728d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2010197d80 con 0x7f20100ff8b0 2024-09-13T16:29:45.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.298+0000 7f201728d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20101a25d0 con 0x7f20100fef00 2024-09-13T16:29:45.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.298+0000 7f201728d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20101a2710 con 0x7f2010104030 2024-09-13T16:29:45.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.298+0000 7f2014828640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20100ff8b0 0x7f2010197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:45.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.298+0000 7f2014828640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20100ff8b0 0x7f2010197550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47734/0 (socket says 172.21.15.16:47734) 2024-09-13T16:29:45.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.298+0000 7f2014828640 1 -- 172.21.15.16:0/1453831043 learned_addr learned my addr 172.21.15.16:0/1453831043 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:45.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.298+0000 7f2015029640 1 --2- 172.21.15.16:0/1453831043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20100fef00 0x7f2010197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:45.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.298+0000 7f201582a640 1 --2- 172.21.15.16:0/1453831043 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2010104030 0x7f20101a2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:45.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.298+0000 7f2014828640 1 -- 172.21.15.16:0/1453831043 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2010104030 msgr2=0x7f20101a2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:45.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.298+0000 7f2014828640 1 --2- 172.21.15.16:0/1453831043 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2010104030 0x7f20101a2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:45.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.298+0000 7f2014828640 1 -- 172.21.15.16:0/1453831043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20100fef00 msgr2=0x7f2010197010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:45.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.298+0000 7f2014828640 1 --2- 172.21.15.16:0/1453831043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20100fef00 0x7f2010197010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:45.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.298+0000 7f2014828640 1 -- 172.21.15.16:0/1453831043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2000009560 con 0x7f20100ff8b0 2024-09-13T16:29:45.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.298+0000 7f2015029640 1 --2- 172.21.15.16:0/1453831043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20100fef00 0x7f2010197010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:29:45.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.299+0000 7f2014828640 1 --2- 172.21.15.16:0/1453831043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20100ff8b0 0x7f2010197550 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2000005ab0 tx=0x7f2000032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:45.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.299+0000 7f20067fc640 1 -- 172.21.15.16:0/1453831043 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2000032e10 con 0x7f20100ff8b0 2024-09-13T16:29:45.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.299+0000 7f20067fc640 1 -- 172.21.15.16:0/1453831043 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f20000046c0 con 0x7f20100ff8b0 2024-09-13T16:29:45.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.299+0000 7f201728d640 1 -- 172.21.15.16:0/1453831043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20101a2910 con 0x7f20100ff8b0 2024-09-13T16:29:45.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.299+0000 7f20067fc640 1 -- 172.21.15.16:0/1453831043 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2000003850 con 0x7f20100ff8b0 2024-09-13T16:29:45.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.299+0000 7f201728d640 1 -- 172.21.15.16:0/1453831043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20101a2d80 con 0x7f20100ff8b0 2024-09-13T16:29:45.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.300+0000 7f201728d640 1 -- 172.21.15.16:0/1453831043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f20101003e0 con 0x7f20100ff8b0 2024-09-13T16:29:45.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.301+0000 7f20067fc640 1 -- 172.21.15.16:0/1453831043 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f2000004830 con 0x7f20100ff8b0 2024-09-13T16:29:45.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.301+0000 7f20067fc640 1 --2- 172.21.15.16:0/1453831043 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f1ffc0a1110 0x7f1ffc0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:45.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.302+0000 7f20067fc640 1 -- 172.21.15.16:0/1453831043 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f20000edea0 con 0x7f20100ff8b0 2024-09-13T16:29:45.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.302+0000 7f2015029640 1 --2- 172.21.15.16:0/1453831043 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f1ffc0a1110 0x7f1ffc0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:45.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.302+0000 7f20067fc640 1 --2- 172.21.15.16:0/1453831043 >> [v2:172.21.15.111:6824/1633548683,v1:172.21.15.111:6825/1633548683] conn(0x7f1ffc0ac1a0 0x7f1ffc0ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:45.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.302+0000 7f20067fc640 1 -- 172.21.15.16:0/1453831043 --> [v2:172.21.15.111:6824/1633548683,v1:172.21.15.111:6825/1633548683] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1ffc0aed10 con 0x7f1ffc0ac1a0 2024-09-13T16:29:45.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.302+0000 7f20067fc640 1 -- 172.21.15.16:0/1453831043 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20000ee2b0 con 0x7f20100ff8b0 2024-09-13T16:29:45.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.302+0000 7f2015029640 1 --2- 172.21.15.16:0/1453831043 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f1ffc0a1110 0x7f1ffc0a35d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f200c0058c0 tx=0x7f200c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:45.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.302+0000 7f201582a640 1 --2- 172.21.15.16:0/1453831043 >> [v2:172.21.15.111:6824/1633548683,v1:172.21.15.111:6825/1633548683] conn(0x7f1ffc0ac1a0 0x7f1ffc0ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:45.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.303+0000 7f201582a640 1 --2- 172.21.15.16:0/1453831043 >> [v2:172.21.15.111:6824/1633548683,v1:172.21.15.111:6825/1633548683] conn(0x7f1ffc0ac1a0 0x7f1ffc0ae640 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:45.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.303+0000 7f20067fc640 1 -- 172.21.15.16:0/1453831043 <== osd.11 v2:172.21.15.111:6824/1633548683 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f1ffc0aed10 con 0x7f1ffc0ac1a0 2024-09-13T16:29:45.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.316+0000 7f201728d640 1 -- 172.21.15.16:0/1453831043 --> [v2:172.21.15.111:6824/1633548683,v1:172.21.15.111:6825/1633548683] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2010060b30 con 0x7f1ffc0ac1a0 2024-09-13T16:29:45.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.317+0000 7f20067fc640 1 -- 172.21.15.16:0/1453831043 <== osd.11 v2:172.21.15.111:6824/1633548683 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f2010060b30 con 0x7f1ffc0ac1a0 2024-09-13T16:29:45.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.317+0000 7f201728d640 1 -- 172.21.15.16:0/1453831043 >> [v2:172.21.15.111:6824/1633548683,v1:172.21.15.111:6825/1633548683] conn(0x7f1ffc0ac1a0 msgr2=0x7f1ffc0ae640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:45.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.317+0000 7f201728d640 1 --2- 172.21.15.16:0/1453831043 >> [v2:172.21.15.111:6824/1633548683,v1:172.21.15.111:6825/1633548683] conn(0x7f1ffc0ac1a0 0x7f1ffc0ae640 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:45.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.317+0000 7f201728d640 1 -- 172.21.15.16:0/1453831043 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f1ffc0a1110 msgr2=0x7f1ffc0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:45.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.317+0000 7f201728d640 1 --2- 172.21.15.16:0/1453831043 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f1ffc0a1110 0x7f1ffc0a35d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f200c0058c0 tx=0x7f200c005850 comp rx=0 tx=0).stop 2024-09-13T16:29:45.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.317+0000 7f201728d640 1 -- 172.21.15.16:0/1453831043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20100ff8b0 msgr2=0x7f2010197550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:45.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.317+0000 7f201728d640 1 --2- 172.21.15.16:0/1453831043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20100ff8b0 0x7f2010197550 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2000005ab0 tx=0x7f2000032c20 comp rx=0 tx=0).stop 2024-09-13T16:29:45.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.317+0000 7f201582a640 1 -- 172.21.15.16:0/1453831043 reap_dead start 2024-09-13T16:29:45.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.318+0000 7f201728d640 1 -- 172.21.15.16:0/1453831043 shutdown_connections 2024-09-13T16:29:45.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.318+0000 7f201728d640 1 -- 172.21.15.16:0/1453831043 >> 172.21.15.16:0/1453831043 conn(0x7f20100fa9a0 msgr2=0x7f20100fbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:45.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.318+0000 7f201728d640 1 -- 172.21.15.16:0/1453831043 shutdown_connections 2024-09-13T16:29:45.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.318+0000 7f201728d640 1 -- 172.21.15.16:0/1453831043 wait complete. 2024-09-13T16:29:45.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.507+0000 7f379a2c1640 1 -- 172.21.15.16:0/3993914769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3794101880 msgr2=0x7f3794109d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:45.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.507+0000 7f379a2c1640 1 --2- 172.21.15.16:0/3993914769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3794101880 0x7f3794109d70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f377c009900 tx=0x7f377c02a480 comp rx=0 tx=0).stop 2024-09-13T16:29:45.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.507+0000 7f379a2c1640 1 -- 172.21.15.16:0/3993914769 shutdown_connections 2024-09-13T16:29:45.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.507+0000 7f379a2c1640 1 --2- 172.21.15.16:0/3993914769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f379410a390 0x7f379410c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:45.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.507+0000 7f379a2c1640 1 --2- 172.21.15.16:0/3993914769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3794101880 0x7f3794109d70 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:45.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.507+0000 7f379a2c1640 1 --2- 172.21.15.16:0/3993914769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3794100f60 0x7f3794101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:45.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.507+0000 7f379a2c1640 1 -- 172.21.15.16:0/3993914769 >> 172.21.15.16:0/3993914769 conn(0x7f37940fa9a0 msgr2=0x7f37940fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:45.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.507+0000 7f379a2c1640 1 -- 172.21.15.16:0/3993914769 shutdown_connections 2024-09-13T16:29:45.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.507+0000 7f379a2c1640 1 -- 172.21.15.16:0/3993914769 wait complete. 2024-09-13T16:29:45.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.508+0000 7f379a2c1640 1 Processor -- start 2024-09-13T16:29:45.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.508+0000 7f379a2c1640 1 -- start start 2024-09-13T16:29:45.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.508+0000 7f379a2c1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3794100f60 0x7f37940febc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:45.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.508+0000 7f379a2c1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3794101880 0x7f37940ff100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:45.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.508+0000 7f379a2c1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f379410a390 0x7f37940ff640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:45.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.508+0000 7f379a2c1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37941a4600 con 0x7f3794100f60 2024-09-13T16:29:45.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.508+0000 7f379a2c1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37941a4770 con 0x7f379410a390 2024-09-13T16:29:45.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.508+0000 7f379a2c1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37941a4930 con 0x7f3794101880 2024-09-13T16:29:45.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.508+0000 7f3793fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3794100f60 0x7f37940febc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:45.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.508+0000 7f3793fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3794100f60 0x7f37940febc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47752/0 (socket says 172.21.15.16:47752) 2024-09-13T16:29:45.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.508+0000 7f3793fff640 1 -- 172.21.15.16:0/3779703407 learned_addr learned my addr 172.21.15.16:0/3779703407 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:45.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.508+0000 7f379885e640 1 --2- 172.21.15.16:0/3779703407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f379410a390 0x7f37940ff640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:45.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.508+0000 7f378bfff640 1 --2- 172.21.15.16:0/3779703407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3794101880 0x7f37940ff100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:45.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.509+0000 7f3793fff640 1 -- 172.21.15.16:0/3779703407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3794101880 msgr2=0x7f37940ff100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:45.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.509+0000 7f3793fff640 1 --2- 172.21.15.16:0/3779703407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3794101880 0x7f37940ff100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:45.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.509+0000 7f3793fff640 1 -- 172.21.15.16:0/3779703407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f379410a390 msgr2=0x7f37940ff640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:45.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.509+0000 7f3793fff640 1 --2- 172.21.15.16:0/3779703407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f379410a390 0x7f37940ff640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:45.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.509+0000 7f3793fff640 1 -- 172.21.15.16:0/3779703407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377c009560 con 0x7f3794100f60 2024-09-13T16:29:45.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.509+0000 7f378bfff640 1 --2- 172.21.15.16:0/3779703407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3794101880 0x7f37940ff100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:29:45.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.509+0000 7f3793fff640 1 --2- 172.21.15.16:0/3779703407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3794100f60 0x7f37940febc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f378400d950 tx=0x7f378400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:45.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.509+0000 7f3791ffb640 1 -- 172.21.15.16:0/3779703407 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3784009c30 con 0x7f3794100f60 2024-09-13T16:29:45.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.509+0000 7f3791ffb640 1 -- 172.21.15.16:0/3779703407 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3784009d90 con 0x7f3794100f60 2024-09-13T16:29:45.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.509+0000 7f379a2c1640 1 -- 172.21.15.16:0/3779703407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37941a4c10 con 0x7f3794100f60 2024-09-13T16:29:45.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.509+0000 7f3791ffb640 1 -- 172.21.15.16:0/3779703407 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378400f620 con 0x7f3794100f60 2024-09-13T16:29:45.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.509+0000 7f379a2c1640 1 -- 172.21.15.16:0/3779703407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37941a5160 con 0x7f3794100f60 2024-09-13T16:29:45.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.510+0000 7f379a2c1640 1 -- 172.21.15.16:0/3779703407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f379410ff40 con 0x7f3794100f60 2024-09-13T16:29:45.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.511+0000 7f3791ffb640 1 -- 172.21.15.16:0/3779703407 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f37840109f0 con 0x7f3794100f60 2024-09-13T16:29:45.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.512+0000 7f3791ffb640 1 --2- 172.21.15.16:0/3779703407 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f37800a1190 0x7f37800a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:45.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.512+0000 7f378bfff640 1 --2- 172.21.15.16:0/3779703407 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f37800a1190 0x7f37800a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:45.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.512+0000 7f3791ffb640 1 -- 172.21.15.16:0/3779703407 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f37840ccd90 con 0x7f3794100f60 2024-09-13T16:29:45.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.512+0000 7f3791ffb640 1 --2- 172.21.15.16:0/3779703407 >> [v2:172.21.15.107:6800/4021360958,v1:172.21.15.107:6801/4021360958] conn(0x7f37800ac220 0x7f37800ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:45.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.512+0000 7f3791ffb640 1 -- 172.21.15.16:0/3779703407 --> [v2:172.21.15.107:6800/4021360958,v1:172.21.15.107:6801/4021360958] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f37800aed90 con 0x7f37800ac220 2024-09-13T16:29:45.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.512+0000 7f3791ffb640 1 -- 172.21.15.16:0/3779703407 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37840cd100 con 0x7f3794100f60 2024-09-13T16:29:45.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.512+0000 7f378bfff640 1 --2- 172.21.15.16:0/3779703407 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f37800a1190 0x7f37800a3650 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f377c0057c0 tx=0x7f377c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:45.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.513+0000 7f379885e640 1 --2- 172.21.15.16:0/3779703407 >> [v2:172.21.15.107:6800/4021360958,v1:172.21.15.107:6801/4021360958] conn(0x7f37800ac220 0x7f37800ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:45.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.513+0000 7f379885e640 1 --2- 172.21.15.16:0/3779703407 >> [v2:172.21.15.107:6800/4021360958,v1:172.21.15.107:6801/4021360958] conn(0x7f37800ac220 0x7f37800ae6c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:45.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.514+0000 7f3791ffb640 1 -- 172.21.15.16:0/3779703407 <== osd.1 v2:172.21.15.107:6800/4021360958 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f37800aed90 con 0x7f37800ac220 2024-09-13T16:29:45.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.527+0000 7f379a2c1640 1 -- 172.21.15.16:0/3779703407 --> [v2:172.21.15.107:6800/4021360958,v1:172.21.15.107:6801/4021360958] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3794069000 con 0x7f37800ac220 2024-09-13T16:29:45.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.527+0000 7f3791ffb640 1 -- 172.21.15.16:0/3779703407 <== osd.1 v2:172.21.15.107:6800/4021360958 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3794069000 con 0x7f37800ac220 2024-09-13T16:29:45.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.528+0000 7f379a2c1640 1 -- 172.21.15.16:0/3779703407 >> [v2:172.21.15.107:6800/4021360958,v1:172.21.15.107:6801/4021360958] conn(0x7f37800ac220 msgr2=0x7f37800ae6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:45.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.528+0000 7f379a2c1640 1 --2- 172.21.15.16:0/3779703407 >> [v2:172.21.15.107:6800/4021360958,v1:172.21.15.107:6801/4021360958] conn(0x7f37800ac220 0x7f37800ae6c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:45.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.528+0000 7f379a2c1640 1 -- 172.21.15.16:0/3779703407 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f37800a1190 msgr2=0x7f37800a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:45.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.528+0000 7f379a2c1640 1 --2- 172.21.15.16:0/3779703407 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f37800a1190 0x7f37800a3650 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f377c0057c0 tx=0x7f377c005a20 comp rx=0 tx=0).stop 2024-09-13T16:29:45.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.528+0000 7f379a2c1640 1 -- 172.21.15.16:0/3779703407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3794100f60 msgr2=0x7f37940febc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:45.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.528+0000 7f379a2c1640 1 --2- 172.21.15.16:0/3779703407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3794100f60 0x7f37940febc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f378400d950 tx=0x7f378400de10 comp rx=0 tx=0).stop 2024-09-13T16:29:45.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.528+0000 7f379885e640 1 -- 172.21.15.16:0/3779703407 reap_dead start 2024-09-13T16:29:45.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.528+0000 7f379a2c1640 1 -- 172.21.15.16:0/3779703407 shutdown_connections 2024-09-13T16:29:45.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.528+0000 7f379a2c1640 1 -- 172.21.15.16:0/3779703407 >> 172.21.15.16:0/3779703407 conn(0x7f37940fa9a0 msgr2=0x7f37940fc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:45.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.528+0000 7f379a2c1640 1 -- 172.21.15.16:0/3779703407 shutdown_connections 2024-09-13T16:29:45.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:45.528+0000 7f379a2c1640 1 -- 172.21.15.16:0/3779703407 wait complete. 2024-09-13T16:29:46.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:46 smithi016 ceph-mon[31187]: pgmap v312: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-13T16:29:46.907 INFO:teuthology.orchestra.run.smithi016.stdout:343597383704 2024-09-13T16:29:46.908 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-13T16:29:46.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:46 smithi111 ceph-mon[34600]: pgmap v312: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-13T16:29:47.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:46 smithi107 ceph-mon[34139]: pgmap v312: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-13T16:29:47.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.952+0000 7fa81a584640 1 -- 172.21.15.16:0/1546767579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa814104000 msgr2=0x7fa81410bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:47.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.952+0000 7fa81a584640 1 --2- 172.21.15.16:0/1546767579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa814104000 0x7fa81410bb80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa80c009cd0 tx=0x7fa80c02a8e0 comp rx=0 tx=0).stop 2024-09-13T16:29:47.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.952+0000 7fa81a584640 1 -- 172.21.15.16:0/1546767579 shutdown_connections 2024-09-13T16:29:47.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.952+0000 7fa81a584640 1 --2- 172.21.15.16:0/1546767579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa814104000 0x7fa81410bb80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:47.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.952+0000 7fa81a584640 1 --2- 172.21.15.16:0/1546767579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8140ff880 0x7fa8141038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:47.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.952+0000 7fa81a584640 1 --2- 172.21.15.16:0/1546767579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8140feed0 0x7fa8140ff2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:47.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.952+0000 7fa81a584640 1 -- 172.21.15.16:0/1546767579 >> 172.21.15.16:0/1546767579 conn(0x7fa8140fa9b0 msgr2=0x7fa8140fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:47.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.952+0000 7fa81a584640 1 -- 172.21.15.16:0/1546767579 shutdown_connections 2024-09-13T16:29:47.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.952+0000 7fa81a584640 1 -- 172.21.15.16:0/1546767579 wait complete. 2024-09-13T16:29:47.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.953+0000 7fa81a584640 1 Processor -- start 2024-09-13T16:29:47.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.953+0000 7fa81a584640 1 -- start start 2024-09-13T16:29:47.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.953+0000 7fa81a584640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8140feed0 0x7fa8141983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:47.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.953+0000 7fa81a584640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8140ff880 0x7fa814198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:47.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.953+0000 7fa81a584640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa814104000 0x7fa81419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:47.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.953+0000 7fa81a584640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81419d320 con 0x7fa8140feed0 2024-09-13T16:29:47.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.953+0000 7fa81a584640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81419d490 con 0x7fa814104000 2024-09-13T16:29:47.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.953+0000 7fa81a584640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81419d600 con 0x7fa8140ff880 2024-09-13T16:29:47.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.953+0000 7fa813fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8140feed0 0x7fa8141983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:47.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.953+0000 7fa813fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8140feed0 0x7fa8141983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42950/0 (socket says 172.21.15.16:42950) 2024-09-13T16:29:47.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.953+0000 7fa813fff640 1 -- 172.21.15.16:0/3343732293 learned_addr learned my addr 172.21.15.16:0/3343732293 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:47.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.954+0000 7fa818b21640 1 --2- 172.21.15.16:0/3343732293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa814104000 0x7fa81419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:47.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.954+0000 7fa8137fe640 1 --2- 172.21.15.16:0/3343732293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8140ff880 0x7fa814198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:47.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.954+0000 7fa813fff640 1 -- 172.21.15.16:0/3343732293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8140ff880 msgr2=0x7fa814198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:47.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.954+0000 7fa813fff640 1 --2- 172.21.15.16:0/3343732293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8140ff880 0x7fa814198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:47.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.954+0000 7fa813fff640 1 -- 172.21.15.16:0/3343732293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa814104000 msgr2=0x7fa81419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:47.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.954+0000 7fa813fff640 1 --2- 172.21.15.16:0/3343732293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa814104000 0x7fa81419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:47.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.954+0000 7fa813fff640 1 -- 172.21.15.16:0/3343732293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa80c009900 con 0x7fa8140feed0 2024-09-13T16:29:47.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.954+0000 7fa813fff640 1 --2- 172.21.15.16:0/3343732293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8140feed0 0x7fa8141983e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa80400c890 tx=0x7fa80400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:47.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.954+0000 7fa8117fa640 1 -- 172.21.15.16:0/3343732293 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa804004220 con 0x7fa8140feed0 2024-09-13T16:29:47.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.954+0000 7fa8117fa640 1 -- 172.21.15.16:0/3343732293 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa804007e40 con 0x7fa8140feed0 2024-09-13T16:29:47.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.954+0000 7fa81a584640 1 -- 172.21.15.16:0/3343732293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa81419d8e0 con 0x7fa8140feed0 2024-09-13T16:29:47.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.954+0000 7fa8117fa640 1 -- 172.21.15.16:0/3343732293 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa804004a00 con 0x7fa8140feed0 2024-09-13T16:29:47.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.954+0000 7fa81a584640 1 -- 172.21.15.16:0/3343732293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8141a51a0 con 0x7fa8140feed0 2024-09-13T16:29:47.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.955+0000 7fa81a584640 1 -- 172.21.15.16:0/3343732293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa8141003b0 con 0x7fa8140feed0 2024-09-13T16:29:47.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.956+0000 7fa8117fa640 1 -- 172.21.15.16:0/3343732293 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fa8040079b0 con 0x7fa8140feed0 2024-09-13T16:29:47.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.957+0000 7fa8117fa640 1 --2- 172.21.15.16:0/3343732293 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa7fc0a1190 0x7fa7fc0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:47.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.957+0000 7fa8117fa640 1 -- 172.21.15.16:0/3343732293 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa8040ccde0 con 0x7fa8140feed0 2024-09-13T16:29:47.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.957+0000 7fa8137fe640 1 --2- 172.21.15.16:0/3343732293 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa7fc0a1190 0x7fa7fc0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:47.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.957+0000 7fa8117fa640 1 --2- 172.21.15.16:0/3343732293 >> [v2:172.21.15.111:6808/483858448,v1:172.21.15.111:6809/483858448] conn(0x7fa7fc0ac220 0x7fa7fc0ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:47.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.957+0000 7fa8117fa640 1 -- 172.21.15.16:0/3343732293 --> [v2:172.21.15.111:6808/483858448,v1:172.21.15.111:6809/483858448] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa7fc0aed90 con 0x7fa7fc0ac220 2024-09-13T16:29:47.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.957+0000 7fa8117fa640 1 -- 172.21.15.16:0/3343732293 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8040cf460 con 0x7fa8140feed0 2024-09-13T16:29:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.957+0000 7fa8137fe640 1 --2- 172.21.15.16:0/3343732293 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa7fc0a1190 0x7fa7fc0a3650 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa808005d80 tx=0x7fa808005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.958+0000 7fa818b21640 1 --2- 172.21.15.16:0/3343732293 >> [v2:172.21.15.111:6808/483858448,v1:172.21.15.111:6809/483858448] conn(0x7fa7fc0ac220 0x7fa7fc0ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.958+0000 7fa818b21640 1 --2- 172.21.15.16:0/3343732293 >> [v2:172.21.15.111:6808/483858448,v1:172.21.15.111:6809/483858448] conn(0x7fa7fc0ac220 0x7fa7fc0ae6c0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.959+0000 7fa8117fa640 1 -- 172.21.15.16:0/3343732293 <== osd.5 v2:172.21.15.111:6808/483858448 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fa7fc0aed90 con 0x7fa7fc0ac220 2024-09-13T16:29:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.972+0000 7fa81a584640 1 -- 172.21.15.16:0/3343732293 --> [v2:172.21.15.111:6808/483858448,v1:172.21.15.111:6809/483858448] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa81410fec0 con 0x7fa7fc0ac220 2024-09-13T16:29:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.973+0000 7fa8117fa640 1 -- 172.21.15.16:0/3343732293 <== osd.5 v2:172.21.15.111:6808/483858448 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa81410fec0 con 0x7fa7fc0ac220 2024-09-13T16:29:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.973+0000 7fa81a584640 1 -- 172.21.15.16:0/3343732293 >> [v2:172.21.15.111:6808/483858448,v1:172.21.15.111:6809/483858448] conn(0x7fa7fc0ac220 msgr2=0x7fa7fc0ae6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.973+0000 7fa81a584640 1 --2- 172.21.15.16:0/3343732293 >> [v2:172.21.15.111:6808/483858448,v1:172.21.15.111:6809/483858448] conn(0x7fa7fc0ac220 0x7fa7fc0ae6c0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.973+0000 7fa81a584640 1 -- 172.21.15.16:0/3343732293 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa7fc0a1190 msgr2=0x7fa7fc0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.973+0000 7fa81a584640 1 --2- 172.21.15.16:0/3343732293 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa7fc0a1190 0x7fa7fc0a3650 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa808005d80 tx=0x7fa808005c90 comp rx=0 tx=0).stop 2024-09-13T16:29:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.973+0000 7fa81a584640 1 -- 172.21.15.16:0/3343732293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8140feed0 msgr2=0x7fa8141983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.973+0000 7fa81a584640 1 --2- 172.21.15.16:0/3343732293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8140feed0 0x7fa8141983e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa80400c890 tx=0x7fa80400cd50 comp rx=0 tx=0).stop 2024-09-13T16:29:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.973+0000 7fa818b21640 1 -- 172.21.15.16:0/3343732293 reap_dead start 2024-09-13T16:29:47.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.973+0000 7fa81a584640 1 -- 172.21.15.16:0/3343732293 shutdown_connections 2024-09-13T16:29:47.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.973+0000 7fa81a584640 1 -- 172.21.15.16:0/3343732293 >> 172.21.15.16:0/3343732293 conn(0x7fa8140fa9b0 msgr2=0x7fa8140fbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:47.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.973+0000 7fa81a584640 1 -- 172.21.15.16:0/3343732293 shutdown_connections 2024-09-13T16:29:47.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:42.973+0000 7fa81a584640 1 -- 172.21.15.16:0/3343732293 wait complete. 2024-09-13T16:29:47.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.704+0000 7f4b5c7eb640 1 -- 172.21.15.16:0/2562026277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b540fee70 msgr2=0x7f4b540ff250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:47.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.704+0000 7f4b5c7eb640 1 --2- 172.21.15.16:0/2562026277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b540fee70 0x7f4b540ff250 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4b50009930 tx=0x7f4b5002a420 comp rx=0 tx=0).stop 2024-09-13T16:29:47.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.704+0000 7f4b5c7eb640 1 -- 172.21.15.16:0/2562026277 shutdown_connections 2024-09-13T16:29:47.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.704+0000 7f4b5c7eb640 1 --2- 172.21.15.16:0/2562026277 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b54103fa0 0x7f4b5410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:47.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.704+0000 7f4b5c7eb640 1 --2- 172.21.15.16:0/2562026277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b540ff820 0x7f4b54103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:47.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.704+0000 7f4b5c7eb640 1 --2- 172.21.15.16:0/2562026277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b540fee70 0x7f4b540ff250 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:47.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.704+0000 7f4b5c7eb640 1 -- 172.21.15.16:0/2562026277 >> 172.21.15.16:0/2562026277 conn(0x7f4b540fa930 msgr2=0x7f4b540fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:47.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.704+0000 7f4b5c7eb640 1 -- 172.21.15.16:0/2562026277 shutdown_connections 2024-09-13T16:29:47.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.704+0000 7f4b5c7eb640 1 -- 172.21.15.16:0/2562026277 wait complete. 2024-09-13T16:29:47.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.705+0000 7f4b5c7eb640 1 Processor -- start 2024-09-13T16:29:47.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.705+0000 7f4b5c7eb640 1 -- start start 2024-09-13T16:29:47.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.705+0000 7f4b5c7eb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b540fee70 0x7f4b54196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:47.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.705+0000 7f4b5c7eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b540ff820 0x7f4b541967b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:47.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.705+0000 7f4b5c7eb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b54103fa0 0x7f4b5419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:47.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.705+0000 7f4b5c7eb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5419b1b0 con 0x7f4b540ff820 2024-09-13T16:29:47.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.705+0000 7f4b5c7eb640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5419b320 con 0x7f4b540fee70 2024-09-13T16:29:47.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.705+0000 7f4b5c7eb640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5419b490 con 0x7f4b54103fa0 2024-09-13T16:29:47.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.705+0000 7f4b59d86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b540ff820 0x7f4b541967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:47.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.705+0000 7f4b59d86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b540ff820 0x7f4b541967b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47770/0 (socket says 172.21.15.16:47770) 2024-09-13T16:29:47.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.705+0000 7f4b59d86640 1 -- 172.21.15.16:0/987593099 learned_addr learned my addr 172.21.15.16:0/987593099 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:47.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.705+0000 7f4b5ad88640 1 --2- 172.21.15.16:0/987593099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b54103fa0 0x7f4b5419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:47.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.706+0000 7f4b5a587640 1 --2- 172.21.15.16:0/987593099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b540fee70 0x7f4b54196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:47.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.706+0000 7f4b59d86640 1 -- 172.21.15.16:0/987593099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b54103fa0 msgr2=0x7f4b5419abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:47.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.706+0000 7f4b59d86640 1 --2- 172.21.15.16:0/987593099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b54103fa0 0x7f4b5419abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:47.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.706+0000 7f4b59d86640 1 -- 172.21.15.16:0/987593099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b540fee70 msgr2=0x7f4b54196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:47.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.706+0000 7f4b59d86640 1 --2- 172.21.15.16:0/987593099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b540fee70 0x7f4b54196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:47.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.706+0000 7f4b59d86640 1 -- 172.21.15.16:0/987593099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b50009560 con 0x7f4b540ff820 2024-09-13T16:29:47.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.706+0000 7f4b59d86640 1 --2- 172.21.15.16:0/987593099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b540ff820 0x7f4b541967b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4b4400c890 tx=0x7f4b4400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:47.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.706+0000 7f4b4b7fe640 1 -- 172.21.15.16:0/987593099 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b44004220 con 0x7f4b540ff820 2024-09-13T16:29:47.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.706+0000 7f4b4b7fe640 1 -- 172.21.15.16:0/987593099 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b44007e40 con 0x7f4b540ff820 2024-09-13T16:29:47.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.706+0000 7f4b5c7eb640 1 -- 172.21.15.16:0/987593099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b5419b770 con 0x7f4b540ff820 2024-09-13T16:29:47.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.706+0000 7f4b4b7fe640 1 -- 172.21.15.16:0/987593099 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b44004a00 con 0x7f4b540ff820 2024-09-13T16:29:47.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.706+0000 7f4b5c7eb640 1 -- 172.21.15.16:0/987593099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b541a3030 con 0x7f4b540ff820 2024-09-13T16:29:47.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.707+0000 7f4b5c7eb640 1 -- 172.21.15.16:0/987593099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4b5410ff40 con 0x7f4b540ff820 2024-09-13T16:29:47.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.708+0000 7f4b4b7fe640 1 -- 172.21.15.16:0/987593099 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f4b440079b0 con 0x7f4b540ff820 2024-09-13T16:29:47.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.709+0000 7f4b4b7fe640 1 --2- 172.21.15.16:0/987593099 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f4b400a1190 0x7f4b400a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:47.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.709+0000 7f4b4b7fe640 1 -- 172.21.15.16:0/987593099 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4b440cce70 con 0x7f4b540ff820 2024-09-13T16:29:47.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.709+0000 7f4b5a587640 1 --2- 172.21.15.16:0/987593099 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f4b400a1190 0x7f4b400a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:47.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.709+0000 7f4b4b7fe640 1 --2- 172.21.15.16:0/987593099 >> [v2:172.21.15.107:6824/2659550426,v1:172.21.15.107:6825/2659550426] conn(0x7f4b400ac220 0x7f4b400ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:47.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.709+0000 7f4b4b7fe640 1 -- 172.21.15.16:0/987593099 --> [v2:172.21.15.107:6824/2659550426,v1:172.21.15.107:6825/2659550426] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4b400aed90 con 0x7f4b400ac220 2024-09-13T16:29:47.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.709+0000 7f4b4b7fe640 1 -- 172.21.15.16:0/987593099 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b440cd280 con 0x7f4b540ff820 2024-09-13T16:29:47.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.710+0000 7f4b5a587640 1 --2- 172.21.15.16:0/987593099 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f4b400a1190 0x7f4b400a3650 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4b50006010 tx=0x7f4b50000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:47.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.710+0000 7f4b5ad88640 1 --2- 172.21.15.16:0/987593099 >> [v2:172.21.15.107:6824/2659550426,v1:172.21.15.107:6825/2659550426] conn(0x7f4b400ac220 0x7f4b400ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:47.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.710+0000 7f4b5ad88640 1 --2- 172.21.15.16:0/987593099 >> [v2:172.21.15.107:6824/2659550426,v1:172.21.15.107:6825/2659550426] conn(0x7f4b400ac220 0x7f4b400ae6c0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:47.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.711+0000 7f4b4b7fe640 1 -- 172.21.15.16:0/987593099 <== osd.10 v2:172.21.15.107:6824/2659550426 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f4b400aed90 con 0x7f4b400ac220 2024-09-13T16:29:47.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.725+0000 7f4b5c7eb640 1 -- 172.21.15.16:0/987593099 --> [v2:172.21.15.107:6824/2659550426,v1:172.21.15.107:6825/2659550426] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4b54060b30 con 0x7f4b400ac220 2024-09-13T16:29:47.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.726+0000 7f4b4b7fe640 1 -- 172.21.15.16:0/987593099 <== osd.10 v2:172.21.15.107:6824/2659550426 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4b54060b30 con 0x7f4b400ac220 2024-09-13T16:29:47.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.726+0000 7f4b5c7eb640 1 -- 172.21.15.16:0/987593099 >> [v2:172.21.15.107:6824/2659550426,v1:172.21.15.107:6825/2659550426] conn(0x7f4b400ac220 msgr2=0x7f4b400ae6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:47.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.726+0000 7f4b5c7eb640 1 --2- 172.21.15.16:0/987593099 >> [v2:172.21.15.107:6824/2659550426,v1:172.21.15.107:6825/2659550426] conn(0x7f4b400ac220 0x7f4b400ae6c0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:47.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.726+0000 7f4b5c7eb640 1 -- 172.21.15.16:0/987593099 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f4b400a1190 msgr2=0x7f4b400a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:47.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.726+0000 7f4b5c7eb640 1 --2- 172.21.15.16:0/987593099 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f4b400a1190 0x7f4b400a3650 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4b50006010 tx=0x7f4b50000bc0 comp rx=0 tx=0).stop 2024-09-13T16:29:47.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.726+0000 7f4b5c7eb640 1 -- 172.21.15.16:0/987593099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b540ff820 msgr2=0x7f4b541967b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:47.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.726+0000 7f4b5c7eb640 1 --2- 172.21.15.16:0/987593099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b540ff820 0x7f4b541967b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4b4400c890 tx=0x7f4b4400cd50 comp rx=0 tx=0).stop 2024-09-13T16:29:47.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.726+0000 7f4b5ad88640 1 -- 172.21.15.16:0/987593099 reap_dead start 2024-09-13T16:29:47.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.726+0000 7f4b5c7eb640 1 -- 172.21.15.16:0/987593099 shutdown_connections 2024-09-13T16:29:47.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.726+0000 7f4b5c7eb640 1 -- 172.21.15.16:0/987593099 >> 172.21.15.16:0/987593099 conn(0x7f4b540fa930 msgr2=0x7f4b540fbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:47.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.726+0000 7f4b5c7eb640 1 -- 172.21.15.16:0/987593099 shutdown_connections 2024-09-13T16:29:47.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:47.727+0000 7f4b5c7eb640 1 -- 172.21.15.16:0/987593099 wait complete. 2024-09-13T16:29:48.175 INFO:teuthology.orchestra.run.smithi016.stdout:55834574920 2024-09-13T16:29:48.175 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-13T16:29:48.743 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:48 smithi016 ceph-mon[31187]: pgmap v313: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-13T16:29:48.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:48 smithi111 ceph-mon[34600]: pgmap v313: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-13T16:29:49.041 INFO:teuthology.orchestra.run.smithi016.stdout:309237645343 2024-09-13T16:29:49.042 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-13T16:29:49.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:48 smithi107 ceph-mon[34139]: pgmap v313: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-13T16:29:49.544 INFO:teuthology.orchestra.run.smithi016.stdout:287762808865 2024-09-13T16:29:49.545 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-13T16:29:49.948 INFO:teuthology.orchestra.run.smithi016.stdout:77309411394 2024-09-13T16:29:49.949 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-13T16:29:50.445 INFO:teuthology.orchestra.run.smithi016.stdout:141733920826 2024-09-13T16:29:50.446 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-13T16:29:50.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:50 smithi016 ceph-mon[31187]: pgmap v314: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-09-13T16:29:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.879+0000 7fcf86bed640 1 -- 172.21.15.16:0/3827716537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf80100f10 msgr2=0x7fcf801012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.879+0000 7fcf86bed640 1 --2- 172.21.15.16:0/3827716537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf80100f10 0x7fcf801012f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcf74009900 tx=0x7fcf7402a480 comp rx=0 tx=0).stop 2024-09-13T16:29:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.879+0000 7fcf86bed640 1 -- 172.21.15.16:0/3827716537 shutdown_connections 2024-09-13T16:29:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.879+0000 7fcf86bed640 1 --2- 172.21.15.16:0/3827716537 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf8010a340 0x7fcf8010c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.879+0000 7fcf86bed640 1 --2- 172.21.15.16:0/3827716537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf80101830 0x7fcf80109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.879+0000 7fcf86bed640 1 --2- 172.21.15.16:0/3827716537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf80100f10 0x7fcf801012f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:50.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.879+0000 7fcf86bed640 1 -- 172.21.15.16:0/3827716537 >> 172.21.15.16:0/3827716537 conn(0x7fcf800fa990 msgr2=0x7fcf800fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.879+0000 7fcf86bed640 1 -- 172.21.15.16:0/3827716537 shutdown_connections 2024-09-13T16:29:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.879+0000 7fcf86bed640 1 -- 172.21.15.16:0/3827716537 wait complete. 2024-09-13T16:29:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.880+0000 7fcf86bed640 1 Processor -- start 2024-09-13T16:29:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.880+0000 7fcf86bed640 1 -- start start 2024-09-13T16:29:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.880+0000 7fcf86bed640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf80100f10 0x7fcf80196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.880+0000 7fcf86bed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf80101830 0x7fcf80196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.880+0000 7fcf86bed640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf8010a340 0x7fcf8019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.880+0000 7fcf86bed640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8019b160 con 0x7fcf80101830 2024-09-13T16:29:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.880+0000 7fcf86bed640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8019b2d0 con 0x7fcf8010a340 2024-09-13T16:29:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.880+0000 7fcf86bed640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8019b440 con 0x7fcf80100f10 2024-09-13T16:29:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.880+0000 7fcf7ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf80101830 0x7fcf80196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.880+0000 7fcf7ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf80101830 0x7fcf80196760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47776/0 (socket says 172.21.15.16:47776) 2024-09-13T16:29:50.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.880+0000 7fcf7ffff640 1 -- 172.21.15.16:0/2552757914 learned_addr learned my addr 172.21.15.16:0/2552757914 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:50.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.880+0000 7fcf84989640 1 --2- 172.21.15.16:0/2552757914 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf80100f10 0x7fcf80196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:50.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.880+0000 7fcf8518a640 1 --2- 172.21.15.16:0/2552757914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf8010a340 0x7fcf8019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:50.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.881+0000 7fcf7ffff640 1 -- 172.21.15.16:0/2552757914 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf80100f10 msgr2=0x7fcf80196220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:50.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.881+0000 7fcf7ffff640 1 --2- 172.21.15.16:0/2552757914 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf80100f10 0x7fcf80196220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:50.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.881+0000 7fcf7ffff640 1 -- 172.21.15.16:0/2552757914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf8010a340 msgr2=0x7fcf8019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:50.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.881+0000 7fcf7ffff640 1 --2- 172.21.15.16:0/2552757914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf8010a340 0x7fcf8019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:50.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.881+0000 7fcf7ffff640 1 -- 172.21.15.16:0/2552757914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf74009560 con 0x7fcf80101830 2024-09-13T16:29:50.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.881+0000 7fcf7ffff640 1 --2- 172.21.15.16:0/2552757914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf80101830 0x7fcf80196760 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00c940 tx=0x7fcf6c00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:50.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.881+0000 7fcf7dffb640 1 -- 172.21.15.16:0/2552757914 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c004220 con 0x7fcf80101830 2024-09-13T16:29:50.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.881+0000 7fcf7dffb640 1 -- 172.21.15.16:0/2552757914 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcf6c007e40 con 0x7fcf80101830 2024-09-13T16:29:50.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.881+0000 7fcf86bed640 1 -- 172.21.15.16:0/2552757914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf8019b720 con 0x7fcf80101830 2024-09-13T16:29:50.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.881+0000 7fcf7dffb640 1 -- 172.21.15.16:0/2552757914 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c0047a0 con 0x7fcf80101830 2024-09-13T16:29:50.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.881+0000 7fcf86bed640 1 -- 172.21.15.16:0/2552757914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf801a2fe0 con 0x7fcf80101830 2024-09-13T16:29:50.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.882+0000 7fcf86bed640 1 -- 172.21.15.16:0/2552757914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcf8010ff30 con 0x7fcf80101830 2024-09-13T16:29:50.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.883+0000 7fcf7dffb640 1 -- 172.21.15.16:0/2552757914 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fcf6c0079b0 con 0x7fcf80101830 2024-09-13T16:29:50.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.884+0000 7fcf7dffb640 1 --2- 172.21.15.16:0/2552757914 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fcf700a1190 0x7fcf700a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:50.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.884+0000 7fcf7dffb640 1 -- 172.21.15.16:0/2552757914 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fcf6c0ccc80 con 0x7fcf80101830 2024-09-13T16:29:50.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.884+0000 7fcf84989640 1 --2- 172.21.15.16:0/2552757914 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fcf700a1190 0x7fcf700a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:50.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.884+0000 7fcf7dffb640 1 --2- 172.21.15.16:0/2552757914 >> [v2:172.21.15.107:6816/1133876754,v1:172.21.15.107:6817/1133876754] conn(0x7fcf700ac220 0x7fcf700ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:50.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.884+0000 7fcf7dffb640 1 -- 172.21.15.16:0/2552757914 --> [v2:172.21.15.107:6816/1133876754,v1:172.21.15.107:6817/1133876754] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcf700aed90 con 0x7fcf700ac220 2024-09-13T16:29:50.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.884+0000 7fcf7dffb640 1 -- 172.21.15.16:0/2552757914 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf6c0d12a0 con 0x7fcf80101830 2024-09-13T16:29:50.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.885+0000 7fcf84989640 1 --2- 172.21.15.16:0/2552757914 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fcf700a1190 0x7fcf700a3650 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcf74006010 tx=0x7fcf74005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:50.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.885+0000 7fcf8518a640 1 --2- 172.21.15.16:0/2552757914 >> [v2:172.21.15.107:6816/1133876754,v1:172.21.15.107:6817/1133876754] conn(0x7fcf700ac220 0x7fcf700ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:50.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.885+0000 7fcf8518a640 1 --2- 172.21.15.16:0/2552757914 >> [v2:172.21.15.107:6816/1133876754,v1:172.21.15.107:6817/1133876754] conn(0x7fcf700ac220 0x7fcf700ae6c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:50.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.886+0000 7fcf7dffb640 1 -- 172.21.15.16:0/2552757914 <== osd.7 v2:172.21.15.107:6816/1133876754 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fcf700aed90 con 0x7fcf700ac220 2024-09-13T16:29:50.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.899+0000 7fcf86bed640 1 -- 172.21.15.16:0/2552757914 --> [v2:172.21.15.107:6816/1133876754,v1:172.21.15.107:6817/1133876754] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcf80060b30 con 0x7fcf700ac220 2024-09-13T16:29:50.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.900+0000 7fcf7dffb640 1 -- 172.21.15.16:0/2552757914 <== osd.7 v2:172.21.15.107:6816/1133876754 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fcf80060b30 con 0x7fcf700ac220 2024-09-13T16:29:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.901+0000 7fcf86bed640 1 -- 172.21.15.16:0/2552757914 >> [v2:172.21.15.107:6816/1133876754,v1:172.21.15.107:6817/1133876754] conn(0x7fcf700ac220 msgr2=0x7fcf700ae6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.901+0000 7fcf86bed640 1 --2- 172.21.15.16:0/2552757914 >> [v2:172.21.15.107:6816/1133876754,v1:172.21.15.107:6817/1133876754] conn(0x7fcf700ac220 0x7fcf700ae6c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.901+0000 7fcf86bed640 1 -- 172.21.15.16:0/2552757914 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fcf700a1190 msgr2=0x7fcf700a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.901+0000 7fcf86bed640 1 --2- 172.21.15.16:0/2552757914 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fcf700a1190 0x7fcf700a3650 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcf74006010 tx=0x7fcf74005a20 comp rx=0 tx=0).stop 2024-09-13T16:29:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.901+0000 7fcf86bed640 1 -- 172.21.15.16:0/2552757914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf80101830 msgr2=0x7fcf80196760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.901+0000 7fcf86bed640 1 --2- 172.21.15.16:0/2552757914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf80101830 0x7fcf80196760 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00c940 tx=0x7fcf6c00ce00 comp rx=0 tx=0).stop 2024-09-13T16:29:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.901+0000 7fcf8518a640 1 -- 172.21.15.16:0/2552757914 reap_dead start 2024-09-13T16:29:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.901+0000 7fcf86bed640 1 -- 172.21.15.16:0/2552757914 shutdown_connections 2024-09-13T16:29:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.901+0000 7fcf86bed640 1 -- 172.21.15.16:0/2552757914 >> 172.21.15.16:0/2552757914 conn(0x7fcf800fa990 msgr2=0x7fcf800fc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:50.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.901+0000 7fcf86bed640 1 -- 172.21.15.16:0/2552757914 shutdown_connections 2024-09-13T16:29:50.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:50.901+0000 7fcf86bed640 1 -- 172.21.15.16:0/2552757914 wait complete. 2024-09-13T16:29:50.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:50 smithi111 ceph-mon[34600]: pgmap v314: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-09-13T16:29:51.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.086+0000 7fa4e0283640 1 -- 172.21.15.16:0/4208551169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4d8105110 msgr2=0x7fa4d810cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:51.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.086+0000 7fa4e0283640 1 --2- 172.21.15.16:0/4208551169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4d8105110 0x7fa4d810cc90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0009cd0 tx=0x7fa4d002a8e0 comp rx=0 tx=0).stop 2024-09-13T16:29:51.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.086+0000 7fa4e0283640 1 -- 172.21.15.16:0/4208551169 shutdown_connections 2024-09-13T16:29:51.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.086+0000 7fa4e0283640 1 --2- 172.21.15.16:0/4208551169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4d8105110 0x7fa4d810cc90 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:51.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.086+0000 7fa4e0283640 1 --2- 172.21.15.16:0/4208551169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4d8100990 0x7fa4d81049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:51.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.086+0000 7fa4e0283640 1 --2- 172.21.15.16:0/4208551169 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4d80fffe0 0x7fa4d81003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:51.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.086+0000 7fa4e0283640 1 -- 172.21.15.16:0/4208551169 >> 172.21.15.16:0/4208551169 conn(0x7fa4d80fbae0 msgr2=0x7fa4d80fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:51.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.086+0000 7fa4e0283640 1 -- 172.21.15.16:0/4208551169 shutdown_connections 2024-09-13T16:29:51.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.086+0000 7fa4e0283640 1 -- 172.21.15.16:0/4208551169 wait complete. 2024-09-13T16:29:51.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.086+0000 7fa4e0283640 1 Processor -- start 2024-09-13T16:29:51.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.086+0000 7fa4e0283640 1 -- start start 2024-09-13T16:29:51.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.087+0000 7fa4e0283640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4d80fffe0 0x7fa4d8196100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:51.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.087+0000 7fa4e0283640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4d8100990 0x7fa4d8196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:51.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.087+0000 7fa4e0283640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4d8105110 0x7fa4d819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:51.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.087+0000 7fa4e0283640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d819b040 con 0x7fa4d8105110 2024-09-13T16:29:51.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.087+0000 7fa4e0283640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d819b1b0 con 0x7fa4d8100990 2024-09-13T16:29:51.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.087+0000 7fa4e0283640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d819b320 con 0x7fa4d80fffe0 2024-09-13T16:29:51.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.087+0000 7fa4de820640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4d8105110 0x7fa4d819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:51.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.087+0000 7fa4de820640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4d8105110 0x7fa4d819aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47790/0 (socket says 172.21.15.16:47790) 2024-09-13T16:29:51.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.087+0000 7fa4de820640 1 -- 172.21.15.16:0/1583321408 learned_addr learned my addr 172.21.15.16:0/1583321408 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.087+0000 7fa4dd81e640 1 --2- 172.21.15.16:0/1583321408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4d8100990 0x7fa4d8196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.087+0000 7fa4de01f640 1 --2- 172.21.15.16:0/1583321408 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4d80fffe0 0x7fa4d8196100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.087+0000 7fa4de820640 1 -- 172.21.15.16:0/1583321408 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4d80fffe0 msgr2=0x7fa4d8196100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.087+0000 7fa4de820640 1 --2- 172.21.15.16:0/1583321408 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4d80fffe0 0x7fa4d8196100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.087+0000 7fa4de820640 1 -- 172.21.15.16:0/1583321408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4d8100990 msgr2=0x7fa4d8196640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.087+0000 7fa4de820640 1 --2- 172.21.15.16:0/1583321408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4d8100990 0x7fa4d8196640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.087+0000 7fa4de820640 1 -- 172.21.15.16:0/1583321408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d0009900 con 0x7fa4d8105110 2024-09-13T16:29:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.087+0000 7fa4de820640 1 --2- 172.21.15.16:0/1583321408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4d8105110 0x7fa4d819aa40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0004af0 tx=0x7fa4d0030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.088+0000 7fa4cf7fe640 1 -- 172.21.15.16:0/1583321408 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0003970 con 0x7fa4d8105110 2024-09-13T16:29:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.088+0000 7fa4cf7fe640 1 -- 172.21.15.16:0/1583321408 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa4d0004040 con 0x7fa4d8105110 2024-09-13T16:29:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.088+0000 7fa4e0283640 1 -- 172.21.15.16:0/1583321408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4d819b5a0 con 0x7fa4d8105110 2024-09-13T16:29:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.088+0000 7fa4cf7fe640 1 -- 172.21.15.16:0/1583321408 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0045650 con 0x7fa4d8105110 2024-09-13T16:29:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.088+0000 7fa4e0283640 1 -- 172.21.15.16:0/1583321408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4d81a2ec0 con 0x7fa4d8105110 2024-09-13T16:29:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.088+0000 7fa4e0283640 1 -- 172.21.15.16:0/1583321408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa4d81014c0 con 0x7fa4d8105110 2024-09-13T16:29:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.090+0000 7fa4cf7fe640 1 -- 172.21.15.16:0/1583321408 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fa4d000a040 con 0x7fa4d8105110 2024-09-13T16:29:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.090+0000 7fa4cf7fe640 1 --2- 172.21.15.16:0/1583321408 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa4c40a1110 0x7fa4c40a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.091+0000 7fa4de01f640 1 --2- 172.21.15.16:0/1583321408 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa4c40a1110 0x7fa4c40a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.091+0000 7fa4cf7fe640 1 -- 172.21.15.16:0/1583321408 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa4d00ee7c0 con 0x7fa4d8105110 2024-09-13T16:29:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.091+0000 7fa4cf7fe640 1 --2- 172.21.15.16:0/1583321408 >> [v2:172.21.15.111:6816/837548190,v1:172.21.15.111:6817/837548190] conn(0x7fa4c40ac1a0 0x7fa4c40ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.091+0000 7fa4cf7fe640 1 -- 172.21.15.16:0/1583321408 --> [v2:172.21.15.111:6816/837548190,v1:172.21.15.111:6817/837548190] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa4c40aed10 con 0x7fa4c40ac1a0 2024-09-13T16:29:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.091+0000 7fa4cf7fe640 1 -- 172.21.15.16:0/1583321408 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4d00eeb30 con 0x7fa4d8105110 2024-09-13T16:29:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.091+0000 7fa4de01f640 1 --2- 172.21.15.16:0/1583321408 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa4c40a1110 0x7fa4c40a35d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4005e10 tx=0x7fa4d4005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.091+0000 7fa4dd81e640 1 --2- 172.21.15.16:0/1583321408 >> [v2:172.21.15.111:6816/837548190,v1:172.21.15.111:6817/837548190] conn(0x7fa4c40ac1a0 0x7fa4c40ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.092+0000 7fa4dd81e640 1 --2- 172.21.15.16:0/1583321408 >> [v2:172.21.15.111:6816/837548190,v1:172.21.15.111:6817/837548190] conn(0x7fa4c40ac1a0 0x7fa4c40ae640 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.092+0000 7fa4cf7fe640 1 -- 172.21.15.16:0/1583321408 <== osd.8 v2:172.21.15.111:6816/837548190 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fa4c40aed10 con 0x7fa4c40ac1a0 2024-09-13T16:29:51.101 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:50 smithi107 ceph-mon[34139]: pgmap v314: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-09-13T16:29:51.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.106+0000 7fa4e0283640 1 -- 172.21.15.16:0/1583321408 --> [v2:172.21.15.111:6816/837548190,v1:172.21.15.111:6817/837548190] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa4d810ff90 con 0x7fa4c40ac1a0 2024-09-13T16:29:51.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.106+0000 7fa4cf7fe640 1 -- 172.21.15.16:0/1583321408 <== osd.8 v2:172.21.15.111:6816/837548190 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa4c800b7b0 con 0x7fa4c40ac1a0 2024-09-13T16:29:51.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.107+0000 7fa4e0283640 1 -- 172.21.15.16:0/1583321408 >> [v2:172.21.15.111:6816/837548190,v1:172.21.15.111:6817/837548190] conn(0x7fa4c40ac1a0 msgr2=0x7fa4c40ae640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:51.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.107+0000 7fa4e0283640 1 --2- 172.21.15.16:0/1583321408 >> [v2:172.21.15.111:6816/837548190,v1:172.21.15.111:6817/837548190] conn(0x7fa4c40ac1a0 0x7fa4c40ae640 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:51.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.107+0000 7fa4e0283640 1 -- 172.21.15.16:0/1583321408 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa4c40a1110 msgr2=0x7fa4c40a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:51.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.107+0000 7fa4e0283640 1 --2- 172.21.15.16:0/1583321408 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa4c40a1110 0x7fa4c40a35d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4005e10 tx=0x7fa4d4005da0 comp rx=0 tx=0).stop 2024-09-13T16:29:51.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.107+0000 7fa4e0283640 1 -- 172.21.15.16:0/1583321408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4d8105110 msgr2=0x7fa4d819aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:51.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.107+0000 7fa4e0283640 1 --2- 172.21.15.16:0/1583321408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4d8105110 0x7fa4d819aa40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0004af0 tx=0x7fa4d0030ec0 comp rx=0 tx=0).stop 2024-09-13T16:29:51.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.107+0000 7fa4de820640 1 -- 172.21.15.16:0/1583321408 reap_dead start 2024-09-13T16:29:51.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.107+0000 7fa4e0283640 1 -- 172.21.15.16:0/1583321408 shutdown_connections 2024-09-13T16:29:51.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.107+0000 7fa4e0283640 1 -- 172.21.15.16:0/1583321408 >> 172.21.15.16:0/1583321408 conn(0x7fa4d80fbae0 msgr2=0x7fa4d80fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:51.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.107+0000 7fa4e0283640 1 -- 172.21.15.16:0/1583321408 shutdown_connections 2024-09-13T16:29:51.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.107+0000 7fa4e0283640 1 -- 172.21.15.16:0/1583321408 wait complete. 2024-09-13T16:29:51.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.522+0000 7fd50ec41640 1 -- 172.21.15.16:0/1310990789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd508100990 msgr2=0x7fd5081049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:51.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.522+0000 7fd50ec41640 1 --2- 172.21.15.16:0/1310990789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd508100990 0x7fd5081049e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8009cd0 tx=0x7fd4f802a8e0 comp rx=0 tx=0).stop 2024-09-13T16:29:51.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.522+0000 7fd50ec41640 1 -- 172.21.15.16:0/1310990789 shutdown_connections 2024-09-13T16:29:51.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.522+0000 7fd50ec41640 1 --2- 172.21.15.16:0/1310990789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd508105110 0x7fd50810cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:51.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.522+0000 7fd50ec41640 1 --2- 172.21.15.16:0/1310990789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd508100990 0x7fd5081049e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:51.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.522+0000 7fd50ec41640 1 --2- 172.21.15.16:0/1310990789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5080fffe0 0x7fd5081003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:51.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.522+0000 7fd50ec41640 1 -- 172.21.15.16:0/1310990789 >> 172.21.15.16:0/1310990789 conn(0x7fd5080fbae0 msgr2=0x7fd5080fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:51.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.522+0000 7fd50ec41640 1 -- 172.21.15.16:0/1310990789 shutdown_connections 2024-09-13T16:29:51.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.522+0000 7fd50ec41640 1 -- 172.21.15.16:0/1310990789 wait complete. 2024-09-13T16:29:51.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.523+0000 7fd50ec41640 1 Processor -- start 2024-09-13T16:29:51.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.523+0000 7fd50ec41640 1 -- start start 2024-09-13T16:29:51.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.523+0000 7fd50ec41640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5080fffe0 0x7fd5080727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:51.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.523+0000 7fd50ec41640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd508100990 0x7fd508072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:51.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.523+0000 7fd50ec41640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd508105110 0x7fd508073230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:51.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.523+0000 7fd50ec41640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5081a4610 con 0x7fd5080fffe0 2024-09-13T16:29:51.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.523+0000 7fd50ec41640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5081a4780 con 0x7fd508105110 2024-09-13T16:29:51.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.523+0000 7fd50ec41640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5081a4940 con 0x7fd508100990 2024-09-13T16:29:51.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.523+0000 7fd50c9dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5080fffe0 0x7fd5080727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:51.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.523+0000 7fd50c9dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5080fffe0 0x7fd5080727b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47816/0 (socket says 172.21.15.16:47816) 2024-09-13T16:29:51.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.523+0000 7fd50c9dd640 1 -- 172.21.15.16:0/12573913 learned_addr learned my addr 172.21.15.16:0/12573913 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:51.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.523+0000 7fd507fff640 1 --2- 172.21.15.16:0/12573913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd508100990 0x7fd508072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:51.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.523+0000 7fd50c9dd640 1 -- 172.21.15.16:0/12573913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd508100990 msgr2=0x7fd508072cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:51.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.523+0000 7fd50c9dd640 1 --2- 172.21.15.16:0/12573913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd508100990 0x7fd508072cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:51.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.523+0000 7fd50c9dd640 1 -- 172.21.15.16:0/12573913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd508105110 msgr2=0x7fd508073230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:51.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.523+0000 7fd50d1de640 1 --2- 172.21.15.16:0/12573913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd508105110 0x7fd508073230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:51.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.524+0000 7fd50c9dd640 1 --2- 172.21.15.16:0/12573913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd508105110 0x7fd508073230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:51.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.524+0000 7fd50c9dd640 1 -- 172.21.15.16:0/12573913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f8009900 con 0x7fd5080fffe0 2024-09-13T16:29:51.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.524+0000 7fd50c9dd640 1 --2- 172.21.15.16:0/12573913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5080fffe0 0x7fd5080727b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd4f000b4a0 tx=0x7fd4f000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:51.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.524+0000 7fd505ffb640 1 -- 172.21.15.16:0/12573913 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0010610 con 0x7fd5080fffe0 2024-09-13T16:29:51.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.524+0000 7fd50ec41640 1 -- 172.21.15.16:0/12573913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5081a4c20 con 0x7fd5080fffe0 2024-09-13T16:29:51.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.524+0000 7fd505ffb640 1 -- 172.21.15.16:0/12573913 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4f0010c00 con 0x7fd5080fffe0 2024-09-13T16:29:51.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.524+0000 7fd505ffb640 1 -- 172.21.15.16:0/12573913 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0017ca0 con 0x7fd5080fffe0 2024-09-13T16:29:51.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.524+0000 7fd50ec41640 1 -- 172.21.15.16:0/12573913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5081a5170 con 0x7fd5080fffe0 2024-09-13T16:29:51.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.525+0000 7fd50ec41640 1 -- 172.21.15.16:0/12573913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd5081014c0 con 0x7fd5080fffe0 2024-09-13T16:29:51.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.526+0000 7fd505ffb640 1 -- 172.21.15.16:0/12573913 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fd4f0010770 con 0x7fd5080fffe0 2024-09-13T16:29:51.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.527+0000 7fd505ffb640 1 --2- 172.21.15.16:0/12573913 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd4f40a1110 0x7fd4f40a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:51.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.527+0000 7fd507fff640 1 --2- 172.21.15.16:0/12573913 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd4f40a1110 0x7fd4f40a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:51.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.527+0000 7fd505ffb640 1 -- 172.21.15.16:0/12573913 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd4f00cc760 con 0x7fd5080fffe0 2024-09-13T16:29:51.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.527+0000 7fd505ffb640 1 --2- 172.21.15.16:0/12573913 >> [v2:172.21.15.16:6818/1330214141,v1:172.21.15.16:6819/1330214141] conn(0x7fd4f40ac1a0 0x7fd4f40ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:51.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.527+0000 7fd505ffb640 1 -- 172.21.15.16:0/12573913 --> [v2:172.21.15.16:6818/1330214141,v1:172.21.15.16:6819/1330214141] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd4f40aed10 con 0x7fd4f40ac1a0 2024-09-13T16:29:51.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.527+0000 7fd505ffb640 1 -- 172.21.15.16:0/12573913 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4f00ced40 con 0x7fd5080fffe0 2024-09-13T16:29:51.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.527+0000 7fd50d1de640 1 --2- 172.21.15.16:0/12573913 >> [v2:172.21.15.16:6818/1330214141,v1:172.21.15.16:6819/1330214141] conn(0x7fd4f40ac1a0 0x7fd4f40ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:51.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.527+0000 7fd507fff640 1 --2- 172.21.15.16:0/12573913 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd4f40a1110 0x7fd4f40a35d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd508073f30 tx=0x7fd4f80316d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:51.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.528+0000 7fd50d1de640 1 --2- 172.21.15.16:0/12573913 >> [v2:172.21.15.16:6818/1330214141,v1:172.21.15.16:6819/1330214141] conn(0x7fd4f40ac1a0 0x7fd4f40ae640 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:51.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.528+0000 7fd505ffb640 1 -- 172.21.15.16:0/12573913 <== osd.6 v2:172.21.15.16:6818/1330214141 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fd4f40aed10 con 0x7fd4f40ac1a0 2024-09-13T16:29:51.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.542+0000 7fd50ec41640 1 -- 172.21.15.16:0/12573913 --> [v2:172.21.15.16:6818/1330214141,v1:172.21.15.16:6819/1330214141] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd50810fef0 con 0x7fd4f40ac1a0 2024-09-13T16:29:51.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.542+0000 7fd505ffb640 1 -- 172.21.15.16:0/12573913 <== osd.6 v2:172.21.15.16:6818/1330214141 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd50810fef0 con 0x7fd4f40ac1a0 2024-09-13T16:29:51.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.542+0000 7fd50ec41640 1 -- 172.21.15.16:0/12573913 >> [v2:172.21.15.16:6818/1330214141,v1:172.21.15.16:6819/1330214141] conn(0x7fd4f40ac1a0 msgr2=0x7fd4f40ae640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.542+0000 7fd50ec41640 1 --2- 172.21.15.16:0/12573913 >> [v2:172.21.15.16:6818/1330214141,v1:172.21.15.16:6819/1330214141] conn(0x7fd4f40ac1a0 0x7fd4f40ae640 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.542+0000 7fd50ec41640 1 -- 172.21.15.16:0/12573913 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd4f40a1110 msgr2=0x7fd4f40a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.542+0000 7fd50ec41640 1 --2- 172.21.15.16:0/12573913 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd4f40a1110 0x7fd4f40a35d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd508073f30 tx=0x7fd4f80316d0 comp rx=0 tx=0).stop 2024-09-13T16:29:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.542+0000 7fd50ec41640 1 -- 172.21.15.16:0/12573913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5080fffe0 msgr2=0x7fd5080727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.542+0000 7fd50ec41640 1 --2- 172.21.15.16:0/12573913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5080fffe0 0x7fd5080727b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd4f000b4a0 tx=0x7fd4f000b960 comp rx=0 tx=0).stop 2024-09-13T16:29:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.542+0000 7fd50d1de640 1 -- 172.21.15.16:0/12573913 reap_dead start 2024-09-13T16:29:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.543+0000 7fd50ec41640 1 -- 172.21.15.16:0/12573913 shutdown_connections 2024-09-13T16:29:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.543+0000 7fd50ec41640 1 -- 172.21.15.16:0/12573913 >> 172.21.15.16:0/12573913 conn(0x7fd5080fbae0 msgr2=0x7fd5080fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.543+0000 7fd50ec41640 1 -- 172.21.15.16:0/12573913 shutdown_connections 2024-09-13T16:29:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:51.543+0000 7fd50ec41640 1 -- 172.21.15.16:0/12573913 wait complete. 2024-09-13T16:29:52.182 INFO:teuthology.orchestra.run.smithi016.stdout:219043332141 2024-09-13T16:29:52.183 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-13T16:29:52.711 INFO:teuthology.orchestra.run.smithi016.stdout:163208757302 2024-09-13T16:29:52.711 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-13T16:29:52.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:52 smithi111 ceph-mon[34600]: pgmap v315: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-09-13T16:29:52.984 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:52 smithi016 ceph-mon[31187]: pgmap v315: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-09-13T16:29:53.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:52 smithi107 ceph-mon[34139]: pgmap v315: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-09-13T16:29:53.324 INFO:teuthology.orchestra.run.smithi016.stdout:253403070504 2024-09-13T16:29:53.325 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-13T16:29:53.749 INFO:teuthology.orchestra.run.smithi016.stdout:197568495667 2024-09-13T16:29:53.750 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-13T16:29:53.755 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:29:53.763 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:29:53.809 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:29:53.836 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:29:54.610 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:29:54.614 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:29:54.616 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:29:54.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:54 smithi111 ceph-mon[34600]: pgmap v316: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T16:29:54.941 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:54 smithi016 ceph-mon[31187]: pgmap v316: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T16:29:55.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:54 smithi107 ceph-mon[34139]: pgmap v316: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T16:29:56.749 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:56 smithi016 ceph-mon[31187]: pgmap v317: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T16:29:56.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:56 smithi111 ceph-mon[34600]: pgmap v317: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T16:29:57.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:56 smithi107 ceph-mon[34139]: pgmap v317: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T16:29:58.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:29:58 smithi016 ceph-mon[31187]: pgmap v318: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-13T16:29:58.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:29:58 smithi111 ceph-mon[34600]: pgmap v318: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-13T16:29:59.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:29:58 smithi107 ceph-mon[34139]: pgmap v318: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-13T16:29:59.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.860+0000 7f568860e640 1 -- 172.21.15.16:0/4282226886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5680104030 msgr2=0x7f568010bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:59.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.860+0000 7f568860e640 1 --2- 172.21.15.16:0/4282226886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5680104030 0x7f568010bbb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5678009cd0 tx=0x7f567802a8e0 comp rx=0 tx=0).stop 2024-09-13T16:29:59.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.860+0000 7f568860e640 1 -- 172.21.15.16:0/4282226886 shutdown_connections 2024-09-13T16:29:59.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.860+0000 7f568860e640 1 --2- 172.21.15.16:0/4282226886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5680104030 0x7f568010bbb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:59.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.860+0000 7f568860e640 1 --2- 172.21.15.16:0/4282226886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56800ff8b0 0x7f5680103900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:59.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.860+0000 7f568860e640 1 --2- 172.21.15.16:0/4282226886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56800fef00 0x7f56800ff2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:59.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.860+0000 7f568860e640 1 -- 172.21.15.16:0/4282226886 >> 172.21.15.16:0/4282226886 conn(0x7f56800fa9a0 msgr2=0x7f56800fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:29:59.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.860+0000 7f568860e640 1 -- 172.21.15.16:0/4282226886 shutdown_connections 2024-09-13T16:29:59.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.860+0000 7f568860e640 1 -- 172.21.15.16:0/4282226886 wait complete. 2024-09-13T16:29:59.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.861+0000 7f568860e640 1 Processor -- start 2024-09-13T16:29:59.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.861+0000 7f568860e640 1 -- start start 2024-09-13T16:29:59.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.861+0000 7f568860e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56800fef00 0x7f5680196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:59.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.861+0000 7f568860e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56800ff8b0 0x7f56801974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:59.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.862+0000 7f568860e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5680104030 0x7f56801a1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:59.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.862+0000 7f568860e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5680197d50 con 0x7f56800fef00 2024-09-13T16:29:59.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.862+0000 7f568860e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56801a2530 con 0x7f56800ff8b0 2024-09-13T16:29:59.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.862+0000 7f568860e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56801a2670 con 0x7f5680104030 2024-09-13T16:29:59.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.862+0000 7f56863aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56800fef00 0x7f5680196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:59.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.862+0000 7f56863aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56800fef00 0x7f5680196f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43624/0 (socket says 172.21.15.16:43624) 2024-09-13T16:29:59.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.862+0000 7f56863aa640 1 -- 172.21.15.16:0/3380534387 learned_addr learned my addr 172.21.15.16:0/3380534387 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:29:59.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.862+0000 7f5685ba9640 1 --2- 172.21.15.16:0/3380534387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56800ff8b0 0x7f56801974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.862+0000 7f5686bab640 1 --2- 172.21.15.16:0/3380534387 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5680104030 0x7f56801a1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.862+0000 7f56863aa640 1 -- 172.21.15.16:0/3380534387 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5680104030 msgr2=0x7f56801a1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.862+0000 7f56863aa640 1 --2- 172.21.15.16:0/3380534387 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5680104030 0x7f56801a1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.862+0000 7f56863aa640 1 -- 172.21.15.16:0/3380534387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56800ff8b0 msgr2=0x7f56801974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:29:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.862+0000 7f56863aa640 1 --2- 172.21.15.16:0/3380534387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56800ff8b0 0x7f56801974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:29:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.862+0000 7f56863aa640 1 -- 172.21.15.16:0/3380534387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5678009900 con 0x7f56800fef00 2024-09-13T16:29:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.862+0000 7f56863aa640 1 --2- 172.21.15.16:0/3380534387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56800fef00 0x7f5680196f70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f567c00d980 tx=0x7f567c00dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.862+0000 7f56777fe640 1 -- 172.21.15.16:0/3380534387 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c009c30 con 0x7f56800fef00 2024-09-13T16:29:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.862+0000 7f56777fe640 1 -- 172.21.15.16:0/3380534387 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f567c00bea0 con 0x7f56800fef00 2024-09-13T16:29:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.862+0000 7f568860e640 1 -- 172.21.15.16:0/3380534387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56801a2920 con 0x7f56800fef00 2024-09-13T16:29:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.863+0000 7f56777fe640 1 -- 172.21.15.16:0/3380534387 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c00f620 con 0x7f56800fef00 2024-09-13T16:29:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.863+0000 7f568860e640 1 -- 172.21.15.16:0/3380534387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56801a2e70 con 0x7f56800fef00 2024-09-13T16:29:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.864+0000 7f568860e640 1 -- 172.21.15.16:0/3380534387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56801003e0 con 0x7f56800fef00 2024-09-13T16:29:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.865+0000 7f56777fe640 1 -- 172.21.15.16:0/3380534387 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f567c010460 con 0x7f56800fef00 2024-09-13T16:29:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.865+0000 7f56777fe640 1 --2- 172.21.15.16:0/3380534387 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f566c0a1190 0x7f566c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:29:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.865+0000 7f5685ba9640 1 --2- 172.21.15.16:0/3380534387 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f566c0a1190 0x7f566c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:29:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.865+0000 7f56777fe640 1 -- 172.21.15.16:0/3380534387 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f567c0cc850 con 0x7f56800fef00 2024-09-13T16:29:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.866+0000 7f5685ba9640 1 --2- 172.21.15.16:0/3380534387 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f566c0a1190 0x7f566c0a3650 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f56800692e0 tx=0x7f567000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:29:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:29:59.870+0000 7f56777fe640 1 -- 172.21.15.16:0/3380534387 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f567c095b70 con 0x7f56800fef00 2024-09-13T16:30:00.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:00.044+0000 7f568860e640 1 -- 172.21.15.16:0/3380534387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f56800ff2e0 con 0x7f56800fef00 2024-09-13T16:30:00.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:00.045+0000 7f56777fe640 1 -- 172.21.15.16:0/3380534387 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f567c099190 con 0x7f56800fef00 2024-09-13T16:30:00.047 INFO:teuthology.orchestra.run.smithi016.stdout:141733920830 2024-09-13T16:30:00.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:00.047+0000 7f568860e640 1 -- 172.21.15.16:0/3380534387 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f566c0a1190 msgr2=0x7f566c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:00.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:00.047+0000 7f568860e640 1 --2- 172.21.15.16:0/3380534387 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f566c0a1190 0x7f566c0a3650 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f56800692e0 tx=0x7f567000a2d0 comp rx=0 tx=0).stop 2024-09-13T16:30:00.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:00.047+0000 7f568860e640 1 -- 172.21.15.16:0/3380534387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56800fef00 msgr2=0x7f5680196f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:00.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:00.047+0000 7f568860e640 1 --2- 172.21.15.16:0/3380534387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56800fef00 0x7f5680196f70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f567c00d980 tx=0x7f567c00dd80 comp rx=0 tx=0).stop 2024-09-13T16:30:00.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:00.047+0000 7f568860e640 1 -- 172.21.15.16:0/3380534387 shutdown_connections 2024-09-13T16:30:00.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:00.047+0000 7f568860e640 1 --2- 172.21.15.16:0/3380534387 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5680104030 0x7f56801a1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:00.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:00.047+0000 7f568860e640 1 --2- 172.21.15.16:0/3380534387 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f566c0a1190 0x7f566c0a3650 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:00.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:00.047+0000 7f568860e640 1 --2- 172.21.15.16:0/3380534387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56800ff8b0 0x7f56801974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:00.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:00.047+0000 7f568860e640 1 --2- 172.21.15.16:0/3380534387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56800fef00 0x7f5680196f70 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:00.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:00.047+0000 7f568860e640 1 -- 172.21.15.16:0/3380534387 >> 172.21.15.16:0/3380534387 conn(0x7f56800fa9a0 msgr2=0x7f56800fbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:00.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:00.047+0000 7f568860e640 1 -- 172.21.15.16:0/3380534387 shutdown_connections 2024-09-13T16:30:00.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:00.047+0000 7f568860e640 1 -- 172.21.15.16:0/3380534387 wait complete. 2024-09-13T16:30:00.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:00 smithi016 ceph-mon[31187]: overall HEALTH_OK 2024-09-13T16:30:00.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:00 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3380534387' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-13T16:30:00.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:00 smithi107 ceph-mon[34139]: overall HEALTH_OK 2024-09-13T16:30:00.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:00 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3380534387' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-13T16:30:00.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:00 smithi111 ceph-mon[34600]: overall HEALTH_OK 2024-09-13T16:30:00.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:00 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3380534387' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-13T16:30:01.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:01 smithi107 ceph-mon[34139]: pgmap v319: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-13T16:30:01.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:01 smithi016 ceph-mon[31187]: pgmap v319: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-13T16:30:01.386 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:30:01.387 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:30:01.388 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:30:01.403 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920826 got 141733920830 for osd.4 2024-09-13T16:30:01.403 DEBUG:teuthology.parallel:result is None 2024-09-13T16:30:01.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:01 smithi111 ceph-mon[34600]: pgmap v319: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-13T16:30:02.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:02 smithi016 ceph-mon[31187]: pgmap v320: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-13T16:30:02.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:02 smithi111 ceph-mon[34600]: pgmap v320: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-13T16:30:03.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:02 smithi107 ceph-mon[34139]: pgmap v320: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-13T16:30:04.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.609+0000 7f899a986640 1 -- 172.21.15.16:0/1565955600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8994100990 msgr2=0x7f89941049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:04.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.609+0000 7f899a986640 1 --2- 172.21.15.16:0/1565955600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8994100990 0x7f89941049e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8988009930 tx=0x7f898802a420 comp rx=0 tx=0).stop 2024-09-13T16:30:04.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.609+0000 7f899a986640 1 -- 172.21.15.16:0/1565955600 shutdown_connections 2024-09-13T16:30:04.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.609+0000 7f899a986640 1 --2- 172.21.15.16:0/1565955600 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8994105110 0x7f899410cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:04.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.609+0000 7f899a986640 1 --2- 172.21.15.16:0/1565955600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8994100990 0x7f89941049e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:04.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.609+0000 7f899a986640 1 --2- 172.21.15.16:0/1565955600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89940fffe0 0x7f89941003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:04.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.609+0000 7f899a986640 1 -- 172.21.15.16:0/1565955600 >> 172.21.15.16:0/1565955600 conn(0x7f89940fbae0 msgr2=0x7f89940fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:04.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.609+0000 7f899a986640 1 -- 172.21.15.16:0/1565955600 shutdown_connections 2024-09-13T16:30:04.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.609+0000 7f899a986640 1 -- 172.21.15.16:0/1565955600 wait complete. 2024-09-13T16:30:04.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.610+0000 7f899a986640 1 Processor -- start 2024-09-13T16:30:04.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.610+0000 7f899a986640 1 -- start start 2024-09-13T16:30:04.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.610+0000 7f899a986640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89940fffe0 0x7f89940727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:04.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.610+0000 7f899a986640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8994100990 0x7f8994072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:04.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.610+0000 7f899a986640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8994105110 0x7f8994073230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:04.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.610+0000 7f899a986640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89941a4530 con 0x7f8994105110 2024-09-13T16:30:04.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.610+0000 7f899a986640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89941a46a0 con 0x7f8994100990 2024-09-13T16:30:04.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.610+0000 7f899a986640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89941a4860 con 0x7f89940fffe0 2024-09-13T16:30:04.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.610+0000 7f8998f23640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8994105110 0x7f8994073230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:04.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.610+0000 7f8998f23640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8994105110 0x7f8994073230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55714/0 (socket says 172.21.15.16:55714) 2024-09-13T16:30:04.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.610+0000 7f8998f23640 1 -- 172.21.15.16:0/2512353856 learned_addr learned my addr 172.21.15.16:0/2512353856 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:04.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.610+0000 7f89937fe640 1 --2- 172.21.15.16:0/2512353856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8994100990 0x7f8994072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:04.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.610+0000 7f8993fff640 1 --2- 172.21.15.16:0/2512353856 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89940fffe0 0x7f89940727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:04.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.611+0000 7f8998f23640 1 -- 172.21.15.16:0/2512353856 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89940fffe0 msgr2=0x7f89940727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:04.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.611+0000 7f8998f23640 1 --2- 172.21.15.16:0/2512353856 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89940fffe0 0x7f89940727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:04.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.611+0000 7f8998f23640 1 -- 172.21.15.16:0/2512353856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8994100990 msgr2=0x7f8994072cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:04.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.611+0000 7f8998f23640 1 --2- 172.21.15.16:0/2512353856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8994100990 0x7f8994072cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:04.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.611+0000 7f8998f23640 1 -- 172.21.15.16:0/2512353856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8988009560 con 0x7f8994105110 2024-09-13T16:30:04.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.611+0000 7f8993fff640 1 --2- 172.21.15.16:0/2512353856 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89940fffe0 0x7f89940727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:30:04.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.611+0000 7f89937fe640 1 --2- 172.21.15.16:0/2512353856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8994100990 0x7f8994072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:30:04.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.611+0000 7f8998f23640 1 --2- 172.21.15.16:0/2512353856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8994105110 0x7f8994073230 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f898c00eb10 tx=0x7f898c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:04.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.611+0000 7f89917fa640 1 -- 172.21.15.16:0/2512353856 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898c004b20 con 0x7f8994105110 2024-09-13T16:30:04.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.611+0000 7f89917fa640 1 -- 172.21.15.16:0/2512353856 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f898c004c80 con 0x7f8994105110 2024-09-13T16:30:04.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.611+0000 7f899a986640 1 -- 172.21.15.16:0/2512353856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89941a4b40 con 0x7f8994105110 2024-09-13T16:30:04.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.611+0000 7f89917fa640 1 -- 172.21.15.16:0/2512353856 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898c005a10 con 0x7f8994105110 2024-09-13T16:30:04.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.611+0000 7f899a986640 1 -- 172.21.15.16:0/2512353856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89941a4fc0 con 0x7f8994105110 2024-09-13T16:30:04.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.612+0000 7f899a986640 1 -- 172.21.15.16:0/2512353856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89941014c0 con 0x7f8994105110 2024-09-13T16:30:04.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.614+0000 7f89917fa640 1 -- 172.21.15.16:0/2512353856 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f898c004fd0 con 0x7f8994105110 2024-09-13T16:30:04.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.614+0000 7f89917fa640 1 --2- 172.21.15.16:0/2512353856 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f897c0a1110 0x7f897c0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:04.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.615+0000 7f8993fff640 1 --2- 172.21.15.16:0/2512353856 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f897c0a1110 0x7f897c0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:04.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.615+0000 7f89917fa640 1 -- 172.21.15.16:0/2512353856 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f898c014070 con 0x7f8994105110 2024-09-13T16:30:04.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.615+0000 7f8993fff640 1 --2- 172.21.15.16:0/2512353856 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f897c0a1110 0x7f897c0a35d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f898400bfd0 tx=0x7f898400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:04.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.618+0000 7f89917fa640 1 -- 172.21.15.16:0/2512353856 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f898c0967a0 con 0x7f8994105110 2024-09-13T16:30:04.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.804+0000 7f899a986640 1 -- 172.21.15.16:0/2512353856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8994073ce0 con 0x7f8994105110 2024-09-13T16:30:04.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.804+0000 7f89917fa640 1 -- 172.21.15.16:0/2512353856 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f898c099dc0 con 0x7f8994105110 2024-09-13T16:30:04.807 INFO:teuthology.orchestra.run.smithi016.stdout:55834574924 2024-09-13T16:30:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.806+0000 7f899a986640 1 -- 172.21.15.16:0/2512353856 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f897c0a1110 msgr2=0x7f897c0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.806+0000 7f899a986640 1 --2- 172.21.15.16:0/2512353856 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f897c0a1110 0x7f897c0a35d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f898400bfd0 tx=0x7f898400b390 comp rx=0 tx=0).stop 2024-09-13T16:30:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.806+0000 7f899a986640 1 -- 172.21.15.16:0/2512353856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8994105110 msgr2=0x7f8994073230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.806+0000 7f899a986640 1 --2- 172.21.15.16:0/2512353856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8994105110 0x7f8994073230 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f898c00eb10 tx=0x7f898c00c2d0 comp rx=0 tx=0).stop 2024-09-13T16:30:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.807+0000 7f899a986640 1 -- 172.21.15.16:0/2512353856 shutdown_connections 2024-09-13T16:30:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.807+0000 7f899a986640 1 --2- 172.21.15.16:0/2512353856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8994105110 0x7f8994073230 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.807+0000 7f899a986640 1 --2- 172.21.15.16:0/2512353856 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f897c0a1110 0x7f897c0a35d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.807+0000 7f899a986640 1 --2- 172.21.15.16:0/2512353856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8994100990 0x7f8994072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.807+0000 7f899a986640 1 --2- 172.21.15.16:0/2512353856 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89940fffe0 0x7f89940727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.807+0000 7f899a986640 1 -- 172.21.15.16:0/2512353856 >> 172.21.15.16:0/2512353856 conn(0x7f89940fbae0 msgr2=0x7f89940fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.807+0000 7f899a986640 1 -- 172.21.15.16:0/2512353856 shutdown_connections 2024-09-13T16:30:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.807+0000 7f899a986640 1 -- 172.21.15.16:0/2512353856 wait complete. 2024-09-13T16:30:04.901 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:04 smithi016 ceph-mon[31187]: pgmap v321: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-13T16:30:04.901 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-09-13T16:30:04.901 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:04 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 10]}]: dispatch 2024-09-13T16:30:04.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:04 smithi111 ceph-mon[34600]: pgmap v321: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-13T16:30:04.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-09-13T16:30:04.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:04 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 10]}]: dispatch 2024-09-13T16:30:05.105 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:04 smithi107 ceph-mon[34139]: pgmap v321: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-13T16:30:05.105 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-09-13T16:30:05.105 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:04 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 10]}]: dispatch 2024-09-13T16:30:05.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.516+0000 7f4938095640 1 -- 172.21.15.16:0/1635038730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4930105110 msgr2=0x7f493010cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:05.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.516+0000 7f4938095640 1 --2- 172.21.15.16:0/1635038730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4930105110 0x7f493010cc90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4928009930 tx=0x7f492802a420 comp rx=0 tx=0).stop 2024-09-13T16:30:05.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.516+0000 7f4938095640 1 -- 172.21.15.16:0/1635038730 shutdown_connections 2024-09-13T16:30:05.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.516+0000 7f4938095640 1 --2- 172.21.15.16:0/1635038730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4930105110 0x7f493010cc90 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:05.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.516+0000 7f4938095640 1 --2- 172.21.15.16:0/1635038730 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4930100990 0x7f49301049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:05.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.516+0000 7f4938095640 1 --2- 172.21.15.16:0/1635038730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49300fffe0 0x7f49301003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:05.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.516+0000 7f4938095640 1 -- 172.21.15.16:0/1635038730 >> 172.21.15.16:0/1635038730 conn(0x7f49300fbae0 msgr2=0x7f49300fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:05.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.516+0000 7f4938095640 1 -- 172.21.15.16:0/1635038730 shutdown_connections 2024-09-13T16:30:05.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.516+0000 7f4938095640 1 -- 172.21.15.16:0/1635038730 wait complete. 2024-09-13T16:30:05.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.517+0000 7f4938095640 1 Processor -- start 2024-09-13T16:30:05.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.517+0000 7f4938095640 1 -- start start 2024-09-13T16:30:05.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.517+0000 7f4938095640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49300fffe0 0x7f49300727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:05.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.517+0000 7f4938095640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4930100990 0x7f4930072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:05.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.517+0000 7f4938095640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4930105110 0x7f4930073230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:05.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.517+0000 7f4938095640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49301a44d0 con 0x7f4930100990 2024-09-13T16:30:05.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.517+0000 7f4938095640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49301a4640 con 0x7f4930105110 2024-09-13T16:30:05.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.517+0000 7f4938095640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49301a4800 con 0x7f49300fffe0 2024-09-13T16:30:05.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.517+0000 7f4935630640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4930100990 0x7f4930072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:05.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.517+0000 7f4935630640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4930100990 0x7f4930072cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55696/0 (socket says 172.21.15.16:55696) 2024-09-13T16:30:05.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.517+0000 7f4935630640 1 -- 172.21.15.16:0/105072311 learned_addr learned my addr 172.21.15.16:0/105072311 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:05.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.517+0000 7f4936632640 1 --2- 172.21.15.16:0/105072311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4930105110 0x7f4930073230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:05.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.517+0000 7f4935e31640 1 --2- 172.21.15.16:0/105072311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49300fffe0 0x7f49300727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:05.663 INFO:teuthology.orchestra.run.smithi016.stdout:34359738449 2024-09-13T16:30:05.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.517+0000 7f4935630640 1 -- 172.21.15.16:0/105072311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49300fffe0 msgr2=0x7f49300727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:05.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.517+0000 7f4935630640 1 --2- 172.21.15.16:0/105072311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49300fffe0 0x7f49300727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:05.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.517+0000 7f4935630640 1 -- 172.21.15.16:0/105072311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4930105110 msgr2=0x7f4930073230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:05.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.517+0000 7f4935630640 1 --2- 172.21.15.16:0/105072311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4930105110 0x7f4930073230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:05.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.517+0000 7f4935630640 1 -- 172.21.15.16:0/105072311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4928009560 con 0x7f4930100990 2024-09-13T16:30:05.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.518+0000 7f4935630640 1 --2- 172.21.15.16:0/105072311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4930100990 0x7f4930072cf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f492c00d950 tx=0x7f492c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:05.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.518+0000 7f4922ffd640 1 -- 172.21.15.16:0/105072311 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c009c30 con 0x7f4930100990 2024-09-13T16:30:05.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.518+0000 7f4922ffd640 1 -- 172.21.15.16:0/105072311 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f492c009d90 con 0x7f4930100990 2024-09-13T16:30:05.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.518+0000 7f4922ffd640 1 -- 172.21.15.16:0/105072311 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c00f620 con 0x7f4930100990 2024-09-13T16:30:05.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.518+0000 7f4938095640 1 -- 172.21.15.16:0/105072311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49301a4ae0 con 0x7f4930100990 2024-09-13T16:30:05.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.518+0000 7f4938095640 1 -- 172.21.15.16:0/105072311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49301a4fe0 con 0x7f4930100990 2024-09-13T16:30:05.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.519+0000 7f4938095640 1 -- 172.21.15.16:0/105072311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49301014c0 con 0x7f4930100990 2024-09-13T16:30:05.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.520+0000 7f4922ffd640 1 -- 172.21.15.16:0/105072311 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f492c0109f0 con 0x7f4930100990 2024-09-13T16:30:05.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.521+0000 7f4922ffd640 1 --2- 172.21.15.16:0/105072311 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f491c0a1190 0x7f491c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:05.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.521+0000 7f4935e31640 1 --2- 172.21.15.16:0/105072311 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f491c0a1190 0x7f491c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:05.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.521+0000 7f4922ffd640 1 -- 172.21.15.16:0/105072311 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f492c0cc140 con 0x7f4930100990 2024-09-13T16:30:05.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.521+0000 7f4935e31640 1 --2- 172.21.15.16:0/105072311 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f491c0a1190 0x7f491c0a3650 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4924005c40 tx=0x7f492400a590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:05.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.526+0000 7f4922ffd640 1 -- 172.21.15.16:0/105072311 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f492c095510 con 0x7f4930100990 2024-09-13T16:30:05.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.714+0000 7f4938095640 1 -- 172.21.15.16:0/105072311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f49301003c0 con 0x7f4930100990 2024-09-13T16:30:05.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.715+0000 7f4922ffd640 1 -- 172.21.15.16:0/105072311 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f492c017090 con 0x7f4930100990 2024-09-13T16:30:05.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.717+0000 7f4938095640 1 -- 172.21.15.16:0/105072311 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f491c0a1190 msgr2=0x7f491c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:05.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.717+0000 7f4938095640 1 --2- 172.21.15.16:0/105072311 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f491c0a1190 0x7f491c0a3650 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4924005c40 tx=0x7f492400a590 comp rx=0 tx=0).stop 2024-09-13T16:30:05.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.717+0000 7f4938095640 1 -- 172.21.15.16:0/105072311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4930100990 msgr2=0x7f4930072cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:05.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.717+0000 7f4938095640 1 --2- 172.21.15.16:0/105072311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4930100990 0x7f4930072cf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f492c00d950 tx=0x7f492c00de10 comp rx=0 tx=0).stop 2024-09-13T16:30:05.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.717+0000 7f4938095640 1 -- 172.21.15.16:0/105072311 shutdown_connections 2024-09-13T16:30:05.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.717+0000 7f4938095640 1 --2- 172.21.15.16:0/105072311 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f491c0a1190 0x7f491c0a3650 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:05.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.717+0000 7f4938095640 1 --2- 172.21.15.16:0/105072311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4930105110 0x7f4930073230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:05.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.717+0000 7f4938095640 1 --2- 172.21.15.16:0/105072311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4930100990 0x7f4930072cf0 secure :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f492c00d950 tx=0x7f492c00de10 comp rx=0 tx=0).stop 2024-09-13T16:30:05.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.717+0000 7f4938095640 1 --2- 172.21.15.16:0/105072311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49300fffe0 0x7f49300727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:05.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.717+0000 7f4938095640 1 -- 172.21.15.16:0/105072311 >> 172.21.15.16:0/105072311 conn(0x7f49300fbae0 msgr2=0x7f49300fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:05.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.718+0000 7f4938095640 1 -- 172.21.15.16:0/105072311 shutdown_connections 2024-09-13T16:30:05.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:04.718+0000 7f4938095640 1 -- 172.21.15.16:0/105072311 wait complete. 2024-09-13T16:30:05.964 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:05 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-13T16:30:05.965 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:05 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 10]}]': finished 2024-09-13T16:30:05.965 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:05 smithi016 ceph-mon[31187]: osdmap e92: 12 total, 12 up, 12 in 2024-09-13T16:30:05.965 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:05 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/105072311' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-13T16:30:05.965 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:05 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2512353856' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-13T16:30:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.052+0000 7f56b246b640 1 -- 172.21.15.16:0/1086112312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56ac100990 msgr2=0x7f56ac1049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.052+0000 7f56b246b640 1 --2- 172.21.15.16:0/1086112312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56ac100990 0x7f56ac1049e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f56a0009930 tx=0x7f56a002a420 comp rx=0 tx=0).stop 2024-09-13T16:30:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.052+0000 7f56b246b640 1 -- 172.21.15.16:0/1086112312 shutdown_connections 2024-09-13T16:30:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.052+0000 7f56b246b640 1 --2- 172.21.15.16:0/1086112312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56ac105110 0x7f56ac10cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.052+0000 7f56b246b640 1 --2- 172.21.15.16:0/1086112312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56ac100990 0x7f56ac1049e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.052+0000 7f56b246b640 1 --2- 172.21.15.16:0/1086112312 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56ac0fffe0 0x7f56ac1003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.052+0000 7f56b246b640 1 -- 172.21.15.16:0/1086112312 >> 172.21.15.16:0/1086112312 conn(0x7f56ac0fbae0 msgr2=0x7f56ac0fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.052+0000 7f56b246b640 1 -- 172.21.15.16:0/1086112312 shutdown_connections 2024-09-13T16:30:06.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.052+0000 7f56b246b640 1 -- 172.21.15.16:0/1086112312 wait complete. 2024-09-13T16:30:06.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.052+0000 7f56b246b640 1 Processor -- start 2024-09-13T16:30:06.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.052+0000 7f56b246b640 1 -- start start 2024-09-13T16:30:06.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.053+0000 7f56b246b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56ac0fffe0 0x7f56ac196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:06.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.053+0000 7f56b246b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56ac100990 0x7f56ac1967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:06.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.053+0000 7f56b246b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56ac105110 0x7f56ac19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:06.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.053+0000 7f56b246b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac19b100 con 0x7f56ac100990 2024-09-13T16:30:06.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.053+0000 7f56b246b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac19b270 con 0x7f56ac0fffe0 2024-09-13T16:30:06.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.053+0000 7f56b246b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac19b3e0 con 0x7f56ac105110 2024-09-13T16:30:06.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.053+0000 7f56ab7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56ac100990 0x7f56ac1967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:06.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.053+0000 7f56ab7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56ac100990 0x7f56ac1967a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55728/0 (socket says 172.21.15.16:55728) 2024-09-13T16:30:06.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.053+0000 7f56ab7fe640 1 -- 172.21.15.16:0/2490039936 learned_addr learned my addr 172.21.15.16:0/2490039936 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:06.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.053+0000 7f56abfff640 1 --2- 172.21.15.16:0/2490039936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56ac0fffe0 0x7f56ac196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:06.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.053+0000 7f56b0a08640 1 --2- 172.21.15.16:0/2490039936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56ac105110 0x7f56ac19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:06.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.053+0000 7f56ab7fe640 1 -- 172.21.15.16:0/2490039936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56ac105110 msgr2=0x7f56ac19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:06.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.053+0000 7f56ab7fe640 1 --2- 172.21.15.16:0/2490039936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56ac105110 0x7f56ac19ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:06.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.053+0000 7f56ab7fe640 1 -- 172.21.15.16:0/2490039936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56ac0fffe0 msgr2=0x7f56ac196260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:06.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.053+0000 7f56ab7fe640 1 --2- 172.21.15.16:0/2490039936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56ac0fffe0 0x7f56ac196260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:06.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.053+0000 7f56ab7fe640 1 -- 172.21.15.16:0/2490039936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a0009560 con 0x7f56ac100990 2024-09-13T16:30:06.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.053+0000 7f56b0a08640 1 --2- 172.21.15.16:0/2490039936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56ac105110 0x7f56ac19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:30:06.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.054+0000 7f56ab7fe640 1 --2- 172.21.15.16:0/2490039936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56ac100990 0x7f56ac1967a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f56a002aa10 tx=0x7f56a0032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:06.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.054+0000 7f56a97fa640 1 -- 172.21.15.16:0/2490039936 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a0032e10 con 0x7f56ac100990 2024-09-13T16:30:06.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.054+0000 7f56b246b640 1 -- 172.21.15.16:0/2490039936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56ac19b660 con 0x7f56ac100990 2024-09-13T16:30:06.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.054+0000 7f56a97fa640 1 -- 172.21.15.16:0/2490039936 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f56a00046c0 con 0x7f56ac100990 2024-09-13T16:30:06.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.054+0000 7f56a97fa640 1 -- 172.21.15.16:0/2490039936 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a0003850 con 0x7f56ac100990 2024-09-13T16:30:06.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.054+0000 7f56b246b640 1 -- 172.21.15.16:0/2490039936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56ac19b980 con 0x7f56ac100990 2024-09-13T16:30:06.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.055+0000 7f56b246b640 1 -- 172.21.15.16:0/2490039936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56ac1014c0 con 0x7f56ac100990 2024-09-13T16:30:06.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.056+0000 7f56a97fa640 1 -- 172.21.15.16:0/2490039936 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f56a0004b30 con 0x7f56ac100990 2024-09-13T16:30:06.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.057+0000 7f56a97fa640 1 --2- 172.21.15.16:0/2490039936 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f56940a1110 0x7f56940a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:06.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.057+0000 7f56a97fa640 1 -- 172.21.15.16:0/2490039936 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f56a00edf90 con 0x7f56ac100990 2024-09-13T16:30:06.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.057+0000 7f56abfff640 1 --2- 172.21.15.16:0/2490039936 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f56940a1110 0x7f56940a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:06.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.057+0000 7f56abfff640 1 --2- 172.21.15.16:0/2490039936 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f56940a1110 0x7f56940a35d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f569c00abe0 tx=0x7f569c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:06.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.060+0000 7f56a97fa640 1 -- 172.21.15.16:0/2490039936 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f56a00b7210 con 0x7f56ac100990 2024-09-13T16:30:06.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:05 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-13T16:30:06.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:05 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 10]}]': finished 2024-09-13T16:30:06.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:05 smithi107 ceph-mon[34139]: osdmap e92: 12 total, 12 up, 12 in 2024-09-13T16:30:06.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:05 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/105072311' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-13T16:30:06.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:05 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/2512353856' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-13T16:30:06.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:05 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-13T16:30:06.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:05 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 10]}]': finished 2024-09-13T16:30:06.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:05 smithi111 ceph-mon[34600]: osdmap e92: 12 total, 12 up, 12 in 2024-09-13T16:30:06.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:05 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/105072311' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-13T16:30:06.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:05 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/2512353856' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-13T16:30:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.236+0000 7f56b246b640 1 -- 172.21.15.16:0/2490039936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f56ac1003c0 con 0x7f56ac100990 2024-09-13T16:30:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.237+0000 7f56a97fa640 1 -- 172.21.15.16:0/2490039936 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f56a00ba830 con 0x7f56ac100990 2024-09-13T16:30:06.240 INFO:teuthology.orchestra.run.smithi016.stdout:309237645346 2024-09-13T16:30:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.239+0000 7f56b246b640 1 -- 172.21.15.16:0/2490039936 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f56940a1110 msgr2=0x7f56940a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.240+0000 7f56b246b640 1 --2- 172.21.15.16:0/2490039936 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f56940a1110 0x7f56940a35d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f569c00abe0 tx=0x7f569c00a2d0 comp rx=0 tx=0).stop 2024-09-13T16:30:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.240+0000 7f56b246b640 1 -- 172.21.15.16:0/2490039936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56ac100990 msgr2=0x7f56ac1967a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.240+0000 7f56b246b640 1 --2- 172.21.15.16:0/2490039936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56ac100990 0x7f56ac1967a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f56a002aa10 tx=0x7f56a0032c20 comp rx=0 tx=0).stop 2024-09-13T16:30:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.240+0000 7f56b246b640 1 -- 172.21.15.16:0/2490039936 shutdown_connections 2024-09-13T16:30:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.240+0000 7f56b246b640 1 --2- 172.21.15.16:0/2490039936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56ac105110 0x7f56ac19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.240+0000 7f56b246b640 1 --2- 172.21.15.16:0/2490039936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56ac100990 0x7f56ac1967a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.240+0000 7f56b246b640 1 --2- 172.21.15.16:0/2490039936 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f56940a1110 0x7f56940a35d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.240+0000 7f56b246b640 1 --2- 172.21.15.16:0/2490039936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56ac0fffe0 0x7f56ac196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.240+0000 7f56b246b640 1 -- 172.21.15.16:0/2490039936 >> 172.21.15.16:0/2490039936 conn(0x7f56ac0fbae0 msgr2=0x7f56ac0fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.240+0000 7f56b246b640 1 -- 172.21.15.16:0/2490039936 shutdown_connections 2024-09-13T16:30:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:06.240+0000 7f56b246b640 1 -- 172.21.15.16:0/2490039936 wait complete. 2024-09-13T16:30:06.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:06 smithi016 ceph-mon[31187]: osdmap e93: 12 total, 12 up, 12 in 2024-09-13T16:30:06.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:06 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2490039936' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-13T16:30:06.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:06 smithi016 ceph-mon[31187]: pgmap v324: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-13T16:30:07.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:06 smithi107 ceph-mon[34139]: osdmap e93: 12 total, 12 up, 12 in 2024-09-13T16:30:07.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:06 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/2490039936' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-13T16:30:07.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:06 smithi107 ceph-mon[34139]: pgmap v324: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-13T16:30:07.146 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645343 got 309237645346 for osd.10 2024-09-13T16:30:07.146 DEBUG:teuthology.parallel:result is None 2024-09-13T16:30:07.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:06 smithi111 ceph-mon[34600]: osdmap e93: 12 total, 12 up, 12 in 2024-09-13T16:30:07.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:06 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/2490039936' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-13T16:30:07.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:06 smithi111 ceph-mon[34600]: pgmap v324: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-13T16:30:07.415 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:30:07.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.415+0000 7f47b62c2640 1 -- 172.21.15.16:0/3356765182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b0100f30 msgr2=0x7f47b0101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:07.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.415+0000 7f47b62c2640 1 --2- 172.21.15.16:0/3356765182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b0100f30 0x7f47b0101310 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f47a0009900 tx=0x7f47a002a480 comp rx=0 tx=0).stop 2024-09-13T16:30:07.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.415+0000 7f47b62c2640 1 -- 172.21.15.16:0/3356765182 shutdown_connections 2024-09-13T16:30:07.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.415+0000 7f47b62c2640 1 --2- 172.21.15.16:0/3356765182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47b010a360 0x7f47b010c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:07.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.415+0000 7f47b62c2640 1 --2- 172.21.15.16:0/3356765182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47b0101850 0x7f47b0109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:07.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.415+0000 7f47b62c2640 1 --2- 172.21.15.16:0/3356765182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b0100f30 0x7f47b0101310 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:07.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.415+0000 7f47b62c2640 1 -- 172.21.15.16:0/3356765182 >> 172.21.15.16:0/3356765182 conn(0x7f47b00fa9d0 msgr2=0x7f47b00fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:07.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.415+0000 7f47b62c2640 1 -- 172.21.15.16:0/3356765182 shutdown_connections 2024-09-13T16:30:07.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.415+0000 7f47b62c2640 1 -- 172.21.15.16:0/3356765182 wait complete. 2024-09-13T16:30:07.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.416+0000 7f47b62c2640 1 Processor -- start 2024-09-13T16:30:07.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.416+0000 7f47b62c2640 1 -- start start 2024-09-13T16:30:07.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.417+0000 7f47b62c2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47b0100f30 0x7f47b00feb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:07.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.417+0000 7f47b62c2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b0101850 0x7f47b00ff0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:07.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.417+0000 7f47b62c2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47b010a360 0x7f47b00ff630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:07.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.417+0000 7f47b62c2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b01a4660 con 0x7f47b0101850 2024-09-13T16:30:07.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.417+0000 7f47b62c2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b01a47d0 con 0x7f47b010a360 2024-09-13T16:30:07.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.417+0000 7f47b62c2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b01a4990 con 0x7f47b0100f30 2024-09-13T16:30:07.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.417+0000 7f47affff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47b0100f30 0x7f47b00feb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:07.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.417+0000 7f47af7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b0101850 0x7f47b00ff0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:07.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.417+0000 7f47af7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b0101850 0x7f47b00ff0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55756/0 (socket says 172.21.15.16:55756) 2024-09-13T16:30:07.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.417+0000 7f47af7fe640 1 -- 172.21.15.16:0/4136325729 learned_addr learned my addr 172.21.15.16:0/4136325729 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:07.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.417+0000 7f47b485f640 1 --2- 172.21.15.16:0/4136325729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47b010a360 0x7f47b00ff630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:07.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.417+0000 7f47af7fe640 1 -- 172.21.15.16:0/4136325729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47b0100f30 msgr2=0x7f47b00feb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:07.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.417+0000 7f47af7fe640 1 --2- 172.21.15.16:0/4136325729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47b0100f30 0x7f47b00feb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:07.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.417+0000 7f47af7fe640 1 -- 172.21.15.16:0/4136325729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47b010a360 msgr2=0x7f47b00ff630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:07.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.417+0000 7f47af7fe640 1 --2- 172.21.15.16:0/4136325729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47b010a360 0x7f47b00ff630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:07.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.418+0000 7f47af7fe640 1 -- 172.21.15.16:0/4136325729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a0009560 con 0x7f47b0101850 2024-09-13T16:30:07.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.418+0000 7f47af7fe640 1 --2- 172.21.15.16:0/4136325729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b0101850 0x7f47b00ff0d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f47a400c970 tx=0x7f47a400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:07.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.418+0000 7f47ad7fa640 1 -- 172.21.15.16:0/4136325729 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a4007bb0 con 0x7f47b0101850 2024-09-13T16:30:07.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.418+0000 7f47ad7fa640 1 -- 172.21.15.16:0/4136325729 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f47a4007d10 con 0x7f47b0101850 2024-09-13T16:30:07.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.418+0000 7f47b62c2640 1 -- 172.21.15.16:0/4136325729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b01a4c70 con 0x7f47b0101850 2024-09-13T16:30:07.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.418+0000 7f47ad7fa640 1 -- 172.21.15.16:0/4136325729 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a4018660 con 0x7f47b0101850 2024-09-13T16:30:07.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.418+0000 7f47b62c2640 1 -- 172.21.15.16:0/4136325729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b01a51c0 con 0x7f47b0101850 2024-09-13T16:30:07.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.419+0000 7f47b62c2640 1 -- 172.21.15.16:0/4136325729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b0069000 con 0x7f47b0101850 2024-09-13T16:30:07.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.421+0000 7f47ad7fa640 1 -- 172.21.15.16:0/4136325729 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f47a400f450 con 0x7f47b0101850 2024-09-13T16:30:07.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.422+0000 7f47ad7fa640 1 --2- 172.21.15.16:0/4136325729 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f47980a1190 0x7f47980a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:07.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.422+0000 7f47ad7fa640 1 -- 172.21.15.16:0/4136325729 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f47a40ccff0 con 0x7f47b0101850 2024-09-13T16:30:07.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.422+0000 7f47affff640 1 --2- 172.21.15.16:0/4136325729 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f47980a1190 0x7f47980a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:07.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.423+0000 7f47affff640 1 --2- 172.21.15.16:0/4136325729 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f47980a1190 0x7f47980a3650 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f47a0000c00 tx=0x7f47a0004040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:07.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.425+0000 7f47ad7fa640 1 -- 172.21.15.16:0/4136325729 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f47a40963e0 con 0x7f47b0101850 2024-09-13T16:30:07.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.599+0000 7f47b62c2640 1 -- 172.21.15.16:0/4136325729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f47b0105400 con 0x7f47b0101850 2024-09-13T16:30:07.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.599+0000 7f47ad7fa640 1 -- 172.21.15.16:0/4136325729 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f47a4099a00 con 0x7f47b0101850 2024-09-13T16:30:07.602 INFO:teuthology.orchestra.run.smithi016.stdout:107374182466 2024-09-13T16:30:07.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.602+0000 7f47b62c2640 1 -- 172.21.15.16:0/4136325729 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f47980a1190 msgr2=0x7f47980a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:07.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.602+0000 7f47b62c2640 1 --2- 172.21.15.16:0/4136325729 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f47980a1190 0x7f47980a3650 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f47a0000c00 tx=0x7f47a0004040 comp rx=0 tx=0).stop 2024-09-13T16:30:07.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.602+0000 7f47b62c2640 1 -- 172.21.15.16:0/4136325729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b0101850 msgr2=0x7f47b00ff0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:07.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.602+0000 7f47b62c2640 1 --2- 172.21.15.16:0/4136325729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b0101850 0x7f47b00ff0d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f47a400c970 tx=0x7f47a400ce30 comp rx=0 tx=0).stop 2024-09-13T16:30:07.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.602+0000 7f47b62c2640 1 -- 172.21.15.16:0/4136325729 shutdown_connections 2024-09-13T16:30:07.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.602+0000 7f47b62c2640 1 --2- 172.21.15.16:0/4136325729 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f47980a1190 0x7f47980a3650 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:07.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.602+0000 7f47b62c2640 1 --2- 172.21.15.16:0/4136325729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47b010a360 0x7f47b00ff630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:07.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.602+0000 7f47b62c2640 1 --2- 172.21.15.16:0/4136325729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47b0101850 0x7f47b00ff0d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:07.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.602+0000 7f47b62c2640 1 --2- 172.21.15.16:0/4136325729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47b0100f30 0x7f47b00feb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:07.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.602+0000 7f47b62c2640 1 -- 172.21.15.16:0/4136325729 >> 172.21.15.16:0/4136325729 conn(0x7f47b00fa9d0 msgr2=0x7f47b00fc260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:07.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.602+0000 7f47b62c2640 1 -- 172.21.15.16:0/4136325729 shutdown_connections 2024-09-13T16:30:07.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:07.602+0000 7f47b62c2640 1 -- 172.21.15.16:0/4136325729 wait complete. 2024-09-13T16:30:07.836 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738444 got 34359738449 for osd.0 2024-09-13T16:30:07.836 DEBUG:teuthology.parallel:result is None 2024-09-13T16:30:08.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:07 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/4136325729' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-13T16:30:08.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:07 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/4136325729' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-13T16:30:08.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:07 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/4136325729' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-13T16:30:08.466 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574920 got 55834574924 for osd.1 2024-09-13T16:30:08.466 DEBUG:teuthology.parallel:result is None 2024-09-13T16:30:09.080 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:08 smithi016 ceph-mon[31187]: pgmap v325: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-13T16:30:09.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:08 smithi107 ceph-mon[34139]: pgmap v325: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-13T16:30:09.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:08 smithi111 ceph-mon[34600]: pgmap v325: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-13T16:30:09.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.786+0000 7fe2b6fea640 1 -- 172.21.15.16:0/1843183598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b010a390 msgr2=0x7fe2b010c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:09.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.786+0000 7fe2b6fea640 1 --2- 172.21.15.16:0/1843183598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b010a390 0x7fe2b010c830 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009ca0 tx=0x7fe2a802a950 comp rx=0 tx=0).stop 2024-09-13T16:30:09.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.786+0000 7fe2b6fea640 1 -- 172.21.15.16:0/1843183598 shutdown_connections 2024-09-13T16:30:09.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.786+0000 7fe2b6fea640 1 --2- 172.21.15.16:0/1843183598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b010a390 0x7fe2b010c830 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:09.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.786+0000 7fe2b6fea640 1 --2- 172.21.15.16:0/1843183598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2b0101880 0x7fe2b0109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:09.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.786+0000 7fe2b6fea640 1 --2- 172.21.15.16:0/1843183598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2b0100f60 0x7fe2b0101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:09.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.786+0000 7fe2b6fea640 1 -- 172.21.15.16:0/1843183598 >> 172.21.15.16:0/1843183598 conn(0x7fe2b00fa9a0 msgr2=0x7fe2b00fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:09.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.786+0000 7fe2b6fea640 1 -- 172.21.15.16:0/1843183598 shutdown_connections 2024-09-13T16:30:09.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.786+0000 7fe2b6fea640 1 -- 172.21.15.16:0/1843183598 wait complete. 2024-09-13T16:30:09.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.786+0000 7fe2b6fea640 1 Processor -- start 2024-09-13T16:30:09.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.786+0000 7fe2b6fea640 1 -- start start 2024-09-13T16:30:09.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.787+0000 7fe2b6fea640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2b0100f60 0x7fe2b00febc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:09.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.787+0000 7fe2b6fea640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2b0101880 0x7fe2b00ff100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:09.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.787+0000 7fe2b6fea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b010a390 0x7fe2b00ff640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:09.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.787+0000 7fe2b6fea640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b01a4460 con 0x7fe2b010a390 2024-09-13T16:30:09.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.787+0000 7fe2b6fea640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b01a45d0 con 0x7fe2b0101880 2024-09-13T16:30:09.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.787+0000 7fe2b6fea640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b01a4790 con 0x7fe2b0100f60 2024-09-13T16:30:09.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.787+0000 7fe2b5587640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b010a390 0x7fe2b00ff640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:09.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.787+0000 7fe2b5587640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b010a390 0x7fe2b00ff640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55784/0 (socket says 172.21.15.16:55784) 2024-09-13T16:30:09.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.787+0000 7fe2b5587640 1 -- 172.21.15.16:0/1927922066 learned_addr learned my addr 172.21.15.16:0/1927922066 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:09.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.787+0000 7fe2affff640 1 --2- 172.21.15.16:0/1927922066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2b0101880 0x7fe2b00ff100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:09.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.787+0000 7fe2b4d86640 1 --2- 172.21.15.16:0/1927922066 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2b0100f60 0x7fe2b00febc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:09.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.787+0000 7fe2b5587640 1 -- 172.21.15.16:0/1927922066 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2b0100f60 msgr2=0x7fe2b00febc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:09.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.787+0000 7fe2b5587640 1 --2- 172.21.15.16:0/1927922066 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2b0100f60 0x7fe2b00febc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:09.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.787+0000 7fe2b5587640 1 -- 172.21.15.16:0/1927922066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2b0101880 msgr2=0x7fe2b00ff100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:09.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.787+0000 7fe2b5587640 1 --2- 172.21.15.16:0/1927922066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2b0101880 0x7fe2b00ff100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:09.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.787+0000 7fe2b5587640 1 -- 172.21.15.16:0/1927922066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a8009900 con 0x7fe2b010a390 2024-09-13T16:30:09.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.787+0000 7fe2b5587640 1 --2- 172.21.15.16:0/1927922066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b010a390 0x7fe2b00ff640 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe2a802af40 tx=0x7fe2a802fd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:09.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.788+0000 7fe2adffb640 1 -- 172.21.15.16:0/1927922066 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8003d80 con 0x7fe2b010a390 2024-09-13T16:30:09.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.788+0000 7fe2adffb640 1 -- 172.21.15.16:0/1927922066 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe2a80308d0 con 0x7fe2b010a390 2024-09-13T16:30:09.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.788+0000 7fe2adffb640 1 -- 172.21.15.16:0/1927922066 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8004810 con 0x7fe2b010a390 2024-09-13T16:30:09.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.788+0000 7fe2b6fea640 1 -- 172.21.15.16:0/1927922066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b01a4a10 con 0x7fe2b010a390 2024-09-13T16:30:09.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.788+0000 7fe2b6fea640 1 -- 172.21.15.16:0/1927922066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b01a4e60 con 0x7fe2b010a390 2024-09-13T16:30:09.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.789+0000 7fe2b6fea640 1 -- 172.21.15.16:0/1927922066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b0069000 con 0x7fe2b010a390 2024-09-13T16:30:09.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.790+0000 7fe2adffb640 1 -- 172.21.15.16:0/1927922066 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fe2a802e050 con 0x7fe2b010a390 2024-09-13T16:30:09.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.791+0000 7fe2adffb640 1 --2- 172.21.15.16:0/1927922066 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe2980a1110 0x7fe2980a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:09.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.791+0000 7fe2b4d86640 1 --2- 172.21.15.16:0/1927922066 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe2980a1110 0x7fe2980a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:09.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.791+0000 7fe2adffb640 1 -- 172.21.15.16:0/1927922066 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fe2a80edc30 con 0x7fe2b010a390 2024-09-13T16:30:09.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.791+0000 7fe2b4d86640 1 --2- 172.21.15.16:0/1927922066 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe2980a1110 0x7fe2980a35d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0005e10 tx=0x7fe2a0005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:09.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.795+0000 7fe2adffb640 1 -- 172.21.15.16:0/1927922066 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fe2a80b6f10 con 0x7fe2b010a390 2024-09-13T16:30:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.973+0000 7fe2b6fea640 1 -- 172.21.15.16:0/1927922066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7fe2b0101340 con 0x7fe2b010a390 2024-09-13T16:30:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.973+0000 7fe2adffb640 1 -- 172.21.15.16:0/1927922066 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe2a802c020 con 0x7fe2b010a390 2024-09-13T16:30:09.974 INFO:teuthology.orchestra.run.smithi016.stdout:287762808871 2024-09-13T16:30:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.975+0000 7fe2b6fea640 1 -- 172.21.15.16:0/1927922066 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe2980a1110 msgr2=0x7fe2980a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.975+0000 7fe2b6fea640 1 --2- 172.21.15.16:0/1927922066 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe2980a1110 0x7fe2980a35d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0005e10 tx=0x7fe2a0005da0 comp rx=0 tx=0).stop 2024-09-13T16:30:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.975+0000 7fe2b6fea640 1 -- 172.21.15.16:0/1927922066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b010a390 msgr2=0x7fe2b00ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.975+0000 7fe2b6fea640 1 --2- 172.21.15.16:0/1927922066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b010a390 0x7fe2b00ff640 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe2a802af40 tx=0x7fe2a802fd30 comp rx=0 tx=0).stop 2024-09-13T16:30:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.975+0000 7fe2b6fea640 1 -- 172.21.15.16:0/1927922066 shutdown_connections 2024-09-13T16:30:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.975+0000 7fe2b6fea640 1 --2- 172.21.15.16:0/1927922066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b010a390 0x7fe2b00ff640 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.975+0000 7fe2b6fea640 1 --2- 172.21.15.16:0/1927922066 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe2980a1110 0x7fe2980a35d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.975+0000 7fe2b6fea640 1 --2- 172.21.15.16:0/1927922066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2b0101880 0x7fe2b00ff100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.975+0000 7fe2b6fea640 1 --2- 172.21.15.16:0/1927922066 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2b0100f60 0x7fe2b00febc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.975+0000 7fe2b6fea640 1 -- 172.21.15.16:0/1927922066 >> 172.21.15.16:0/1927922066 conn(0x7fe2b00fa9a0 msgr2=0x7fe2b00fc130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.975+0000 7fe2b6fea640 1 -- 172.21.15.16:0/1927922066 shutdown_connections 2024-09-13T16:30:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:09.975+0000 7fe2b6fea640 1 -- 172.21.15.16:0/1927922066 wait complete. 2024-09-13T16:30:10.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:10 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1927922066' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-13T16:30:10.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:10 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1927922066' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-13T16:30:10.440 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:10 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1927922066' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-13T16:30:11.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.030+0000 7f102f241640 1 -- 172.21.15.16:0/583577661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10281009b0 msgr2=0x7f1028104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:11.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.030+0000 7f102f241640 1 --2- 172.21.15.16:0/583577661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10281009b0 0x7f1028104a00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f101c009930 tx=0x7f101c02a420 comp rx=0 tx=0).stop 2024-09-13T16:30:11.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.030+0000 7f102f241640 1 -- 172.21.15.16:0/583577661 shutdown_connections 2024-09-13T16:30:11.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.030+0000 7f102f241640 1 --2- 172.21.15.16:0/583577661 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1028105130 0x7f102810ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:11.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.030+0000 7f102f241640 1 --2- 172.21.15.16:0/583577661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10281009b0 0x7f1028104a00 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:11.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.030+0000 7f102f241640 1 --2- 172.21.15.16:0/583577661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1028100000 0x7f10281003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:11.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.030+0000 7f102f241640 1 -- 172.21.15.16:0/583577661 >> 172.21.15.16:0/583577661 conn(0x7f10280fba80 msgr2=0x7f10280fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:11.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.030+0000 7f102f241640 1 -- 172.21.15.16:0/583577661 shutdown_connections 2024-09-13T16:30:11.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.030+0000 7f102f241640 1 -- 172.21.15.16:0/583577661 wait complete. 2024-09-13T16:30:11.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.031+0000 7f102f241640 1 Processor -- start 2024-09-13T16:30:11.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.031+0000 7f102f241640 1 -- start start 2024-09-13T16:30:11.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.031+0000 7f102f241640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1028100000 0x7f10281962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:11.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.031+0000 7f102f241640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10281009b0 0x7f1028196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:11.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.031+0000 7f102f241640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1028105130 0x7f102819ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:11.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.031+0000 7f102f241640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102819b210 con 0x7f1028100000 2024-09-13T16:30:11.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.031+0000 7f102f241640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102819b380 con 0x7f1028105130 2024-09-13T16:30:11.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.031+0000 7f102f241640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102819b4f0 con 0x7f10281009b0 2024-09-13T16:30:11.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.031+0000 7f102cfdd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1028100000 0x7f10281962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:11.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.032+0000 7f102cfdd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1028100000 0x7f10281962d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55810/0 (socket says 172.21.15.16:55810) 2024-09-13T16:30:11.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.032+0000 7f102cfdd640 1 -- 172.21.15.16:0/2168261972 learned_addr learned my addr 172.21.15.16:0/2168261972 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:11.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.032+0000 7f1027fff640 1 --2- 172.21.15.16:0/2168261972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10281009b0 0x7f1028196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:11.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.032+0000 7f102d7de640 1 --2- 172.21.15.16:0/2168261972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1028105130 0x7f102819ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:11.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.032+0000 7f102cfdd640 1 -- 172.21.15.16:0/2168261972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10281009b0 msgr2=0x7f1028196810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:11.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.032+0000 7f102cfdd640 1 --2- 172.21.15.16:0/2168261972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10281009b0 0x7f1028196810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:11.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.032+0000 7f102cfdd640 1 -- 172.21.15.16:0/2168261972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1028105130 msgr2=0x7f102819ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:11.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.032+0000 7f102cfdd640 1 --2- 172.21.15.16:0/2168261972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1028105130 0x7f102819ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:11.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.032+0000 7f102cfdd640 1 -- 172.21.15.16:0/2168261972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f101c009560 con 0x7f1028100000 2024-09-13T16:30:11.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.032+0000 7f102cfdd640 1 --2- 172.21.15.16:0/2168261972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1028100000 0x7f10281962d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f101800d950 tx=0x7f101800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:11.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.033+0000 7f1025ffb640 1 -- 172.21.15.16:0/2168261972 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1018009c30 con 0x7f1028100000 2024-09-13T16:30:11.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.033+0000 7f1025ffb640 1 -- 172.21.15.16:0/2168261972 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1018009d90 con 0x7f1028100000 2024-09-13T16:30:11.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.033+0000 7f102f241640 1 -- 172.21.15.16:0/2168261972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f102819b7d0 con 0x7f1028100000 2024-09-13T16:30:11.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.033+0000 7f1025ffb640 1 -- 172.21.15.16:0/2168261972 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101800f620 con 0x7f1028100000 2024-09-13T16:30:11.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.033+0000 7f102f241640 1 -- 172.21.15.16:0/2168261972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10281a3090 con 0x7f1028100000 2024-09-13T16:30:11.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.034+0000 7f102f241640 1 -- 172.21.15.16:0/2168261972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10281014e0 con 0x7f1028100000 2024-09-13T16:30:11.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.035+0000 7f1025ffb640 1 -- 172.21.15.16:0/2168261972 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f10180109f0 con 0x7f1028100000 2024-09-13T16:30:11.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.036+0000 7f1025ffb640 1 --2- 172.21.15.16:0/2168261972 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f10100a1110 0x7f10100a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:11.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.036+0000 7f1027fff640 1 --2- 172.21.15.16:0/2168261972 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f10100a1110 0x7f10100a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:11.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.036+0000 7f1025ffb640 1 -- 172.21.15.16:0/2168261972 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f10180cc900 con 0x7f1028100000 2024-09-13T16:30:11.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.036+0000 7f1027fff640 1 --2- 172.21.15.16:0/2168261972 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f10100a1110 0x7f10100a35d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f101c006010 tx=0x7f101c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:11.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.040+0000 7f1025ffb640 1 -- 172.21.15.16:0/2168261972 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f1018095c90 con 0x7f1028100000 2024-09-13T16:30:11.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:11 smithi016 ceph-mon[31187]: pgmap v326: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-09-13T16:30:11.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.214+0000 7f102f241640 1 -- 172.21.15.16:0/2168261972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f1028108370 con 0x7f1028100000 2024-09-13T16:30:11.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.215+0000 7f1025ffb640 1 -- 172.21.15.16:0/2168261972 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f10180992b0 con 0x7f1028100000 2024-09-13T16:30:11.216 INFO:teuthology.orchestra.run.smithi016.stdout:343597383709 2024-09-13T16:30:11.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.217+0000 7f102f241640 1 -- 172.21.15.16:0/2168261972 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f10100a1110 msgr2=0x7f10100a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:11.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.217+0000 7f102f241640 1 --2- 172.21.15.16:0/2168261972 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f10100a1110 0x7f10100a35d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f101c006010 tx=0x7f101c000bc0 comp rx=0 tx=0).stop 2024-09-13T16:30:11.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.217+0000 7f102f241640 1 -- 172.21.15.16:0/2168261972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1028100000 msgr2=0x7f10281962d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:11.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.217+0000 7f102f241640 1 --2- 172.21.15.16:0/2168261972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1028100000 0x7f10281962d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f101800d950 tx=0x7f101800de10 comp rx=0 tx=0).stop 2024-09-13T16:30:11.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.217+0000 7f102f241640 1 -- 172.21.15.16:0/2168261972 shutdown_connections 2024-09-13T16:30:11.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.217+0000 7f102f241640 1 --2- 172.21.15.16:0/2168261972 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f10100a1110 0x7f10100a35d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:11.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.217+0000 7f102f241640 1 --2- 172.21.15.16:0/2168261972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1028105130 0x7f102819ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:11.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.217+0000 7f102f241640 1 --2- 172.21.15.16:0/2168261972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10281009b0 0x7f1028196810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:11.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.217+0000 7f102f241640 1 --2- 172.21.15.16:0/2168261972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1028100000 0x7f10281962d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:11.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.217+0000 7f102f241640 1 -- 172.21.15.16:0/2168261972 >> 172.21.15.16:0/2168261972 conn(0x7f10280fba80 msgr2=0x7f10280fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:11.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.217+0000 7f102f241640 1 -- 172.21.15.16:0/2168261972 shutdown_connections 2024-09-13T16:30:11.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:11.217+0000 7f102f241640 1 -- 172.21.15.16:0/2168261972 wait complete. 2024-09-13T16:30:11.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:11 smithi107 ceph-mon[34139]: pgmap v326: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-09-13T16:30:11.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:11 smithi111 ceph-mon[34600]: pgmap v326: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-09-13T16:30:11.906 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383704 got 343597383709 for osd.11 2024-09-13T16:30:11.906 DEBUG:teuthology.parallel:result is None 2024-09-13T16:30:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.075+0000 7f47ed1da640 1 -- 172.21.15.16:0/425711202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e81044f0 msgr2=0x7f47e81048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.075+0000 7f47ed1da640 1 --2- 172.21.15.16:0/425711202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e81044f0 0x7f47e81048d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f47d8009900 tx=0x7f47d802a480 comp rx=0 tx=0).stop 2024-09-13T16:30:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.075+0000 7f47ed1da640 1 -- 172.21.15.16:0/425711202 shutdown_connections 2024-09-13T16:30:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.075+0000 7f47ed1da640 1 --2- 172.21.15.16:0/425711202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47e80ff2b0 0x7f47e810b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.075+0000 7f47ed1da640 1 --2- 172.21.15.16:0/425711202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47e80fe930 0x7f47e80fed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.075+0000 7f47ed1da640 1 --2- 172.21.15.16:0/425711202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e81044f0 0x7f47e81048d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.075+0000 7f47ed1da640 1 -- 172.21.15.16:0/425711202 >> 172.21.15.16:0/425711202 conn(0x7f47e8074130 msgr2=0x7f47e8074540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.075+0000 7f47ed1da640 1 -- 172.21.15.16:0/425711202 shutdown_connections 2024-09-13T16:30:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.075+0000 7f47ed1da640 1 -- 172.21.15.16:0/425711202 wait complete. 2024-09-13T16:30:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.076+0000 7f47ed1da640 1 Processor -- start 2024-09-13T16:30:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.076+0000 7f47ed1da640 1 -- start start 2024-09-13T16:30:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.076+0000 7f47ed1da640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47e80fe930 0x7f47e810cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.076+0000 7f47ed1da640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e80ff2b0 0x7f47e810d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.076+0000 7f47ed1da640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47e81044f0 0x7f47e810d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.076+0000 7f47ed1da640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e81a4590 con 0x7f47e80ff2b0 2024-09-13T16:30:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.076+0000 7f47ed1da640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e81a4700 con 0x7f47e81044f0 2024-09-13T16:30:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.076+0000 7f47ed1da640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e81a48c0 con 0x7f47e80fe930 2024-09-13T16:30:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.076+0000 7f47e659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e80ff2b0 0x7f47e810d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.076+0000 7f47e659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e80ff2b0 0x7f47e810d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55828/0 (socket says 172.21.15.16:55828) 2024-09-13T16:30:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.076+0000 7f47e659c640 1 -- 172.21.15.16:0/3635466553 learned_addr learned my addr 172.21.15.16:0/3635466553 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.077+0000 7f47e6d9d640 1 --2- 172.21.15.16:0/3635466553 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47e80fe930 0x7f47e810cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.077+0000 7f47e759e640 1 --2- 172.21.15.16:0/3635466553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47e81044f0 0x7f47e810d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.077+0000 7f47e659c640 1 -- 172.21.15.16:0/3635466553 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47e80fe930 msgr2=0x7f47e810cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.077+0000 7f47e659c640 1 --2- 172.21.15.16:0/3635466553 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47e80fe930 0x7f47e810cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.077+0000 7f47e659c640 1 -- 172.21.15.16:0/3635466553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47e81044f0 msgr2=0x7f47e810d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.077+0000 7f47e659c640 1 --2- 172.21.15.16:0/3635466553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47e81044f0 0x7f47e810d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.077+0000 7f47e659c640 1 -- 172.21.15.16:0/3635466553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d8009560 con 0x7f47e80ff2b0 2024-09-13T16:30:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.077+0000 7f47e6d9d640 1 --2- 172.21.15.16:0/3635466553 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47e80fe930 0x7f47e810cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:30:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.077+0000 7f47e759e640 1 --2- 172.21.15.16:0/3635466553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47e81044f0 0x7f47e810d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:30:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.077+0000 7f47e659c640 1 --2- 172.21.15.16:0/3635466553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e80ff2b0 0x7f47e810d060 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f47dc00d770 tx=0x7f47dc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.077+0000 7f47cffff640 1 -- 172.21.15.16:0/3635466553 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47dc009c30 con 0x7f47e80ff2b0 2024-09-13T16:30:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.077+0000 7f47cffff640 1 -- 172.21.15.16:0/3635466553 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f47dc00de80 con 0x7f47e80ff2b0 2024-09-13T16:30:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.077+0000 7f47ed1da640 1 -- 172.21.15.16:0/3635466553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47e81a4ba0 con 0x7f47e80ff2b0 2024-09-13T16:30:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.077+0000 7f47cffff640 1 -- 172.21.15.16:0/3635466553 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47dc004d40 con 0x7f47e80ff2b0 2024-09-13T16:30:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.077+0000 7f47ed1da640 1 -- 172.21.15.16:0/3635466553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e81a50a0 con 0x7f47e80ff2b0 2024-09-13T16:30:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.078+0000 7f47ed1da640 1 -- 172.21.15.16:0/3635466553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47e80ffb90 con 0x7f47e80ff2b0 2024-09-13T16:30:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.079+0000 7f47cffff640 1 -- 172.21.15.16:0/3635466553 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f47dc004ea0 con 0x7f47e80ff2b0 2024-09-13T16:30:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.080+0000 7f47cffff640 1 --2- 172.21.15.16:0/3635466553 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f47d00a1190 0x7f47d00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.080+0000 7f47e6d9d640 1 --2- 172.21.15.16:0/3635466553 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f47d00a1190 0x7f47d00a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.081+0000 7f47cffff640 1 -- 172.21.15.16:0/3635466553 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f47dc0cc8b0 con 0x7f47e80ff2b0 2024-09-13T16:30:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.081+0000 7f47e6d9d640 1 --2- 172.21.15.16:0/3635466553 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f47d00a1190 0x7f47d00a3650 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f47d8000c00 tx=0x7f47d8005e00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.085+0000 7f47cffff640 1 -- 172.21.15.16:0/3635466553 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f47dc095b30 con 0x7f47e80ff2b0 2024-09-13T16:30:12.132 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:30:12.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.273+0000 7f47ed1da640 1 -- 172.21.15.16:0/3635466553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f47e80fed70 con 0x7f47e80ff2b0 2024-09-13T16:30:12.274 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:12 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2168261972' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-13T16:30:12.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.273+0000 7f47cffff640 1 -- 172.21.15.16:0/3635466553 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f47dc020090 con 0x7f47e80ff2b0 2024-09-13T16:30:12.275 INFO:teuthology.orchestra.run.smithi016.stdout:163208757307 2024-09-13T16:30:12.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.276+0000 7f47ed1da640 1 -- 172.21.15.16:0/3635466553 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f47d00a1190 msgr2=0x7f47d00a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:12.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.276+0000 7f47ed1da640 1 --2- 172.21.15.16:0/3635466553 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f47d00a1190 0x7f47d00a3650 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f47d8000c00 tx=0x7f47d8005e00 comp rx=0 tx=0).stop 2024-09-13T16:30:12.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.276+0000 7f47ed1da640 1 -- 172.21.15.16:0/3635466553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e80ff2b0 msgr2=0x7f47e810d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:12.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.276+0000 7f47ed1da640 1 --2- 172.21.15.16:0/3635466553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e80ff2b0 0x7f47e810d060 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f47dc00d770 tx=0x7f47dc00dc30 comp rx=0 tx=0).stop 2024-09-13T16:30:12.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.276+0000 7f47ed1da640 1 -- 172.21.15.16:0/3635466553 shutdown_connections 2024-09-13T16:30:12.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.276+0000 7f47ed1da640 1 --2- 172.21.15.16:0/3635466553 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f47d00a1190 0x7f47d00a3650 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:12.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.276+0000 7f47ed1da640 1 --2- 172.21.15.16:0/3635466553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47e81044f0 0x7f47e810d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:12.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.276+0000 7f47ed1da640 1 --2- 172.21.15.16:0/3635466553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e80ff2b0 0x7f47e810d060 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:12.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.276+0000 7f47ed1da640 1 --2- 172.21.15.16:0/3635466553 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47e80fe930 0x7f47e810cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:12.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.276+0000 7f47ed1da640 1 -- 172.21.15.16:0/3635466553 >> 172.21.15.16:0/3635466553 conn(0x7f47e8074130 msgr2=0x7f47e80fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:12.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.276+0000 7f47ed1da640 1 -- 172.21.15.16:0/3635466553 shutdown_connections 2024-09-13T16:30:12.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:12.276+0000 7f47ed1da640 1 -- 172.21.15.16:0/3635466553 wait complete. 2024-09-13T16:30:12.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:12 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/2168261972' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-13T16:30:12.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:12 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/2168261972' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-13T16:30:12.621 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182461 got 107374182466 for osd.3 2024-09-13T16:30:12.621 DEBUG:teuthology.parallel:result is None 2024-09-13T16:30:13.281 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757302 got 163208757307 for osd.5 2024-09-13T16:30:13.281 DEBUG:teuthology.parallel:result is None 2024-09-13T16:30:13.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:13 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3635466553' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-13T16:30:13.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:13 smithi107 ceph-mon[34139]: pgmap v327: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-13T16:30:13.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:13 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:30:13.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:13 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:30:13.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:13 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:30:13.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:13 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:30:13.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:13 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:30:13.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:13 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3635466553' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-13T16:30:13.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:13 smithi016 ceph-mon[31187]: pgmap v327: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-13T16:30:13.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:13 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:30:13.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:13 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:30:13.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:13 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:30:13.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:13 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:30:13.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:13 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:30:13.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:13 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3635466553' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-13T16:30:13.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:13 smithi111 ceph-mon[34600]: pgmap v327: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-13T16:30:13.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:13 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:30:13.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:13 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:30:13.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:13 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:30:13.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:13 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:30:13.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:13 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:30:13.729 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808865 got 287762808871 for osd.9 2024-09-13T16:30:13.730 DEBUG:teuthology.parallel:result is None 2024-09-13T16:30:14.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:14 smithi016 ceph-mon[31187]: pgmap v328: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-13T16:30:14.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:14 smithi111 ceph-mon[34600]: pgmap v328: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-13T16:30:15.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:14 smithi107 ceph-mon[34139]: pgmap v328: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-13T16:30:15.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.687+0000 7f48ba092640 1 -- 172.21.15.16:0/582995331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48b4100990 msgr2=0x7f48b41049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:15.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.687+0000 7f48ba092640 1 --2- 172.21.15.16:0/582995331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48b4100990 0x7f48b41049e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f48a8009930 tx=0x7f48a802a420 comp rx=0 tx=0).stop 2024-09-13T16:30:15.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.687+0000 7f48ba092640 1 -- 172.21.15.16:0/582995331 shutdown_connections 2024-09-13T16:30:15.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.687+0000 7f48ba092640 1 --2- 172.21.15.16:0/582995331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48b4105110 0x7f48b410cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:15.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.687+0000 7f48ba092640 1 --2- 172.21.15.16:0/582995331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48b4100990 0x7f48b41049e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:15.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.687+0000 7f48ba092640 1 --2- 172.21.15.16:0/582995331 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48b40fffe0 0x7f48b41003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:15.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.687+0000 7f48ba092640 1 -- 172.21.15.16:0/582995331 >> 172.21.15.16:0/582995331 conn(0x7f48b40fbae0 msgr2=0x7f48b40fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:15.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.687+0000 7f48ba092640 1 -- 172.21.15.16:0/582995331 shutdown_connections 2024-09-13T16:30:15.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.687+0000 7f48ba092640 1 -- 172.21.15.16:0/582995331 wait complete. 2024-09-13T16:30:15.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.687+0000 7f48ba092640 1 Processor -- start 2024-09-13T16:30:15.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.688+0000 7f48ba092640 1 -- start start 2024-09-13T16:30:15.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.688+0000 7f48ba092640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48b40fffe0 0x7f48b4196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:15.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.688+0000 7f48ba092640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48b4100990 0x7f48b4196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:15.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.688+0000 7f48ba092640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48b4105110 0x7f48b419ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:15.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.688+0000 7f48ba092640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b419b260 con 0x7f48b4100990 2024-09-13T16:30:15.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.688+0000 7f48ba092640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b419b3d0 con 0x7f48b4105110 2024-09-13T16:30:15.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.688+0000 7f48ba092640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b419b540 con 0x7f48b40fffe0 2024-09-13T16:30:15.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.688+0000 7f48b2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48b4100990 0x7f48b4196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:15.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.688+0000 7f48b2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48b4100990 0x7f48b4196860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38684/0 (socket says 172.21.15.16:38684) 2024-09-13T16:30:15.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.688+0000 7f48b2ffd640 1 -- 172.21.15.16:0/3450569212 learned_addr learned my addr 172.21.15.16:0/3450569212 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:15.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.688+0000 7f48b3fff640 1 --2- 172.21.15.16:0/3450569212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48b4105110 0x7f48b419ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:15.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.688+0000 7f48b37fe640 1 --2- 172.21.15.16:0/3450569212 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48b40fffe0 0x7f48b4196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:15.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.688+0000 7f48b2ffd640 1 -- 172.21.15.16:0/3450569212 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48b40fffe0 msgr2=0x7f48b4196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:15.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.688+0000 7f48b2ffd640 1 --2- 172.21.15.16:0/3450569212 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48b40fffe0 0x7f48b4196320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:15.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.688+0000 7f48b2ffd640 1 -- 172.21.15.16:0/3450569212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48b4105110 msgr2=0x7f48b419ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:15.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.688+0000 7f48b2ffd640 1 --2- 172.21.15.16:0/3450569212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48b4105110 0x7f48b419ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:15.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.688+0000 7f48b2ffd640 1 -- 172.21.15.16:0/3450569212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a8009560 con 0x7f48b4100990 2024-09-13T16:30:15.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.688+0000 7f48b37fe640 1 --2- 172.21.15.16:0/3450569212 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48b40fffe0 0x7f48b4196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:30:15.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.689+0000 7f48b2ffd640 1 --2- 172.21.15.16:0/3450569212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48b4100990 0x7f48b4196860 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f48a8005ab0 tx=0x7f48a8032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:15.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.689+0000 7f48b0ff9640 1 -- 172.21.15.16:0/3450569212 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a8032e10 con 0x7f48b4100990 2024-09-13T16:30:15.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.689+0000 7f48b0ff9640 1 -- 172.21.15.16:0/3450569212 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f48a80046c0 con 0x7f48b4100990 2024-09-13T16:30:15.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.689+0000 7f48ba092640 1 -- 172.21.15.16:0/3450569212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b419b7c0 con 0x7f48b4100990 2024-09-13T16:30:15.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.689+0000 7f48b0ff9640 1 -- 172.21.15.16:0/3450569212 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a8003850 con 0x7f48b4100990 2024-09-13T16:30:15.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.689+0000 7f48ba092640 1 -- 172.21.15.16:0/3450569212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b419bae0 con 0x7f48b4100990 2024-09-13T16:30:15.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.691+0000 7f48ba092640 1 -- 172.21.15.16:0/3450569212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b41014c0 con 0x7f48b4100990 2024-09-13T16:30:15.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.692+0000 7f48b0ff9640 1 -- 172.21.15.16:0/3450569212 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f48a8004830 con 0x7f48b4100990 2024-09-13T16:30:15.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.693+0000 7f48b0ff9640 1 --2- 172.21.15.16:0/3450569212 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f489c0a1190 0x7f489c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:15.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.693+0000 7f48b37fe640 1 --2- 172.21.15.16:0/3450569212 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f489c0a1190 0x7f489c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:15.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.693+0000 7f48b0ff9640 1 -- 172.21.15.16:0/3450569212 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f48a80ed9c0 con 0x7f48b4100990 2024-09-13T16:30:15.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.693+0000 7f48b37fe640 1 --2- 172.21.15.16:0/3450569212 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f489c0a1190 0x7f489c0a3650 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f48a40058c0 tx=0x7f48a4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:15.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.697+0000 7f48b0ff9640 1 -- 172.21.15.16:0/3450569212 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f48a80b6c40 con 0x7f48b4100990 2024-09-13T16:30:15.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.872+0000 7f48ba092640 1 -- 172.21.15.16:0/3450569212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f48b41003c0 con 0x7f48b4100990 2024-09-13T16:30:15.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.873+0000 7f48b0ff9640 1 -- 172.21.15.16:0/3450569212 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f48a80ba260 con 0x7f48b4100990 2024-09-13T16:30:15.874 INFO:teuthology.orchestra.run.smithi016.stdout:77309411402 2024-09-13T16:30:15.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.875+0000 7f48ba092640 1 -- 172.21.15.16:0/3450569212 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f489c0a1190 msgr2=0x7f489c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:15.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.875+0000 7f48ba092640 1 --2- 172.21.15.16:0/3450569212 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f489c0a1190 0x7f489c0a3650 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f48a40058c0 tx=0x7f48a4005850 comp rx=0 tx=0).stop 2024-09-13T16:30:15.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.875+0000 7f48ba092640 1 -- 172.21.15.16:0/3450569212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48b4100990 msgr2=0x7f48b4196860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:15.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.875+0000 7f48ba092640 1 --2- 172.21.15.16:0/3450569212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48b4100990 0x7f48b4196860 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f48a8005ab0 tx=0x7f48a8032c20 comp rx=0 tx=0).stop 2024-09-13T16:30:15.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.875+0000 7f48ba092640 1 -- 172.21.15.16:0/3450569212 shutdown_connections 2024-09-13T16:30:15.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.875+0000 7f48ba092640 1 --2- 172.21.15.16:0/3450569212 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f489c0a1190 0x7f489c0a3650 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:15.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.875+0000 7f48ba092640 1 --2- 172.21.15.16:0/3450569212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48b4105110 0x7f48b419ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:15.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.875+0000 7f48ba092640 1 --2- 172.21.15.16:0/3450569212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48b4100990 0x7f48b4196860 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:15.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.875+0000 7f48ba092640 1 --2- 172.21.15.16:0/3450569212 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48b40fffe0 0x7f48b4196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:15.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.875+0000 7f48ba092640 1 -- 172.21.15.16:0/3450569212 >> 172.21.15.16:0/3450569212 conn(0x7f48b40fbae0 msgr2=0x7f48b40fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:15.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.875+0000 7f48ba092640 1 -- 172.21.15.16:0/3450569212 shutdown_connections 2024-09-13T16:30:15.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:15.875+0000 7f48ba092640 1 -- 172.21.15.16:0/3450569212 wait complete. 2024-09-13T16:30:16.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:15 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3450569212' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-13T16:30:16.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:15 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3450569212' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-13T16:30:16.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:15 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3450569212' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-13T16:30:16.896 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411394 got 77309411402 for osd.2 2024-09-13T16:30:16.896 DEBUG:teuthology.parallel:result is None 2024-09-13T16:30:17.125 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:16 smithi016 ceph-mon[31187]: pgmap v329: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-13T16:30:17.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:16 smithi111 ceph-mon[34600]: pgmap v329: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-13T16:30:17.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:16 smithi107 ceph-mon[34139]: pgmap v329: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-13T16:30:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.177+0000 7f3540cd2640 1 -- 172.21.15.16:0/1440346438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f353c104000 msgr2=0x7f353c10bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.177+0000 7f3540cd2640 1 --2- 172.21.15.16:0/1440346438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f353c104000 0x7f353c10bb80 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3534009cd0 tx=0x7f353402a8e0 comp rx=0 tx=0).stop 2024-09-13T16:30:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.177+0000 7f3540cd2640 1 -- 172.21.15.16:0/1440346438 shutdown_connections 2024-09-13T16:30:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.177+0000 7f3540cd2640 1 --2- 172.21.15.16:0/1440346438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f353c104000 0x7f353c10bb80 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.177+0000 7f3540cd2640 1 --2- 172.21.15.16:0/1440346438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f353c0ff880 0x7f353c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.177+0000 7f3540cd2640 1 --2- 172.21.15.16:0/1440346438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f353c0feed0 0x7f353c0ff2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.177+0000 7f3540cd2640 1 -- 172.21.15.16:0/1440346438 >> 172.21.15.16:0/1440346438 conn(0x7f353c0fa9b0 msgr2=0x7f353c0fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.177+0000 7f3540cd2640 1 -- 172.21.15.16:0/1440346438 shutdown_connections 2024-09-13T16:30:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.177+0000 7f3540cd2640 1 -- 172.21.15.16:0/1440346438 wait complete. 2024-09-13T16:30:18.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.178+0000 7f3540cd2640 1 Processor -- start 2024-09-13T16:30:18.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.178+0000 7f3540cd2640 1 -- start start 2024-09-13T16:30:18.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.178+0000 7f3540cd2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f353c0feed0 0x7f353c196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:18.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.178+0000 7f3540cd2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f353c0ff880 0x7f353c1965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:18.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.178+0000 7f3540cd2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f353c104000 0x7f353c19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:18.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.178+0000 7f3540cd2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f353c19afd0 con 0x7f353c0ff880 2024-09-13T16:30:18.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.178+0000 7f3540cd2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f353c19b140 con 0x7f353c0feed0 2024-09-13T16:30:18.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.178+0000 7f3540cd2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f353c19b2b0 con 0x7f353c104000 2024-09-13T16:30:18.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.178+0000 7f3539d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f353c0ff880 0x7f353c1965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:18.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.178+0000 7f3539d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f353c0ff880 0x7f353c1965d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38694/0 (socket says 172.21.15.16:38694) 2024-09-13T16:30:18.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.178+0000 7f3539d9b640 1 -- 172.21.15.16:0/3916637635 learned_addr learned my addr 172.21.15.16:0/3916637635 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:18.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.178+0000 7f353a59c640 1 --2- 172.21.15.16:0/3916637635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f353c0feed0 0x7f353c196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:18.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.178+0000 7f3539d9b640 1 -- 172.21.15.16:0/3916637635 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f353c104000 msgr2=0x7f353c19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:18.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.178+0000 7f3539d9b640 1 --2- 172.21.15.16:0/3916637635 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f353c104000 0x7f353c19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:18.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.178+0000 7f3539d9b640 1 -- 172.21.15.16:0/3916637635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f353c0feed0 msgr2=0x7f353c196090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:18.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.178+0000 7f3539d9b640 1 --2- 172.21.15.16:0/3916637635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f353c0feed0 0x7f353c196090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:18.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.178+0000 7f3539d9b640 1 -- 172.21.15.16:0/3916637635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3534009900 con 0x7f353c0ff880 2024-09-13T16:30:18.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.179+0000 7f3539d9b640 1 --2- 172.21.15.16:0/3916637635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f353c0ff880 0x7f353c1965d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f353000c920 tx=0x7f353000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:18.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.179+0000 7f352b7fe640 1 -- 172.21.15.16:0/3916637635 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3530007c70 con 0x7f353c0ff880 2024-09-13T16:30:18.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.179+0000 7f3540cd2640 1 -- 172.21.15.16:0/3916637635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f353c19b590 con 0x7f353c0ff880 2024-09-13T16:30:18.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.179+0000 7f352b7fe640 1 -- 172.21.15.16:0/3916637635 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3530007dd0 con 0x7f353c0ff880 2024-09-13T16:30:18.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.179+0000 7f352b7fe640 1 -- 172.21.15.16:0/3916637635 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3530018730 con 0x7f353c0ff880 2024-09-13T16:30:18.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.179+0000 7f3540cd2640 1 -- 172.21.15.16:0/3916637635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f353c1a2e50 con 0x7f353c0ff880 2024-09-13T16:30:18.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.180+0000 7f3540cd2640 1 -- 172.21.15.16:0/3916637635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f353c1003b0 con 0x7f353c0ff880 2024-09-13T16:30:18.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.181+0000 7f352b7fe640 1 -- 172.21.15.16:0/3916637635 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f353000f450 con 0x7f353c0ff880 2024-09-13T16:30:18.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.182+0000 7f352b7fe640 1 --2- 172.21.15.16:0/3916637635 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f35240a1110 0x7f35240a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:18.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.182+0000 7f353a59c640 1 --2- 172.21.15.16:0/3916637635 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f35240a1110 0x7f35240a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:18.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.182+0000 7f352b7fe640 1 -- 172.21.15.16:0/3916637635 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f35300ccf50 con 0x7f353c0ff880 2024-09-13T16:30:18.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.182+0000 7f353a59c640 1 --2- 172.21.15.16:0/3916637635 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f35240a1110 0x7f35240a35d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f352c005e10 tx=0x7f352c005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:18.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.186+0000 7f352b7fe640 1 -- 172.21.15.16:0/3916637635 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f3530096230 con 0x7f353c0ff880 2024-09-13T16:30:18.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.358+0000 7f3540cd2640 1 -- 172.21.15.16:0/3916637635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f353c10ebd0 con 0x7f353c0ff880 2024-09-13T16:30:18.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.359+0000 7f352b7fe640 1 -- 172.21.15.16:0/3916637635 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3530099850 con 0x7f353c0ff880 2024-09-13T16:30:18.360 INFO:teuthology.orchestra.run.smithi016.stdout:219043332146 2024-09-13T16:30:18.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.361+0000 7f3540cd2640 1 -- 172.21.15.16:0/3916637635 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f35240a1110 msgr2=0x7f35240a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:18.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.361+0000 7f3540cd2640 1 --2- 172.21.15.16:0/3916637635 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f35240a1110 0x7f35240a35d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f352c005e10 tx=0x7f352c005da0 comp rx=0 tx=0).stop 2024-09-13T16:30:18.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.361+0000 7f3540cd2640 1 -- 172.21.15.16:0/3916637635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f353c0ff880 msgr2=0x7f353c1965d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:18.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.361+0000 7f3540cd2640 1 --2- 172.21.15.16:0/3916637635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f353c0ff880 0x7f353c1965d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f353000c920 tx=0x7f353000cde0 comp rx=0 tx=0).stop 2024-09-13T16:30:18.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.361+0000 7f3540cd2640 1 -- 172.21.15.16:0/3916637635 shutdown_connections 2024-09-13T16:30:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.361+0000 7f3540cd2640 1 --2- 172.21.15.16:0/3916637635 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f353c104000 0x7f353c19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.361+0000 7f3540cd2640 1 --2- 172.21.15.16:0/3916637635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f353c0ff880 0x7f353c1965d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.361+0000 7f3540cd2640 1 --2- 172.21.15.16:0/3916637635 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f35240a1110 0x7f35240a35d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.361+0000 7f3540cd2640 1 --2- 172.21.15.16:0/3916637635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f353c0feed0 0x7f353c196090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.361+0000 7f3540cd2640 1 -- 172.21.15.16:0/3916637635 >> 172.21.15.16:0/3916637635 conn(0x7f353c0fa9b0 msgr2=0x7f353c0fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.361+0000 7f3540cd2640 1 -- 172.21.15.16:0/3916637635 shutdown_connections 2024-09-13T16:30:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:18.361+0000 7f3540cd2640 1 -- 172.21.15.16:0/3916637635 wait complete. 2024-09-13T16:30:18.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:18 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3916637635' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-13T16:30:18.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:18 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3916637635' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-13T16:30:18.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:18 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3916637635' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-13T16:30:19.042 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332141 got 219043332146 for osd.7 2024-09-13T16:30:19.042 DEBUG:teuthology.parallel:result is None 2024-09-13T16:30:19.540 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:19 smithi016 ceph-mon[31187]: pgmap v330: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-13T16:30:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.623+0000 7fd374f98640 1 -- 172.21.15.16:0/4074645397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3700ff880 msgr2=0x7fd3701038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.623+0000 7fd374f98640 1 --2- 172.21.15.16:0/4074645397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3700ff880 0x7fd3701038d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd364009930 tx=0x7fd36402a420 comp rx=0 tx=0).stop 2024-09-13T16:30:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.623+0000 7fd374f98640 1 -- 172.21.15.16:0/4074645397 shutdown_connections 2024-09-13T16:30:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.623+0000 7fd374f98640 1 --2- 172.21.15.16:0/4074645397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd370104000 0x7fd37010bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.623+0000 7fd374f98640 1 --2- 172.21.15.16:0/4074645397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3700ff880 0x7fd3701038d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.623+0000 7fd374f98640 1 --2- 172.21.15.16:0/4074645397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3700feed0 0x7fd3700ff2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.623+0000 7fd374f98640 1 -- 172.21.15.16:0/4074645397 >> 172.21.15.16:0/4074645397 conn(0x7fd3700fa9b0 msgr2=0x7fd3700fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.623+0000 7fd374f98640 1 -- 172.21.15.16:0/4074645397 shutdown_connections 2024-09-13T16:30:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.624+0000 7fd374f98640 1 -- 172.21.15.16:0/4074645397 wait complete. 2024-09-13T16:30:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.624+0000 7fd374f98640 1 Processor -- start 2024-09-13T16:30:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.624+0000 7fd374f98640 1 -- start start 2024-09-13T16:30:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.625+0000 7fd374f98640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3700feed0 0x7fd370100970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.625+0000 7fd374f98640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3700ff880 0x7fd370100eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.625+0000 7fd374f98640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd370104000 0x7fd370101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.625+0000 7fd374f98640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3701a4550 con 0x7fd3700ff880 2024-09-13T16:30:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.625+0000 7fd374f98640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3701a46c0 con 0x7fd3700feed0 2024-09-13T16:30:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.625+0000 7fd374f98640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3701a4880 con 0x7fd370104000 2024-09-13T16:30:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.625+0000 7fd36dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3700ff880 0x7fd370100eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.625+0000 7fd36dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3700ff880 0x7fd370100eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38716/0 (socket says 172.21.15.16:38716) 2024-09-13T16:30:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.625+0000 7fd36dd9b640 1 -- 172.21.15.16:0/1019283482 learned_addr learned my addr 172.21.15.16:0/1019283482 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.625+0000 7fd36e59c640 1 --2- 172.21.15.16:0/1019283482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3700feed0 0x7fd370100970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.625+0000 7fd36ed9d640 1 --2- 172.21.15.16:0/1019283482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd370104000 0x7fd370101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.625+0000 7fd36dd9b640 1 -- 172.21.15.16:0/1019283482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd370104000 msgr2=0x7fd370101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.625+0000 7fd36dd9b640 1 --2- 172.21.15.16:0/1019283482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd370104000 0x7fd370101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.625+0000 7fd36dd9b640 1 -- 172.21.15.16:0/1019283482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3700feed0 msgr2=0x7fd370100970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.625+0000 7fd36dd9b640 1 --2- 172.21.15.16:0/1019283482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3700feed0 0x7fd370100970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.625+0000 7fd36dd9b640 1 -- 172.21.15.16:0/1019283482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd364009560 con 0x7fd3700ff880 2024-09-13T16:30:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.625+0000 7fd36dd9b640 1 --2- 172.21.15.16:0/1019283482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3700ff880 0x7fd370100eb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd36402aa10 tx=0x7fd364032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.625+0000 7fd35f7fe640 1 -- 172.21.15.16:0/1019283482 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd364005210 con 0x7fd3700ff880 2024-09-13T16:30:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.625+0000 7fd374f98640 1 -- 172.21.15.16:0/1019283482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3701a4b00 con 0x7fd3700ff880 2024-09-13T16:30:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.625+0000 7fd35f7fe640 1 -- 172.21.15.16:0/1019283482 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd364005370 con 0x7fd3700ff880 2024-09-13T16:30:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.626+0000 7fd35f7fe640 1 -- 172.21.15.16:0/1019283482 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3640038e0 con 0x7fd3700ff880 2024-09-13T16:30:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.626+0000 7fd374f98640 1 -- 172.21.15.16:0/1019283482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3701a4fa0 con 0x7fd3700ff880 2024-09-13T16:30:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.627+0000 7fd374f98640 1 -- 172.21.15.16:0/1019283482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3701003b0 con 0x7fd3700ff880 2024-09-13T16:30:19.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.628+0000 7fd35f7fe640 1 -- 172.21.15.16:0/1019283482 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fd364003a40 con 0x7fd3700ff880 2024-09-13T16:30:19.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.628+0000 7fd35f7fe640 1 --2- 172.21.15.16:0/1019283482 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd3580a1110 0x7fd3580a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:19.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.629+0000 7fd35f7fe640 1 -- 172.21.15.16:0/1019283482 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd3640eda60 con 0x7fd3700ff880 2024-09-13T16:30:19.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.629+0000 7fd36e59c640 1 --2- 172.21.15.16:0/1019283482 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd3580a1110 0x7fd3580a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:19.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.629+0000 7fd36e59c640 1 --2- 172.21.15.16:0/1019283482 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd3580a1110 0x7fd3580a35d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd3600058c0 tx=0x7fd36000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:19.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.632+0000 7fd35f7fe640 1 -- 172.21.15.16:0/1019283482 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd3640b6d40 con 0x7fd3700ff880 2024-09-13T16:30:19.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:19 smithi111 ceph-mon[34600]: pgmap v330: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-13T16:30:19.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.805+0000 7fd374f98640 1 -- 172.21.15.16:0/1019283482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fd370108250 con 0x7fd3700ff880 2024-09-13T16:30:19.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.805+0000 7fd35f7fe640 1 -- 172.21.15.16:0/1019283482 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd364035090 con 0x7fd3700ff880 2024-09-13T16:30:19.806 INFO:teuthology.orchestra.run.smithi016.stdout:253403070510 2024-09-13T16:30:19.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.807+0000 7fd374f98640 1 -- 172.21.15.16:0/1019283482 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd3580a1110 msgr2=0x7fd3580a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:19.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.807+0000 7fd374f98640 1 --2- 172.21.15.16:0/1019283482 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd3580a1110 0x7fd3580a35d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd3600058c0 tx=0x7fd36000a2d0 comp rx=0 tx=0).stop 2024-09-13T16:30:19.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.807+0000 7fd374f98640 1 -- 172.21.15.16:0/1019283482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3700ff880 msgr2=0x7fd370100eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:19.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.807+0000 7fd374f98640 1 --2- 172.21.15.16:0/1019283482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3700ff880 0x7fd370100eb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd36402aa10 tx=0x7fd364032c20 comp rx=0 tx=0).stop 2024-09-13T16:30:19.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.807+0000 7fd374f98640 1 -- 172.21.15.16:0/1019283482 shutdown_connections 2024-09-13T16:30:19.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.807+0000 7fd374f98640 1 --2- 172.21.15.16:0/1019283482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd370104000 0x7fd370101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:19.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.807+0000 7fd374f98640 1 --2- 172.21.15.16:0/1019283482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3700ff880 0x7fd370100eb0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:19.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.807+0000 7fd374f98640 1 --2- 172.21.15.16:0/1019283482 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd3580a1110 0x7fd3580a35d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:19.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.807+0000 7fd374f98640 1 --2- 172.21.15.16:0/1019283482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3700feed0 0x7fd370100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:19.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.807+0000 7fd374f98640 1 -- 172.21.15.16:0/1019283482 >> 172.21.15.16:0/1019283482 conn(0x7fd3700fa9b0 msgr2=0x7fd370068240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:19.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.807+0000 7fd374f98640 1 -- 172.21.15.16:0/1019283482 shutdown_connections 2024-09-13T16:30:19.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.807+0000 7fd374f98640 1 -- 172.21.15.16:0/1019283482 wait complete. 2024-09-13T16:30:19.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:19 smithi107 ceph-mon[34139]: pgmap v330: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-13T16:30:19.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.926+0000 7f01c0562640 1 -- 172.21.15.16:0/2610562607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01b8101880 msgr2=0x7f01b8109d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:19.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.926+0000 7f01c0562640 1 --2- 172.21.15.16:0/2610562607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01b8101880 0x7f01b8109d70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f01a8009900 tx=0x7f01a802a480 comp rx=0 tx=0).stop 2024-09-13T16:30:19.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.926+0000 7f01c0562640 1 -- 172.21.15.16:0/2610562607 shutdown_connections 2024-09-13T16:30:19.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.926+0000 7f01c0562640 1 --2- 172.21.15.16:0/2610562607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01b810a390 0x7f01b810c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.926+0000 7f01c0562640 1 --2- 172.21.15.16:0/2610562607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01b8101880 0x7f01b8109d70 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.926+0000 7f01c0562640 1 --2- 172.21.15.16:0/2610562607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01b8100f60 0x7f01b8101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.926+0000 7f01c0562640 1 -- 172.21.15.16:0/2610562607 >> 172.21.15.16:0/2610562607 conn(0x7f01b80fa9a0 msgr2=0x7f01b80fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.926+0000 7f01c0562640 1 -- 172.21.15.16:0/2610562607 shutdown_connections 2024-09-13T16:30:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.926+0000 7f01c0562640 1 -- 172.21.15.16:0/2610562607 wait complete. 2024-09-13T16:30:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.927+0000 7f01c0562640 1 Processor -- start 2024-09-13T16:30:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.927+0000 7f01c0562640 1 -- start start 2024-09-13T16:30:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.927+0000 7f01c0562640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01b8100f60 0x7f01b81961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.927+0000 7f01c0562640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01b8101880 0x7f01b8196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.927+0000 7f01c0562640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01b810a390 0x7f01b819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.927+0000 7f01c0562640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b819b130 con 0x7f01b8100f60 2024-09-13T16:30:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.927+0000 7f01c0562640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b819b2a0 con 0x7f01b8101880 2024-09-13T16:30:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.927+0000 7f01c0562640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b819b410 con 0x7f01b810a390 2024-09-13T16:30:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.927+0000 7f01be2fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01b8100f60 0x7f01b81961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.927+0000 7f01be2fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01b8100f60 0x7f01b81961f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38746/0 (socket says 172.21.15.16:38746) 2024-09-13T16:30:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.927+0000 7f01be2fe640 1 -- 172.21.15.16:0/2485424435 learned_addr learned my addr 172.21.15.16:0/2485424435 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.927+0000 7f01bdafd640 1 --2- 172.21.15.16:0/2485424435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01b8101880 0x7f01b8196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.927+0000 7f01beaff640 1 --2- 172.21.15.16:0/2485424435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01b810a390 0x7f01b819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.927+0000 7f01be2fe640 1 -- 172.21.15.16:0/2485424435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01b810a390 msgr2=0x7f01b819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.927+0000 7f01be2fe640 1 --2- 172.21.15.16:0/2485424435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01b810a390 0x7f01b819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.927+0000 7f01be2fe640 1 -- 172.21.15.16:0/2485424435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01b8101880 msgr2=0x7f01b8196730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.927+0000 7f01be2fe640 1 --2- 172.21.15.16:0/2485424435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01b8101880 0x7f01b8196730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.927+0000 7f01be2fe640 1 -- 172.21.15.16:0/2485424435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a8009560 con 0x7f01b8100f60 2024-09-13T16:30:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.927+0000 7f01beaff640 1 --2- 172.21.15.16:0/2485424435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01b810a390 0x7f01b819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:30:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.927+0000 7f01bdafd640 1 --2- 172.21.15.16:0/2485424435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01b8101880 0x7f01b8196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:30:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.928+0000 7f01be2fe640 1 --2- 172.21.15.16:0/2485424435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01b8100f60 0x7f01b81961f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f01b400e9f0 tx=0x7f01b400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.928+0000 7f01af7fe640 1 -- 172.21.15.16:0/2485424435 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b4009720 con 0x7f01b8100f60 2024-09-13T16:30:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.928+0000 7f01af7fe640 1 -- 172.21.15.16:0/2485424435 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f01b400cd80 con 0x7f01b8100f60 2024-09-13T16:30:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.928+0000 7f01c0562640 1 -- 172.21.15.16:0/2485424435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b819b6f0 con 0x7f01b8100f60 2024-09-13T16:30:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.928+0000 7f01af7fe640 1 -- 172.21.15.16:0/2485424435 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b4010430 con 0x7f01b8100f60 2024-09-13T16:30:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.928+0000 7f01c0562640 1 -- 172.21.15.16:0/2485424435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b81a2fb0 con 0x7f01b8100f60 2024-09-13T16:30:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.929+0000 7f01c0562640 1 -- 172.21.15.16:0/2485424435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01b8069000 con 0x7f01b8100f60 2024-09-13T16:30:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.930+0000 7f01af7fe640 1 -- 172.21.15.16:0/2485424435 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f01b4004fb0 con 0x7f01b8100f60 2024-09-13T16:30:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.931+0000 7f01af7fe640 1 --2- 172.21.15.16:0/2485424435 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f01a40a1190 0x7f01a40a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.931+0000 7f01bdafd640 1 --2- 172.21.15.16:0/2485424435 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f01a40a1190 0x7f01a40a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.931+0000 7f01af7fe640 1 -- 172.21.15.16:0/2485424435 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f01b4014070 con 0x7f01b8100f60 2024-09-13T16:30:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.931+0000 7f01bdafd640 1 --2- 172.21.15.16:0/2485424435 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f01a40a1190 0x7f01a40a3650 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f01a8000c00 tx=0x7f01a8005e00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:19.935+0000 7f01af7fe640 1 -- 172.21.15.16:0/2485424435 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f01b40964d0 con 0x7f01b8100f60 2024-09-13T16:30:20.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:20.110+0000 7f01c0562640 1 -- 172.21.15.16:0/2485424435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f01b8101340 con 0x7f01b8100f60 2024-09-13T16:30:20.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:20.111+0000 7f01af7fe640 1 -- 172.21.15.16:0/2485424435 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f01b4099af0 con 0x7f01b8100f60 2024-09-13T16:30:20.112 INFO:teuthology.orchestra.run.smithi016.stdout:197568495672 2024-09-13T16:30:20.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:20.112+0000 7f01c0562640 1 -- 172.21.15.16:0/2485424435 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f01a40a1190 msgr2=0x7f01a40a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:20.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:20.112+0000 7f01c0562640 1 --2- 172.21.15.16:0/2485424435 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f01a40a1190 0x7f01a40a3650 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f01a8000c00 tx=0x7f01a8005e00 comp rx=0 tx=0).stop 2024-09-13T16:30:20.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:20.113+0000 7f01c0562640 1 -- 172.21.15.16:0/2485424435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01b8100f60 msgr2=0x7f01b81961f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:20.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:20.113+0000 7f01c0562640 1 --2- 172.21.15.16:0/2485424435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01b8100f60 0x7f01b81961f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f01b400e9f0 tx=0x7f01b400edf0 comp rx=0 tx=0).stop 2024-09-13T16:30:20.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:20.113+0000 7f01c0562640 1 -- 172.21.15.16:0/2485424435 shutdown_connections 2024-09-13T16:30:20.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:20.113+0000 7f01c0562640 1 --2- 172.21.15.16:0/2485424435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01b810a390 0x7f01b819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:20.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:20.113+0000 7f01c0562640 1 --2- 172.21.15.16:0/2485424435 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f01a40a1190 0x7f01a40a3650 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:20.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:20.113+0000 7f01c0562640 1 --2- 172.21.15.16:0/2485424435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01b8101880 0x7f01b8196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:20.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:20.113+0000 7f01c0562640 1 --2- 172.21.15.16:0/2485424435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01b8100f60 0x7f01b81961f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:20.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:20.113+0000 7f01c0562640 1 -- 172.21.15.16:0/2485424435 >> 172.21.15.16:0/2485424435 conn(0x7f01b80fa9a0 msgr2=0x7f01b80fc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:20.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:20.113+0000 7f01c0562640 1 -- 172.21.15.16:0/2485424435 shutdown_connections 2024-09-13T16:30:20.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:20.113+0000 7f01c0562640 1 -- 172.21.15.16:0/2485424435 wait complete. 2024-09-13T16:30:20.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:20 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1019283482' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-13T16:30:20.685 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:20 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2485424435' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-13T16:30:20.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:20 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1019283482' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-13T16:30:20.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:20 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/2485424435' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-13T16:30:20.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:20 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1019283482' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-13T16:30:20.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:20 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/2485424435' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-13T16:30:21.028 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070504 got 253403070510 for osd.8 2024-09-13T16:30:21.029 DEBUG:teuthology.parallel:result is None 2024-09-13T16:30:21.325 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495667 got 197568495672 for osd.6 2024-09-13T16:30:21.325 DEBUG:teuthology.parallel:result is None 2024-09-13T16:30:21.325 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-13T16:30:21.325 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-13T16:30:21.543 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:30:21.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:21 smithi016 ceph-mon[31187]: pgmap v331: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T16:30:21.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:21 smithi107 ceph-mon[34139]: pgmap v331: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T16:30:21.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:21 smithi111 ceph-mon[34600]: pgmap v331: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T16:30:22.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:22 smithi111 ceph-mon[34600]: pgmap v332: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-13T16:30:22.966 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:22 smithi016 ceph-mon[31187]: pgmap v332: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-13T16:30:23.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:22 smithi107 ceph-mon[34139]: pgmap v332: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-13T16:30:23.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.318+0000 7fcac1ef8640 1 -- 172.21.15.16:0/2927374072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcabc0ff880 msgr2=0x7fcabc1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:23.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.318+0000 7fcac1ef8640 1 --2- 172.21.15.16:0/2927374072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcabc0ff880 0x7fcabc1038d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009930 tx=0x7fcab002a420 comp rx=0 tx=0).stop 2024-09-13T16:30:23.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.318+0000 7fcac1ef8640 1 -- 172.21.15.16:0/2927374072 shutdown_connections 2024-09-13T16:30:23.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.318+0000 7fcac1ef8640 1 --2- 172.21.15.16:0/2927374072 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcabc104000 0x7fcabc10bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:23.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.318+0000 7fcac1ef8640 1 --2- 172.21.15.16:0/2927374072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcabc0ff880 0x7fcabc1038d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:23.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.318+0000 7fcac1ef8640 1 --2- 172.21.15.16:0/2927374072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcabc0feed0 0x7fcabc0ff2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:23.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.318+0000 7fcac1ef8640 1 -- 172.21.15.16:0/2927374072 >> 172.21.15.16:0/2927374072 conn(0x7fcabc0fa9b0 msgr2=0x7fcabc0fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:23.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.318+0000 7fcac1ef8640 1 -- 172.21.15.16:0/2927374072 shutdown_connections 2024-09-13T16:30:23.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.318+0000 7fcac1ef8640 1 -- 172.21.15.16:0/2927374072 wait complete. 2024-09-13T16:30:23.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.318+0000 7fcac1ef8640 1 Processor -- start 2024-09-13T16:30:23.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.319+0000 7fcac1ef8640 1 -- start start 2024-09-13T16:30:23.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.319+0000 7fcac1ef8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcabc0feed0 0x7fcabc196180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:23.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.319+0000 7fcac1ef8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcabc0ff880 0x7fcabc1966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:23.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.319+0000 7fcac1ef8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcabc104000 0x7fcabc19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:23.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.319+0000 7fcac1ef8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc19b0c0 con 0x7fcabc0ff880 2024-09-13T16:30:23.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.319+0000 7fcac1ef8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc19b230 con 0x7fcabc0feed0 2024-09-13T16:30:23.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.319+0000 7fcac1ef8640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc19b3a0 con 0x7fcabc104000 2024-09-13T16:30:23.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.319+0000 7fcabaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcabc0ff880 0x7fcabc1966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:23.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.319+0000 7fcabaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcabc0ff880 0x7fcabc1966c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38766/0 (socket says 172.21.15.16:38766) 2024-09-13T16:30:23.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.319+0000 7fcabaffd640 1 -- 172.21.15.16:0/698607503 learned_addr learned my addr 172.21.15.16:0/698607503 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:23.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.319+0000 7fcabbfff640 1 --2- 172.21.15.16:0/698607503 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcabc104000 0x7fcabc19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:23.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.319+0000 7fcabb7fe640 1 --2- 172.21.15.16:0/698607503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcabc0feed0 0x7fcabc196180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:23.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.319+0000 7fcabaffd640 1 -- 172.21.15.16:0/698607503 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcabc104000 msgr2=0x7fcabc19aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:23.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.319+0000 7fcabaffd640 1 --2- 172.21.15.16:0/698607503 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcabc104000 0x7fcabc19aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:23.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.319+0000 7fcabaffd640 1 -- 172.21.15.16:0/698607503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcabc0feed0 msgr2=0x7fcabc196180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:23.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.319+0000 7fcabaffd640 1 --2- 172.21.15.16:0/698607503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcabc0feed0 0x7fcabc196180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:23.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.319+0000 7fcabaffd640 1 -- 172.21.15.16:0/698607503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab0009560 con 0x7fcabc0ff880 2024-09-13T16:30:23.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.320+0000 7fcabaffd640 1 --2- 172.21.15.16:0/698607503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcabc0ff880 0x7fcabc1966c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcab0005ab0 tx=0x7fcab0032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:23.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.320+0000 7fcab8ff9640 1 -- 172.21.15.16:0/698607503 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab0032e10 con 0x7fcabc0ff880 2024-09-13T16:30:23.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.320+0000 7fcab8ff9640 1 -- 172.21.15.16:0/698607503 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcab00046c0 con 0x7fcabc0ff880 2024-09-13T16:30:23.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.320+0000 7fcac1ef8640 1 -- 172.21.15.16:0/698607503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcabc19b620 con 0x7fcabc0ff880 2024-09-13T16:30:23.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.320+0000 7fcab8ff9640 1 -- 172.21.15.16:0/698607503 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab0003850 con 0x7fcabc0ff880 2024-09-13T16:30:23.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.320+0000 7fcac1ef8640 1 -- 172.21.15.16:0/698607503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcabc19b940 con 0x7fcabc0ff880 2024-09-13T16:30:23.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.321+0000 7fcac1ef8640 1 -- 172.21.15.16:0/698607503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcabc1003b0 con 0x7fcabc0ff880 2024-09-13T16:30:23.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.322+0000 7fcab8ff9640 1 -- 172.21.15.16:0/698607503 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fcab0004830 con 0x7fcabc0ff880 2024-09-13T16:30:23.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.323+0000 7fcab8ff9640 1 --2- 172.21.15.16:0/698607503 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fcaa40a1190 0x7fcaa40a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:23.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.323+0000 7fcabb7fe640 1 --2- 172.21.15.16:0/698607503 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fcaa40a1190 0x7fcaa40a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:23.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.323+0000 7fcab8ff9640 1 -- 172.21.15.16:0/698607503 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fcab00ed8c0 con 0x7fcabc0ff880 2024-09-13T16:30:23.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.323+0000 7fcabb7fe640 1 --2- 172.21.15.16:0/698607503 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fcaa40a1190 0x7fcaa40a3650 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00bbe0 tx=0x7fcaac00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:23.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.327+0000 7fcab8ff9640 1 -- 172.21.15.16:0/698607503 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fcab00b6c50 con 0x7fcabc0ff880 2024-09-13T16:30:23.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.494+0000 7fcac1ef8640 1 -- 172.21.15.16:0/698607503 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcabc10eb80 con 0x7fcaa40a1190 2024-09-13T16:30:23.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.506+0000 7fcab8ff9640 1 -- 172.21.15.16:0/698607503 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+428600 (secure 0 0 0) 0x7fcabc10eb80 con 0x7fcaa40a1190 2024-09-13T16:30:23.513 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:30:23.515 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-13T16:30:23.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.510+0000 7fcac1ef8640 1 -- 172.21.15.16:0/698607503 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fcaa40a1190 msgr2=0x7fcaa40a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:23.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.510+0000 7fcac1ef8640 1 --2- 172.21.15.16:0/698607503 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fcaa40a1190 0x7fcaa40a3650 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00bbe0 tx=0x7fcaac00b390 comp rx=0 tx=0).stop 2024-09-13T16:30:23.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.510+0000 7fcac1ef8640 1 -- 172.21.15.16:0/698607503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcabc0ff880 msgr2=0x7fcabc1966c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:23.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.510+0000 7fcac1ef8640 1 --2- 172.21.15.16:0/698607503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcabc0ff880 0x7fcabc1966c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcab0005ab0 tx=0x7fcab0032c20 comp rx=0 tx=0).stop 2024-09-13T16:30:23.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.510+0000 7fcac1ef8640 1 -- 172.21.15.16:0/698607503 shutdown_connections 2024-09-13T16:30:23.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.510+0000 7fcac1ef8640 1 --2- 172.21.15.16:0/698607503 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcabc104000 0x7fcabc19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:23.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.510+0000 7fcac1ef8640 1 --2- 172.21.15.16:0/698607503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcabc0ff880 0x7fcabc1966c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:23.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.510+0000 7fcac1ef8640 1 --2- 172.21.15.16:0/698607503 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fcaa40a1190 0x7fcaa40a3650 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:23.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.510+0000 7fcac1ef8640 1 --2- 172.21.15.16:0/698607503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcabc0feed0 0x7fcabc196180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:23.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.510+0000 7fcac1ef8640 1 -- 172.21.15.16:0/698607503 >> 172.21.15.16:0/698607503 conn(0x7fcabc0fa9b0 msgr2=0x7fcabc068200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:23.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.510+0000 7fcac1ef8640 1 -- 172.21.15.16:0/698607503 shutdown_connections 2024-09-13T16:30:23.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:23.510+0000 7fcac1ef8640 1 -- 172.21.15.16:0/698607503 wait complete. 2024-09-13T16:30:23.849 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:23 smithi016 ceph-mon[31187]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T16:30:23.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:23 smithi111 ceph-mon[34600]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T16:30:24.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:23 smithi107 ceph-mon[34139]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T16:30:24.217 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-09-13T16:30:22.593909+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661318,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":930,"num_read_kb":595,"num_write":419,"num_write_kb":3079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1181498,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":347,"ondisk_log_size":347,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365500,"kb_used_data":12568,"kb_used_omap":18,"kb_used_meta":352749,"kb_avail":1124576324,"statfs":{"total":1151940427776,"available":1151566155776,"internally_reserved":0,"allocated":12869632,"data_stored":8182406,"data_compressed":398091,"data_compressed_allocated":540672,"data_compressed_original":2580480,"omap_allocated":19131,"internal_metadata":361215301},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003605"},"pg_stats":[{"pgid":"2.1f","version":"91'13","reported_seq":131,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T16:30:20.740633+0000","last_change":"2024-09-13T16:29:05.647872+0000","last_active":"2024-09-13T16:30:20.740633+0000","last_peered":"2024-09-13T16:30:20.740633+0000","last_clean":"2024-09-13T16:30:20.740633+0000","last_became_active":"2024-09-13T16:29:05.637050+0000","last_became_peered":"2024-09-13T16:29:05.637050+0000","last_unstale":"2024-09-13T16:30:20.740633+0000","last_undegraded":"2024-09-13T16:30:20.740633+0000","last_fullsized":"2024-09-13T16:30:20.740633+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:04.113840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":77,"num_read_kb":77,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.690256+0000","last_change":"2024-09-13T16:28:39.210024+0000","last_active":"2024-09-13T16:30:04.690256+0000","last_peered":"2024-09-13T16:30:04.690256+0000","last_clean":"2024-09-13T16:30:04.690256+0000","last_became_active":"2024-09-13T16:28:39.209751+0000","last_became_peered":"2024-09-13T16:28:39.209751+0000","last_unstale":"2024-09-13T16:30:04.690256+0000","last_undegraded":"2024-09-13T16:30:04.690256+0000","last_fullsized":"2024-09-13T16:30:04.690256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:34:24.357779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.726086+0000","last_change":"2024-09-13T16:28:43.230223+0000","last_active":"2024-09-13T16:30:04.726086+0000","last_peered":"2024-09-13T16:30:04.726086+0000","last_clean":"2024-09-13T16:30:04.726086+0000","last_became_active":"2024-09-13T16:28:43.228992+0000","last_became_peered":"2024-09-13T16:28:43.228992+0000","last_unstale":"2024-09-13T16:30:04.726086+0000","last_undegraded":"2024-09-13T16:30:04.726086+0000","last_fullsized":"2024-09-13T16:30:04.726086+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:45:37.252865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.690413+0000","last_change":"2024-09-13T16:28:38.850198+0000","last_active":"2024-09-13T16:30:04.690413+0000","last_peered":"2024-09-13T16:30:04.690413+0000","last_clean":"2024-09-13T16:30:04.690413+0000","last_became_active":"2024-09-13T16:28:38.849780+0000","last_became_peered":"2024-09-13T16:28:38.849780+0000","last_unstale":"2024-09-13T16:30:04.690413+0000","last_undegraded":"2024-09-13T16:30:04.690413+0000","last_fullsized":"2024-09-13T16:30:04.690413+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:09.653474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689328+0000","last_change":"2024-09-13T16:28:39.203030+0000","last_active":"2024-09-13T16:30:04.689328+0000","last_peered":"2024-09-13T16:30:04.689328+0000","last_clean":"2024-09-13T16:30:04.689328+0000","last_became_active":"2024-09-13T16:28:39.202464+0000","last_became_peered":"2024-09-13T16:28:39.202464+0000","last_unstale":"2024-09-13T16:30:04.689328+0000","last_undegraded":"2024-09-13T16:30:04.689328+0000","last_fullsized":"2024-09-13T16:30:04.689328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:57.528836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.684640+0000","last_change":"2024-09-13T16:28:43.224358+0000","last_active":"2024-09-13T16:30:04.684640+0000","last_peered":"2024-09-13T16:30:04.684640+0000","last_clean":"2024-09-13T16:30:04.684640+0000","last_became_active":"2024-09-13T16:28:43.223955+0000","last_became_peered":"2024-09-13T16:28:43.223955+0000","last_unstale":"2024-09-13T16:30:04.684640+0000","last_undegraded":"2024-09-13T16:30:04.684640+0000","last_fullsized":"2024-09-13T16:30:04.684640+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:53:09.169704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'12","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.685060+0000","last_change":"2024-09-13T16:28:38.846051+0000","last_active":"2024-09-13T16:30:04.685060+0000","last_peered":"2024-09-13T16:30:04.685060+0000","last_clean":"2024-09-13T16:30:04.685060+0000","last_became_active":"2024-09-13T16:28:38.845762+0000","last_became_peered":"2024-09-13T16:28:38.845762+0000","last_unstale":"2024-09-13T16:30:04.685060+0000","last_undegraded":"2024-09-13T16:30:04.685060+0000","last_fullsized":"2024-09-13T16:30:04.685060+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:33:47.048582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.966213+0000","last_change":"2024-09-13T16:28:39.210516+0000","last_active":"2024-09-13T16:30:04.966213+0000","last_peered":"2024-09-13T16:30:04.966213+0000","last_clean":"2024-09-13T16:30:04.966213+0000","last_became_active":"2024-09-13T16:28:39.210214+0000","last_became_peered":"2024-09-13T16:28:39.210214+0000","last_unstale":"2024-09-13T16:30:04.966213+0000","last_undegraded":"2024-09-13T16:30:04.966213+0000","last_fullsized":"2024-09-13T16:30:04.966213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:51:58.212685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.684668+0000","last_change":"2024-09-13T16:28:43.230530+0000","last_active":"2024-09-13T16:30:04.684668+0000","last_peered":"2024-09-13T16:30:04.684668+0000","last_clean":"2024-09-13T16:30:04.684668+0000","last_became_active":"2024-09-13T16:28:43.230154+0000","last_became_peered":"2024-09-13T16:28:43.230154+0000","last_unstale":"2024-09-13T16:30:04.684668+0000","last_undegraded":"2024-09-13T16:30:04.684668+0000","last_fullsized":"2024-09-13T16:30:04.684668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:39:52.362177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.965635+0000","last_change":"2024-09-13T16:29:05.653460+0000","last_active":"2024-09-13T16:30:04.965635+0000","last_peered":"2024-09-13T16:30:04.965635+0000","last_clean":"2024-09-13T16:30:04.965635+0000","last_became_active":"2024-09-13T16:29:05.641836+0000","last_became_peered":"2024-09-13T16:29:05.641836+0000","last_unstale":"2024-09-13T16:30:04.965635+0000","last_undegraded":"2024-09-13T16:30:04.965635+0000","last_fullsized":"2024-09-13T16:30:04.965635+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:22:30.051033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.687612+0000","last_change":"2024-09-13T16:28:39.202381+0000","last_active":"2024-09-13T16:30:04.687612+0000","last_peered":"2024-09-13T16:30:04.687612+0000","last_clean":"2024-09-13T16:30:04.687612+0000","last_became_active":"2024-09-13T16:28:39.202023+0000","last_became_peered":"2024-09-13T16:28:39.202023+0000","last_unstale":"2024-09-13T16:30:04.687612+0000","last_undegraded":"2024-09-13T16:30:04.687612+0000","last_fullsized":"2024-09-13T16:30:04.687612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:55:46.445085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.684511+0000","last_change":"2024-09-13T16:28:43.230623+0000","last_active":"2024-09-13T16:30:04.684511+0000","last_peered":"2024-09-13T16:30:04.684511+0000","last_clean":"2024-09-13T16:30:04.684511+0000","last_became_active":"2024-09-13T16:28:43.230240+0000","last_became_peered":"2024-09-13T16:28:43.230240+0000","last_unstale":"2024-09-13T16:30:04.684511+0000","last_undegraded":"2024-09-13T16:30:04.684511+0000","last_fullsized":"2024-09-13T16:30:04.684511+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:04:07.723269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689169+0000","last_change":"2024-09-13T16:28:39.201896+0000","last_active":"2024-09-13T16:30:04.689169+0000","last_peered":"2024-09-13T16:30:04.689169+0000","last_clean":"2024-09-13T16:30:04.689169+0000","last_became_active":"2024-09-13T16:28:39.201570+0000","last_became_peered":"2024-09-13T16:28:39.201570+0000","last_unstale":"2024-09-13T16:30:04.689169+0000","last_undegraded":"2024-09-13T16:30:04.689169+0000","last_fullsized":"2024-09-13T16:30:04.689169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:07.532058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.684677+0000","last_change":"2024-09-13T16:28:38.845998+0000","last_active":"2024-09-13T16:30:04.684677+0000","last_peered":"2024-09-13T16:30:04.684677+0000","last_clean":"2024-09-13T16:30:04.684677+0000","last_became_active":"2024-09-13T16:28:38.845451+0000","last_became_peered":"2024-09-13T16:28:38.845451+0000","last_unstale":"2024-09-13T16:30:04.684677+0000","last_undegraded":"2024-09-13T16:30:04.684677+0000","last_fullsized":"2024-09-13T16:30:04.684677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:15.696276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.685320+0000","last_change":"2024-09-13T16:28:43.293598+0000","last_active":"2024-09-13T16:30:04.685320+0000","last_peered":"2024-09-13T16:30:04.685320+0000","last_clean":"2024-09-13T16:30:04.685320+0000","last_became_active":"2024-09-13T16:28:43.293045+0000","last_became_peered":"2024-09-13T16:28:43.293045+0000","last_unstale":"2024-09-13T16:30:04.685320+0000","last_undegraded":"2024-09-13T16:30:04.685320+0000","last_fullsized":"2024-09-13T16:30:04.685320+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:01.839626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689878+0000","last_change":"2024-09-13T16:28:39.214302+0000","last_active":"2024-09-13T16:30:04.689878+0000","last_peered":"2024-09-13T16:30:04.689878+0000","last_clean":"2024-09-13T16:30:04.689878+0000","last_became_active":"2024-09-13T16:28:39.213672+0000","last_became_peered":"2024-09-13T16:28:39.213672+0000","last_unstale":"2024-09-13T16:30:04.689878+0000","last_undegraded":"2024-09-13T16:30:04.689878+0000","last_fullsized":"2024-09-13T16:30:04.689878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:03:25.440603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"91'3","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.726254+0000","last_change":"2024-09-13T16:28:38.840626+0000","last_active":"2024-09-13T16:30:04.726254+0000","last_peered":"2024-09-13T16:30:04.726254+0000","last_clean":"2024-09-13T16:30:04.726254+0000","last_became_active":"2024-09-13T16:28:38.840344+0000","last_became_peered":"2024-09-13T16:28:38.840344+0000","last_unstale":"2024-09-13T16:30:04.726254+0000","last_undegraded":"2024-09-13T16:30:04.726254+0000","last_fullsized":"2024-09-13T16:30:04.726254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:03:10.850301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.684639+0000","last_change":"2024-09-13T16:28:43.229163+0000","last_active":"2024-09-13T16:30:04.684639+0000","last_peered":"2024-09-13T16:30:04.684639+0000","last_clean":"2024-09-13T16:30:04.684639+0000","last_became_active":"2024-09-13T16:28:43.228783+0000","last_became_peered":"2024-09-13T16:28:43.228783+0000","last_unstale":"2024-09-13T16:30:04.684639+0000","last_undegraded":"2024-09-13T16:30:04.684639+0000","last_fullsized":"2024-09-13T16:30:04.684639+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:48.183837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.965630+0000","last_change":"2024-09-13T16:28:39.209247+0000","last_active":"2024-09-13T16:30:04.965630+0000","last_peered":"2024-09-13T16:30:04.965630+0000","last_clean":"2024-09-13T16:30:04.965630+0000","last_became_active":"2024-09-13T16:28:39.208921+0000","last_became_peered":"2024-09-13T16:28:39.208921+0000","last_unstale":"2024-09-13T16:30:04.965630+0000","last_undegraded":"2024-09-13T16:30:04.965630+0000","last_fullsized":"2024-09-13T16:30:04.965630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:32:41.383029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689356+0000","last_change":"2024-09-13T16:28:38.845904+0000","last_active":"2024-09-13T16:30:04.689356+0000","last_peered":"2024-09-13T16:30:04.689356+0000","last_clean":"2024-09-13T16:30:04.689356+0000","last_became_active":"2024-09-13T16:28:38.845639+0000","last_became_peered":"2024-09-13T16:28:38.845639+0000","last_unstale":"2024-09-13T16:30:04.689356+0000","last_undegraded":"2024-09-13T16:30:04.689356+0000","last_fullsized":"2024-09-13T16:30:04.689356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:25:34.148607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.685063+0000","last_change":"2024-09-13T16:29:05.640063+0000","last_active":"2024-09-13T16:30:04.685063+0000","last_peered":"2024-09-13T16:30:04.685063+0000","last_clean":"2024-09-13T16:30:04.685063+0000","last_became_active":"2024-09-13T16:29:05.639596+0000","last_became_peered":"2024-09-13T16:29:05.639596+0000","last_unstale":"2024-09-13T16:30:04.685063+0000","last_undegraded":"2024-09-13T16:30:04.685063+0000","last_fullsized":"2024-09-13T16:30:04.685063+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:41:38.351162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.684450+0000","last_change":"2024-09-13T16:28:39.212947+0000","last_active":"2024-09-13T16:30:04.684450+0000","last_peered":"2024-09-13T16:30:04.684450+0000","last_clean":"2024-09-13T16:30:04.684450+0000","last_became_active":"2024-09-13T16:28:39.212622+0000","last_became_peered":"2024-09-13T16:28:39.212622+0000","last_unstale":"2024-09-13T16:30:04.684450+0000","last_undegraded":"2024-09-13T16:30:04.684450+0000","last_fullsized":"2024-09-13T16:30:04.684450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:16.425154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.965961+0000","last_change":"2024-09-13T16:28:38.846173+0000","last_active":"2024-09-13T16:30:04.965961+0000","last_peered":"2024-09-13T16:30:04.965961+0000","last_clean":"2024-09-13T16:30:04.965961+0000","last_became_active":"2024-09-13T16:28:38.845873+0000","last_became_peered":"2024-09-13T16:28:38.845873+0000","last_unstale":"2024-09-13T16:30:04.965961+0000","last_undegraded":"2024-09-13T16:30:04.965961+0000","last_fullsized":"2024-09-13T16:30:04.965961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:19:55.817524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T16:30:06.362655+0000","last_change":"2024-09-13T16:30:06.362655+0000","last_active":"2024-09-13T16:30:06.362655+0000","last_peered":"2024-09-13T16:30:06.362655+0000","last_clean":"2024-09-13T16:30:06.362655+0000","last_became_active":"2024-09-13T16:30:06.362123+0000","last_became_peered":"2024-09-13T16:30:06.362123+0000","last_unstale":"2024-09-13T16:30:06.362655+0000","last_undegraded":"2024-09-13T16:30:06.362655+0000","last_fullsized":"2024-09-13T16:30:06.362655+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:14:22.445244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.690105+0000","last_change":"2024-09-13T16:28:39.205817+0000","last_active":"2024-09-13T16:30:04.690105+0000","last_peered":"2024-09-13T16:30:04.690105+0000","last_clean":"2024-09-13T16:30:04.690105+0000","last_became_active":"2024-09-13T16:28:39.205534+0000","last_became_peered":"2024-09-13T16:28:39.205534+0000","last_unstale":"2024-09-13T16:30:04.690105+0000","last_undegraded":"2024-09-13T16:30:04.690105+0000","last_fullsized":"2024-09-13T16:30:04.690105+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:09:16.294842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.685510+0000","last_change":"2024-09-13T16:28:38.847582+0000","last_active":"2024-09-13T16:30:04.685510+0000","last_peered":"2024-09-13T16:30:04.685510+0000","last_clean":"2024-09-13T16:30:04.685510+0000","last_became_active":"2024-09-13T16:28:38.847353+0000","last_became_peered":"2024-09-13T16:28:38.847353+0000","last_unstale":"2024-09-13T16:30:04.685510+0000","last_undegraded":"2024-09-13T16:30:04.685510+0000","last_fullsized":"2024-09-13T16:30:04.685510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:24.944088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.726258+0000","last_change":"2024-09-13T16:28:43.225202+0000","last_active":"2024-09-13T16:30:04.726258+0000","last_peered":"2024-09-13T16:30:04.726258+0000","last_clean":"2024-09-13T16:30:04.726258+0000","last_became_active":"2024-09-13T16:28:43.224802+0000","last_became_peered":"2024-09-13T16:28:43.224802+0000","last_unstale":"2024-09-13T16:30:04.726258+0000","last_undegraded":"2024-09-13T16:30:04.726258+0000","last_fullsized":"2024-09-13T16:30:04.726258+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:51:19.325859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689238+0000","last_change":"2024-09-13T16:28:39.209630+0000","last_active":"2024-09-13T16:30:04.689238+0000","last_peered":"2024-09-13T16:30:04.689238+0000","last_clean":"2024-09-13T16:30:04.689238+0000","last_became_active":"2024-09-13T16:28:39.209303+0000","last_became_peered":"2024-09-13T16:28:39.209303+0000","last_unstale":"2024-09-13T16:30:04.689238+0000","last_undegraded":"2024-09-13T16:30:04.689238+0000","last_fullsized":"2024-09-13T16:30:04.689238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:01:31.822673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.684231+0000","last_change":"2024-09-13T16:28:38.837858+0000","last_active":"2024-09-13T16:30:04.684231+0000","last_peered":"2024-09-13T16:30:04.684231+0000","last_clean":"2024-09-13T16:30:04.684231+0000","last_became_active":"2024-09-13T16:28:38.837378+0000","last_became_peered":"2024-09-13T16:28:38.837378+0000","last_unstale":"2024-09-13T16:30:04.684231+0000","last_undegraded":"2024-09-13T16:30:04.684231+0000","last_fullsized":"2024-09-13T16:30:04.684231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:59:41.055902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.688716+0000","last_change":"2024-09-13T16:28:43.294241+0000","last_active":"2024-09-13T16:30:04.688716+0000","last_peered":"2024-09-13T16:30:04.688716+0000","last_clean":"2024-09-13T16:30:04.688716+0000","last_became_active":"2024-09-13T16:28:43.293873+0000","last_became_peered":"2024-09-13T16:28:43.293873+0000","last_unstale":"2024-09-13T16:30:04.688716+0000","last_undegraded":"2024-09-13T16:30:04.688716+0000","last_fullsized":"2024-09-13T16:30:04.688716+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:34.018162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.719362+0000","last_change":"2024-09-13T16:28:39.218071+0000","last_active":"2024-09-13T16:30:04.719362+0000","last_peered":"2024-09-13T16:30:04.719362+0000","last_clean":"2024-09-13T16:30:04.719362+0000","last_became_active":"2024-09-13T16:28:39.217798+0000","last_became_peered":"2024-09-13T16:28:39.217798+0000","last_unstale":"2024-09-13T16:30:04.719362+0000","last_undegraded":"2024-09-13T16:30:04.719362+0000","last_fullsized":"2024-09-13T16:30:04.719362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:39:09.769384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.685074+0000","last_change":"2024-09-13T16:28:38.847238+0000","last_active":"2024-09-13T16:30:04.685074+0000","last_peered":"2024-09-13T16:30:04.685074+0000","last_clean":"2024-09-13T16:30:04.685074+0000","last_became_active":"2024-09-13T16:28:38.846741+0000","last_became_peered":"2024-09-13T16:28:38.846741+0000","last_unstale":"2024-09-13T16:30:04.685074+0000","last_undegraded":"2024-09-13T16:30:04.685074+0000","last_fullsized":"2024-09-13T16:30:04.685074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:52:51.946158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.965790+0000","last_change":"2024-09-13T16:28:43.228431+0000","last_active":"2024-09-13T16:30:04.965790+0000","last_peered":"2024-09-13T16:30:04.965790+0000","last_clean":"2024-09-13T16:30:04.965790+0000","last_became_active":"2024-09-13T16:28:43.228030+0000","last_became_peered":"2024-09-13T16:28:43.228030+0000","last_unstale":"2024-09-13T16:30:04.965790+0000","last_undegraded":"2024-09-13T16:30:04.965790+0000","last_fullsized":"2024-09-13T16:30:04.965790+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:14:50.573033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.688142+0000","last_change":"2024-09-13T16:28:39.209485+0000","last_active":"2024-09-13T16:30:04.688142+0000","last_peered":"2024-09-13T16:30:04.688142+0000","last_clean":"2024-09-13T16:30:04.688142+0000","last_became_active":"2024-09-13T16:28:39.209157+0000","last_became_peered":"2024-09-13T16:28:39.209157+0000","last_unstale":"2024-09-13T16:30:04.688142+0000","last_undegraded":"2024-09-13T16:30:04.688142+0000","last_fullsized":"2024-09-13T16:30:04.688142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:12:19.398255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.683935+0000","last_change":"2024-09-13T16:28:38.842551+0000","last_active":"2024-09-13T16:30:04.683935+0000","last_peered":"2024-09-13T16:30:04.683935+0000","last_clean":"2024-09-13T16:30:04.683935+0000","last_became_active":"2024-09-13T16:28:38.841741+0000","last_became_peered":"2024-09-13T16:28:38.841741+0000","last_unstale":"2024-09-13T16:30:04.683935+0000","last_undegraded":"2024-09-13T16:30:04.683935+0000","last_fullsized":"2024-09-13T16:30:04.683935+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:35:25.131693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.684450+0000","last_change":"2024-09-13T16:28:43.294300+0000","last_active":"2024-09-13T16:30:04.684450+0000","last_peered":"2024-09-13T16:30:04.684450+0000","last_clean":"2024-09-13T16:30:04.684450+0000","last_became_active":"2024-09-13T16:28:43.293918+0000","last_became_peered":"2024-09-13T16:28:43.293918+0000","last_unstale":"2024-09-13T16:30:04.684450+0000","last_undegraded":"2024-09-13T16:30:04.684450+0000","last_fullsized":"2024-09-13T16:30:04.684450+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:19:21.959617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.687836+0000","last_change":"2024-09-13T16:28:39.212001+0000","last_active":"2024-09-13T16:30:04.687836+0000","last_peered":"2024-09-13T16:30:04.687836+0000","last_clean":"2024-09-13T16:30:04.687836+0000","last_became_active":"2024-09-13T16:28:39.211716+0000","last_became_peered":"2024-09-13T16:28:39.211716+0000","last_unstale":"2024-09-13T16:30:04.687836+0000","last_undegraded":"2024-09-13T16:30:04.687836+0000","last_fullsized":"2024-09-13T16:30:04.687836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:56.753677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'6","reported_seq":98,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T16:30:16.736045+0000","last_change":"2024-09-13T16:28:38.851229+0000","last_active":"2024-09-13T16:30:16.736045+0000","last_peered":"2024-09-13T16:30:16.736045+0000","last_clean":"2024-09-13T16:30:16.736045+0000","last_became_active":"2024-09-13T16:28:38.850937+0000","last_became_peered":"2024-09-13T16:28:38.850937+0000","last_unstale":"2024-09-13T16:30:16.736045+0000","last_undegraded":"2024-09-13T16:30:16.736045+0000","last_fullsized":"2024-09-13T16:30:16.736045+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:19.460864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.719658+0000","last_change":"2024-09-13T16:28:43.224825+0000","last_active":"2024-09-13T16:30:04.719658+0000","last_peered":"2024-09-13T16:30:04.719658+0000","last_clean":"2024-09-13T16:30:04.719658+0000","last_became_active":"2024-09-13T16:28:43.223903+0000","last_became_peered":"2024-09-13T16:28:43.223903+0000","last_unstale":"2024-09-13T16:30:04.719658+0000","last_undegraded":"2024-09-13T16:30:04.719658+0000","last_fullsized":"2024-09-13T16:30:04.719658+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:08:46.850668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.683242+0000","last_change":"2024-09-13T16:29:05.636787+0000","last_active":"2024-09-13T16:30:04.683242+0000","last_peered":"2024-09-13T16:30:04.683242+0000","last_clean":"2024-09-13T16:30:04.683242+0000","last_became_active":"2024-09-13T16:29:05.636399+0000","last_became_peered":"2024-09-13T16:29:05.636399+0000","last_unstale":"2024-09-13T16:30:04.683242+0000","last_undegraded":"2024-09-13T16:30:04.683242+0000","last_fullsized":"2024-09-13T16:30:04.683242+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:38:25.545325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":103,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T16:30:18.738009+0000","last_change":"2024-09-13T16:28:38.845374+0000","last_active":"2024-09-13T16:30:18.738009+0000","last_peered":"2024-09-13T16:30:18.738009+0000","last_clean":"2024-09-13T16:30:18.738009+0000","last_became_active":"2024-09-13T16:28:38.845076+0000","last_became_peered":"2024-09-13T16:28:38.845076+0000","last_unstale":"2024-09-13T16:30:18.738009+0000","last_undegraded":"2024-09-13T16:30:18.738009+0000","last_fullsized":"2024-09-13T16:30:18.738009+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:37:56.225682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":78,"num_read_kb":78,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.719371+0000","last_change":"2024-09-13T16:28:43.293529+0000","last_active":"2024-09-13T16:30:04.719371+0000","last_peered":"2024-09-13T16:30:04.719371+0000","last_clean":"2024-09-13T16:30:04.719371+0000","last_became_active":"2024-09-13T16:28:43.293083+0000","last_became_peered":"2024-09-13T16:28:43.293083+0000","last_unstale":"2024-09-13T16:30:04.719371+0000","last_undegraded":"2024-09-13T16:30:04.719371+0000","last_fullsized":"2024-09-13T16:30:04.719371+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:05:55.572984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.718742+0000","last_change":"2024-09-13T16:28:39.217546+0000","last_active":"2024-09-13T16:30:04.718742+0000","last_peered":"2024-09-13T16:30:04.718742+0000","last_clean":"2024-09-13T16:30:04.718742+0000","last_became_active":"2024-09-13T16:28:39.217242+0000","last_became_peered":"2024-09-13T16:28:39.217242+0000","last_unstale":"2024-09-13T16:30:04.718742+0000","last_undegraded":"2024-09-13T16:30:04.718742+0000","last_fullsized":"2024-09-13T16:30:04.718742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:23:18.183616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.688989+0000","last_change":"2024-09-13T16:28:38.846436+0000","last_active":"2024-09-13T16:30:04.688989+0000","last_peered":"2024-09-13T16:30:04.688989+0000","last_clean":"2024-09-13T16:30:04.688989+0000","last_became_active":"2024-09-13T16:28:38.845953+0000","last_became_peered":"2024-09-13T16:28:38.845953+0000","last_unstale":"2024-09-13T16:30:04.688989+0000","last_undegraded":"2024-09-13T16:30:04.688989+0000","last_fullsized":"2024-09-13T16:30:04.688989+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:33.998462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.687895+0000","last_change":"2024-09-13T16:28:43.294504+0000","last_active":"2024-09-13T16:30:04.687895+0000","last_peered":"2024-09-13T16:30:04.687895+0000","last_clean":"2024-09-13T16:30:04.687895+0000","last_became_active":"2024-09-13T16:28:43.294063+0000","last_became_peered":"2024-09-13T16:28:43.294063+0000","last_unstale":"2024-09-13T16:30:04.687895+0000","last_undegraded":"2024-09-13T16:30:04.687895+0000","last_fullsized":"2024-09-13T16:30:04.687895+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:51:35.774467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689668+0000","last_change":"2024-09-13T16:28:39.206491+0000","last_active":"2024-09-13T16:30:04.689668+0000","last_peered":"2024-09-13T16:30:04.689668+0000","last_clean":"2024-09-13T16:30:04.689668+0000","last_became_active":"2024-09-13T16:28:39.206187+0000","last_became_peered":"2024-09-13T16:28:39.206187+0000","last_unstale":"2024-09-13T16:30:04.689668+0000","last_undegraded":"2024-09-13T16:30:04.689668+0000","last_fullsized":"2024-09-13T16:30:04.689668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:20:28.457994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.684882+0000","last_change":"2024-09-13T16:28:38.836270+0000","last_active":"2024-09-13T16:30:04.684882+0000","last_peered":"2024-09-13T16:30:04.684882+0000","last_clean":"2024-09-13T16:30:04.684882+0000","last_became_active":"2024-09-13T16:28:38.835966+0000","last_became_peered":"2024-09-13T16:28:38.835966+0000","last_unstale":"2024-09-13T16:30:04.684882+0000","last_undegraded":"2024-09-13T16:30:04.684882+0000","last_fullsized":"2024-09-13T16:30:04.684882+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:03:11.388654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":333,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689087+0000","last_change":"2024-09-13T16:27:34.264484+0000","last_active":"2024-09-13T16:30:04.689087+0000","last_peered":"2024-09-13T16:30:04.689087+0000","last_clean":"2024-09-13T16:30:04.689087+0000","last_became_active":"2024-09-13T16:27:34.263995+0000","last_became_peered":"2024-09-13T16:27:34.263995+0000","last_unstale":"2024-09-13T16:30:04.689087+0000","last_undegraded":"2024-09-13T16:30:04.689087+0000","last_fullsized":"2024-09-13T16:30:04.689087+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:24:23.015578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:24:23.015578+0000","last_clean_scrub_stamp":"2024-09-13T16:24:23.015578+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:25:04.376312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689366+0000","last_change":"2024-09-13T16:28:43.229347+0000","last_active":"2024-09-13T16:30:04.689366+0000","last_peered":"2024-09-13T16:30:04.689366+0000","last_clean":"2024-09-13T16:30:04.689366+0000","last_became_active":"2024-09-13T16:28:43.228981+0000","last_became_peered":"2024-09-13T16:28:43.228981+0000","last_unstale":"2024-09-13T16:30:04.689366+0000","last_undegraded":"2024-09-13T16:30:04.689366+0000","last_fullsized":"2024-09-13T16:30:04.689366+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:46:15.151295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689569+0000","last_change":"2024-09-13T16:28:39.203195+0000","last_active":"2024-09-13T16:30:04.689569+0000","last_peered":"2024-09-13T16:30:04.689569+0000","last_clean":"2024-09-13T16:30:04.689569+0000","last_became_active":"2024-09-13T16:28:39.202878+0000","last_became_peered":"2024-09-13T16:28:39.202878+0000","last_unstale":"2024-09-13T16:30:04.689569+0000","last_undegraded":"2024-09-13T16:30:04.689569+0000","last_fullsized":"2024-09-13T16:30:04.689569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:17.615531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":98,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T16:30:15.746792+0000","last_change":"2024-09-13T16:28:38.844856+0000","last_active":"2024-09-13T16:30:15.746792+0000","last_peered":"2024-09-13T16:30:15.746792+0000","last_clean":"2024-09-13T16:30:15.746792+0000","last_became_active":"2024-09-13T16:28:38.844578+0000","last_became_peered":"2024-09-13T16:28:38.844578+0000","last_unstale":"2024-09-13T16:30:15.746792+0000","last_undegraded":"2024-09-13T16:30:15.746792+0000","last_fullsized":"2024-09-13T16:30:15.746792+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:41:48.253174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.965911+0000","last_change":"2024-09-13T16:28:43.228472+0000","last_active":"2024-09-13T16:30:04.965911+0000","last_peered":"2024-09-13T16:30:04.965911+0000","last_clean":"2024-09-13T16:30:04.965911+0000","last_became_active":"2024-09-13T16:28:43.228056+0000","last_became_peered":"2024-09-13T16:28:43.228056+0000","last_unstale":"2024-09-13T16:30:04.965911+0000","last_undegraded":"2024-09-13T16:30:04.965911+0000","last_fullsized":"2024-09-13T16:30:04.965911+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:07:55.980691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689417+0000","last_change":"2024-09-13T16:28:39.203278+0000","last_active":"2024-09-13T16:30:04.689417+0000","last_peered":"2024-09-13T16:30:04.689417+0000","last_clean":"2024-09-13T16:30:04.689417+0000","last_became_active":"2024-09-13T16:28:39.202936+0000","last_became_peered":"2024-09-13T16:28:39.202936+0000","last_unstale":"2024-09-13T16:30:04.689417+0000","last_undegraded":"2024-09-13T16:30:04.689417+0000","last_fullsized":"2024-09-13T16:30:04.689417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:44.210612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.687892+0000","last_change":"2024-09-13T16:28:38.851085+0000","last_active":"2024-09-13T16:30:04.687892+0000","last_peered":"2024-09-13T16:30:04.687892+0000","last_clean":"2024-09-13T16:30:04.687892+0000","last_became_active":"2024-09-13T16:28:38.850785+0000","last_became_peered":"2024-09-13T16:28:38.850785+0000","last_unstale":"2024-09-13T16:30:04.687892+0000","last_undegraded":"2024-09-13T16:30:04.687892+0000","last_fullsized":"2024-09-13T16:30:04.687892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:52.713431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689588+0000","last_change":"2024-09-13T16:28:43.229980+0000","last_active":"2024-09-13T16:30:04.689588+0000","last_peered":"2024-09-13T16:30:04.689588+0000","last_clean":"2024-09-13T16:30:04.689588+0000","last_became_active":"2024-09-13T16:28:43.229647+0000","last_became_peered":"2024-09-13T16:28:43.229647+0000","last_unstale":"2024-09-13T16:30:04.689588+0000","last_undegraded":"2024-09-13T16:30:04.689588+0000","last_fullsized":"2024-09-13T16:30:04.689588+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:39:42.871613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689552+0000","last_change":"2024-09-13T16:28:39.204167+0000","last_active":"2024-09-13T16:30:04.689552+0000","last_peered":"2024-09-13T16:30:04.689552+0000","last_clean":"2024-09-13T16:30:04.689552+0000","last_became_active":"2024-09-13T16:28:39.203776+0000","last_became_peered":"2024-09-13T16:28:39.203776+0000","last_unstale":"2024-09-13T16:30:04.689552+0000","last_undegraded":"2024-09-13T16:30:04.689552+0000","last_fullsized":"2024-09-13T16:30:04.689552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:53:36.410365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.719572+0000","last_change":"2024-09-13T16:28:38.846331+0000","last_active":"2024-09-13T16:30:04.719572+0000","last_peered":"2024-09-13T16:30:04.719572+0000","last_clean":"2024-09-13T16:30:04.719572+0000","last_became_active":"2024-09-13T16:28:38.846012+0000","last_became_peered":"2024-09-13T16:28:38.846012+0000","last_unstale":"2024-09-13T16:30:04.719572+0000","last_undegraded":"2024-09-13T16:30:04.719572+0000","last_fullsized":"2024-09-13T16:30:04.719572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:23:54.282736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.687072+0000","last_change":"2024-09-13T16:28:43.227886+0000","last_active":"2024-09-13T16:30:04.687072+0000","last_peered":"2024-09-13T16:30:04.687072+0000","last_clean":"2024-09-13T16:30:04.687072+0000","last_became_active":"2024-09-13T16:28:43.227346+0000","last_became_peered":"2024-09-13T16:28:43.227346+0000","last_unstale":"2024-09-13T16:30:04.687072+0000","last_undegraded":"2024-09-13T16:30:04.687072+0000","last_fullsized":"2024-09-13T16:30:04.687072+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:41:36.559064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.965886+0000","last_change":"2024-09-13T16:28:39.205975+0000","last_active":"2024-09-13T16:30:04.965886+0000","last_peered":"2024-09-13T16:30:04.965886+0000","last_clean":"2024-09-13T16:30:04.965886+0000","last_became_active":"2024-09-13T16:28:39.205657+0000","last_became_peered":"2024-09-13T16:28:39.205657+0000","last_unstale":"2024-09-13T16:30:04.965886+0000","last_undegraded":"2024-09-13T16:30:04.965886+0000","last_fullsized":"2024-09-13T16:30:04.965886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:31:47.309482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.685059+0000","last_change":"2024-09-13T16:28:38.841589+0000","last_active":"2024-09-13T16:30:04.685059+0000","last_peered":"2024-09-13T16:30:04.685059+0000","last_clean":"2024-09-13T16:30:04.685059+0000","last_became_active":"2024-09-13T16:28:38.841298+0000","last_became_peered":"2024-09-13T16:28:38.841298+0000","last_unstale":"2024-09-13T16:30:04.685059+0000","last_undegraded":"2024-09-13T16:30:04.685059+0000","last_fullsized":"2024-09-13T16:30:04.685059+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:27:38.358320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.687185+0000","last_change":"2024-09-13T16:28:43.233790+0000","last_active":"2024-09-13T16:30:04.687185+0000","last_peered":"2024-09-13T16:30:04.687185+0000","last_clean":"2024-09-13T16:30:04.687185+0000","last_became_active":"2024-09-13T16:28:43.231963+0000","last_became_peered":"2024-09-13T16:28:43.231963+0000","last_unstale":"2024-09-13T16:30:04.687185+0000","last_undegraded":"2024-09-13T16:30:04.687185+0000","last_fullsized":"2024-09-13T16:30:04.687185+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:45:23.406395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.687352+0000","last_change":"2024-09-13T16:28:39.201476+0000","last_active":"2024-09-13T16:30:04.687352+0000","last_peered":"2024-09-13T16:30:04.687352+0000","last_clean":"2024-09-13T16:30:04.687352+0000","last_became_active":"2024-09-13T16:28:39.201168+0000","last_became_peered":"2024-09-13T16:28:39.201168+0000","last_unstale":"2024-09-13T16:30:04.687352+0000","last_undegraded":"2024-09-13T16:30:04.687352+0000","last_fullsized":"2024-09-13T16:30:04.687352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:26:43.570295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":64,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.684447+0000","last_change":"2024-09-13T16:29:05.652225+0000","last_active":"2024-09-13T16:30:04.684447+0000","last_peered":"2024-09-13T16:30:04.684447+0000","last_clean":"2024-09-13T16:30:04.684447+0000","last_became_active":"2024-09-13T16:29:05.639325+0000","last_became_peered":"2024-09-13T16:29:05.639325+0000","last_unstale":"2024-09-13T16:30:04.684447+0000","last_undegraded":"2024-09-13T16:30:04.684447+0000","last_fullsized":"2024-09-13T16:30:04.684447+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:18:30.083447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.719391+0000","last_change":"2024-09-13T16:28:43.228563+0000","last_active":"2024-09-13T16:30:04.719391+0000","last_peered":"2024-09-13T16:30:04.719391+0000","last_clean":"2024-09-13T16:30:04.719391+0000","last_became_active":"2024-09-13T16:28:43.228184+0000","last_became_peered":"2024-09-13T16:28:43.228184+0000","last_unstale":"2024-09-13T16:30:04.719391+0000","last_undegraded":"2024-09-13T16:30:04.719391+0000","last_fullsized":"2024-09-13T16:30:04.719391+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:05:33.368302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.688901+0000","last_change":"2024-09-13T16:28:39.210750+0000","last_active":"2024-09-13T16:30:04.688901+0000","last_peered":"2024-09-13T16:30:04.688901+0000","last_clean":"2024-09-13T16:30:04.688901+0000","last_became_active":"2024-09-13T16:28:39.210401+0000","last_became_peered":"2024-09-13T16:28:39.210401+0000","last_unstale":"2024-09-13T16:30:04.688901+0000","last_undegraded":"2024-09-13T16:30:04.688901+0000","last_fullsized":"2024-09-13T16:30:04.688901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:36.670420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":104,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T16:30:16.736133+0000","last_change":"2024-09-13T16:28:38.843573+0000","last_active":"2024-09-13T16:30:16.736133+0000","last_peered":"2024-09-13T16:30:16.736133+0000","last_clean":"2024-09-13T16:30:16.736133+0000","last_became_active":"2024-09-13T16:28:38.843245+0000","last_became_peered":"2024-09-13T16:28:38.843245+0000","last_unstale":"2024-09-13T16:30:16.736133+0000","last_undegraded":"2024-09-13T16:30:16.736133+0000","last_fullsized":"2024-09-13T16:30:16.736133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:45.983123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T16:30:05.835858+0000","last_change":"2024-09-13T16:30:05.835462+0000","last_active":"2024-09-13T16:30:05.835858+0000","last_peered":"2024-09-13T16:30:05.835858+0000","last_clean":"2024-09-13T16:30:05.835858+0000","last_became_active":"2024-09-13T16:30:05.835050+0000","last_became_peered":"2024-09-13T16:30:05.835050+0000","last_unstale":"2024-09-13T16:30:05.835858+0000","last_undegraded":"2024-09-13T16:30:05.835858+0000","last_fullsized":"2024-09-13T16:30:05.835858+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:30:43.427684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7,0,3],"acting":[10,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689446+0000","last_change":"2024-09-13T16:28:39.201471+0000","last_active":"2024-09-13T16:30:04.689446+0000","last_peered":"2024-09-13T16:30:04.689446+0000","last_clean":"2024-09-13T16:30:04.689446+0000","last_became_active":"2024-09-13T16:28:39.201140+0000","last_became_peered":"2024-09-13T16:28:39.201140+0000","last_unstale":"2024-09-13T16:30:04.689446+0000","last_undegraded":"2024-09-13T16:30:04.689446+0000","last_fullsized":"2024-09-13T16:30:04.689446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:37:25.037490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.687912+0000","last_change":"2024-09-13T16:28:38.845703+0000","last_active":"2024-09-13T16:30:04.687912+0000","last_peered":"2024-09-13T16:30:04.687912+0000","last_clean":"2024-09-13T16:30:04.687912+0000","last_became_active":"2024-09-13T16:28:38.845354+0000","last_became_peered":"2024-09-13T16:28:38.845354+0000","last_unstale":"2024-09-13T16:30:04.687912+0000","last_undegraded":"2024-09-13T16:30:04.687912+0000","last_fullsized":"2024-09-13T16:30:04.687912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:18.208723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.965726+0000","last_change":"2024-09-13T16:28:43.223924+0000","last_active":"2024-09-13T16:30:04.965726+0000","last_peered":"2024-09-13T16:30:04.965726+0000","last_clean":"2024-09-13T16:30:04.965726+0000","last_became_active":"2024-09-13T16:28:43.223535+0000","last_became_peered":"2024-09-13T16:28:43.223535+0000","last_unstale":"2024-09-13T16:30:04.965726+0000","last_undegraded":"2024-09-13T16:30:04.965726+0000","last_fullsized":"2024-09-13T16:30:04.965726+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:10.622436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.719026+0000","last_change":"2024-09-13T16:28:39.203943+0000","last_active":"2024-09-13T16:30:04.719026+0000","last_peered":"2024-09-13T16:30:04.719026+0000","last_clean":"2024-09-13T16:30:04.719026+0000","last_became_active":"2024-09-13T16:28:39.203657+0000","last_became_peered":"2024-09-13T16:28:39.203657+0000","last_unstale":"2024-09-13T16:30:04.719026+0000","last_undegraded":"2024-09-13T16:30:04.719026+0000","last_fullsized":"2024-09-13T16:30:04.719026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:52:48.523939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.965628+0000","last_change":"2024-09-13T16:29:05.651015+0000","last_active":"2024-09-13T16:30:04.965628+0000","last_peered":"2024-09-13T16:30:04.965628+0000","last_clean":"2024-09-13T16:30:04.965628+0000","last_became_active":"2024-09-13T16:29:05.640006+0000","last_became_peered":"2024-09-13T16:29:05.640006+0000","last_unstale":"2024-09-13T16:30:04.965628+0000","last_undegraded":"2024-09-13T16:30:04.965628+0000","last_fullsized":"2024-09-13T16:30:04.965628+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:12:25.508262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.687619+0000","last_change":"2024-09-13T16:28:43.232634+0000","last_active":"2024-09-13T16:30:04.687619+0000","last_peered":"2024-09-13T16:30:04.687619+0000","last_clean":"2024-09-13T16:30:04.687619+0000","last_became_active":"2024-09-13T16:28:43.232107+0000","last_became_peered":"2024-09-13T16:28:43.232107+0000","last_unstale":"2024-09-13T16:30:04.687619+0000","last_undegraded":"2024-09-13T16:30:04.687619+0000","last_fullsized":"2024-09-13T16:30:04.687619+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:45.952933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.684953+0000","last_change":"2024-09-13T16:28:39.212883+0000","last_active":"2024-09-13T16:30:04.684953+0000","last_peered":"2024-09-13T16:30:04.684953+0000","last_clean":"2024-09-13T16:30:04.684953+0000","last_became_active":"2024-09-13T16:28:39.212591+0000","last_became_peered":"2024-09-13T16:28:39.212591+0000","last_unstale":"2024-09-13T16:30:04.684953+0000","last_undegraded":"2024-09-13T16:30:04.684953+0000","last_fullsized":"2024-09-13T16:30:04.684953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:17:30.926940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'34","reported_seq":222,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T16:30:21.748536+0000","last_change":"2024-09-13T16:28:38.846564+0000","last_active":"2024-09-13T16:30:21.748536+0000","last_peered":"2024-09-13T16:30:21.748536+0000","last_clean":"2024-09-13T16:30:21.748536+0000","last_became_active":"2024-09-13T16:28:38.846165+0000","last_became_peered":"2024-09-13T16:28:38.846165+0000","last_unstale":"2024-09-13T16:30:21.748536+0000","last_undegraded":"2024-09-13T16:30:21.748536+0000","last_fullsized":"2024-09-13T16:30:21.748536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:33:06.124592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63310,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":168,"num_read_kb":146,"num_write":36,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.683680+0000","last_change":"2024-09-13T16:28:43.231464+0000","last_active":"2024-09-13T16:30:04.683680+0000","last_peered":"2024-09-13T16:30:04.683680+0000","last_clean":"2024-09-13T16:30:04.683680+0000","last_became_active":"2024-09-13T16:28:43.231040+0000","last_became_peered":"2024-09-13T16:28:43.231040+0000","last_unstale":"2024-09-13T16:30:04.683680+0000","last_undegraded":"2024-09-13T16:30:04.683680+0000","last_fullsized":"2024-09-13T16:30:04.683680+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:11:21.847375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.966262+0000","last_change":"2024-09-13T16:29:05.639415+0000","last_active":"2024-09-13T16:30:04.966262+0000","last_peered":"2024-09-13T16:30:04.966262+0000","last_clean":"2024-09-13T16:30:04.966262+0000","last_became_active":"2024-09-13T16:29:05.639101+0000","last_became_peered":"2024-09-13T16:29:05.639101+0000","last_unstale":"2024-09-13T16:30:04.966262+0000","last_undegraded":"2024-09-13T16:30:04.966262+0000","last_fullsized":"2024-09-13T16:30:04.966262+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:32:42.630013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.687774+0000","last_change":"2024-09-13T16:28:38.841637+0000","last_active":"2024-09-13T16:30:04.687774+0000","last_peered":"2024-09-13T16:30:04.687774+0000","last_clean":"2024-09-13T16:30:04.687774+0000","last_became_active":"2024-09-13T16:28:38.841359+0000","last_became_peered":"2024-09-13T16:28:38.841359+0000","last_unstale":"2024-09-13T16:30:04.687774+0000","last_undegraded":"2024-09-13T16:30:04.687774+0000","last_fullsized":"2024-09-13T16:30:04.687774+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:47:05.284987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.684791+0000","last_change":"2024-09-13T16:29:05.638870+0000","last_active":"2024-09-13T16:30:04.684791+0000","last_peered":"2024-09-13T16:30:04.684791+0000","last_clean":"2024-09-13T16:30:04.684791+0000","last_became_active":"2024-09-13T16:29:05.638422+0000","last_became_peered":"2024-09-13T16:29:05.638422+0000","last_unstale":"2024-09-13T16:30:04.684791+0000","last_undegraded":"2024-09-13T16:30:04.684791+0000","last_fullsized":"2024-09-13T16:30:04.684791+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:41.263661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.690269+0000","last_change":"2024-09-13T16:28:39.203595+0000","last_active":"2024-09-13T16:30:04.690269+0000","last_peered":"2024-09-13T16:30:04.690269+0000","last_clean":"2024-09-13T16:30:04.690269+0000","last_became_active":"2024-09-13T16:28:39.203283+0000","last_became_peered":"2024-09-13T16:28:39.203283+0000","last_unstale":"2024-09-13T16:30:04.690269+0000","last_undegraded":"2024-09-13T16:30:04.690269+0000","last_fullsized":"2024-09-13T16:30:04.690269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:58:45.970761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689460+0000","last_change":"2024-09-13T16:28:38.850203+0000","last_active":"2024-09-13T16:30:04.689460+0000","last_peered":"2024-09-13T16:30:04.689460+0000","last_clean":"2024-09-13T16:30:04.689460+0000","last_became_active":"2024-09-13T16:28:38.849963+0000","last_became_peered":"2024-09-13T16:28:38.849963+0000","last_unstale":"2024-09-13T16:30:04.689460+0000","last_undegraded":"2024-09-13T16:30:04.689460+0000","last_fullsized":"2024-09-13T16:30:04.689460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:15.849627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.683250+0000","last_change":"2024-09-13T16:28:43.227679+0000","last_active":"2024-09-13T16:30:04.683250+0000","last_peered":"2024-09-13T16:30:04.683250+0000","last_clean":"2024-09-13T16:30:04.683250+0000","last_became_active":"2024-09-13T16:28:43.227269+0000","last_became_peered":"2024-09-13T16:28:43.227269+0000","last_unstale":"2024-09-13T16:30:04.683250+0000","last_undegraded":"2024-09-13T16:30:04.683250+0000","last_fullsized":"2024-09-13T16:30:04.683250+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:05:00.186305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.966381+0000","last_change":"2024-09-13T16:28:39.211401+0000","last_active":"2024-09-13T16:30:04.966381+0000","last_peered":"2024-09-13T16:30:04.966381+0000","last_clean":"2024-09-13T16:30:04.966381+0000","last_became_active":"2024-09-13T16:28:39.211133+0000","last_became_peered":"2024-09-13T16:28:39.211133+0000","last_unstale":"2024-09-13T16:30:04.966381+0000","last_undegraded":"2024-09-13T16:30:04.966381+0000","last_fullsized":"2024-09-13T16:30:04.966381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:36.468666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":179,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T16:30:21.748567+0000","last_change":"2024-09-13T16:28:38.845132+0000","last_active":"2024-09-13T16:30:21.748567+0000","last_peered":"2024-09-13T16:30:21.748567+0000","last_clean":"2024-09-13T16:30:21.748567+0000","last_became_active":"2024-09-13T16:28:38.844804+0000","last_became_peered":"2024-09-13T16:28:38.844804+0000","last_unstale":"2024-09-13T16:30:21.748567+0000","last_undegraded":"2024-09-13T16:30:21.748567+0000","last_fullsized":"2024-09-13T16:30:21.748567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:15:08.643227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":155,"num_read_kb":155,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689066+0000","last_change":"2024-09-13T16:28:43.230829+0000","last_active":"2024-09-13T16:30:04.689066+0000","last_peered":"2024-09-13T16:30:04.689066+0000","last_clean":"2024-09-13T16:30:04.689066+0000","last_became_active":"2024-09-13T16:28:43.230405+0000","last_became_peered":"2024-09-13T16:28:43.230405+0000","last_unstale":"2024-09-13T16:30:04.689066+0000","last_undegraded":"2024-09-13T16:30:04.689066+0000","last_fullsized":"2024-09-13T16:30:04.689066+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:04:56.083193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689956+0000","last_change":"2024-09-13T16:28:43.226662+0000","last_active":"2024-09-13T16:30:04.689956+0000","last_peered":"2024-09-13T16:30:04.689956+0000","last_clean":"2024-09-13T16:30:04.689956+0000","last_became_active":"2024-09-13T16:28:43.226232+0000","last_became_peered":"2024-09-13T16:28:43.226232+0000","last_unstale":"2024-09-13T16:30:04.689956+0000","last_undegraded":"2024-09-13T16:30:04.689956+0000","last_fullsized":"2024-09-13T16:30:04.689956+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:03.568500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.683392+0000","last_change":"2024-09-13T16:28:39.208909+0000","last_active":"2024-09-13T16:30:04.683392+0000","last_peered":"2024-09-13T16:30:04.683392+0000","last_clean":"2024-09-13T16:30:04.683392+0000","last_became_active":"2024-09-13T16:28:39.208572+0000","last_became_peered":"2024-09-13T16:28:39.208572+0000","last_unstale":"2024-09-13T16:30:04.683392+0000","last_undegraded":"2024-09-13T16:30:04.683392+0000","last_fullsized":"2024-09-13T16:30:04.683392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:01:40.173308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.687385+0000","last_change":"2024-09-13T16:28:38.844259+0000","last_active":"2024-09-13T16:30:04.687385+0000","last_peered":"2024-09-13T16:30:04.687385+0000","last_clean":"2024-09-13T16:30:04.687385+0000","last_became_active":"2024-09-13T16:28:38.843956+0000","last_became_peered":"2024-09-13T16:28:38.843956+0000","last_unstale":"2024-09-13T16:30:04.687385+0000","last_undegraded":"2024-09-13T16:30:04.687385+0000","last_fullsized":"2024-09-13T16:30:04.687385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:05.652852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.965953+0000","last_change":"2024-09-13T16:28:43.229385+0000","last_active":"2024-09-13T16:30:04.965953+0000","last_peered":"2024-09-13T16:30:04.965953+0000","last_clean":"2024-09-13T16:30:04.965953+0000","last_became_active":"2024-09-13T16:28:43.229033+0000","last_became_peered":"2024-09-13T16:28:43.229033+0000","last_unstale":"2024-09-13T16:30:04.965953+0000","last_undegraded":"2024-09-13T16:30:04.965953+0000","last_fullsized":"2024-09-13T16:30:04.965953+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:46:41.611758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.690367+0000","last_change":"2024-09-13T16:28:39.208100+0000","last_active":"2024-09-13T16:30:04.690367+0000","last_peered":"2024-09-13T16:30:04.690367+0000","last_clean":"2024-09-13T16:30:04.690367+0000","last_became_active":"2024-09-13T16:28:39.207808+0000","last_became_peered":"2024-09-13T16:28:39.207808+0000","last_unstale":"2024-09-13T16:30:04.690367+0000","last_undegraded":"2024-09-13T16:30:04.690367+0000","last_fullsized":"2024-09-13T16:30:04.690367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:55.446568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.718842+0000","last_change":"2024-09-13T16:28:38.844689+0000","last_active":"2024-09-13T16:30:04.718842+0000","last_peered":"2024-09-13T16:30:04.718842+0000","last_clean":"2024-09-13T16:30:04.718842+0000","last_became_active":"2024-09-13T16:28:38.844326+0000","last_became_peered":"2024-09-13T16:28:38.844326+0000","last_unstale":"2024-09-13T16:30:04.718842+0000","last_undegraded":"2024-09-13T16:30:04.718842+0000","last_fullsized":"2024-09-13T16:30:04.718842+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:02.616518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.687781+0000","last_change":"2024-09-13T16:28:38.849233+0000","last_active":"2024-09-13T16:30:04.687781+0000","last_peered":"2024-09-13T16:30:04.687781+0000","last_clean":"2024-09-13T16:30:04.687781+0000","last_became_active":"2024-09-13T16:28:38.848911+0000","last_became_peered":"2024-09-13T16:28:38.848911+0000","last_unstale":"2024-09-13T16:30:04.687781+0000","last_undegraded":"2024-09-13T16:30:04.687781+0000","last_fullsized":"2024-09-13T16:30:04.687781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:02:35.951263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.687469+0000","last_change":"2024-09-13T16:28:39.203855+0000","last_active":"2024-09-13T16:30:04.687469+0000","last_peered":"2024-09-13T16:30:04.687469+0000","last_clean":"2024-09-13T16:30:04.687469+0000","last_became_active":"2024-09-13T16:28:39.203532+0000","last_became_peered":"2024-09-13T16:28:39.203532+0000","last_unstale":"2024-09-13T16:30:04.687469+0000","last_undegraded":"2024-09-13T16:30:04.687469+0000","last_fullsized":"2024-09-13T16:30:04.687469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:15.048930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.726375+0000","last_change":"2024-09-13T16:28:43.228554+0000","last_active":"2024-09-13T16:30:04.726375+0000","last_peered":"2024-09-13T16:30:04.726375+0000","last_clean":"2024-09-13T16:30:04.726375+0000","last_became_active":"2024-09-13T16:28:43.228204+0000","last_became_peered":"2024-09-13T16:28:43.228204+0000","last_unstale":"2024-09-13T16:30:04.726375+0000","last_undegraded":"2024-09-13T16:30:04.726375+0000","last_fullsized":"2024-09-13T16:30:04.726375+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:16:15.383932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":107,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T16:30:19.746116+0000","last_change":"2024-09-13T16:28:38.848425+0000","last_active":"2024-09-13T16:30:19.746116+0000","last_peered":"2024-09-13T16:30:19.746116+0000","last_clean":"2024-09-13T16:30:19.746116+0000","last_became_active":"2024-09-13T16:28:38.848136+0000","last_became_peered":"2024-09-13T16:28:38.848136+0000","last_unstale":"2024-09-13T16:30:19.746116+0000","last_undegraded":"2024-09-13T16:30:19.746116+0000","last_fullsized":"2024-09-13T16:30:19.746116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:47:03.904315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.684661+0000","last_change":"2024-09-13T16:28:39.210565+0000","last_active":"2024-09-13T16:30:04.684661+0000","last_peered":"2024-09-13T16:30:04.684661+0000","last_clean":"2024-09-13T16:30:04.684661+0000","last_became_active":"2024-09-13T16:28:39.210272+0000","last_became_peered":"2024-09-13T16:28:39.210272+0000","last_unstale":"2024-09-13T16:30:04.684661+0000","last_undegraded":"2024-09-13T16:30:04.684661+0000","last_fullsized":"2024-09-13T16:30:04.684661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:12.746087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.726440+0000","last_change":"2024-09-13T16:28:43.231623+0000","last_active":"2024-09-13T16:30:04.726440+0000","last_peered":"2024-09-13T16:30:04.726440+0000","last_clean":"2024-09-13T16:30:04.726440+0000","last_became_active":"2024-09-13T16:28:43.231255+0000","last_became_peered":"2024-09-13T16:28:43.231255+0000","last_unstale":"2024-09-13T16:30:04.726440+0000","last_undegraded":"2024-09-13T16:30:04.726440+0000","last_fullsized":"2024-09-13T16:30:04.726440+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:29.755772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.965567+0000","last_change":"2024-09-13T16:28:39.208438+0000","last_active":"2024-09-13T16:30:04.965567+0000","last_peered":"2024-09-13T16:30:04.965567+0000","last_clean":"2024-09-13T16:30:04.965567+0000","last_became_active":"2024-09-13T16:28:39.208156+0000","last_became_peered":"2024-09-13T16:28:39.208156+0000","last_unstale":"2024-09-13T16:30:04.965567+0000","last_undegraded":"2024-09-13T16:30:04.965567+0000","last_fullsized":"2024-09-13T16:30:04.965567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:53:46.923166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.726374+0000","last_change":"2024-09-13T16:28:39.208529+0000","last_active":"2024-09-13T16:30:04.726374+0000","last_peered":"2024-09-13T16:30:04.726374+0000","last_clean":"2024-09-13T16:30:04.726374+0000","last_became_active":"2024-09-13T16:28:39.208202+0000","last_became_peered":"2024-09-13T16:28:39.208202+0000","last_unstale":"2024-09-13T16:30:04.726374+0000","last_undegraded":"2024-09-13T16:30:04.726374+0000","last_fullsized":"2024-09-13T16:30:04.726374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:17:11.357410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.726166+0000","last_change":"2024-09-13T16:28:39.212958+0000","last_active":"2024-09-13T16:30:04.726166+0000","last_peered":"2024-09-13T16:30:04.726166+0000","last_clean":"2024-09-13T16:30:04.726166+0000","last_became_active":"2024-09-13T16:28:39.212646+0000","last_became_peered":"2024-09-13T16:28:39.212646+0000","last_unstale":"2024-09-13T16:30:04.726166+0000","last_undegraded":"2024-09-13T16:30:04.726166+0000","last_fullsized":"2024-09-13T16:30:04.726166+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:31:04.047778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689683+0000","last_change":"2024-09-13T16:28:39.200449+0000","last_active":"2024-09-13T16:30:04.689683+0000","last_peered":"2024-09-13T16:30:04.689683+0000","last_clean":"2024-09-13T16:30:04.689683+0000","last_became_active":"2024-09-13T16:28:39.200133+0000","last_became_peered":"2024-09-13T16:28:39.200133+0000","last_unstale":"2024-09-13T16:30:04.689683+0000","last_undegraded":"2024-09-13T16:30:04.689683+0000","last_fullsized":"2024-09-13T16:30:04.689683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:18:06.480691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.719276+0000","last_change":"2024-09-13T16:28:39.200274+0000","last_active":"2024-09-13T16:30:04.719276+0000","last_peered":"2024-09-13T16:30:04.719276+0000","last_clean":"2024-09-13T16:30:04.719276+0000","last_became_active":"2024-09-13T16:28:39.199951+0000","last_became_peered":"2024-09-13T16:28:39.199951+0000","last_unstale":"2024-09-13T16:30:04.719276+0000","last_undegraded":"2024-09-13T16:30:04.719276+0000","last_fullsized":"2024-09-13T16:30:04.719276+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:13:31.199857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.965945+0000","last_change":"2024-09-13T16:28:39.213591+0000","last_active":"2024-09-13T16:30:04.965945+0000","last_peered":"2024-09-13T16:30:04.965945+0000","last_clean":"2024-09-13T16:30:04.965945+0000","last_became_active":"2024-09-13T16:28:39.213324+0000","last_became_peered":"2024-09-13T16:28:39.213324+0000","last_unstale":"2024-09-13T16:30:04.965945+0000","last_undegraded":"2024-09-13T16:30:04.965945+0000","last_fullsized":"2024-09-13T16:30:04.965945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:54:03.172919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.726681+0000","last_change":"2024-09-13T16:28:39.217442+0000","last_active":"2024-09-13T16:30:04.726681+0000","last_peered":"2024-09-13T16:30:04.726681+0000","last_clean":"2024-09-13T16:30:04.726681+0000","last_became_active":"2024-09-13T16:28:39.217151+0000","last_became_peered":"2024-09-13T16:28:39.217151+0000","last_unstale":"2024-09-13T16:30:04.726681+0000","last_undegraded":"2024-09-13T16:30:04.726681+0000","last_fullsized":"2024-09-13T16:30:04.726681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:17.132278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689157+0000","last_change":"2024-09-13T16:28:39.203938+0000","last_active":"2024-09-13T16:30:04.689157+0000","last_peered":"2024-09-13T16:30:04.689157+0000","last_clean":"2024-09-13T16:30:04.689157+0000","last_became_active":"2024-09-13T16:28:39.203655+0000","last_became_peered":"2024-09-13T16:28:39.203655+0000","last_unstale":"2024-09-13T16:30:04.689157+0000","last_undegraded":"2024-09-13T16:30:04.689157+0000","last_fullsized":"2024-09-13T16:30:04.689157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:03:25.415256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.687258+0000","last_change":"2024-09-13T16:28:39.209592+0000","last_active":"2024-09-13T16:30:04.687258+0000","last_peered":"2024-09-13T16:30:04.687258+0000","last_clean":"2024-09-13T16:30:04.687258+0000","last_became_active":"2024-09-13T16:28:39.209299+0000","last_became_peered":"2024-09-13T16:28:39.209299+0000","last_unstale":"2024-09-13T16:30:04.687258+0000","last_undegraded":"2024-09-13T16:30:04.687258+0000","last_fullsized":"2024-09-13T16:30:04.687258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:04.053405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689202+0000","last_change":"2024-09-13T16:28:39.208437+0000","last_active":"2024-09-13T16:30:04.689202+0000","last_peered":"2024-09-13T16:30:04.689202+0000","last_clean":"2024-09-13T16:30:04.689202+0000","last_became_active":"2024-09-13T16:28:39.208136+0000","last_became_peered":"2024-09-13T16:28:39.208136+0000","last_unstale":"2024-09-13T16:30:04.689202+0000","last_undegraded":"2024-09-13T16:30:04.689202+0000","last_fullsized":"2024-09-13T16:30:04.689202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:21.304037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.966094+0000","last_change":"2024-09-13T16:28:39.203970+0000","last_active":"2024-09-13T16:30:04.966094+0000","last_peered":"2024-09-13T16:30:04.966094+0000","last_clean":"2024-09-13T16:30:04.966094+0000","last_became_active":"2024-09-13T16:28:39.203559+0000","last_became_peered":"2024-09-13T16:28:39.203559+0000","last_unstale":"2024-09-13T16:30:04.966094+0000","last_undegraded":"2024-09-13T16:30:04.966094+0000","last_fullsized":"2024-09-13T16:30:04.966094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:26:25.158393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.719221+0000","last_change":"2024-09-13T16:28:39.205584+0000","last_active":"2024-09-13T16:30:04.719221+0000","last_peered":"2024-09-13T16:30:04.719221+0000","last_clean":"2024-09-13T16:30:04.719221+0000","last_became_active":"2024-09-13T16:28:39.205281+0000","last_became_peered":"2024-09-13T16:28:39.205281+0000","last_unstale":"2024-09-13T16:30:04.719221+0000","last_undegraded":"2024-09-13T16:30:04.719221+0000","last_fullsized":"2024-09-13T16:30:04.719221+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:58:10.109647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.965875+0000","last_change":"2024-09-13T16:28:39.214479+0000","last_active":"2024-09-13T16:30:04.965875+0000","last_peered":"2024-09-13T16:30:04.965875+0000","last_clean":"2024-09-13T16:30:04.965875+0000","last_became_active":"2024-09-13T16:28:39.213984+0000","last_became_peered":"2024-09-13T16:28:39.213984+0000","last_unstale":"2024-09-13T16:30:04.965875+0000","last_undegraded":"2024-09-13T16:30:04.965875+0000","last_fullsized":"2024-09-13T16:30:04.965875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:53:43.137726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.726557+0000","last_change":"2024-09-13T16:28:39.209595+0000","last_active":"2024-09-13T16:30:04.726557+0000","last_peered":"2024-09-13T16:30:04.726557+0000","last_clean":"2024-09-13T16:30:04.726557+0000","last_became_active":"2024-09-13T16:28:39.209282+0000","last_became_peered":"2024-09-13T16:28:39.209282+0000","last_unstale":"2024-09-13T16:30:04.726557+0000","last_undegraded":"2024-09-13T16:30:04.726557+0000","last_fullsized":"2024-09-13T16:30:04.726557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:26:41.452462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.684096+0000","last_change":"2024-09-13T16:28:39.210610+0000","last_active":"2024-09-13T16:30:04.684096+0000","last_peered":"2024-09-13T16:30:04.684096+0000","last_clean":"2024-09-13T16:30:04.684096+0000","last_became_active":"2024-09-13T16:28:39.210296+0000","last_became_peered":"2024-09-13T16:28:39.210296+0000","last_unstale":"2024-09-13T16:30:04.684096+0000","last_undegraded":"2024-09-13T16:30:04.684096+0000","last_fullsized":"2024-09-13T16:30:04.684096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:06.385733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.966115+0000","last_change":"2024-09-13T16:28:39.207609+0000","last_active":"2024-09-13T16:30:04.966115+0000","last_peered":"2024-09-13T16:30:04.966115+0000","last_clean":"2024-09-13T16:30:04.966115+0000","last_became_active":"2024-09-13T16:28:39.206893+0000","last_became_peered":"2024-09-13T16:28:39.206893+0000","last_unstale":"2024-09-13T16:30:04.966115+0000","last_undegraded":"2024-09-13T16:30:04.966115+0000","last_fullsized":"2024-09-13T16:30:04.966115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:51.582666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.965896+0000","last_change":"2024-09-13T16:28:39.201856+0000","last_active":"2024-09-13T16:30:04.965896+0000","last_peered":"2024-09-13T16:30:04.965896+0000","last_clean":"2024-09-13T16:30:04.965896+0000","last_became_active":"2024-09-13T16:28:39.201511+0000","last_became_peered":"2024-09-13T16:28:39.201511+0000","last_unstale":"2024-09-13T16:30:04.965896+0000","last_undegraded":"2024-09-13T16:30:04.965896+0000","last_fullsized":"2024-09-13T16:30:04.965896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:05:19.722354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689066+0000","last_change":"2024-09-13T16:28:39.216106+0000","last_active":"2024-09-13T16:30:04.689066+0000","last_peered":"2024-09-13T16:30:04.689066+0000","last_clean":"2024-09-13T16:30:04.689066+0000","last_became_active":"2024-09-13T16:28:39.215432+0000","last_became_peered":"2024-09-13T16:28:39.215432+0000","last_unstale":"2024-09-13T16:30:04.689066+0000","last_undegraded":"2024-09-13T16:30:04.689066+0000","last_fullsized":"2024-09-13T16:30:04.689066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:21:18.439891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689291+0000","last_change":"2024-09-13T16:28:39.210578+0000","last_active":"2024-09-13T16:30:04.689291+0000","last_peered":"2024-09-13T16:30:04.689291+0000","last_clean":"2024-09-13T16:30:04.689291+0000","last_became_active":"2024-09-13T16:28:39.210269+0000","last_became_peered":"2024-09-13T16:28:39.210269+0000","last_unstale":"2024-09-13T16:30:04.689291+0000","last_undegraded":"2024-09-13T16:30:04.689291+0000","last_fullsized":"2024-09-13T16:30:04.689291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:27:27.078880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.719142+0000","last_change":"2024-09-13T16:28:39.213409+0000","last_active":"2024-09-13T16:30:04.719142+0000","last_peered":"2024-09-13T16:30:04.719142+0000","last_clean":"2024-09-13T16:30:04.719142+0000","last_became_active":"2024-09-13T16:28:39.213077+0000","last_became_peered":"2024-09-13T16:28:39.213077+0000","last_unstale":"2024-09-13T16:30:04.719142+0000","last_undegraded":"2024-09-13T16:30:04.719142+0000","last_fullsized":"2024-09-13T16:30:04.719142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:32:24.747516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.684840+0000","last_change":"2024-09-13T16:28:39.215216+0000","last_active":"2024-09-13T16:30:04.684840+0000","last_peered":"2024-09-13T16:30:04.684840+0000","last_clean":"2024-09-13T16:30:04.684840+0000","last_became_active":"2024-09-13T16:28:39.214920+0000","last_became_peered":"2024-09-13T16:28:39.214920+0000","last_unstale":"2024-09-13T16:30:04.684840+0000","last_undegraded":"2024-09-13T16:30:04.684840+0000","last_fullsized":"2024-09-13T16:30:04.684840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:25:38.132833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689532+0000","last_change":"2024-09-13T16:28:39.202506+0000","last_active":"2024-09-13T16:30:04.689532+0000","last_peered":"2024-09-13T16:30:04.689532+0000","last_clean":"2024-09-13T16:30:04.689532+0000","last_became_active":"2024-09-13T16:28:39.202224+0000","last_became_peered":"2024-09-13T16:28:39.202224+0000","last_unstale":"2024-09-13T16:30:04.689532+0000","last_undegraded":"2024-09-13T16:30:04.689532+0000","last_fullsized":"2024-09-13T16:30:04.689532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:00:37.208442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.688211+0000","last_change":"2024-09-13T16:29:05.640477+0000","last_active":"2024-09-13T16:30:04.688211+0000","last_peered":"2024-09-13T16:30:04.688211+0000","last_clean":"2024-09-13T16:30:04.688211+0000","last_became_active":"2024-09-13T16:29:05.640014+0000","last_became_peered":"2024-09-13T16:29:05.640014+0000","last_unstale":"2024-09-13T16:30:04.688211+0000","last_undegraded":"2024-09-13T16:30:04.688211+0000","last_fullsized":"2024-09-13T16:30:04.688211+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:47:45.202113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.966433+0000","last_change":"2024-09-13T16:28:39.200217+0000","last_active":"2024-09-13T16:30:04.966433+0000","last_peered":"2024-09-13T16:30:04.966433+0000","last_clean":"2024-09-13T16:30:04.966433+0000","last_became_active":"2024-09-13T16:28:39.199946+0000","last_became_peered":"2024-09-13T16:28:39.199946+0000","last_unstale":"2024-09-13T16:30:04.966433+0000","last_undegraded":"2024-09-13T16:30:04.966433+0000","last_fullsized":"2024-09-13T16:30:04.966433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:20:02.906225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.688846+0000","last_change":"2024-09-13T16:28:39.215013+0000","last_active":"2024-09-13T16:30:04.688846+0000","last_peered":"2024-09-13T16:30:04.688846+0000","last_clean":"2024-09-13T16:30:04.688846+0000","last_became_active":"2024-09-13T16:28:39.214731+0000","last_became_peered":"2024-09-13T16:28:39.214731+0000","last_unstale":"2024-09-13T16:30:04.688846+0000","last_undegraded":"2024-09-13T16:30:04.688846+0000","last_fullsized":"2024-09-13T16:30:04.688846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:35:01.836001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689055+0000","last_change":"2024-09-13T16:28:39.200300+0000","last_active":"2024-09-13T16:30:04.689055+0000","last_peered":"2024-09-13T16:30:04.689055+0000","last_clean":"2024-09-13T16:30:04.689055+0000","last_became_active":"2024-09-13T16:28:39.200006+0000","last_became_peered":"2024-09-13T16:28:39.200006+0000","last_unstale":"2024-09-13T16:30:04.689055+0000","last_undegraded":"2024-09-13T16:30:04.689055+0000","last_fullsized":"2024-09-13T16:30:04.689055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:35:14.783905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.687736+0000","last_change":"2024-09-13T16:28:39.213552+0000","last_active":"2024-09-13T16:30:04.687736+0000","last_peered":"2024-09-13T16:30:04.687736+0000","last_clean":"2024-09-13T16:30:04.687736+0000","last_became_active":"2024-09-13T16:28:39.213287+0000","last_became_peered":"2024-09-13T16:28:39.213287+0000","last_unstale":"2024-09-13T16:30:04.687736+0000","last_undegraded":"2024-09-13T16:30:04.687736+0000","last_fullsized":"2024-09-13T16:30:04.687736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:30.588770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.684568+0000","last_change":"2024-09-13T16:28:39.211126+0000","last_active":"2024-09-13T16:30:04.684568+0000","last_peered":"2024-09-13T16:30:04.684568+0000","last_clean":"2024-09-13T16:30:04.684568+0000","last_became_active":"2024-09-13T16:28:39.210810+0000","last_became_peered":"2024-09-13T16:28:39.210810+0000","last_unstale":"2024-09-13T16:30:04.684568+0000","last_undegraded":"2024-09-13T16:30:04.684568+0000","last_fullsized":"2024-09-13T16:30:04.684568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:52:45.249309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.683625+0000","last_change":"2024-09-13T16:29:05.636535+0000","last_active":"2024-09-13T16:30:04.683625+0000","last_peered":"2024-09-13T16:30:04.683625+0000","last_clean":"2024-09-13T16:30:04.683625+0000","last_became_active":"2024-09-13T16:29:05.636161+0000","last_became_peered":"2024-09-13T16:29:05.636161+0000","last_unstale":"2024-09-13T16:30:04.683625+0000","last_undegraded":"2024-09-13T16:30:04.683625+0000","last_fullsized":"2024-09-13T16:30:04.683625+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:09.606922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689775+0000","last_change":"2024-09-13T16:28:39.202222+0000","last_active":"2024-09-13T16:30:04.689775+0000","last_peered":"2024-09-13T16:30:04.689775+0000","last_clean":"2024-09-13T16:30:04.689775+0000","last_became_active":"2024-09-13T16:28:39.201897+0000","last_became_peered":"2024-09-13T16:28:39.201897+0000","last_unstale":"2024-09-13T16:30:04.689775+0000","last_undegraded":"2024-09-13T16:30:04.689775+0000","last_fullsized":"2024-09-13T16:30:04.689775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:41:09.087185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.690038+0000","last_change":"2024-09-13T16:28:39.211164+0000","last_active":"2024-09-13T16:30:04.690038+0000","last_peered":"2024-09-13T16:30:04.690038+0000","last_clean":"2024-09-13T16:30:04.690038+0000","last_became_active":"2024-09-13T16:28:39.210826+0000","last_became_peered":"2024-09-13T16:28:39.210826+0000","last_unstale":"2024-09-13T16:30:04.690038+0000","last_undegraded":"2024-09-13T16:30:04.690038+0000","last_fullsized":"2024-09-13T16:30:04.690038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:40:30.663867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.966070+0000","last_change":"2024-09-13T16:28:39.207991+0000","last_active":"2024-09-13T16:30:04.966070+0000","last_peered":"2024-09-13T16:30:04.966070+0000","last_clean":"2024-09-13T16:30:04.966070+0000","last_became_active":"2024-09-13T16:28:39.207332+0000","last_became_peered":"2024-09-13T16:28:39.207332+0000","last_unstale":"2024-09-13T16:30:04.966070+0000","last_undegraded":"2024-09-13T16:30:04.966070+0000","last_fullsized":"2024-09-13T16:30:04.966070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:30:54.271361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70716,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":804,"num_read_kb":486,"num_write":236,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":762,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":212148,"data_compressed":4776,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250749,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383712,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":519861,"data_compressed":23704,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.504,"15min":0.495},"min":{"1min":0.371,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":13.802},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54100000000000004,"15min":0.53900000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.39700000000000002}]},{"osd":1,"last update":"Fri Sep 13 16:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.78300000000000003,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55700000000000005,"15min":0.55800000000000005},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69799999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.503}]},{"osd":2,"last update":"Fri Sep 13 16:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.436,"15min":0.438},"min":{"1min":0.29599999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.55400000000000005,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.441,"15min":0.438},"min":{"1min":0.32200000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.627,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.61299999999999999}]},{"osd":3,"last update":"Fri Sep 13 16:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.34200000000000003}]},{"osd":4,"last update":"Fri Sep 13 16:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.40699999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.58899999999999997,"15min":0.58299999999999996},"min":{"1min":0.443,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.83099999999999996,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.59999999999999998}]},{"osd":5,"last update":"Fri Sep 13 16:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54400000000000004,"15min":0.54300000000000004},"min":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70599999999999996,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.505,"15min":0.495},"min":{"1min":0.38300000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.67200000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.41999999999999998}]},{"osd":6,"last update":"Fri Sep 13 16:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.48999999999999999,"5min":0.438,"15min":0.438},"max":{"1min":0.89600000000000002,"5min":1.032,"15min":1.032},"last":13.763999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65400000000000003,"15min":0.65500000000000003},"min":{"1min":0.48999999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77800000000000002,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":13.747}]},{"osd":7,"last update":"Fri Sep 13 16:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.65900000000000003,"15min":0.65300000000000002},"min":{"1min":0.55900000000000005,"5min":0.436,"15min":0.436},"max":{"1min":0.84499999999999997,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.65000000000000002,"15min":0.64400000000000002},"min":{"1min":0.54000000000000004,"5min":0.502,"15min":0.502},"max":{"1min":0.84899999999999998,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.63800000000000001}]},{"osd":8,"last update":"Fri Sep 13 16:29:39 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.622},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.65200000000000002}]},{"osd":9,"last update":"Fri Sep 13 16:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72799999999999998,"15min":0.72999999999999998},"min":{"1min":0.54800000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.89000000000000001,"5min":1.716,"15min":1.716},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.68899999999999995,"15min":0.67800000000000005},"min":{"1min":0.56000000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.87,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.56699999999999995}]},{"osd":10,"last update":"Fri Sep 13 16:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.71299999999999997,"15min":0.70899999999999996},"min":{"1min":0.58299999999999996,"5min":0.505,"15min":0.505},"max":{"1min":0.90400000000000003,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73399999999999999,"15min":0.73499999999999999},"min":{"1min":0.54300000000000004,"5min":0.498,"15min":0.498},"max":{"1min":0.91800000000000004,"5min":1.756,"15min":1.756},"last":0.71499999999999997}]}]},{"osd":10,"up_from":72,"seq":309237645350,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1171456,"data_stored":1169398,"data_compressed":61585,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:29:39 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.53500000000000003}]},{"osd":1,"last update":"Fri Sep 13 16:29:34 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.40999999999999998,"15min":0.40300000000000002},"min":{"1min":0.315,"5min":0.249,"15min":0.249},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.439,"15min":0.437},"min":{"1min":0.312,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.36899999999999999}]},{"osd":2,"last update":"Fri Sep 13 16:29:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.38300000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.76300000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.57199999999999995,"15min":0.56200000000000006},"min":{"1min":0.41199999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.66300000000000003}]},{"osd":3,"last update":"Fri Sep 13 16:29:34 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60399999999999998,"15min":0.60799999999999998},"min":{"1min":0.40799999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80500000000000005,"5min":1.04,"15min":1.04},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.89100000000000001,"15min":0.94599999999999995},"min":{"1min":0.46600000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.70799999999999996,"5min":7.2320000000000002,"15min":7.2320000000000002},"last":0.68000000000000005}]},{"osd":4,"last update":"Fri Sep 13 16:29:34 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48599999999999999,"15min":0.48299999999999998},"min":{"1min":0.39000000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.435},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.46800000000000003,"15min":0.46500000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.65000000000000002,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.39300000000000002}]},{"osd":5,"last update":"Fri Sep 13 16:29:34 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.49399999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78400000000000003,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.63},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.63100000000000001,"15min":0.627},"min":{"1min":0.45000000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.879,"5min":1.006,"15min":1.006},"last":0.58399999999999996}]},{"osd":6,"last update":"Fri Sep 13 16:29:34 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.48399999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.90300000000000002,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63200000000000001,"15min":0.628},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.82299999999999995,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.55300000000000005}]},{"osd":7,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.56999999999999995}]},{"osd":8,"last update":"Fri Sep 13 16:29:34 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.95999999999999996,"15min":1.01},"min":{"1min":0.48499999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.91300000000000003,"5min":7.2569999999999997,"15min":7.2569999999999997},"last":0.746},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.69899999999999995,"15min":0.70299999999999996},"min":{"1min":0.496,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.92200000000000004,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.69599999999999995}]},{"osd":9,"last update":"Fri Sep 13 16:29:34 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89100000000000001,"5min":1.167,"15min":1.167},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.73699999999999999,"15min":0.74299999999999999},"min":{"1min":0.61399999999999999,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.82699999999999996,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.73199999999999998}]},{"osd":11,"last update":"Fri Sep 13 16:30:00 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71099999999999997,"15min":0.71299999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.86399999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.97499999999999998,"15min":1.0209999999999999},"min":{"1min":0.55800000000000005,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.83799999999999997,"5min":7.2199999999999998,"15min":7.2199999999999998},"last":0.76600000000000001}]}]},{"osd":9,"up_from":67,"seq":287762808874,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":523285,"data_compressed":23704,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:30:10 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.40500000000000003,"15min":0.42299999999999999},"min":{"1min":0.28499999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.51600000000000001,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.45500000000000002,"15min":0.48099999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.55000000000000004,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.33000000000000002}]},{"osd":1,"last update":"Fri Sep 13 16:30:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.38800000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.79400000000000004,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58499999999999996,"15min":0.59699999999999998},"min":{"1min":0.40300000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70999999999999996,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.42099999999999999}]},{"osd":2,"last update":"Fri Sep 13 16:30:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.34399999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.629},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.60699999999999998}]},{"osd":3,"last update":"Fri Sep 13 16:30:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.46200000000000002,"15min":0.47999999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65000000000000002,"5min":0.754,"15min":0.754},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.443,"5min":0.50800000000000001,"15min":0.53900000000000003},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.30099999999999999}]},{"osd":4,"last update":"Fri Sep 13 16:30:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.60799999999999998,"15min":0.623},"min":{"1min":0.40200000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.74399999999999999,"5min":0.997,"15min":0.997},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.63400000000000001,"15min":0.65100000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.76400000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.35499999999999998}]},{"osd":5,"last update":"Fri Sep 13 16:30:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60899999999999999,"15min":0.61199999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.63100000000000001,"15min":0.63800000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.58399999999999996}]},{"osd":6,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.45700000000000002}]},{"osd":7,"last update":"Fri Sep 13 16:30:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.70999999999999996,"15min":0.73699999999999999},"min":{"1min":0.47799999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.81599999999999995,"5min":1.107,"15min":1.107},"last":0.373},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.72099999999999997,"15min":0.749},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.98599999999999999,"5min":1.139,"15min":1.139},"last":0.502}]},{"osd":8,"last update":"Fri Sep 13 16:30:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.70799999999999996,"15min":0.73199999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.82199999999999995,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.72099999999999997,"15min":0.73999999999999999},"min":{"1min":0.503,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.83699999999999997,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.56999999999999995}]},{"osd":10,"last update":"Fri Sep 13 16:29:33 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.751,"15min":0.76300000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.022,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.73699999999999999,"15min":0.745},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.006,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.65300000000000002}]},{"osd":11,"last update":"Fri Sep 13 16:29:44 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.55600000000000005}]}]},{"osd":8,"up_from":59,"seq":253403070511,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":1036288,"data_stored":518313,"data_compressed":23704,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.51500000000000001,"15min":0.51100000000000001},"min":{"1min":0.42199999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.71099999999999997,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52000000000000002,"15min":0.51300000000000001},"min":{"1min":0.378,"5min":0.313,"15min":0.313},"max":{"1min":0.76500000000000001,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.66600000000000004}]},{"osd":1,"last update":"Fri Sep 13 16:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.56899999999999995,"15min":0.56399999999999995},"min":{"1min":0.45800000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.72799999999999998,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.45000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.70199999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.48699999999999999}]},{"osd":2,"last update":"Fri Sep 13 16:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.41999999999999998,"15min":0.41299999999999998},"min":{"1min":0.33500000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.59099999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.313},{"interface":"front","average":{"1min":0.433,"5min":0.45200000000000001,"15min":0.45600000000000002},"min":{"1min":0.33300000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57799999999999996,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.34000000000000002}]},{"osd":3,"last update":"Fri Sep 13 16:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.55900000000000005,"15min":0.55300000000000005},"min":{"1min":0.378,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76000000000000001,"5min":0.871,"15min":0.871},"last":0.499},{"interface":"front","average":{"1min":0.621,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.46100000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.85699999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.747}]},{"osd":4,"last update":"Fri Sep 13 16:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57499999999999996,"15min":0.56699999999999995},"min":{"1min":0.46500000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.72399999999999998,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.62,"15min":0.61199999999999999},"min":{"1min":0.47999999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.81200000000000006,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.54400000000000004}]},{"osd":5,"last update":"Fri Sep 13 16:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.378,"5min":0.314,"15min":0.314},"max":{"1min":0.70599999999999996,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.378},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53800000000000003,"15min":0.54300000000000004},"min":{"1min":0.38,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.74299999999999999,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.39200000000000002}]},{"osd":6,"last update":"Fri Sep 13 16:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63800000000000001,"15min":0.63300000000000001},"min":{"1min":0.48899999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.89000000000000001,"5min":1.002,"15min":1.002},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66300000000000003,"15min":0.66400000000000003},"min":{"1min":0.53300000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.86899999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.57899999999999996}]},{"osd":7,"last update":"Fri Sep 13 16:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.67000000000000004,"15min":0.67100000000000004},"min":{"1min":0.52100000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.84899999999999998,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.66200000000000003,"15min":0.66300000000000003},"min":{"1min":0.49199999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.83199999999999996,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.61199999999999999}]},{"osd":9,"last update":"Fri Sep 13 16:30:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.63800000000000001,"15min":0.61399999999999999},"min":{"1min":0.47099999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85199999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.67000000000000004,"15min":0.64200000000000002},"min":{"1min":0.48899999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95699999999999996,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.63}]},{"osd":10,"last update":"Fri Sep 13 16:29:34 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.73499999999999999,"15min":0.73099999999999998},"min":{"1min":0.61299999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0529999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.67100000000000004,"15min":0.64900000000000002},"min":{"1min":0.69299999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.76800000000000002}]},{"osd":11,"last update":"Fri Sep 13 16:29:43 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.68999999999999995}]}]},{"osd":7,"up_from":51,"seq":219043332147,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":1116705,"data_compressed":59993,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.39600000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.41199999999999998}]},{"osd":1,"last update":"Fri Sep 13 16:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.39500000000000002,"15min":0.39000000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.76400000000000001,"5min":1.02,"15min":1.02},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.443,"5min":0.41399999999999998,"15min":0.41099999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.48499999999999999}]},{"osd":2,"last update":"Fri Sep 13 16:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.69499999999999995,"5min":0.748,"15min":0.748},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55300000000000005,"15min":0.55000000000000004},"min":{"1min":0.42499999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.58199999999999996}]},{"osd":3,"last update":"Fri Sep 13 16:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.52900000000000003,"15min":0.52800000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.82399999999999995,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.56799999999999995}]},{"osd":4,"last update":"Fri Sep 13 16:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.45500000000000002,"15min":0.45200000000000001},"min":{"1min":0.29499999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.45400000000000001,"15min":0.46200000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.77700000000000002,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.69099999999999995}]},{"osd":5,"last update":"Fri Sep 13 16:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.58699999999999997,"15min":0.58099999999999996},"min":{"1min":0.41199999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.81299999999999994,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.44500000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85399999999999998,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.64600000000000002}]},{"osd":6,"last update":"Fri Sep 13 16:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.60199999999999998,"15min":0.60799999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61099999999999999,"15min":0.61499999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.73799999999999999}]},{"osd":8,"last update":"Fri Sep 13 16:29:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.628,"15min":0.627},"min":{"1min":0.46400000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.72499999999999998,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66200000000000003,"15min":0.66600000000000004},"min":{"1min":0.50800000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77000000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.71099999999999997}]},{"osd":9,"last update":"Fri Sep 13 16:30:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.628,"15min":0.625},"min":{"1min":0.52100000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89100000000000001,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.66200000000000003,"15min":0.65000000000000002},"min":{"1min":0.58999999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.88300000000000001,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.72299999999999998}]},{"osd":10,"last update":"Fri Sep 13 16:29:34 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.621,"15min":0.62},"min":{"1min":0.433,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78700000000000003,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.64300000000000002,"15min":0.63600000000000001},"min":{"1min":0.52100000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84399999999999997,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.67800000000000005}]},{"osd":11,"last update":"Fri Sep 13 16:29:42 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.749},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.83799999999999997}]}]},{"osd":6,"up_from":46,"seq":197568495673,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":1028096,"data_stored":515037,"data_compressed":23704,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.39100000000000001,"15min":0.375},"min":{"1min":0.252,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39900000000000002,"15min":0.39100000000000001},"min":{"1min":0.25800000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.373}]},{"osd":1,"last update":"Fri Sep 13 16:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54000000000000004,"15min":0.51400000000000001},"min":{"1min":0.27400000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.79600000000000004,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.625},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56999999999999995,"15min":0.54800000000000004},"min":{"1min":0.34499999999999997,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.54900000000000004}]},{"osd":2,"last update":"Fri Sep 13 16:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55700000000000005,"15min":0.55000000000000004},"min":{"1min":0.38400000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83499999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60099999999999998,"15min":0.58899999999999997},"min":{"1min":0.39100000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.92000000000000004,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.42699999999999999}]},{"osd":3,"last update":"Fri Sep 13 16:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.46700000000000003,"15min":0.46600000000000003},"min":{"1min":0.28499999999999998,"5min":0.251,"15min":0.251},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.501,"15min":0.48999999999999999},"min":{"1min":0.26200000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.65500000000000003,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.40699999999999997}]},{"osd":4,"last update":"Fri Sep 13 16:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.317,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.81599999999999995,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63300000000000001,"15min":0.63100000000000001},"min":{"1min":0.42699999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0029999999999999,"5min":1.079,"15min":1.079},"last":0.70999999999999996}]},{"osd":5,"last update":"Fri Sep 13 16:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.625,"15min":0.60799999999999998},"min":{"1min":0.43099999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.92400000000000004,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.499},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.44600000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85899999999999999,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.48899999999999999}]},{"osd":7,"last update":"Fri Sep 13 16:29:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.63,"15min":0.629},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.93500000000000005,"5min":1.048,"15min":1.048},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.66400000000000003,"15min":0.65200000000000002},"min":{"1min":0.45600000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.94999999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.65300000000000002}]},{"osd":8,"last update":"Fri Sep 13 16:29:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.49099999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90400000000000003,"5min":1.054,"15min":1.054},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.71199999999999997,"15min":0.72299999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.038,"5min":1.042,"15min":1.042},"last":0.53000000000000003}]},{"osd":9,"last update":"Fri Sep 13 16:30:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.68999999999999995,"15min":0.71599999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.95799999999999996,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.68000000000000005,"15min":0.69799999999999995},"min":{"1min":0.436,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.86099999999999999,"5min":1.113,"15min":1.113},"last":0.64300000000000002}]},{"osd":10,"last update":"Fri Sep 13 16:29:36 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.71999999999999997,"15min":0.72899999999999998},"min":{"1min":0.5,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.751},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.76300000000000001,"15min":0.77300000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94499999999999995,"5min":1.089,"15min":1.089},"last":0.71999999999999997}]},{"osd":11,"last update":"Fri Sep 13 16:29:43 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.61399999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757310,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1105920,"data_stored":1111268,"data_compressed":59993,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:29:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.68600000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.54600000000000004,"15min":0.53700000000000003},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.435}]},{"osd":1,"last update":"Fri Sep 13 16:29:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.502,"15min":0.48199999999999998},"min":{"1min":0.32500000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52500000000000002,"15min":0.50700000000000001},"min":{"1min":0.41299999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.73499999999999999,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.61599999999999999}]},{"osd":2,"last update":"Fri Sep 13 16:29:42 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.39300000000000002,"15min":0.36599999999999999},"min":{"1min":0.33500000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.64000000000000001,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.39100000000000001,"15min":0.371},"min":{"1min":0.34200000000000003,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.61599999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.46800000000000003}]},{"osd":3,"last update":"Fri Sep 13 16:29:42 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56299999999999994,"15min":0.56200000000000006},"min":{"1min":0.35399999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.754,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.55600000000000005,"15min":0.54400000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.69599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.51500000000000001}]},{"osd":4,"last update":"Fri Sep 13 16:29:42 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.54300000000000004,"15min":0.51700000000000002},"min":{"1min":0.47099999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54800000000000004,"15min":0.53200000000000003},"min":{"1min":0.376,"5min":0.316,"15min":0.316},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.68500000000000005}]},{"osd":6,"last update":"Fri Sep 13 16:30:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84699999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58899999999999997,"15min":0.57599999999999996},"min":{"1min":0.47099999999999997,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.78400000000000003,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.628}]},{"osd":7,"last update":"Fri Sep 13 16:29:23 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60699999999999998,"15min":0.60499999999999998},"min":{"1min":0.42999999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79300000000000004,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79900000000000004,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.66700000000000004}]},{"osd":8,"last update":"Fri Sep 13 16:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.40200000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.76100000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.45300000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.70799999999999996}]},{"osd":9,"last update":"Fri Sep 13 16:30:09 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.66800000000000004,"15min":0.65300000000000002},"min":{"1min":0.56100000000000005,"5min":0.497,"15min":0.497},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.66400000000000003,"15min":0.65600000000000003},"min":{"1min":0.502,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.80900000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.69799999999999995}]},{"osd":10,"last update":"Fri Sep 13 16:29:32 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.67000000000000004,"15min":0.66000000000000003},"min":{"1min":0.53400000000000003,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.64600000000000002,"15min":0.63800000000000001},"min":{"1min":0.51500000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.872,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.72499999999999998}]}]},{"osd":4,"up_from":33,"seq":141733920834,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":519223,"data_compressed":23704,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.48699999999999999,"15min":0.46300000000000002},"min":{"1min":0.32200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.751,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.504,"15min":0.48199999999999998},"min":{"1min":0.41399999999999998,"5min":0.25,"15min":0.25},"max":{"1min":0.80800000000000005,"5min":0.874,"15min":0.874},"last":0.60499999999999998}]},{"osd":1,"last update":"Fri Sep 13 16:29:21 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.36199999999999999,"15min":0.33600000000000002},"min":{"1min":0.29799999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.63400000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.36099999999999999,"15min":0.33800000000000002},"min":{"1min":0.32400000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52300000000000002,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.35799999999999998}]},{"osd":2,"last update":"Fri Sep 13 16:29:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.50600000000000001,"15min":0.48999999999999999},"min":{"1min":0.33800000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73599999999999999,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.501,"15min":0.46400000000000002},"min":{"1min":0.35799999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.53300000000000003}]},{"osd":3,"last update":"Fri Sep 13 16:29:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.51900000000000002,"15min":0.49099999999999999},"min":{"1min":0.435,"5min":0.308,"15min":0.308},"max":{"1min":0.71499999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53400000000000003,"15min":0.5},"min":{"1min":0.377,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.83399999999999996,"5min":0.875,"15min":0.875},"last":0.40100000000000002}]},{"osd":5,"last update":"Fri Sep 13 16:29:38 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.57599999999999996,"15min":0.55400000000000005},"min":{"1min":0.39500000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.57299999999999995,"15min":0.54000000000000004},"min":{"1min":0.50900000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80700000000000005,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.61599999999999999}]},{"osd":6,"last update":"Fri Sep 13 16:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56699999999999995,"15min":0.54100000000000004},"min":{"1min":0.39200000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77100000000000002,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58499999999999996,"15min":0.56899999999999995},"min":{"1min":0.40500000000000003,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71599999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.46100000000000002}]},{"osd":7,"last update":"Fri Sep 13 16:29:26 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.503,"15min":0.48399999999999999},"min":{"1min":0.379,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.83399999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.503,"15min":0.49299999999999999},"min":{"1min":0.37,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76600000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.44800000000000001}]},{"osd":8,"last update":"Fri Sep 13 16:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.59399999999999997,"15min":0.57599999999999996},"min":{"1min":0.47899999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84599999999999997,"5min":0.872,"15min":0.872},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.628,"15min":0.61399999999999999},"min":{"1min":0.49199999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.79500000000000004,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.58899999999999997}]},{"osd":9,"last update":"Fri Sep 13 16:30:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.64200000000000002,"15min":0.628},"min":{"1min":0.51300000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84699999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67600000000000005,"15min":0.67200000000000004},"min":{"1min":0.53500000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.83299999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.501}]},{"osd":10,"last update":"Fri Sep 13 16:29:34 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.48599999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.77900000000000003,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.63300000000000001,"15min":0.63900000000000001},"min":{"1min":0.45500000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.435}]},{"osd":11,"last update":"Fri Sep 13 16:29:38 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.63},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64500000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36228,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708924,"statfs":{"total":95995035648,"available":95957938176,"internally_reserved":0,"allocated":1036288,"data_stored":514070,"data_compressed":23704,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:29:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.371,"15min":0.33200000000000002},"min":{"1min":0.26500000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.38200000000000001,"15min":0.33800000000000002},"min":{"1min":0.23000000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.28799999999999998}]},{"osd":1,"last update":"Fri Sep 13 16:29:57 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57899999999999996,"15min":0.49299999999999999},"min":{"1min":0.39900000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.91800000000000004,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55400000000000005,"15min":0.49299999999999999},"min":{"1min":0.39200000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.46999999999999997}]},{"osd":2,"last update":"Fri Sep 13 16:29:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52900000000000003,"15min":0.51600000000000001},"min":{"1min":0.35799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.57799999999999996,"15min":0.53400000000000003},"min":{"1min":0.42099999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.94599999999999995,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.63400000000000001}]},{"osd":4,"last update":"Fri Sep 13 16:30:12 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57099999999999995,"15min":0.50900000000000001},"min":{"1min":0.42199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.61899999999999999,"15min":0.55700000000000005},"min":{"1min":0.505,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.96799999999999997,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.48899999999999999}]},{"osd":5,"last update":"Fri Sep 13 16:29:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59299999999999997,"15min":0.57499999999999996},"min":{"1min":0.45500000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81599999999999995,"5min":1.038,"15min":1.038},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60899999999999999,"15min":0.59999999999999998},"min":{"1min":0.41599999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.872,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.61599999999999999}]},{"osd":6,"last update":"Fri Sep 13 16:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.52600000000000002,"15min":0.503},"min":{"1min":0.34200000000000003,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.503,"15min":0.48599999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.36299999999999999}]},{"osd":7,"last update":"Fri Sep 13 16:29:23 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.68300000000000005,"15min":0.63300000000000001},"min":{"1min":0.48799999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":5.5099999999999998,"5min":5.5099999999999998,"15min":5.5099999999999998},"last":0.498},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61599999999999999,"15min":0.59599999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.78600000000000003,"5min":1.014,"15min":1.014},"last":0.44800000000000001}]},{"osd":8,"last update":"Fri Sep 13 16:27:45 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.71499999999999997}]},{"osd":9,"last update":"Fri Sep 13 16:30:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65000000000000002,"15min":0.66000000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.68000000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.436}]},{"osd":10,"last update":"Fri Sep 13 16:29:40 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.51300000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79000000000000004,"15min":0.78800000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1140000000000001,"5min":1.157,"15min":1.157},"last":0.74299999999999999}]},{"osd":11,"last update":"Fri Sep 13 16:29:40 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.77000000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":1093632,"data_stored":579096,"data_compressed":25296,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:29:30 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.61699999999999999,"15min":0.502},"min":{"1min":0.35999999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70399999999999996,"5min":16.027999999999999,"15min":16.027999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.49299999999999999,"15min":0.47199999999999998},"min":{"1min":0.38300000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.45000000000000001}]},{"osd":1,"last update":"Fri Sep 13 16:29:30 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.52100000000000002,"15min":0.47699999999999998},"min":{"1min":0.41099999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.55400000000000005,"15min":0.50600000000000001},"min":{"1min":0.49199999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.83699999999999997,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.72199999999999998}]},{"osd":3,"last update":"Fri Sep 13 16:29:54 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56100000000000005,"15min":0.52100000000000002},"min":{"1min":0.436,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77100000000000002,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55700000000000005,"15min":0.52100000000000002},"min":{"1min":0.37,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.70899999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.56000000000000005}]},{"osd":4,"last update":"Fri Sep 13 16:29:39 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.69099999999999995}]},{"osd":5,"last update":"Fri Sep 13 16:29:40 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.432,"15min":0.40899999999999997},"min":{"1min":0.32700000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.76900000000000002,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.501},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.41699999999999998,"15min":0.39700000000000002},"min":{"1min":0.33300000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.40400000000000003}]},{"osd":6,"last update":"Fri Sep 13 16:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59299999999999997,"15min":0.55700000000000005},"min":{"1min":0.43099999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73299999999999998,"5min":1.585,"15min":1.585},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61899999999999999,"15min":0.58799999999999997},"min":{"1min":0.503,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79400000000000004,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.61799999999999999}]},{"osd":7,"last update":"Fri Sep 13 16:29:43 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.75700000000000001}]},{"osd":8,"last update":"Fri Sep 13 16:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.53700000000000003,"15min":0.52100000000000002},"min":{"1min":0.38300000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.95899999999999996,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52300000000000002,"15min":0.51000000000000001},"min":{"1min":0.40100000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76600000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.54200000000000004}]},{"osd":9,"last update":"Fri Sep 13 16:30:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68300000000000005,"15min":0.67100000000000004},"min":{"1min":0.49299999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88900000000000001,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68799999999999994,"15min":0.67300000000000004},"min":{"1min":0.54500000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.83299999999999996,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.78500000000000003}]},{"osd":10,"last update":"Fri Sep 13 16:29:39 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.68600000000000005,"15min":0.67300000000000004},"min":{"1min":0.504,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.74399999999999999,"15min":0.73599999999999999},"min":{"1min":0.56000000000000005,"5min":0.498,"15min":0.498},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.82899999999999996}]},{"osd":11,"last update":"Fri Sep 13 16:29:39 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.70999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":580049,"data_compressed":25296,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:30:15 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.503,"15min":0.48499999999999999},"min":{"1min":0.30299999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.80100000000000005,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.497,"15min":0.48499999999999999},"min":{"1min":0.29799999999999999,"5min":0.29399999999999998,"15min":0.26900000000000002},"max":{"1min":0.69099999999999995,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.36299999999999999}]},{"osd":2,"last update":"Fri Sep 13 16:29:29 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.68700000000000006,"15min":0.54600000000000004},"min":{"1min":0.30399999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.75700000000000001,"5min":17.655000000000001,"15min":17.655000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.50700000000000001,"15min":0.48499999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.76200000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.50600000000000001}]},{"osd":3,"last update":"Fri Sep 13 16:29:55 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53900000000000003,"15min":0.51200000000000001},"min":{"1min":0.38100000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57299999999999995,"15min":0.52700000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.60499999999999998}]},{"osd":4,"last update":"Fri Sep 13 16:28:13 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.40400000000000003,"15min":0.38},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.433,"5min":0.40899999999999997,"15min":0.39100000000000001},"min":{"1min":0.28299999999999997,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.57899999999999996}]},{"osd":5,"last update":"Fri Sep 13 16:29:37 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.89800000000000002,"15min":1.2290000000000001},"min":{"1min":0.441,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.80800000000000005,"5min":17.640000000000001,"15min":17.640000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.56399999999999995,"15min":0.54900000000000004},"min":{"1min":0.42799999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82899999999999996,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.48299999999999998}]},{"osd":6,"last update":"Fri Sep 13 16:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.69899999999999995,"5min":0.872,"15min":0.872},"last":0.52200000000000002}]},{"osd":7,"last update":"Fri Sep 13 16:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.55300000000000005,"15min":0.55400000000000005},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.77400000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.623},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.36899999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78900000000000003,"5min":1.022,"15min":1.022},"last":0.63600000000000001}]},{"osd":8,"last update":"Fri Sep 13 16:29:43 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.48699999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83699999999999997,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.65500000000000003,"15min":0.64500000000000002},"min":{"1min":0.55400000000000005,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84799999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.46700000000000003}]},{"osd":9,"last update":"Fri Sep 13 16:30:15 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.64600000000000002,"15min":0.65000000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.75700000000000001,"5min":1.008,"15min":1.008},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65500000000000003,"15min":0.65600000000000003},"min":{"1min":0.42699999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.85399999999999998,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.59099999999999997}]},{"osd":10,"last update":"Fri Sep 13 16:29:37 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65500000000000003,"15min":0.65700000000000003},"min":{"1min":0.47599999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85599999999999998,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.82199999999999995,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.57699999999999996}]},{"osd":11,"last update":"Fri Sep 13 16:29:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.67100000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":516101,"data_compressed":23704,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 16:30:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.50900000000000001,"15min":0.48199999999999998},"min":{"1min":0.32600000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.81799999999999995,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.53600000000000003,"15min":0.505},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.84399999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.60699999999999998}]},{"osd":2,"last update":"Fri Sep 13 16:29:37 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56499999999999995,"15min":0.53100000000000003},"min":{"1min":0.36499999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.95399999999999996,"5min":1.111,"15min":1.111},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.55300000000000005,"15min":0.56299999999999994},"min":{"1min":0.34499999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77600000000000002,"5min":1.698,"15min":1.698},"last":0.436}]},{"osd":3,"last update":"Fri Sep 13 16:29:39 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.55600000000000005}]},{"osd":4,"last update":"Fri Sep 13 16:30:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.54700000000000004,"15min":0.47599999999999998},"min":{"1min":0.39700000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.84199999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56299999999999994,"15min":0.5},"min":{"1min":0.35499999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.63400000000000001}]},{"osd":5,"last update":"Fri Sep 13 16:29:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55800000000000005,"15min":0.54300000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.91300000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.58499999999999996,"15min":0.56200000000000006},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.88200000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.61299999999999999}]},{"osd":6,"last update":"Fri Sep 13 16:29:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.52800000000000002,"15min":0.48999999999999999},"min":{"1min":0.34799999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56699999999999995,"15min":0.52600000000000002},"min":{"1min":0.32200000000000001,"5min":0.309,"15min":0.309},"max":{"1min":1.2230000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.48899999999999999}]},{"osd":7,"last update":"Fri Sep 13 16:29:46 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.66000000000000003}]},{"osd":8,"last update":"Fri Sep 13 16:29:48 2024","interfaces":[{"interface":"back","average":{"1min":1.5880000000000001,"5min":0.83999999999999997,"15min":0.71499999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":19.501999999999999,"5min":19.501999999999999,"15min":19.501999999999999},"last":0.624},{"interface":"front","average":{"1min":1.665,"5min":0.89100000000000001,"15min":0.76700000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":19.488,"5min":19.488,"15min":19.488},"last":0.68700000000000006}]},{"osd":9,"last update":"Fri Sep 13 16:30:14 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.68600000000000005,"15min":0.72199999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.97799999999999998,"5min":1.01,"15min":1.01},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.68000000000000005,"15min":0.71699999999999997},"min":{"1min":0.39300000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.1930000000000001,"5min":1.236,"15min":1.236},"last":0.52900000000000003}]},{"osd":10,"last update":"Fri Sep 13 16:29:37 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.70599999999999996,"15min":0.70399999999999996},"min":{"1min":0.48599999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.248,"5min":1.284,"15min":1.284},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.75600000000000001,"15min":0.76200000000000001},"min":{"1min":0.56200000000000006,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.032,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.71399999999999997}]},{"osd":11,"last update":"Fri Sep 13 16:29:59 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.69999999999999996,"15min":0.69599999999999995},"min":{"1min":0.56399999999999995,"5min":0.502,"15min":0.502},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.73299999999999998,"15min":0.72499999999999998},"min":{"1min":0.60399999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.74399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":66078,"data_compressed":1592,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65398,"data_compressed":1592,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":65390,"data_compressed":1592,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T16:30:24.226 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-13T16:30:24.435 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:30:24.838 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:24 smithi016 ceph-mon[31187]: pgmap v333: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T16:30:24.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:24 smithi111 ceph-mon[34600]: pgmap v333: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T16:30:25.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:24 smithi107 ceph-mon[34139]: pgmap v333: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T16:30:26.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.184+0000 7fa06171f640 1 -- 172.21.15.16:0/941171942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa05c105110 msgr2=0x7fa05c10cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:26.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.184+0000 7fa06171f640 1 --2- 172.21.15.16:0/941171942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa05c105110 0x7fa05c10cc90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa054009cd0 tx=0x7fa05402a8e0 comp rx=0 tx=0).stop 2024-09-13T16:30:26.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.184+0000 7fa06171f640 1 -- 172.21.15.16:0/941171942 shutdown_connections 2024-09-13T16:30:26.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.184+0000 7fa06171f640 1 --2- 172.21.15.16:0/941171942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa05c105110 0x7fa05c10cc90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:26.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.184+0000 7fa06171f640 1 --2- 172.21.15.16:0/941171942 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa05c100990 0x7fa05c1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:26.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.184+0000 7fa06171f640 1 --2- 172.21.15.16:0/941171942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa05c0fffe0 0x7fa05c1003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:26.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.184+0000 7fa06171f640 1 -- 172.21.15.16:0/941171942 >> 172.21.15.16:0/941171942 conn(0x7fa05c0fbae0 msgr2=0x7fa05c0fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:26.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.184+0000 7fa06171f640 1 -- 172.21.15.16:0/941171942 shutdown_connections 2024-09-13T16:30:26.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.184+0000 7fa06171f640 1 -- 172.21.15.16:0/941171942 wait complete. 2024-09-13T16:30:26.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.185+0000 7fa06171f640 1 Processor -- start 2024-09-13T16:30:26.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.185+0000 7fa06171f640 1 -- start start 2024-09-13T16:30:26.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.185+0000 7fa06171f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa05c0fffe0 0x7fa05c196240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:26.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.185+0000 7fa06171f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa05c100990 0x7fa05c196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:26.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.185+0000 7fa06171f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa05c105110 0x7fa05c19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:26.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.185+0000 7fa06171f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05c19b180 con 0x7fa05c100990 2024-09-13T16:30:26.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.185+0000 7fa06171f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05c19b2f0 con 0x7fa05c0fffe0 2024-09-13T16:30:26.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.185+0000 7fa06171f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05c19b460 con 0x7fa05c105110 2024-09-13T16:30:26.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.185+0000 7fa0527fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa05c100990 0x7fa05c196780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:26.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.186+0000 7fa0527fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa05c100990 0x7fa05c196780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57718/0 (socket says 172.21.15.16:57718) 2024-09-13T16:30:26.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.186+0000 7fa0527fc640 1 -- 172.21.15.16:0/1927067248 learned_addr learned my addr 172.21.15.16:0/1927067248 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:26.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.186+0000 7fa05b7fe640 1 --2- 172.21.15.16:0/1927067248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa05c105110 0x7fa05c19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:26.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.186+0000 7fa05affd640 1 --2- 172.21.15.16:0/1927067248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa05c0fffe0 0x7fa05c196240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:26.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.186+0000 7fa0527fc640 1 -- 172.21.15.16:0/1927067248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa05c105110 msgr2=0x7fa05c19ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:26.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.186+0000 7fa0527fc640 1 --2- 172.21.15.16:0/1927067248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa05c105110 0x7fa05c19ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:26.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.186+0000 7fa0527fc640 1 -- 172.21.15.16:0/1927067248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa05c0fffe0 msgr2=0x7fa05c196240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:26.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.186+0000 7fa0527fc640 1 --2- 172.21.15.16:0/1927067248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa05c0fffe0 0x7fa05c196240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:26.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.186+0000 7fa0527fc640 1 -- 172.21.15.16:0/1927067248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa054009900 con 0x7fa05c100990 2024-09-13T16:30:26.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.186+0000 7fa05b7fe640 1 --2- 172.21.15.16:0/1927067248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa05c105110 0x7fa05c19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:30:26.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.186+0000 7fa05affd640 1 --2- 172.21.15.16:0/1927067248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa05c0fffe0 0x7fa05c196240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:30:26.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.186+0000 7fa0527fc640 1 --2- 172.21.15.16:0/1927067248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa05c100990 0x7fa05c196780 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa04400d770 tx=0x7fa04400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:26.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.186+0000 7fa058ff9640 1 -- 172.21.15.16:0/1927067248 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa044009c30 con 0x7fa05c100990 2024-09-13T16:30:26.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.186+0000 7fa06171f640 1 -- 172.21.15.16:0/1927067248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa05c19b740 con 0x7fa05c100990 2024-09-13T16:30:26.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.186+0000 7fa058ff9640 1 -- 172.21.15.16:0/1927067248 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa04400de80 con 0x7fa05c100990 2024-09-13T16:30:26.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.186+0000 7fa058ff9640 1 -- 172.21.15.16:0/1927067248 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa044004d40 con 0x7fa05c100990 2024-09-13T16:30:26.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.186+0000 7fa06171f640 1 -- 172.21.15.16:0/1927067248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa05c1a3000 con 0x7fa05c100990 2024-09-13T16:30:26.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.188+0000 7fa06171f640 1 -- 172.21.15.16:0/1927067248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa05c1014c0 con 0x7fa05c100990 2024-09-13T16:30:26.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.189+0000 7fa058ff9640 1 -- 172.21.15.16:0/1927067248 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fa044004ea0 con 0x7fa05c100990 2024-09-13T16:30:26.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.189+0000 7fa058ff9640 1 --2- 172.21.15.16:0/1927067248 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa0480a1110 0x7fa0480a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:26.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.189+0000 7fa058ff9640 1 -- 172.21.15.16:0/1927067248 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fa0440cc8d0 con 0x7fa05c100990 2024-09-13T16:30:26.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.189+0000 7fa05affd640 1 --2- 172.21.15.16:0/1927067248 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa0480a1110 0x7fa0480a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:26.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.190+0000 7fa05affd640 1 --2- 172.21.15.16:0/1927067248 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa0480a1110 0x7fa0480a35d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa04c009930 tx=0x7fa04c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:26.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.194+0000 7fa058ff9640 1 -- 172.21.15.16:0/1927067248 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa044095bb0 con 0x7fa05c100990 2024-09-13T16:30:26.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.361+0000 7fa06171f640 1 -- 172.21.15.16:0/1927067248 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa05c19b9d0 con 0x7fa0480a1110 2024-09-13T16:30:26.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.372+0000 7fa058ff9640 1 -- 172.21.15.16:0/1927067248 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+428571 (secure 0 0 0) 0x7fa05c19b9d0 con 0x7fa0480a1110 2024-09-13T16:30:26.377 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:30:26.379 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-13T16:30:26.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.375+0000 7fa06171f640 1 -- 172.21.15.16:0/1927067248 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa0480a1110 msgr2=0x7fa0480a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:26.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.375+0000 7fa06171f640 1 --2- 172.21.15.16:0/1927067248 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa0480a1110 0x7fa0480a35d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa04c009930 tx=0x7fa04c008040 comp rx=0 tx=0).stop 2024-09-13T16:30:26.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.375+0000 7fa06171f640 1 -- 172.21.15.16:0/1927067248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa05c100990 msgr2=0x7fa05c196780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:26.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.375+0000 7fa06171f640 1 --2- 172.21.15.16:0/1927067248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa05c100990 0x7fa05c196780 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa04400d770 tx=0x7fa04400dc30 comp rx=0 tx=0).stop 2024-09-13T16:30:26.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.376+0000 7fa06171f640 1 -- 172.21.15.16:0/1927067248 shutdown_connections 2024-09-13T16:30:26.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.376+0000 7fa06171f640 1 --2- 172.21.15.16:0/1927067248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa05c105110 0x7fa05c19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:26.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.376+0000 7fa06171f640 1 --2- 172.21.15.16:0/1927067248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa05c100990 0x7fa05c196780 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:26.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.376+0000 7fa06171f640 1 --2- 172.21.15.16:0/1927067248 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa0480a1110 0x7fa0480a35d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:26.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.376+0000 7fa06171f640 1 --2- 172.21.15.16:0/1927067248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa05c0fffe0 0x7fa05c196240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:26.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.376+0000 7fa06171f640 1 -- 172.21.15.16:0/1927067248 >> 172.21.15.16:0/1927067248 conn(0x7fa05c0fbae0 msgr2=0x7fa05c0fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:26.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.376+0000 7fa06171f640 1 -- 172.21.15.16:0/1927067248 shutdown_connections 2024-09-13T16:30:26.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:26.376+0000 7fa06171f640 1 -- 172.21.15.16:0/1927067248 wait complete. 2024-09-13T16:30:26.852 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:26 smithi016 ceph-mon[31187]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T16:30:26.852 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:26 smithi016 ceph-mon[31187]: pgmap v334: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T16:30:26.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:26 smithi111 ceph-mon[34600]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T16:30:26.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:26 smithi111 ceph-mon[34600]: pgmap v334: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T16:30:27.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:26 smithi107 ceph-mon[34139]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T16:30:27.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:26 smithi107 ceph-mon[34139]: pgmap v334: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T16:30:27.143 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2024-09-13T16:30:24.594548+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661318,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":945,"num_read_kb":595,"num_write":419,"num_write_kb":3079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1181498,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":347,"ondisk_log_size":347,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365500,"kb_used_data":12568,"kb_used_omap":18,"kb_used_meta":352749,"kb_avail":1124576324,"statfs":{"total":1151940427776,"available":1151566155776,"internally_reserved":0,"allocated":12869632,"data_stored":8182406,"data_compressed":398091,"data_compressed_allocated":540672,"data_compressed_original":2580480,"omap_allocated":19131,"internal_metadata":361215301},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003668"},"pg_stats":[{"pgid":"2.1f","version":"91'13","reported_seq":131,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T16:30:20.740633+0000","last_change":"2024-09-13T16:29:05.647872+0000","last_active":"2024-09-13T16:30:20.740633+0000","last_peered":"2024-09-13T16:30:20.740633+0000","last_clean":"2024-09-13T16:30:20.740633+0000","last_became_active":"2024-09-13T16:29:05.637050+0000","last_became_peered":"2024-09-13T16:29:05.637050+0000","last_unstale":"2024-09-13T16:30:20.740633+0000","last_undegraded":"2024-09-13T16:30:20.740633+0000","last_fullsized":"2024-09-13T16:30:20.740633+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:04.113840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":77,"num_read_kb":77,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.690256+0000","last_change":"2024-09-13T16:28:39.210024+0000","last_active":"2024-09-13T16:30:04.690256+0000","last_peered":"2024-09-13T16:30:04.690256+0000","last_clean":"2024-09-13T16:30:04.690256+0000","last_became_active":"2024-09-13T16:28:39.209751+0000","last_became_peered":"2024-09-13T16:28:39.209751+0000","last_unstale":"2024-09-13T16:30:04.690256+0000","last_undegraded":"2024-09-13T16:30:04.690256+0000","last_fullsized":"2024-09-13T16:30:04.690256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:34:24.357779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.726086+0000","last_change":"2024-09-13T16:28:43.230223+0000","last_active":"2024-09-13T16:30:04.726086+0000","last_peered":"2024-09-13T16:30:04.726086+0000","last_clean":"2024-09-13T16:30:04.726086+0000","last_became_active":"2024-09-13T16:28:43.228992+0000","last_became_peered":"2024-09-13T16:28:43.228992+0000","last_unstale":"2024-09-13T16:30:04.726086+0000","last_undegraded":"2024-09-13T16:30:04.726086+0000","last_fullsized":"2024-09-13T16:30:04.726086+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:45:37.252865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.690413+0000","last_change":"2024-09-13T16:28:38.850198+0000","last_active":"2024-09-13T16:30:04.690413+0000","last_peered":"2024-09-13T16:30:04.690413+0000","last_clean":"2024-09-13T16:30:04.690413+0000","last_became_active":"2024-09-13T16:28:38.849780+0000","last_became_peered":"2024-09-13T16:28:38.849780+0000","last_unstale":"2024-09-13T16:30:04.690413+0000","last_undegraded":"2024-09-13T16:30:04.690413+0000","last_fullsized":"2024-09-13T16:30:04.690413+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:09.653474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689328+0000","last_change":"2024-09-13T16:28:39.203030+0000","last_active":"2024-09-13T16:30:04.689328+0000","last_peered":"2024-09-13T16:30:04.689328+0000","last_clean":"2024-09-13T16:30:04.689328+0000","last_became_active":"2024-09-13T16:28:39.202464+0000","last_became_peered":"2024-09-13T16:28:39.202464+0000","last_unstale":"2024-09-13T16:30:04.689328+0000","last_undegraded":"2024-09-13T16:30:04.689328+0000","last_fullsized":"2024-09-13T16:30:04.689328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:57.528836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.684640+0000","last_change":"2024-09-13T16:28:43.224358+0000","last_active":"2024-09-13T16:30:04.684640+0000","last_peered":"2024-09-13T16:30:04.684640+0000","last_clean":"2024-09-13T16:30:04.684640+0000","last_became_active":"2024-09-13T16:28:43.223955+0000","last_became_peered":"2024-09-13T16:28:43.223955+0000","last_unstale":"2024-09-13T16:30:04.684640+0000","last_undegraded":"2024-09-13T16:30:04.684640+0000","last_fullsized":"2024-09-13T16:30:04.684640+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:53:09.169704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'12","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.685060+0000","last_change":"2024-09-13T16:28:38.846051+0000","last_active":"2024-09-13T16:30:04.685060+0000","last_peered":"2024-09-13T16:30:04.685060+0000","last_clean":"2024-09-13T16:30:04.685060+0000","last_became_active":"2024-09-13T16:28:38.845762+0000","last_became_peered":"2024-09-13T16:28:38.845762+0000","last_unstale":"2024-09-13T16:30:04.685060+0000","last_undegraded":"2024-09-13T16:30:04.685060+0000","last_fullsized":"2024-09-13T16:30:04.685060+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:33:47.048582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.966213+0000","last_change":"2024-09-13T16:28:39.210516+0000","last_active":"2024-09-13T16:30:04.966213+0000","last_peered":"2024-09-13T16:30:04.966213+0000","last_clean":"2024-09-13T16:30:04.966213+0000","last_became_active":"2024-09-13T16:28:39.210214+0000","last_became_peered":"2024-09-13T16:28:39.210214+0000","last_unstale":"2024-09-13T16:30:04.966213+0000","last_undegraded":"2024-09-13T16:30:04.966213+0000","last_fullsized":"2024-09-13T16:30:04.966213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:51:58.212685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.684668+0000","last_change":"2024-09-13T16:28:43.230530+0000","last_active":"2024-09-13T16:30:04.684668+0000","last_peered":"2024-09-13T16:30:04.684668+0000","last_clean":"2024-09-13T16:30:04.684668+0000","last_became_active":"2024-09-13T16:28:43.230154+0000","last_became_peered":"2024-09-13T16:28:43.230154+0000","last_unstale":"2024-09-13T16:30:04.684668+0000","last_undegraded":"2024-09-13T16:30:04.684668+0000","last_fullsized":"2024-09-13T16:30:04.684668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:39:52.362177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.965635+0000","last_change":"2024-09-13T16:29:05.653460+0000","last_active":"2024-09-13T16:30:04.965635+0000","last_peered":"2024-09-13T16:30:04.965635+0000","last_clean":"2024-09-13T16:30:04.965635+0000","last_became_active":"2024-09-13T16:29:05.641836+0000","last_became_peered":"2024-09-13T16:29:05.641836+0000","last_unstale":"2024-09-13T16:30:04.965635+0000","last_undegraded":"2024-09-13T16:30:04.965635+0000","last_fullsized":"2024-09-13T16:30:04.965635+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:22:30.051033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.687612+0000","last_change":"2024-09-13T16:28:39.202381+0000","last_active":"2024-09-13T16:30:04.687612+0000","last_peered":"2024-09-13T16:30:04.687612+0000","last_clean":"2024-09-13T16:30:04.687612+0000","last_became_active":"2024-09-13T16:28:39.202023+0000","last_became_peered":"2024-09-13T16:28:39.202023+0000","last_unstale":"2024-09-13T16:30:04.687612+0000","last_undegraded":"2024-09-13T16:30:04.687612+0000","last_fullsized":"2024-09-13T16:30:04.687612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:55:46.445085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.684511+0000","last_change":"2024-09-13T16:28:43.230623+0000","last_active":"2024-09-13T16:30:04.684511+0000","last_peered":"2024-09-13T16:30:04.684511+0000","last_clean":"2024-09-13T16:30:04.684511+0000","last_became_active":"2024-09-13T16:28:43.230240+0000","last_became_peered":"2024-09-13T16:28:43.230240+0000","last_unstale":"2024-09-13T16:30:04.684511+0000","last_undegraded":"2024-09-13T16:30:04.684511+0000","last_fullsized":"2024-09-13T16:30:04.684511+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:04:07.723269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689169+0000","last_change":"2024-09-13T16:28:39.201896+0000","last_active":"2024-09-13T16:30:04.689169+0000","last_peered":"2024-09-13T16:30:04.689169+0000","last_clean":"2024-09-13T16:30:04.689169+0000","last_became_active":"2024-09-13T16:28:39.201570+0000","last_became_peered":"2024-09-13T16:28:39.201570+0000","last_unstale":"2024-09-13T16:30:04.689169+0000","last_undegraded":"2024-09-13T16:30:04.689169+0000","last_fullsized":"2024-09-13T16:30:04.689169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:07.532058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.684677+0000","last_change":"2024-09-13T16:28:38.845998+0000","last_active":"2024-09-13T16:30:04.684677+0000","last_peered":"2024-09-13T16:30:04.684677+0000","last_clean":"2024-09-13T16:30:04.684677+0000","last_became_active":"2024-09-13T16:28:38.845451+0000","last_became_peered":"2024-09-13T16:28:38.845451+0000","last_unstale":"2024-09-13T16:30:04.684677+0000","last_undegraded":"2024-09-13T16:30:04.684677+0000","last_fullsized":"2024-09-13T16:30:04.684677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:15.696276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.685320+0000","last_change":"2024-09-13T16:28:43.293598+0000","last_active":"2024-09-13T16:30:04.685320+0000","last_peered":"2024-09-13T16:30:04.685320+0000","last_clean":"2024-09-13T16:30:04.685320+0000","last_became_active":"2024-09-13T16:28:43.293045+0000","last_became_peered":"2024-09-13T16:28:43.293045+0000","last_unstale":"2024-09-13T16:30:04.685320+0000","last_undegraded":"2024-09-13T16:30:04.685320+0000","last_fullsized":"2024-09-13T16:30:04.685320+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:01.839626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689878+0000","last_change":"2024-09-13T16:28:39.214302+0000","last_active":"2024-09-13T16:30:04.689878+0000","last_peered":"2024-09-13T16:30:04.689878+0000","last_clean":"2024-09-13T16:30:04.689878+0000","last_became_active":"2024-09-13T16:28:39.213672+0000","last_became_peered":"2024-09-13T16:28:39.213672+0000","last_unstale":"2024-09-13T16:30:04.689878+0000","last_undegraded":"2024-09-13T16:30:04.689878+0000","last_fullsized":"2024-09-13T16:30:04.689878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:03:25.440603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"91'3","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.726254+0000","last_change":"2024-09-13T16:28:38.840626+0000","last_active":"2024-09-13T16:30:04.726254+0000","last_peered":"2024-09-13T16:30:04.726254+0000","last_clean":"2024-09-13T16:30:04.726254+0000","last_became_active":"2024-09-13T16:28:38.840344+0000","last_became_peered":"2024-09-13T16:28:38.840344+0000","last_unstale":"2024-09-13T16:30:04.726254+0000","last_undegraded":"2024-09-13T16:30:04.726254+0000","last_fullsized":"2024-09-13T16:30:04.726254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:03:10.850301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.684639+0000","last_change":"2024-09-13T16:28:43.229163+0000","last_active":"2024-09-13T16:30:04.684639+0000","last_peered":"2024-09-13T16:30:04.684639+0000","last_clean":"2024-09-13T16:30:04.684639+0000","last_became_active":"2024-09-13T16:28:43.228783+0000","last_became_peered":"2024-09-13T16:28:43.228783+0000","last_unstale":"2024-09-13T16:30:04.684639+0000","last_undegraded":"2024-09-13T16:30:04.684639+0000","last_fullsized":"2024-09-13T16:30:04.684639+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:48.183837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.965630+0000","last_change":"2024-09-13T16:28:39.209247+0000","last_active":"2024-09-13T16:30:04.965630+0000","last_peered":"2024-09-13T16:30:04.965630+0000","last_clean":"2024-09-13T16:30:04.965630+0000","last_became_active":"2024-09-13T16:28:39.208921+0000","last_became_peered":"2024-09-13T16:28:39.208921+0000","last_unstale":"2024-09-13T16:30:04.965630+0000","last_undegraded":"2024-09-13T16:30:04.965630+0000","last_fullsized":"2024-09-13T16:30:04.965630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:32:41.383029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689356+0000","last_change":"2024-09-13T16:28:38.845904+0000","last_active":"2024-09-13T16:30:04.689356+0000","last_peered":"2024-09-13T16:30:04.689356+0000","last_clean":"2024-09-13T16:30:04.689356+0000","last_became_active":"2024-09-13T16:28:38.845639+0000","last_became_peered":"2024-09-13T16:28:38.845639+0000","last_unstale":"2024-09-13T16:30:04.689356+0000","last_undegraded":"2024-09-13T16:30:04.689356+0000","last_fullsized":"2024-09-13T16:30:04.689356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:25:34.148607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.685063+0000","last_change":"2024-09-13T16:29:05.640063+0000","last_active":"2024-09-13T16:30:04.685063+0000","last_peered":"2024-09-13T16:30:04.685063+0000","last_clean":"2024-09-13T16:30:04.685063+0000","last_became_active":"2024-09-13T16:29:05.639596+0000","last_became_peered":"2024-09-13T16:29:05.639596+0000","last_unstale":"2024-09-13T16:30:04.685063+0000","last_undegraded":"2024-09-13T16:30:04.685063+0000","last_fullsized":"2024-09-13T16:30:04.685063+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:41:38.351162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.684450+0000","last_change":"2024-09-13T16:28:39.212947+0000","last_active":"2024-09-13T16:30:04.684450+0000","last_peered":"2024-09-13T16:30:04.684450+0000","last_clean":"2024-09-13T16:30:04.684450+0000","last_became_active":"2024-09-13T16:28:39.212622+0000","last_became_peered":"2024-09-13T16:28:39.212622+0000","last_unstale":"2024-09-13T16:30:04.684450+0000","last_undegraded":"2024-09-13T16:30:04.684450+0000","last_fullsized":"2024-09-13T16:30:04.684450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:16.425154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.965961+0000","last_change":"2024-09-13T16:28:38.846173+0000","last_active":"2024-09-13T16:30:04.965961+0000","last_peered":"2024-09-13T16:30:04.965961+0000","last_clean":"2024-09-13T16:30:04.965961+0000","last_became_active":"2024-09-13T16:28:38.845873+0000","last_became_peered":"2024-09-13T16:28:38.845873+0000","last_unstale":"2024-09-13T16:30:04.965961+0000","last_undegraded":"2024-09-13T16:30:04.965961+0000","last_fullsized":"2024-09-13T16:30:04.965961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:19:55.817524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T16:30:06.362655+0000","last_change":"2024-09-13T16:30:06.362655+0000","last_active":"2024-09-13T16:30:06.362655+0000","last_peered":"2024-09-13T16:30:06.362655+0000","last_clean":"2024-09-13T16:30:06.362655+0000","last_became_active":"2024-09-13T16:30:06.362123+0000","last_became_peered":"2024-09-13T16:30:06.362123+0000","last_unstale":"2024-09-13T16:30:06.362655+0000","last_undegraded":"2024-09-13T16:30:06.362655+0000","last_fullsized":"2024-09-13T16:30:06.362655+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:14:22.445244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.690105+0000","last_change":"2024-09-13T16:28:39.205817+0000","last_active":"2024-09-13T16:30:04.690105+0000","last_peered":"2024-09-13T16:30:04.690105+0000","last_clean":"2024-09-13T16:30:04.690105+0000","last_became_active":"2024-09-13T16:28:39.205534+0000","last_became_peered":"2024-09-13T16:28:39.205534+0000","last_unstale":"2024-09-13T16:30:04.690105+0000","last_undegraded":"2024-09-13T16:30:04.690105+0000","last_fullsized":"2024-09-13T16:30:04.690105+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:09:16.294842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.685510+0000","last_change":"2024-09-13T16:28:38.847582+0000","last_active":"2024-09-13T16:30:04.685510+0000","last_peered":"2024-09-13T16:30:04.685510+0000","last_clean":"2024-09-13T16:30:04.685510+0000","last_became_active":"2024-09-13T16:28:38.847353+0000","last_became_peered":"2024-09-13T16:28:38.847353+0000","last_unstale":"2024-09-13T16:30:04.685510+0000","last_undegraded":"2024-09-13T16:30:04.685510+0000","last_fullsized":"2024-09-13T16:30:04.685510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:24.944088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.726258+0000","last_change":"2024-09-13T16:28:43.225202+0000","last_active":"2024-09-13T16:30:04.726258+0000","last_peered":"2024-09-13T16:30:04.726258+0000","last_clean":"2024-09-13T16:30:04.726258+0000","last_became_active":"2024-09-13T16:28:43.224802+0000","last_became_peered":"2024-09-13T16:28:43.224802+0000","last_unstale":"2024-09-13T16:30:04.726258+0000","last_undegraded":"2024-09-13T16:30:04.726258+0000","last_fullsized":"2024-09-13T16:30:04.726258+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:51:19.325859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689238+0000","last_change":"2024-09-13T16:28:39.209630+0000","last_active":"2024-09-13T16:30:04.689238+0000","last_peered":"2024-09-13T16:30:04.689238+0000","last_clean":"2024-09-13T16:30:04.689238+0000","last_became_active":"2024-09-13T16:28:39.209303+0000","last_became_peered":"2024-09-13T16:28:39.209303+0000","last_unstale":"2024-09-13T16:30:04.689238+0000","last_undegraded":"2024-09-13T16:30:04.689238+0000","last_fullsized":"2024-09-13T16:30:04.689238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:01:31.822673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.684231+0000","last_change":"2024-09-13T16:28:38.837858+0000","last_active":"2024-09-13T16:30:04.684231+0000","last_peered":"2024-09-13T16:30:04.684231+0000","last_clean":"2024-09-13T16:30:04.684231+0000","last_became_active":"2024-09-13T16:28:38.837378+0000","last_became_peered":"2024-09-13T16:28:38.837378+0000","last_unstale":"2024-09-13T16:30:04.684231+0000","last_undegraded":"2024-09-13T16:30:04.684231+0000","last_fullsized":"2024-09-13T16:30:04.684231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:59:41.055902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.688716+0000","last_change":"2024-09-13T16:28:43.294241+0000","last_active":"2024-09-13T16:30:04.688716+0000","last_peered":"2024-09-13T16:30:04.688716+0000","last_clean":"2024-09-13T16:30:04.688716+0000","last_became_active":"2024-09-13T16:28:43.293873+0000","last_became_peered":"2024-09-13T16:28:43.293873+0000","last_unstale":"2024-09-13T16:30:04.688716+0000","last_undegraded":"2024-09-13T16:30:04.688716+0000","last_fullsized":"2024-09-13T16:30:04.688716+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:34.018162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.719362+0000","last_change":"2024-09-13T16:28:39.218071+0000","last_active":"2024-09-13T16:30:04.719362+0000","last_peered":"2024-09-13T16:30:04.719362+0000","last_clean":"2024-09-13T16:30:04.719362+0000","last_became_active":"2024-09-13T16:28:39.217798+0000","last_became_peered":"2024-09-13T16:28:39.217798+0000","last_unstale":"2024-09-13T16:30:04.719362+0000","last_undegraded":"2024-09-13T16:30:04.719362+0000","last_fullsized":"2024-09-13T16:30:04.719362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:39:09.769384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.685074+0000","last_change":"2024-09-13T16:28:38.847238+0000","last_active":"2024-09-13T16:30:04.685074+0000","last_peered":"2024-09-13T16:30:04.685074+0000","last_clean":"2024-09-13T16:30:04.685074+0000","last_became_active":"2024-09-13T16:28:38.846741+0000","last_became_peered":"2024-09-13T16:28:38.846741+0000","last_unstale":"2024-09-13T16:30:04.685074+0000","last_undegraded":"2024-09-13T16:30:04.685074+0000","last_fullsized":"2024-09-13T16:30:04.685074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:52:51.946158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.965790+0000","last_change":"2024-09-13T16:28:43.228431+0000","last_active":"2024-09-13T16:30:04.965790+0000","last_peered":"2024-09-13T16:30:04.965790+0000","last_clean":"2024-09-13T16:30:04.965790+0000","last_became_active":"2024-09-13T16:28:43.228030+0000","last_became_peered":"2024-09-13T16:28:43.228030+0000","last_unstale":"2024-09-13T16:30:04.965790+0000","last_undegraded":"2024-09-13T16:30:04.965790+0000","last_fullsized":"2024-09-13T16:30:04.965790+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:14:50.573033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.688142+0000","last_change":"2024-09-13T16:28:39.209485+0000","last_active":"2024-09-13T16:30:04.688142+0000","last_peered":"2024-09-13T16:30:04.688142+0000","last_clean":"2024-09-13T16:30:04.688142+0000","last_became_active":"2024-09-13T16:28:39.209157+0000","last_became_peered":"2024-09-13T16:28:39.209157+0000","last_unstale":"2024-09-13T16:30:04.688142+0000","last_undegraded":"2024-09-13T16:30:04.688142+0000","last_fullsized":"2024-09-13T16:30:04.688142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:12:19.398255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.683935+0000","last_change":"2024-09-13T16:28:38.842551+0000","last_active":"2024-09-13T16:30:04.683935+0000","last_peered":"2024-09-13T16:30:04.683935+0000","last_clean":"2024-09-13T16:30:04.683935+0000","last_became_active":"2024-09-13T16:28:38.841741+0000","last_became_peered":"2024-09-13T16:28:38.841741+0000","last_unstale":"2024-09-13T16:30:04.683935+0000","last_undegraded":"2024-09-13T16:30:04.683935+0000","last_fullsized":"2024-09-13T16:30:04.683935+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:35:25.131693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.684450+0000","last_change":"2024-09-13T16:28:43.294300+0000","last_active":"2024-09-13T16:30:04.684450+0000","last_peered":"2024-09-13T16:30:04.684450+0000","last_clean":"2024-09-13T16:30:04.684450+0000","last_became_active":"2024-09-13T16:28:43.293918+0000","last_became_peered":"2024-09-13T16:28:43.293918+0000","last_unstale":"2024-09-13T16:30:04.684450+0000","last_undegraded":"2024-09-13T16:30:04.684450+0000","last_fullsized":"2024-09-13T16:30:04.684450+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:19:21.959617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.687836+0000","last_change":"2024-09-13T16:28:39.212001+0000","last_active":"2024-09-13T16:30:04.687836+0000","last_peered":"2024-09-13T16:30:04.687836+0000","last_clean":"2024-09-13T16:30:04.687836+0000","last_became_active":"2024-09-13T16:28:39.211716+0000","last_became_peered":"2024-09-13T16:28:39.211716+0000","last_unstale":"2024-09-13T16:30:04.687836+0000","last_undegraded":"2024-09-13T16:30:04.687836+0000","last_fullsized":"2024-09-13T16:30:04.687836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:56.753677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'6","reported_seq":103,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T16:30:21.742594+0000","last_change":"2024-09-13T16:28:38.851229+0000","last_active":"2024-09-13T16:30:21.742594+0000","last_peered":"2024-09-13T16:30:21.742594+0000","last_clean":"2024-09-13T16:30:21.742594+0000","last_became_active":"2024-09-13T16:28:38.850937+0000","last_became_peered":"2024-09-13T16:28:38.850937+0000","last_unstale":"2024-09-13T16:30:21.742594+0000","last_undegraded":"2024-09-13T16:30:21.742594+0000","last_fullsized":"2024-09-13T16:30:21.742594+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:19.460864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":78,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.719658+0000","last_change":"2024-09-13T16:28:43.224825+0000","last_active":"2024-09-13T16:30:04.719658+0000","last_peered":"2024-09-13T16:30:04.719658+0000","last_clean":"2024-09-13T16:30:04.719658+0000","last_became_active":"2024-09-13T16:28:43.223903+0000","last_became_peered":"2024-09-13T16:28:43.223903+0000","last_unstale":"2024-09-13T16:30:04.719658+0000","last_undegraded":"2024-09-13T16:30:04.719658+0000","last_fullsized":"2024-09-13T16:30:04.719658+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:08:46.850668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.683242+0000","last_change":"2024-09-13T16:29:05.636787+0000","last_active":"2024-09-13T16:30:04.683242+0000","last_peered":"2024-09-13T16:30:04.683242+0000","last_clean":"2024-09-13T16:30:04.683242+0000","last_became_active":"2024-09-13T16:29:05.636399+0000","last_became_peered":"2024-09-13T16:29:05.636399+0000","last_unstale":"2024-09-13T16:30:04.683242+0000","last_undegraded":"2024-09-13T16:30:04.683242+0000","last_fullsized":"2024-09-13T16:30:04.683242+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:38:25.545325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":103,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T16:30:18.738009+0000","last_change":"2024-09-13T16:28:38.845374+0000","last_active":"2024-09-13T16:30:18.738009+0000","last_peered":"2024-09-13T16:30:18.738009+0000","last_clean":"2024-09-13T16:30:18.738009+0000","last_became_active":"2024-09-13T16:28:38.845076+0000","last_became_peered":"2024-09-13T16:28:38.845076+0000","last_unstale":"2024-09-13T16:30:18.738009+0000","last_undegraded":"2024-09-13T16:30:18.738009+0000","last_fullsized":"2024-09-13T16:30:18.738009+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:37:56.225682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":78,"num_read_kb":78,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.719371+0000","last_change":"2024-09-13T16:28:43.293529+0000","last_active":"2024-09-13T16:30:04.719371+0000","last_peered":"2024-09-13T16:30:04.719371+0000","last_clean":"2024-09-13T16:30:04.719371+0000","last_became_active":"2024-09-13T16:28:43.293083+0000","last_became_peered":"2024-09-13T16:28:43.293083+0000","last_unstale":"2024-09-13T16:30:04.719371+0000","last_undegraded":"2024-09-13T16:30:04.719371+0000","last_fullsized":"2024-09-13T16:30:04.719371+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:05:55.572984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.718742+0000","last_change":"2024-09-13T16:28:39.217546+0000","last_active":"2024-09-13T16:30:04.718742+0000","last_peered":"2024-09-13T16:30:04.718742+0000","last_clean":"2024-09-13T16:30:04.718742+0000","last_became_active":"2024-09-13T16:28:39.217242+0000","last_became_peered":"2024-09-13T16:28:39.217242+0000","last_unstale":"2024-09-13T16:30:04.718742+0000","last_undegraded":"2024-09-13T16:30:04.718742+0000","last_fullsized":"2024-09-13T16:30:04.718742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:23:18.183616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.688989+0000","last_change":"2024-09-13T16:28:38.846436+0000","last_active":"2024-09-13T16:30:04.688989+0000","last_peered":"2024-09-13T16:30:04.688989+0000","last_clean":"2024-09-13T16:30:04.688989+0000","last_became_active":"2024-09-13T16:28:38.845953+0000","last_became_peered":"2024-09-13T16:28:38.845953+0000","last_unstale":"2024-09-13T16:30:04.688989+0000","last_undegraded":"2024-09-13T16:30:04.688989+0000","last_fullsized":"2024-09-13T16:30:04.688989+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:33.998462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.687895+0000","last_change":"2024-09-13T16:28:43.294504+0000","last_active":"2024-09-13T16:30:04.687895+0000","last_peered":"2024-09-13T16:30:04.687895+0000","last_clean":"2024-09-13T16:30:04.687895+0000","last_became_active":"2024-09-13T16:28:43.294063+0000","last_became_peered":"2024-09-13T16:28:43.294063+0000","last_unstale":"2024-09-13T16:30:04.687895+0000","last_undegraded":"2024-09-13T16:30:04.687895+0000","last_fullsized":"2024-09-13T16:30:04.687895+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:51:35.774467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689668+0000","last_change":"2024-09-13T16:28:39.206491+0000","last_active":"2024-09-13T16:30:04.689668+0000","last_peered":"2024-09-13T16:30:04.689668+0000","last_clean":"2024-09-13T16:30:04.689668+0000","last_became_active":"2024-09-13T16:28:39.206187+0000","last_became_peered":"2024-09-13T16:28:39.206187+0000","last_unstale":"2024-09-13T16:30:04.689668+0000","last_undegraded":"2024-09-13T16:30:04.689668+0000","last_fullsized":"2024-09-13T16:30:04.689668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:20:28.457994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.684882+0000","last_change":"2024-09-13T16:28:38.836270+0000","last_active":"2024-09-13T16:30:04.684882+0000","last_peered":"2024-09-13T16:30:04.684882+0000","last_clean":"2024-09-13T16:30:04.684882+0000","last_became_active":"2024-09-13T16:28:38.835966+0000","last_became_peered":"2024-09-13T16:28:38.835966+0000","last_unstale":"2024-09-13T16:30:04.684882+0000","last_undegraded":"2024-09-13T16:30:04.684882+0000","last_fullsized":"2024-09-13T16:30:04.684882+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:03:11.388654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":334,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T16:30:05.722935+0000","last_change":"2024-09-13T16:27:34.264484+0000","last_active":"2024-09-13T16:30:05.722935+0000","last_peered":"2024-09-13T16:30:05.722935+0000","last_clean":"2024-09-13T16:30:05.722935+0000","last_became_active":"2024-09-13T16:27:34.263995+0000","last_became_peered":"2024-09-13T16:27:34.263995+0000","last_unstale":"2024-09-13T16:30:05.722935+0000","last_undegraded":"2024-09-13T16:30:05.722935+0000","last_fullsized":"2024-09-13T16:30:05.722935+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:24:23.015578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:24:23.015578+0000","last_clean_scrub_stamp":"2024-09-13T16:24:23.015578+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:25:04.376312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689366+0000","last_change":"2024-09-13T16:28:43.229347+0000","last_active":"2024-09-13T16:30:04.689366+0000","last_peered":"2024-09-13T16:30:04.689366+0000","last_clean":"2024-09-13T16:30:04.689366+0000","last_became_active":"2024-09-13T16:28:43.228981+0000","last_became_peered":"2024-09-13T16:28:43.228981+0000","last_unstale":"2024-09-13T16:30:04.689366+0000","last_undegraded":"2024-09-13T16:30:04.689366+0000","last_fullsized":"2024-09-13T16:30:04.689366+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:46:15.151295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689569+0000","last_change":"2024-09-13T16:28:39.203195+0000","last_active":"2024-09-13T16:30:04.689569+0000","last_peered":"2024-09-13T16:30:04.689569+0000","last_clean":"2024-09-13T16:30:04.689569+0000","last_became_active":"2024-09-13T16:28:39.202878+0000","last_became_peered":"2024-09-13T16:28:39.202878+0000","last_unstale":"2024-09-13T16:30:04.689569+0000","last_undegraded":"2024-09-13T16:30:04.689569+0000","last_fullsized":"2024-09-13T16:30:04.689569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:17.615531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":103,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T16:30:20.755087+0000","last_change":"2024-09-13T16:28:38.844856+0000","last_active":"2024-09-13T16:30:20.755087+0000","last_peered":"2024-09-13T16:30:20.755087+0000","last_clean":"2024-09-13T16:30:20.755087+0000","last_became_active":"2024-09-13T16:28:38.844578+0000","last_became_peered":"2024-09-13T16:28:38.844578+0000","last_unstale":"2024-09-13T16:30:20.755087+0000","last_undegraded":"2024-09-13T16:30:20.755087+0000","last_fullsized":"2024-09-13T16:30:20.755087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:41:48.253174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":78,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.965911+0000","last_change":"2024-09-13T16:28:43.228472+0000","last_active":"2024-09-13T16:30:04.965911+0000","last_peered":"2024-09-13T16:30:04.965911+0000","last_clean":"2024-09-13T16:30:04.965911+0000","last_became_active":"2024-09-13T16:28:43.228056+0000","last_became_peered":"2024-09-13T16:28:43.228056+0000","last_unstale":"2024-09-13T16:30:04.965911+0000","last_undegraded":"2024-09-13T16:30:04.965911+0000","last_fullsized":"2024-09-13T16:30:04.965911+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:07:55.980691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689417+0000","last_change":"2024-09-13T16:28:39.203278+0000","last_active":"2024-09-13T16:30:04.689417+0000","last_peered":"2024-09-13T16:30:04.689417+0000","last_clean":"2024-09-13T16:30:04.689417+0000","last_became_active":"2024-09-13T16:28:39.202936+0000","last_became_peered":"2024-09-13T16:28:39.202936+0000","last_unstale":"2024-09-13T16:30:04.689417+0000","last_undegraded":"2024-09-13T16:30:04.689417+0000","last_fullsized":"2024-09-13T16:30:04.689417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:44.210612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.687892+0000","last_change":"2024-09-13T16:28:38.851085+0000","last_active":"2024-09-13T16:30:04.687892+0000","last_peered":"2024-09-13T16:30:04.687892+0000","last_clean":"2024-09-13T16:30:04.687892+0000","last_became_active":"2024-09-13T16:28:38.850785+0000","last_became_peered":"2024-09-13T16:28:38.850785+0000","last_unstale":"2024-09-13T16:30:04.687892+0000","last_undegraded":"2024-09-13T16:30:04.687892+0000","last_fullsized":"2024-09-13T16:30:04.687892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:52.713431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689588+0000","last_change":"2024-09-13T16:28:43.229980+0000","last_active":"2024-09-13T16:30:04.689588+0000","last_peered":"2024-09-13T16:30:04.689588+0000","last_clean":"2024-09-13T16:30:04.689588+0000","last_became_active":"2024-09-13T16:28:43.229647+0000","last_became_peered":"2024-09-13T16:28:43.229647+0000","last_unstale":"2024-09-13T16:30:04.689588+0000","last_undegraded":"2024-09-13T16:30:04.689588+0000","last_fullsized":"2024-09-13T16:30:04.689588+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:39:42.871613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689552+0000","last_change":"2024-09-13T16:28:39.204167+0000","last_active":"2024-09-13T16:30:04.689552+0000","last_peered":"2024-09-13T16:30:04.689552+0000","last_clean":"2024-09-13T16:30:04.689552+0000","last_became_active":"2024-09-13T16:28:39.203776+0000","last_became_peered":"2024-09-13T16:28:39.203776+0000","last_unstale":"2024-09-13T16:30:04.689552+0000","last_undegraded":"2024-09-13T16:30:04.689552+0000","last_fullsized":"2024-09-13T16:30:04.689552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:53:36.410365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.719572+0000","last_change":"2024-09-13T16:28:38.846331+0000","last_active":"2024-09-13T16:30:04.719572+0000","last_peered":"2024-09-13T16:30:04.719572+0000","last_clean":"2024-09-13T16:30:04.719572+0000","last_became_active":"2024-09-13T16:28:38.846012+0000","last_became_peered":"2024-09-13T16:28:38.846012+0000","last_unstale":"2024-09-13T16:30:04.719572+0000","last_undegraded":"2024-09-13T16:30:04.719572+0000","last_fullsized":"2024-09-13T16:30:04.719572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:23:54.282736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.687072+0000","last_change":"2024-09-13T16:28:43.227886+0000","last_active":"2024-09-13T16:30:04.687072+0000","last_peered":"2024-09-13T16:30:04.687072+0000","last_clean":"2024-09-13T16:30:04.687072+0000","last_became_active":"2024-09-13T16:28:43.227346+0000","last_became_peered":"2024-09-13T16:28:43.227346+0000","last_unstale":"2024-09-13T16:30:04.687072+0000","last_undegraded":"2024-09-13T16:30:04.687072+0000","last_fullsized":"2024-09-13T16:30:04.687072+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:41:36.559064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.965886+0000","last_change":"2024-09-13T16:28:39.205975+0000","last_active":"2024-09-13T16:30:04.965886+0000","last_peered":"2024-09-13T16:30:04.965886+0000","last_clean":"2024-09-13T16:30:04.965886+0000","last_became_active":"2024-09-13T16:28:39.205657+0000","last_became_peered":"2024-09-13T16:28:39.205657+0000","last_unstale":"2024-09-13T16:30:04.965886+0000","last_undegraded":"2024-09-13T16:30:04.965886+0000","last_fullsized":"2024-09-13T16:30:04.965886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:31:47.309482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.685059+0000","last_change":"2024-09-13T16:28:38.841589+0000","last_active":"2024-09-13T16:30:04.685059+0000","last_peered":"2024-09-13T16:30:04.685059+0000","last_clean":"2024-09-13T16:30:04.685059+0000","last_became_active":"2024-09-13T16:28:38.841298+0000","last_became_peered":"2024-09-13T16:28:38.841298+0000","last_unstale":"2024-09-13T16:30:04.685059+0000","last_undegraded":"2024-09-13T16:30:04.685059+0000","last_fullsized":"2024-09-13T16:30:04.685059+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:27:38.358320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.687185+0000","last_change":"2024-09-13T16:28:43.233790+0000","last_active":"2024-09-13T16:30:04.687185+0000","last_peered":"2024-09-13T16:30:04.687185+0000","last_clean":"2024-09-13T16:30:04.687185+0000","last_became_active":"2024-09-13T16:28:43.231963+0000","last_became_peered":"2024-09-13T16:28:43.231963+0000","last_unstale":"2024-09-13T16:30:04.687185+0000","last_undegraded":"2024-09-13T16:30:04.687185+0000","last_fullsized":"2024-09-13T16:30:04.687185+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:45:23.406395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.687352+0000","last_change":"2024-09-13T16:28:39.201476+0000","last_active":"2024-09-13T16:30:04.687352+0000","last_peered":"2024-09-13T16:30:04.687352+0000","last_clean":"2024-09-13T16:30:04.687352+0000","last_became_active":"2024-09-13T16:28:39.201168+0000","last_became_peered":"2024-09-13T16:28:39.201168+0000","last_unstale":"2024-09-13T16:30:04.687352+0000","last_undegraded":"2024-09-13T16:30:04.687352+0000","last_fullsized":"2024-09-13T16:30:04.687352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:26:43.570295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":64,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.684447+0000","last_change":"2024-09-13T16:29:05.652225+0000","last_active":"2024-09-13T16:30:04.684447+0000","last_peered":"2024-09-13T16:30:04.684447+0000","last_clean":"2024-09-13T16:30:04.684447+0000","last_became_active":"2024-09-13T16:29:05.639325+0000","last_became_peered":"2024-09-13T16:29:05.639325+0000","last_unstale":"2024-09-13T16:30:04.684447+0000","last_undegraded":"2024-09-13T16:30:04.684447+0000","last_fullsized":"2024-09-13T16:30:04.684447+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:18:30.083447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.719391+0000","last_change":"2024-09-13T16:28:43.228563+0000","last_active":"2024-09-13T16:30:04.719391+0000","last_peered":"2024-09-13T16:30:04.719391+0000","last_clean":"2024-09-13T16:30:04.719391+0000","last_became_active":"2024-09-13T16:28:43.228184+0000","last_became_peered":"2024-09-13T16:28:43.228184+0000","last_unstale":"2024-09-13T16:30:04.719391+0000","last_undegraded":"2024-09-13T16:30:04.719391+0000","last_fullsized":"2024-09-13T16:30:04.719391+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:05:33.368302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.688901+0000","last_change":"2024-09-13T16:28:39.210750+0000","last_active":"2024-09-13T16:30:04.688901+0000","last_peered":"2024-09-13T16:30:04.688901+0000","last_clean":"2024-09-13T16:30:04.688901+0000","last_became_active":"2024-09-13T16:28:39.210401+0000","last_became_peered":"2024-09-13T16:28:39.210401+0000","last_unstale":"2024-09-13T16:30:04.688901+0000","last_undegraded":"2024-09-13T16:30:04.688901+0000","last_fullsized":"2024-09-13T16:30:04.688901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:36.670420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":109,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T16:30:21.742679+0000","last_change":"2024-09-13T16:28:38.843573+0000","last_active":"2024-09-13T16:30:21.742679+0000","last_peered":"2024-09-13T16:30:21.742679+0000","last_clean":"2024-09-13T16:30:21.742679+0000","last_became_active":"2024-09-13T16:28:38.843245+0000","last_became_peered":"2024-09-13T16:28:38.843245+0000","last_unstale":"2024-09-13T16:30:21.742679+0000","last_undegraded":"2024-09-13T16:30:21.742679+0000","last_fullsized":"2024-09-13T16:30:21.742679+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:45.983123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T16:30:05.835858+0000","last_change":"2024-09-13T16:30:05.835462+0000","last_active":"2024-09-13T16:30:05.835858+0000","last_peered":"2024-09-13T16:30:05.835858+0000","last_clean":"2024-09-13T16:30:05.835858+0000","last_became_active":"2024-09-13T16:30:05.835050+0000","last_became_peered":"2024-09-13T16:30:05.835050+0000","last_unstale":"2024-09-13T16:30:05.835858+0000","last_undegraded":"2024-09-13T16:30:05.835858+0000","last_fullsized":"2024-09-13T16:30:05.835858+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:30:43.427684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7,0,3],"acting":[10,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689446+0000","last_change":"2024-09-13T16:28:39.201471+0000","last_active":"2024-09-13T16:30:04.689446+0000","last_peered":"2024-09-13T16:30:04.689446+0000","last_clean":"2024-09-13T16:30:04.689446+0000","last_became_active":"2024-09-13T16:28:39.201140+0000","last_became_peered":"2024-09-13T16:28:39.201140+0000","last_unstale":"2024-09-13T16:30:04.689446+0000","last_undegraded":"2024-09-13T16:30:04.689446+0000","last_fullsized":"2024-09-13T16:30:04.689446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:37:25.037490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.687912+0000","last_change":"2024-09-13T16:28:38.845703+0000","last_active":"2024-09-13T16:30:04.687912+0000","last_peered":"2024-09-13T16:30:04.687912+0000","last_clean":"2024-09-13T16:30:04.687912+0000","last_became_active":"2024-09-13T16:28:38.845354+0000","last_became_peered":"2024-09-13T16:28:38.845354+0000","last_unstale":"2024-09-13T16:30:04.687912+0000","last_undegraded":"2024-09-13T16:30:04.687912+0000","last_fullsized":"2024-09-13T16:30:04.687912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:18.208723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.965726+0000","last_change":"2024-09-13T16:28:43.223924+0000","last_active":"2024-09-13T16:30:04.965726+0000","last_peered":"2024-09-13T16:30:04.965726+0000","last_clean":"2024-09-13T16:30:04.965726+0000","last_became_active":"2024-09-13T16:28:43.223535+0000","last_became_peered":"2024-09-13T16:28:43.223535+0000","last_unstale":"2024-09-13T16:30:04.965726+0000","last_undegraded":"2024-09-13T16:30:04.965726+0000","last_fullsized":"2024-09-13T16:30:04.965726+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:10.622436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.719026+0000","last_change":"2024-09-13T16:28:39.203943+0000","last_active":"2024-09-13T16:30:04.719026+0000","last_peered":"2024-09-13T16:30:04.719026+0000","last_clean":"2024-09-13T16:30:04.719026+0000","last_became_active":"2024-09-13T16:28:39.203657+0000","last_became_peered":"2024-09-13T16:28:39.203657+0000","last_unstale":"2024-09-13T16:30:04.719026+0000","last_undegraded":"2024-09-13T16:30:04.719026+0000","last_fullsized":"2024-09-13T16:30:04.719026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:52:48.523939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.965628+0000","last_change":"2024-09-13T16:29:05.651015+0000","last_active":"2024-09-13T16:30:04.965628+0000","last_peered":"2024-09-13T16:30:04.965628+0000","last_clean":"2024-09-13T16:30:04.965628+0000","last_became_active":"2024-09-13T16:29:05.640006+0000","last_became_peered":"2024-09-13T16:29:05.640006+0000","last_unstale":"2024-09-13T16:30:04.965628+0000","last_undegraded":"2024-09-13T16:30:04.965628+0000","last_fullsized":"2024-09-13T16:30:04.965628+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:12:25.508262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.687619+0000","last_change":"2024-09-13T16:28:43.232634+0000","last_active":"2024-09-13T16:30:04.687619+0000","last_peered":"2024-09-13T16:30:04.687619+0000","last_clean":"2024-09-13T16:30:04.687619+0000","last_became_active":"2024-09-13T16:28:43.232107+0000","last_became_peered":"2024-09-13T16:28:43.232107+0000","last_unstale":"2024-09-13T16:30:04.687619+0000","last_undegraded":"2024-09-13T16:30:04.687619+0000","last_fullsized":"2024-09-13T16:30:04.687619+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:45.952933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.684953+0000","last_change":"2024-09-13T16:28:39.212883+0000","last_active":"2024-09-13T16:30:04.684953+0000","last_peered":"2024-09-13T16:30:04.684953+0000","last_clean":"2024-09-13T16:30:04.684953+0000","last_became_active":"2024-09-13T16:28:39.212591+0000","last_became_peered":"2024-09-13T16:28:39.212591+0000","last_unstale":"2024-09-13T16:30:04.684953+0000","last_undegraded":"2024-09-13T16:30:04.684953+0000","last_fullsized":"2024-09-13T16:30:04.684953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:17:30.926940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'34","reported_seq":222,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T16:30:21.748536+0000","last_change":"2024-09-13T16:28:38.846564+0000","last_active":"2024-09-13T16:30:21.748536+0000","last_peered":"2024-09-13T16:30:21.748536+0000","last_clean":"2024-09-13T16:30:21.748536+0000","last_became_active":"2024-09-13T16:28:38.846165+0000","last_became_peered":"2024-09-13T16:28:38.846165+0000","last_unstale":"2024-09-13T16:30:21.748536+0000","last_undegraded":"2024-09-13T16:30:21.748536+0000","last_fullsized":"2024-09-13T16:30:21.748536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:33:06.124592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63310,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":168,"num_read_kb":146,"num_write":36,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.683680+0000","last_change":"2024-09-13T16:28:43.231464+0000","last_active":"2024-09-13T16:30:04.683680+0000","last_peered":"2024-09-13T16:30:04.683680+0000","last_clean":"2024-09-13T16:30:04.683680+0000","last_became_active":"2024-09-13T16:28:43.231040+0000","last_became_peered":"2024-09-13T16:28:43.231040+0000","last_unstale":"2024-09-13T16:30:04.683680+0000","last_undegraded":"2024-09-13T16:30:04.683680+0000","last_fullsized":"2024-09-13T16:30:04.683680+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:11:21.847375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.966262+0000","last_change":"2024-09-13T16:29:05.639415+0000","last_active":"2024-09-13T16:30:04.966262+0000","last_peered":"2024-09-13T16:30:04.966262+0000","last_clean":"2024-09-13T16:30:04.966262+0000","last_became_active":"2024-09-13T16:29:05.639101+0000","last_became_peered":"2024-09-13T16:29:05.639101+0000","last_unstale":"2024-09-13T16:30:04.966262+0000","last_undegraded":"2024-09-13T16:30:04.966262+0000","last_fullsized":"2024-09-13T16:30:04.966262+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:32:42.630013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.687774+0000","last_change":"2024-09-13T16:28:38.841637+0000","last_active":"2024-09-13T16:30:04.687774+0000","last_peered":"2024-09-13T16:30:04.687774+0000","last_clean":"2024-09-13T16:30:04.687774+0000","last_became_active":"2024-09-13T16:28:38.841359+0000","last_became_peered":"2024-09-13T16:28:38.841359+0000","last_unstale":"2024-09-13T16:30:04.687774+0000","last_undegraded":"2024-09-13T16:30:04.687774+0000","last_fullsized":"2024-09-13T16:30:04.687774+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:47:05.284987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.684791+0000","last_change":"2024-09-13T16:29:05.638870+0000","last_active":"2024-09-13T16:30:04.684791+0000","last_peered":"2024-09-13T16:30:04.684791+0000","last_clean":"2024-09-13T16:30:04.684791+0000","last_became_active":"2024-09-13T16:29:05.638422+0000","last_became_peered":"2024-09-13T16:29:05.638422+0000","last_unstale":"2024-09-13T16:30:04.684791+0000","last_undegraded":"2024-09-13T16:30:04.684791+0000","last_fullsized":"2024-09-13T16:30:04.684791+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:41.263661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.690269+0000","last_change":"2024-09-13T16:28:39.203595+0000","last_active":"2024-09-13T16:30:04.690269+0000","last_peered":"2024-09-13T16:30:04.690269+0000","last_clean":"2024-09-13T16:30:04.690269+0000","last_became_active":"2024-09-13T16:28:39.203283+0000","last_became_peered":"2024-09-13T16:28:39.203283+0000","last_unstale":"2024-09-13T16:30:04.690269+0000","last_undegraded":"2024-09-13T16:30:04.690269+0000","last_fullsized":"2024-09-13T16:30:04.690269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:58:45.970761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689460+0000","last_change":"2024-09-13T16:28:38.850203+0000","last_active":"2024-09-13T16:30:04.689460+0000","last_peered":"2024-09-13T16:30:04.689460+0000","last_clean":"2024-09-13T16:30:04.689460+0000","last_became_active":"2024-09-13T16:28:38.849963+0000","last_became_peered":"2024-09-13T16:28:38.849963+0000","last_unstale":"2024-09-13T16:30:04.689460+0000","last_undegraded":"2024-09-13T16:30:04.689460+0000","last_fullsized":"2024-09-13T16:30:04.689460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:15.849627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.683250+0000","last_change":"2024-09-13T16:28:43.227679+0000","last_active":"2024-09-13T16:30:04.683250+0000","last_peered":"2024-09-13T16:30:04.683250+0000","last_clean":"2024-09-13T16:30:04.683250+0000","last_became_active":"2024-09-13T16:28:43.227269+0000","last_became_peered":"2024-09-13T16:28:43.227269+0000","last_unstale":"2024-09-13T16:30:04.683250+0000","last_undegraded":"2024-09-13T16:30:04.683250+0000","last_fullsized":"2024-09-13T16:30:04.683250+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:05:00.186305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.966381+0000","last_change":"2024-09-13T16:28:39.211401+0000","last_active":"2024-09-13T16:30:04.966381+0000","last_peered":"2024-09-13T16:30:04.966381+0000","last_clean":"2024-09-13T16:30:04.966381+0000","last_became_active":"2024-09-13T16:28:39.211133+0000","last_became_peered":"2024-09-13T16:28:39.211133+0000","last_unstale":"2024-09-13T16:30:04.966381+0000","last_undegraded":"2024-09-13T16:30:04.966381+0000","last_fullsized":"2024-09-13T16:30:04.966381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:36.468666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":179,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T16:30:21.748567+0000","last_change":"2024-09-13T16:28:38.845132+0000","last_active":"2024-09-13T16:30:21.748567+0000","last_peered":"2024-09-13T16:30:21.748567+0000","last_clean":"2024-09-13T16:30:21.748567+0000","last_became_active":"2024-09-13T16:28:38.844804+0000","last_became_peered":"2024-09-13T16:28:38.844804+0000","last_unstale":"2024-09-13T16:30:21.748567+0000","last_undegraded":"2024-09-13T16:30:21.748567+0000","last_fullsized":"2024-09-13T16:30:21.748567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:15:08.643227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":155,"num_read_kb":155,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689066+0000","last_change":"2024-09-13T16:28:43.230829+0000","last_active":"2024-09-13T16:30:04.689066+0000","last_peered":"2024-09-13T16:30:04.689066+0000","last_clean":"2024-09-13T16:30:04.689066+0000","last_became_active":"2024-09-13T16:28:43.230405+0000","last_became_peered":"2024-09-13T16:28:43.230405+0000","last_unstale":"2024-09-13T16:30:04.689066+0000","last_undegraded":"2024-09-13T16:30:04.689066+0000","last_fullsized":"2024-09-13T16:30:04.689066+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:04:56.083193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689956+0000","last_change":"2024-09-13T16:28:43.226662+0000","last_active":"2024-09-13T16:30:04.689956+0000","last_peered":"2024-09-13T16:30:04.689956+0000","last_clean":"2024-09-13T16:30:04.689956+0000","last_became_active":"2024-09-13T16:28:43.226232+0000","last_became_peered":"2024-09-13T16:28:43.226232+0000","last_unstale":"2024-09-13T16:30:04.689956+0000","last_undegraded":"2024-09-13T16:30:04.689956+0000","last_fullsized":"2024-09-13T16:30:04.689956+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:03.568500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.683392+0000","last_change":"2024-09-13T16:28:39.208909+0000","last_active":"2024-09-13T16:30:04.683392+0000","last_peered":"2024-09-13T16:30:04.683392+0000","last_clean":"2024-09-13T16:30:04.683392+0000","last_became_active":"2024-09-13T16:28:39.208572+0000","last_became_peered":"2024-09-13T16:28:39.208572+0000","last_unstale":"2024-09-13T16:30:04.683392+0000","last_undegraded":"2024-09-13T16:30:04.683392+0000","last_fullsized":"2024-09-13T16:30:04.683392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:01:40.173308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.687385+0000","last_change":"2024-09-13T16:28:38.844259+0000","last_active":"2024-09-13T16:30:04.687385+0000","last_peered":"2024-09-13T16:30:04.687385+0000","last_clean":"2024-09-13T16:30:04.687385+0000","last_became_active":"2024-09-13T16:28:38.843956+0000","last_became_peered":"2024-09-13T16:28:38.843956+0000","last_unstale":"2024-09-13T16:30:04.687385+0000","last_undegraded":"2024-09-13T16:30:04.687385+0000","last_fullsized":"2024-09-13T16:30:04.687385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:05.652852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.965953+0000","last_change":"2024-09-13T16:28:43.229385+0000","last_active":"2024-09-13T16:30:04.965953+0000","last_peered":"2024-09-13T16:30:04.965953+0000","last_clean":"2024-09-13T16:30:04.965953+0000","last_became_active":"2024-09-13T16:28:43.229033+0000","last_became_peered":"2024-09-13T16:28:43.229033+0000","last_unstale":"2024-09-13T16:30:04.965953+0000","last_undegraded":"2024-09-13T16:30:04.965953+0000","last_fullsized":"2024-09-13T16:30:04.965953+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:46:41.611758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.690367+0000","last_change":"2024-09-13T16:28:39.208100+0000","last_active":"2024-09-13T16:30:04.690367+0000","last_peered":"2024-09-13T16:30:04.690367+0000","last_clean":"2024-09-13T16:30:04.690367+0000","last_became_active":"2024-09-13T16:28:39.207808+0000","last_became_peered":"2024-09-13T16:28:39.207808+0000","last_unstale":"2024-09-13T16:30:04.690367+0000","last_undegraded":"2024-09-13T16:30:04.690367+0000","last_fullsized":"2024-09-13T16:30:04.690367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:55.446568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.718842+0000","last_change":"2024-09-13T16:28:38.844689+0000","last_active":"2024-09-13T16:30:04.718842+0000","last_peered":"2024-09-13T16:30:04.718842+0000","last_clean":"2024-09-13T16:30:04.718842+0000","last_became_active":"2024-09-13T16:28:38.844326+0000","last_became_peered":"2024-09-13T16:28:38.844326+0000","last_unstale":"2024-09-13T16:30:04.718842+0000","last_undegraded":"2024-09-13T16:30:04.718842+0000","last_fullsized":"2024-09-13T16:30:04.718842+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:02.616518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.687781+0000","last_change":"2024-09-13T16:28:38.849233+0000","last_active":"2024-09-13T16:30:04.687781+0000","last_peered":"2024-09-13T16:30:04.687781+0000","last_clean":"2024-09-13T16:30:04.687781+0000","last_became_active":"2024-09-13T16:28:38.848911+0000","last_became_peered":"2024-09-13T16:28:38.848911+0000","last_unstale":"2024-09-13T16:30:04.687781+0000","last_undegraded":"2024-09-13T16:30:04.687781+0000","last_fullsized":"2024-09-13T16:30:04.687781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:02:35.951263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.687469+0000","last_change":"2024-09-13T16:28:39.203855+0000","last_active":"2024-09-13T16:30:04.687469+0000","last_peered":"2024-09-13T16:30:04.687469+0000","last_clean":"2024-09-13T16:30:04.687469+0000","last_became_active":"2024-09-13T16:28:39.203532+0000","last_became_peered":"2024-09-13T16:28:39.203532+0000","last_unstale":"2024-09-13T16:30:04.687469+0000","last_undegraded":"2024-09-13T16:30:04.687469+0000","last_fullsized":"2024-09-13T16:30:04.687469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:15.048930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.726375+0000","last_change":"2024-09-13T16:28:43.228554+0000","last_active":"2024-09-13T16:30:04.726375+0000","last_peered":"2024-09-13T16:30:04.726375+0000","last_clean":"2024-09-13T16:30:04.726375+0000","last_became_active":"2024-09-13T16:28:43.228204+0000","last_became_peered":"2024-09-13T16:28:43.228204+0000","last_unstale":"2024-09-13T16:30:04.726375+0000","last_undegraded":"2024-09-13T16:30:04.726375+0000","last_fullsized":"2024-09-13T16:30:04.726375+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:16:15.383932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":107,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T16:30:19.746116+0000","last_change":"2024-09-13T16:28:38.848425+0000","last_active":"2024-09-13T16:30:19.746116+0000","last_peered":"2024-09-13T16:30:19.746116+0000","last_clean":"2024-09-13T16:30:19.746116+0000","last_became_active":"2024-09-13T16:28:38.848136+0000","last_became_peered":"2024-09-13T16:28:38.848136+0000","last_unstale":"2024-09-13T16:30:19.746116+0000","last_undegraded":"2024-09-13T16:30:19.746116+0000","last_fullsized":"2024-09-13T16:30:19.746116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:37.823261+0000","last_clean_scrub_stamp":"2024-09-13T16:28:37.823261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:47:03.904315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.684661+0000","last_change":"2024-09-13T16:28:39.210565+0000","last_active":"2024-09-13T16:30:04.684661+0000","last_peered":"2024-09-13T16:30:04.684661+0000","last_clean":"2024-09-13T16:30:04.684661+0000","last_became_active":"2024-09-13T16:28:39.210272+0000","last_became_peered":"2024-09-13T16:28:39.210272+0000","last_unstale":"2024-09-13T16:30:04.684661+0000","last_undegraded":"2024-09-13T16:30:04.684661+0000","last_fullsized":"2024-09-13T16:30:04.684661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:12.746087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.726440+0000","last_change":"2024-09-13T16:28:43.231623+0000","last_active":"2024-09-13T16:30:04.726440+0000","last_peered":"2024-09-13T16:30:04.726440+0000","last_clean":"2024-09-13T16:30:04.726440+0000","last_became_active":"2024-09-13T16:28:43.231255+0000","last_became_peered":"2024-09-13T16:28:43.231255+0000","last_unstale":"2024-09-13T16:30:04.726440+0000","last_undegraded":"2024-09-13T16:30:04.726440+0000","last_fullsized":"2024-09-13T16:30:04.726440+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:42.198912+0000","last_clean_scrub_stamp":"2024-09-13T16:28:42.198912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:04:29.755772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.965567+0000","last_change":"2024-09-13T16:28:39.208438+0000","last_active":"2024-09-13T16:30:04.965567+0000","last_peered":"2024-09-13T16:30:04.965567+0000","last_clean":"2024-09-13T16:30:04.965567+0000","last_became_active":"2024-09-13T16:28:39.208156+0000","last_became_peered":"2024-09-13T16:28:39.208156+0000","last_unstale":"2024-09-13T16:30:04.965567+0000","last_undegraded":"2024-09-13T16:30:04.965567+0000","last_fullsized":"2024-09-13T16:30:04.965567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:53:46.923166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.726374+0000","last_change":"2024-09-13T16:28:39.208529+0000","last_active":"2024-09-13T16:30:04.726374+0000","last_peered":"2024-09-13T16:30:04.726374+0000","last_clean":"2024-09-13T16:30:04.726374+0000","last_became_active":"2024-09-13T16:28:39.208202+0000","last_became_peered":"2024-09-13T16:28:39.208202+0000","last_unstale":"2024-09-13T16:30:04.726374+0000","last_undegraded":"2024-09-13T16:30:04.726374+0000","last_fullsized":"2024-09-13T16:30:04.726374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:17:11.357410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.726166+0000","last_change":"2024-09-13T16:28:39.212958+0000","last_active":"2024-09-13T16:30:04.726166+0000","last_peered":"2024-09-13T16:30:04.726166+0000","last_clean":"2024-09-13T16:30:04.726166+0000","last_became_active":"2024-09-13T16:28:39.212646+0000","last_became_peered":"2024-09-13T16:28:39.212646+0000","last_unstale":"2024-09-13T16:30:04.726166+0000","last_undegraded":"2024-09-13T16:30:04.726166+0000","last_fullsized":"2024-09-13T16:30:04.726166+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:31:04.047778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689683+0000","last_change":"2024-09-13T16:28:39.200449+0000","last_active":"2024-09-13T16:30:04.689683+0000","last_peered":"2024-09-13T16:30:04.689683+0000","last_clean":"2024-09-13T16:30:04.689683+0000","last_became_active":"2024-09-13T16:28:39.200133+0000","last_became_peered":"2024-09-13T16:28:39.200133+0000","last_unstale":"2024-09-13T16:30:04.689683+0000","last_undegraded":"2024-09-13T16:30:04.689683+0000","last_fullsized":"2024-09-13T16:30:04.689683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:18:06.480691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.719276+0000","last_change":"2024-09-13T16:28:39.200274+0000","last_active":"2024-09-13T16:30:04.719276+0000","last_peered":"2024-09-13T16:30:04.719276+0000","last_clean":"2024-09-13T16:30:04.719276+0000","last_became_active":"2024-09-13T16:28:39.199951+0000","last_became_peered":"2024-09-13T16:28:39.199951+0000","last_unstale":"2024-09-13T16:30:04.719276+0000","last_undegraded":"2024-09-13T16:30:04.719276+0000","last_fullsized":"2024-09-13T16:30:04.719276+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:13:31.199857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.965945+0000","last_change":"2024-09-13T16:28:39.213591+0000","last_active":"2024-09-13T16:30:04.965945+0000","last_peered":"2024-09-13T16:30:04.965945+0000","last_clean":"2024-09-13T16:30:04.965945+0000","last_became_active":"2024-09-13T16:28:39.213324+0000","last_became_peered":"2024-09-13T16:28:39.213324+0000","last_unstale":"2024-09-13T16:30:04.965945+0000","last_undegraded":"2024-09-13T16:30:04.965945+0000","last_fullsized":"2024-09-13T16:30:04.965945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:54:03.172919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.726681+0000","last_change":"2024-09-13T16:28:39.217442+0000","last_active":"2024-09-13T16:30:04.726681+0000","last_peered":"2024-09-13T16:30:04.726681+0000","last_clean":"2024-09-13T16:30:04.726681+0000","last_became_active":"2024-09-13T16:28:39.217151+0000","last_became_peered":"2024-09-13T16:28:39.217151+0000","last_unstale":"2024-09-13T16:30:04.726681+0000","last_undegraded":"2024-09-13T16:30:04.726681+0000","last_fullsized":"2024-09-13T16:30:04.726681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:17.132278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689157+0000","last_change":"2024-09-13T16:28:39.203938+0000","last_active":"2024-09-13T16:30:04.689157+0000","last_peered":"2024-09-13T16:30:04.689157+0000","last_clean":"2024-09-13T16:30:04.689157+0000","last_became_active":"2024-09-13T16:28:39.203655+0000","last_became_peered":"2024-09-13T16:28:39.203655+0000","last_unstale":"2024-09-13T16:30:04.689157+0000","last_undegraded":"2024-09-13T16:30:04.689157+0000","last_fullsized":"2024-09-13T16:30:04.689157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:03:25.415256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.687258+0000","last_change":"2024-09-13T16:28:39.209592+0000","last_active":"2024-09-13T16:30:04.687258+0000","last_peered":"2024-09-13T16:30:04.687258+0000","last_clean":"2024-09-13T16:30:04.687258+0000","last_became_active":"2024-09-13T16:28:39.209299+0000","last_became_peered":"2024-09-13T16:28:39.209299+0000","last_unstale":"2024-09-13T16:30:04.687258+0000","last_undegraded":"2024-09-13T16:30:04.687258+0000","last_fullsized":"2024-09-13T16:30:04.687258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:04.053405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689202+0000","last_change":"2024-09-13T16:28:39.208437+0000","last_active":"2024-09-13T16:30:04.689202+0000","last_peered":"2024-09-13T16:30:04.689202+0000","last_clean":"2024-09-13T16:30:04.689202+0000","last_became_active":"2024-09-13T16:28:39.208136+0000","last_became_peered":"2024-09-13T16:28:39.208136+0000","last_unstale":"2024-09-13T16:30:04.689202+0000","last_undegraded":"2024-09-13T16:30:04.689202+0000","last_fullsized":"2024-09-13T16:30:04.689202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:21.304037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.966094+0000","last_change":"2024-09-13T16:28:39.203970+0000","last_active":"2024-09-13T16:30:04.966094+0000","last_peered":"2024-09-13T16:30:04.966094+0000","last_clean":"2024-09-13T16:30:04.966094+0000","last_became_active":"2024-09-13T16:28:39.203559+0000","last_became_peered":"2024-09-13T16:28:39.203559+0000","last_unstale":"2024-09-13T16:30:04.966094+0000","last_undegraded":"2024-09-13T16:30:04.966094+0000","last_fullsized":"2024-09-13T16:30:04.966094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:26:25.158393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.719221+0000","last_change":"2024-09-13T16:28:39.205584+0000","last_active":"2024-09-13T16:30:04.719221+0000","last_peered":"2024-09-13T16:30:04.719221+0000","last_clean":"2024-09-13T16:30:04.719221+0000","last_became_active":"2024-09-13T16:28:39.205281+0000","last_became_peered":"2024-09-13T16:28:39.205281+0000","last_unstale":"2024-09-13T16:30:04.719221+0000","last_undegraded":"2024-09-13T16:30:04.719221+0000","last_fullsized":"2024-09-13T16:30:04.719221+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:58:10.109647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.965875+0000","last_change":"2024-09-13T16:28:39.214479+0000","last_active":"2024-09-13T16:30:04.965875+0000","last_peered":"2024-09-13T16:30:04.965875+0000","last_clean":"2024-09-13T16:30:04.965875+0000","last_became_active":"2024-09-13T16:28:39.213984+0000","last_became_peered":"2024-09-13T16:28:39.213984+0000","last_unstale":"2024-09-13T16:30:04.965875+0000","last_undegraded":"2024-09-13T16:30:04.965875+0000","last_fullsized":"2024-09-13T16:30:04.965875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:53:43.137726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.726557+0000","last_change":"2024-09-13T16:28:39.209595+0000","last_active":"2024-09-13T16:30:04.726557+0000","last_peered":"2024-09-13T16:30:04.726557+0000","last_clean":"2024-09-13T16:30:04.726557+0000","last_became_active":"2024-09-13T16:28:39.209282+0000","last_became_peered":"2024-09-13T16:28:39.209282+0000","last_unstale":"2024-09-13T16:30:04.726557+0000","last_undegraded":"2024-09-13T16:30:04.726557+0000","last_fullsized":"2024-09-13T16:30:04.726557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:26:41.452462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.684096+0000","last_change":"2024-09-13T16:28:39.210610+0000","last_active":"2024-09-13T16:30:04.684096+0000","last_peered":"2024-09-13T16:30:04.684096+0000","last_clean":"2024-09-13T16:30:04.684096+0000","last_became_active":"2024-09-13T16:28:39.210296+0000","last_became_peered":"2024-09-13T16:28:39.210296+0000","last_unstale":"2024-09-13T16:30:04.684096+0000","last_undegraded":"2024-09-13T16:30:04.684096+0000","last_fullsized":"2024-09-13T16:30:04.684096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:06.385733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.966115+0000","last_change":"2024-09-13T16:28:39.207609+0000","last_active":"2024-09-13T16:30:04.966115+0000","last_peered":"2024-09-13T16:30:04.966115+0000","last_clean":"2024-09-13T16:30:04.966115+0000","last_became_active":"2024-09-13T16:28:39.206893+0000","last_became_peered":"2024-09-13T16:28:39.206893+0000","last_unstale":"2024-09-13T16:30:04.966115+0000","last_undegraded":"2024-09-13T16:30:04.966115+0000","last_fullsized":"2024-09-13T16:30:04.966115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:51.582666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.965896+0000","last_change":"2024-09-13T16:28:39.201856+0000","last_active":"2024-09-13T16:30:04.965896+0000","last_peered":"2024-09-13T16:30:04.965896+0000","last_clean":"2024-09-13T16:30:04.965896+0000","last_became_active":"2024-09-13T16:28:39.201511+0000","last_became_peered":"2024-09-13T16:28:39.201511+0000","last_unstale":"2024-09-13T16:30:04.965896+0000","last_undegraded":"2024-09-13T16:30:04.965896+0000","last_fullsized":"2024-09-13T16:30:04.965896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:05:19.722354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689066+0000","last_change":"2024-09-13T16:28:39.216106+0000","last_active":"2024-09-13T16:30:04.689066+0000","last_peered":"2024-09-13T16:30:04.689066+0000","last_clean":"2024-09-13T16:30:04.689066+0000","last_became_active":"2024-09-13T16:28:39.215432+0000","last_became_peered":"2024-09-13T16:28:39.215432+0000","last_unstale":"2024-09-13T16:30:04.689066+0000","last_undegraded":"2024-09-13T16:30:04.689066+0000","last_fullsized":"2024-09-13T16:30:04.689066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:21:18.439891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689291+0000","last_change":"2024-09-13T16:28:39.210578+0000","last_active":"2024-09-13T16:30:04.689291+0000","last_peered":"2024-09-13T16:30:04.689291+0000","last_clean":"2024-09-13T16:30:04.689291+0000","last_became_active":"2024-09-13T16:28:39.210269+0000","last_became_peered":"2024-09-13T16:28:39.210269+0000","last_unstale":"2024-09-13T16:30:04.689291+0000","last_undegraded":"2024-09-13T16:30:04.689291+0000","last_fullsized":"2024-09-13T16:30:04.689291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:27:27.078880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.719142+0000","last_change":"2024-09-13T16:28:39.213409+0000","last_active":"2024-09-13T16:30:04.719142+0000","last_peered":"2024-09-13T16:30:04.719142+0000","last_clean":"2024-09-13T16:30:04.719142+0000","last_became_active":"2024-09-13T16:28:39.213077+0000","last_became_peered":"2024-09-13T16:28:39.213077+0000","last_unstale":"2024-09-13T16:30:04.719142+0000","last_undegraded":"2024-09-13T16:30:04.719142+0000","last_fullsized":"2024-09-13T16:30:04.719142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:32:24.747516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.684840+0000","last_change":"2024-09-13T16:28:39.215216+0000","last_active":"2024-09-13T16:30:04.684840+0000","last_peered":"2024-09-13T16:30:04.684840+0000","last_clean":"2024-09-13T16:30:04.684840+0000","last_became_active":"2024-09-13T16:28:39.214920+0000","last_became_peered":"2024-09-13T16:28:39.214920+0000","last_unstale":"2024-09-13T16:30:04.684840+0000","last_undegraded":"2024-09-13T16:30:04.684840+0000","last_fullsized":"2024-09-13T16:30:04.684840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:25:38.132833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689532+0000","last_change":"2024-09-13T16:28:39.202506+0000","last_active":"2024-09-13T16:30:04.689532+0000","last_peered":"2024-09-13T16:30:04.689532+0000","last_clean":"2024-09-13T16:30:04.689532+0000","last_became_active":"2024-09-13T16:28:39.202224+0000","last_became_peered":"2024-09-13T16:28:39.202224+0000","last_unstale":"2024-09-13T16:30:04.689532+0000","last_undegraded":"2024-09-13T16:30:04.689532+0000","last_fullsized":"2024-09-13T16:30:04.689532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:00:37.208442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.688211+0000","last_change":"2024-09-13T16:29:05.640477+0000","last_active":"2024-09-13T16:30:04.688211+0000","last_peered":"2024-09-13T16:30:04.688211+0000","last_clean":"2024-09-13T16:30:04.688211+0000","last_became_active":"2024-09-13T16:29:05.640014+0000","last_became_peered":"2024-09-13T16:29:05.640014+0000","last_unstale":"2024-09-13T16:30:04.688211+0000","last_undegraded":"2024-09-13T16:30:04.688211+0000","last_fullsized":"2024-09-13T16:30:04.688211+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:47:45.202113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.966433+0000","last_change":"2024-09-13T16:28:39.200217+0000","last_active":"2024-09-13T16:30:04.966433+0000","last_peered":"2024-09-13T16:30:04.966433+0000","last_clean":"2024-09-13T16:30:04.966433+0000","last_became_active":"2024-09-13T16:28:39.199946+0000","last_became_peered":"2024-09-13T16:28:39.199946+0000","last_unstale":"2024-09-13T16:30:04.966433+0000","last_undegraded":"2024-09-13T16:30:04.966433+0000","last_fullsized":"2024-09-13T16:30:04.966433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:20:02.906225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.688846+0000","last_change":"2024-09-13T16:28:39.215013+0000","last_active":"2024-09-13T16:30:04.688846+0000","last_peered":"2024-09-13T16:30:04.688846+0000","last_clean":"2024-09-13T16:30:04.688846+0000","last_became_active":"2024-09-13T16:28:39.214731+0000","last_became_peered":"2024-09-13T16:28:39.214731+0000","last_unstale":"2024-09-13T16:30:04.688846+0000","last_undegraded":"2024-09-13T16:30:04.688846+0000","last_fullsized":"2024-09-13T16:30:04.688846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:35:01.836001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689055+0000","last_change":"2024-09-13T16:28:39.200300+0000","last_active":"2024-09-13T16:30:04.689055+0000","last_peered":"2024-09-13T16:30:04.689055+0000","last_clean":"2024-09-13T16:30:04.689055+0000","last_became_active":"2024-09-13T16:28:39.200006+0000","last_became_peered":"2024-09-13T16:28:39.200006+0000","last_unstale":"2024-09-13T16:30:04.689055+0000","last_undegraded":"2024-09-13T16:30:04.689055+0000","last_fullsized":"2024-09-13T16:30:04.689055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:35:14.783905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.687736+0000","last_change":"2024-09-13T16:28:39.213552+0000","last_active":"2024-09-13T16:30:04.687736+0000","last_peered":"2024-09-13T16:30:04.687736+0000","last_clean":"2024-09-13T16:30:04.687736+0000","last_became_active":"2024-09-13T16:28:39.213287+0000","last_became_peered":"2024-09-13T16:28:39.213287+0000","last_unstale":"2024-09-13T16:30:04.687736+0000","last_undegraded":"2024-09-13T16:30:04.687736+0000","last_fullsized":"2024-09-13T16:30:04.687736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:30.588770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.684568+0000","last_change":"2024-09-13T16:28:39.211126+0000","last_active":"2024-09-13T16:30:04.684568+0000","last_peered":"2024-09-13T16:30:04.684568+0000","last_clean":"2024-09-13T16:30:04.684568+0000","last_became_active":"2024-09-13T16:28:39.210810+0000","last_became_peered":"2024-09-13T16:28:39.210810+0000","last_unstale":"2024-09-13T16:30:04.684568+0000","last_undegraded":"2024-09-13T16:30:04.684568+0000","last_fullsized":"2024-09-13T16:30:04.684568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:52:45.249309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.683625+0000","last_change":"2024-09-13T16:29:05.636535+0000","last_active":"2024-09-13T16:30:04.683625+0000","last_peered":"2024-09-13T16:30:04.683625+0000","last_clean":"2024-09-13T16:30:04.683625+0000","last_became_active":"2024-09-13T16:29:05.636161+0000","last_became_peered":"2024-09-13T16:29:05.636161+0000","last_unstale":"2024-09-13T16:30:04.683625+0000","last_undegraded":"2024-09-13T16:30:04.683625+0000","last_fullsized":"2024-09-13T16:30:04.683625+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:09.606922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.689775+0000","last_change":"2024-09-13T16:28:39.202222+0000","last_active":"2024-09-13T16:30:04.689775+0000","last_peered":"2024-09-13T16:30:04.689775+0000","last_clean":"2024-09-13T16:30:04.689775+0000","last_became_active":"2024-09-13T16:28:39.201897+0000","last_became_peered":"2024-09-13T16:28:39.201897+0000","last_unstale":"2024-09-13T16:30:04.689775+0000","last_undegraded":"2024-09-13T16:30:04.689775+0000","last_fullsized":"2024-09-13T16:30:04.689775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:41:09.087185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.690038+0000","last_change":"2024-09-13T16:28:39.211164+0000","last_active":"2024-09-13T16:30:04.690038+0000","last_peered":"2024-09-13T16:30:04.690038+0000","last_clean":"2024-09-13T16:30:04.690038+0000","last_became_active":"2024-09-13T16:28:39.210826+0000","last_became_peered":"2024-09-13T16:28:39.210826+0000","last_unstale":"2024-09-13T16:30:04.690038+0000","last_undegraded":"2024-09-13T16:30:04.690038+0000","last_fullsized":"2024-09-13T16:30:04.690038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:40:30.663867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T16:30:04.966070+0000","last_change":"2024-09-13T16:28:39.207991+0000","last_active":"2024-09-13T16:30:04.966070+0000","last_peered":"2024-09-13T16:30:04.966070+0000","last_clean":"2024-09-13T16:30:04.966070+0000","last_became_active":"2024-09-13T16:28:39.207332+0000","last_became_peered":"2024-09-13T16:28:39.207332+0000","last_unstale":"2024-09-13T16:30:04.966070+0000","last_undegraded":"2024-09-13T16:30:04.966070+0000","last_fullsized":"2024-09-13T16:30:04.966070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:28:38.825121+0000","last_clean_scrub_stamp":"2024-09-13T16:28:38.825121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:30:54.271361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70716,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":819,"num_read_kb":486,"num_write":236,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":762,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":212148,"data_compressed":4776,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250749,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383712,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":519861,"data_compressed":23704,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.504,"15min":0.495},"min":{"1min":0.371,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":13.802},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54100000000000004,"15min":0.53900000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.39700000000000002}]},{"osd":1,"last update":"Fri Sep 13 16:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.78300000000000003,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55700000000000005,"15min":0.55800000000000005},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69799999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.503}]},{"osd":2,"last update":"Fri Sep 13 16:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.436,"15min":0.438},"min":{"1min":0.29599999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.55400000000000005,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.441,"15min":0.438},"min":{"1min":0.32200000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.627,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.61299999999999999}]},{"osd":3,"last update":"Fri Sep 13 16:30:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.34200000000000003}]},{"osd":4,"last update":"Fri Sep 13 16:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.40699999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.58899999999999997,"15min":0.58299999999999996},"min":{"1min":0.443,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.83099999999999996,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.59999999999999998}]},{"osd":5,"last update":"Fri Sep 13 16:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54400000000000004,"15min":0.54300000000000004},"min":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70599999999999996,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.505,"15min":0.495},"min":{"1min":0.38300000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.67200000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.41999999999999998}]},{"osd":6,"last update":"Fri Sep 13 16:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.48999999999999999,"5min":0.438,"15min":0.438},"max":{"1min":0.89600000000000002,"5min":1.032,"15min":1.032},"last":13.763999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65400000000000003,"15min":0.65500000000000003},"min":{"1min":0.48999999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77800000000000002,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":13.747}]},{"osd":7,"last update":"Fri Sep 13 16:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.65900000000000003,"15min":0.65300000000000002},"min":{"1min":0.55900000000000005,"5min":0.436,"15min":0.436},"max":{"1min":0.84499999999999997,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.65000000000000002,"15min":0.64400000000000002},"min":{"1min":0.54000000000000004,"5min":0.502,"15min":0.502},"max":{"1min":0.84899999999999998,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.63800000000000001}]},{"osd":8,"last update":"Fri Sep 13 16:29:39 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.622},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.65200000000000002}]},{"osd":9,"last update":"Fri Sep 13 16:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72799999999999998,"15min":0.72999999999999998},"min":{"1min":0.54800000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.89000000000000001,"5min":1.716,"15min":1.716},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.68899999999999995,"15min":0.67800000000000005},"min":{"1min":0.56000000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.87,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.56699999999999995}]},{"osd":10,"last update":"Fri Sep 13 16:29:58 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.71299999999999997,"15min":0.70899999999999996},"min":{"1min":0.58299999999999996,"5min":0.505,"15min":0.505},"max":{"1min":0.90400000000000003,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73399999999999999,"15min":0.73499999999999999},"min":{"1min":0.54300000000000004,"5min":0.498,"15min":0.498},"max":{"1min":0.91800000000000004,"5min":1.756,"15min":1.756},"last":0.71499999999999997}]}]},{"osd":10,"up_from":72,"seq":309237645350,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1171456,"data_stored":1169398,"data_compressed":61585,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:29:39 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.53500000000000003}]},{"osd":1,"last update":"Fri Sep 13 16:29:34 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.40999999999999998,"15min":0.40300000000000002},"min":{"1min":0.315,"5min":0.249,"15min":0.249},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.439,"15min":0.437},"min":{"1min":0.312,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.36899999999999999}]},{"osd":2,"last update":"Fri Sep 13 16:29:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.38300000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.76300000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.57199999999999995,"15min":0.56200000000000006},"min":{"1min":0.41199999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.66300000000000003}]},{"osd":3,"last update":"Fri Sep 13 16:29:34 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60399999999999998,"15min":0.60799999999999998},"min":{"1min":0.40799999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80500000000000005,"5min":1.04,"15min":1.04},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.89100000000000001,"15min":0.94599999999999995},"min":{"1min":0.46600000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.70799999999999996,"5min":7.2320000000000002,"15min":7.2320000000000002},"last":0.68000000000000005}]},{"osd":4,"last update":"Fri Sep 13 16:29:34 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48599999999999999,"15min":0.48299999999999998},"min":{"1min":0.39000000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.435},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.46800000000000003,"15min":0.46500000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.65000000000000002,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.39300000000000002}]},{"osd":5,"last update":"Fri Sep 13 16:29:34 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.49399999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78400000000000003,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.63},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.63100000000000001,"15min":0.627},"min":{"1min":0.45000000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.879,"5min":1.006,"15min":1.006},"last":0.58399999999999996}]},{"osd":6,"last update":"Fri Sep 13 16:29:34 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.48399999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.90300000000000002,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63200000000000001,"15min":0.628},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.82299999999999995,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.55300000000000005}]},{"osd":7,"last update":"Fri Sep 13 16:28:30 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.56999999999999995}]},{"osd":8,"last update":"Fri Sep 13 16:29:34 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.95999999999999996,"15min":1.01},"min":{"1min":0.48499999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.91300000000000003,"5min":7.2569999999999997,"15min":7.2569999999999997},"last":0.746},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.69899999999999995,"15min":0.70299999999999996},"min":{"1min":0.496,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.92200000000000004,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.69599999999999995}]},{"osd":9,"last update":"Fri Sep 13 16:29:34 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89100000000000001,"5min":1.167,"15min":1.167},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.73699999999999999,"15min":0.74299999999999999},"min":{"1min":0.61399999999999999,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.82699999999999996,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.73199999999999998}]},{"osd":11,"last update":"Fri Sep 13 16:30:00 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71099999999999997,"15min":0.71299999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.86399999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.97499999999999998,"15min":1.0209999999999999},"min":{"1min":0.55800000000000005,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.83799999999999997,"5min":7.2199999999999998,"15min":7.2199999999999998},"last":0.76600000000000001}]}]},{"osd":9,"up_from":67,"seq":287762808874,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":523285,"data_compressed":23704,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:30:10 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.40500000000000003,"15min":0.42299999999999999},"min":{"1min":0.28499999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.51600000000000001,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.45500000000000002,"15min":0.48099999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.55000000000000004,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.33000000000000002}]},{"osd":1,"last update":"Fri Sep 13 16:30:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.38800000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.79400000000000004,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58499999999999996,"15min":0.59699999999999998},"min":{"1min":0.40300000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70999999999999996,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.42099999999999999}]},{"osd":2,"last update":"Fri Sep 13 16:30:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.34399999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.629},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.60699999999999998}]},{"osd":3,"last update":"Fri Sep 13 16:30:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.46200000000000002,"15min":0.47999999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65000000000000002,"5min":0.754,"15min":0.754},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.443,"5min":0.50800000000000001,"15min":0.53900000000000003},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.30099999999999999}]},{"osd":4,"last update":"Fri Sep 13 16:30:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.60799999999999998,"15min":0.623},"min":{"1min":0.40200000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.74399999999999999,"5min":0.997,"15min":0.997},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.63400000000000001,"15min":0.65100000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.76400000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.35499999999999998}]},{"osd":5,"last update":"Fri Sep 13 16:30:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60899999999999999,"15min":0.61199999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.63100000000000001,"15min":0.63800000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.58399999999999996}]},{"osd":6,"last update":"Fri Sep 13 16:28:07 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.45700000000000002}]},{"osd":7,"last update":"Fri Sep 13 16:30:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.70999999999999996,"15min":0.73699999999999999},"min":{"1min":0.47799999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.81599999999999995,"5min":1.107,"15min":1.107},"last":0.373},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.72099999999999997,"15min":0.749},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.98599999999999999,"5min":1.139,"15min":1.139},"last":0.502}]},{"osd":8,"last update":"Fri Sep 13 16:30:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.70799999999999996,"15min":0.73199999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.82199999999999995,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.72099999999999997,"15min":0.73999999999999999},"min":{"1min":0.503,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.83699999999999997,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.56999999999999995}]},{"osd":10,"last update":"Fri Sep 13 16:29:33 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.751,"15min":0.76300000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.022,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.73699999999999999,"15min":0.745},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.006,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.65300000000000002}]},{"osd":11,"last update":"Fri Sep 13 16:29:44 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.55600000000000005}]}]},{"osd":8,"up_from":59,"seq":253403070511,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":1036288,"data_stored":518313,"data_compressed":23704,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.51500000000000001,"15min":0.51100000000000001},"min":{"1min":0.42199999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.71099999999999997,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52000000000000002,"15min":0.51300000000000001},"min":{"1min":0.378,"5min":0.313,"15min":0.313},"max":{"1min":0.76500000000000001,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.66600000000000004}]},{"osd":1,"last update":"Fri Sep 13 16:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.56899999999999995,"15min":0.56399999999999995},"min":{"1min":0.45800000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.72799999999999998,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.45000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.70199999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.48699999999999999}]},{"osd":2,"last update":"Fri Sep 13 16:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.41999999999999998,"15min":0.41299999999999998},"min":{"1min":0.33500000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.59099999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.313},{"interface":"front","average":{"1min":0.433,"5min":0.45200000000000001,"15min":0.45600000000000002},"min":{"1min":0.33300000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57799999999999996,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.34000000000000002}]},{"osd":3,"last update":"Fri Sep 13 16:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.55900000000000005,"15min":0.55300000000000005},"min":{"1min":0.378,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76000000000000001,"5min":0.871,"15min":0.871},"last":0.499},{"interface":"front","average":{"1min":0.621,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.46100000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.85699999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.747}]},{"osd":4,"last update":"Fri Sep 13 16:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57499999999999996,"15min":0.56699999999999995},"min":{"1min":0.46500000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.72399999999999998,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.62,"15min":0.61199999999999999},"min":{"1min":0.47999999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.81200000000000006,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.54400000000000004}]},{"osd":5,"last update":"Fri Sep 13 16:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.378,"5min":0.314,"15min":0.314},"max":{"1min":0.70599999999999996,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.378},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53800000000000003,"15min":0.54300000000000004},"min":{"1min":0.38,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.74299999999999999,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.39200000000000002}]},{"osd":6,"last update":"Fri Sep 13 16:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63800000000000001,"15min":0.63300000000000001},"min":{"1min":0.48899999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.89000000000000001,"5min":1.002,"15min":1.002},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66300000000000003,"15min":0.66400000000000003},"min":{"1min":0.53300000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.86899999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.57899999999999996}]},{"osd":7,"last update":"Fri Sep 13 16:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.67000000000000004,"15min":0.67100000000000004},"min":{"1min":0.52100000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.84899999999999998,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.66200000000000003,"15min":0.66300000000000003},"min":{"1min":0.49199999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.83199999999999996,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.61199999999999999}]},{"osd":9,"last update":"Fri Sep 13 16:30:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.63800000000000001,"15min":0.61399999999999999},"min":{"1min":0.47099999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85199999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.67000000000000004,"15min":0.64200000000000002},"min":{"1min":0.48899999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95699999999999996,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.63}]},{"osd":10,"last update":"Fri Sep 13 16:29:34 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.73499999999999999,"15min":0.73099999999999998},"min":{"1min":0.61299999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0529999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.67100000000000004,"15min":0.64900000000000002},"min":{"1min":0.69299999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.76800000000000002}]},{"osd":11,"last update":"Fri Sep 13 16:29:43 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.68999999999999995}]}]},{"osd":7,"up_from":51,"seq":219043332148,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":1116705,"data_compressed":59993,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.39600000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.41199999999999998}]},{"osd":1,"last update":"Fri Sep 13 16:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.40100000000000002,"15min":0.39200000000000002},"min":{"1min":0.315,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.54000000000000004,"5min":1.02,"15min":1.02},"last":0.438},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.42099999999999999,"15min":0.41299999999999998},"min":{"1min":0.29799999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.59099999999999997,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.46500000000000002}]},{"osd":2,"last update":"Fri Sep 13 16:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.55500000000000005,"15min":0.55300000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56200000000000006,"15min":0.55300000000000005},"min":{"1min":0.41299999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.75600000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.67200000000000004}]},{"osd":3,"last update":"Fri Sep 13 16:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.54300000000000004,"15min":0.53300000000000003},"min":{"1min":0.45400000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78200000000000003,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56699999999999995,"15min":0.56399999999999995},"min":{"1min":0.44,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73099999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.57899999999999996}]},{"osd":4,"last update":"Fri Sep 13 16:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.46500000000000002,"15min":0.45500000000000002},"min":{"1min":0.377,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67200000000000004,"5min":0.874,"15min":0.874},"last":0.624},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.45700000000000002,"15min":0.46300000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.69099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.501}]},{"osd":5,"last update":"Fri Sep 13 16:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.59499999999999997,"15min":0.58399999999999996},"min":{"1min":0.42799999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.60499999999999998,"15min":0.59099999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.879,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.81499999999999995}]},{"osd":6,"last update":"Fri Sep 13 16:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.46899999999999997,"5min":0.318,"15min":0.318},"max":{"1min":0.78800000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61499999999999999,"15min":0.61599999999999999},"min":{"1min":0.44900000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.83299999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.56499999999999995}]},{"osd":8,"last update":"Fri Sep 13 16:29:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.628,"15min":0.627},"min":{"1min":0.46400000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.72499999999999998,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66200000000000003,"15min":0.66600000000000004},"min":{"1min":0.50800000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77000000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.77100000000000002}]},{"osd":9,"last update":"Fri Sep 13 16:30:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.628,"15min":0.625},"min":{"1min":0.52100000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89100000000000001,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.66200000000000003,"15min":0.65000000000000002},"min":{"1min":0.58999999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.88300000000000001,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.78200000000000003}]},{"osd":10,"last update":"Fri Sep 13 16:29:34 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.621,"15min":0.62},"min":{"1min":0.433,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78700000000000003,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.64300000000000002,"15min":0.63600000000000001},"min":{"1min":0.52100000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84399999999999997,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.73899999999999999}]},{"osd":11,"last update":"Fri Sep 13 16:29:42 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.79700000000000004}]}]},{"osd":6,"up_from":46,"seq":197568495673,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":1028096,"data_stored":515037,"data_compressed":23704,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.39100000000000001,"15min":0.375},"min":{"1min":0.252,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39900000000000002,"15min":0.39100000000000001},"min":{"1min":0.25800000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.373}]},{"osd":1,"last update":"Fri Sep 13 16:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54000000000000004,"15min":0.51400000000000001},"min":{"1min":0.27400000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.79600000000000004,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.625},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56999999999999995,"15min":0.54800000000000004},"min":{"1min":0.34499999999999997,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.54900000000000004}]},{"osd":2,"last update":"Fri Sep 13 16:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55700000000000005,"15min":0.55000000000000004},"min":{"1min":0.38400000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83499999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60099999999999998,"15min":0.58899999999999997},"min":{"1min":0.39100000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.92000000000000004,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.42699999999999999}]},{"osd":3,"last update":"Fri Sep 13 16:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.46700000000000003,"15min":0.46600000000000003},"min":{"1min":0.28499999999999998,"5min":0.251,"15min":0.251},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.501,"15min":0.48999999999999999},"min":{"1min":0.26200000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.65500000000000003,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.40699999999999997}]},{"osd":4,"last update":"Fri Sep 13 16:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.317,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.81599999999999995,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63300000000000001,"15min":0.63100000000000001},"min":{"1min":0.42699999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0029999999999999,"5min":1.079,"15min":1.079},"last":0.70999999999999996}]},{"osd":5,"last update":"Fri Sep 13 16:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.625,"15min":0.60799999999999998},"min":{"1min":0.43099999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.92400000000000004,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.499},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.44600000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85899999999999999,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.48899999999999999}]},{"osd":7,"last update":"Fri Sep 13 16:29:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.63,"15min":0.629},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.93500000000000005,"5min":1.048,"15min":1.048},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.66400000000000003,"15min":0.65200000000000002},"min":{"1min":0.45600000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.94999999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.65300000000000002}]},{"osd":8,"last update":"Fri Sep 13 16:29:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.49099999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90400000000000003,"5min":1.054,"15min":1.054},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.71199999999999997,"15min":0.72299999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.038,"5min":1.042,"15min":1.042},"last":0.53000000000000003}]},{"osd":9,"last update":"Fri Sep 13 16:30:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.68999999999999995,"15min":0.71599999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.95799999999999996,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.68000000000000005,"15min":0.69799999999999995},"min":{"1min":0.436,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.86099999999999999,"5min":1.113,"15min":1.113},"last":0.64300000000000002}]},{"osd":10,"last update":"Fri Sep 13 16:29:36 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.71999999999999997,"15min":0.72899999999999998},"min":{"1min":0.5,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.751},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.76300000000000001,"15min":0.77300000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94499999999999995,"5min":1.089,"15min":1.089},"last":0.71999999999999997}]},{"osd":11,"last update":"Fri Sep 13 16:29:43 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.61399999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757310,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1105920,"data_stored":1111268,"data_compressed":59993,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:29:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.68600000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.54600000000000004,"15min":0.53700000000000003},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.435}]},{"osd":1,"last update":"Fri Sep 13 16:29:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.502,"15min":0.48199999999999998},"min":{"1min":0.32500000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52500000000000002,"15min":0.50700000000000001},"min":{"1min":0.41299999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.73499999999999999,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.61599999999999999}]},{"osd":2,"last update":"Fri Sep 13 16:29:42 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.39300000000000002,"15min":0.36599999999999999},"min":{"1min":0.33500000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.64000000000000001,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.39100000000000001,"15min":0.371},"min":{"1min":0.34200000000000003,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.61599999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.46800000000000003}]},{"osd":3,"last update":"Fri Sep 13 16:29:42 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56299999999999994,"15min":0.56200000000000006},"min":{"1min":0.35399999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.754,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.55600000000000005,"15min":0.54400000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.69599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.51500000000000001}]},{"osd":4,"last update":"Fri Sep 13 16:29:42 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.54300000000000004,"15min":0.51700000000000002},"min":{"1min":0.47099999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54800000000000004,"15min":0.53200000000000003},"min":{"1min":0.376,"5min":0.316,"15min":0.316},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.68500000000000005}]},{"osd":6,"last update":"Fri Sep 13 16:30:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84699999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58899999999999997,"15min":0.57599999999999996},"min":{"1min":0.47099999999999997,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.78400000000000003,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.628}]},{"osd":7,"last update":"Fri Sep 13 16:29:23 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60699999999999998,"15min":0.60499999999999998},"min":{"1min":0.42999999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79300000000000004,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79900000000000004,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.66700000000000004}]},{"osd":8,"last update":"Fri Sep 13 16:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.40200000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.76100000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.45300000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.70799999999999996}]},{"osd":9,"last update":"Fri Sep 13 16:30:09 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.66800000000000004,"15min":0.65300000000000002},"min":{"1min":0.56100000000000005,"5min":0.497,"15min":0.497},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.66400000000000003,"15min":0.65600000000000003},"min":{"1min":0.502,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.80900000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.69799999999999995}]},{"osd":10,"last update":"Fri Sep 13 16:29:32 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.67000000000000004,"15min":0.66000000000000003},"min":{"1min":0.53400000000000003,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.64600000000000002,"15min":0.63800000000000001},"min":{"1min":0.51500000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.872,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.72499999999999998}]}]},{"osd":4,"up_from":33,"seq":141733920835,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":519223,"data_compressed":23704,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.48699999999999999,"15min":0.46300000000000002},"min":{"1min":0.32200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.751,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.504,"15min":0.48199999999999998},"min":{"1min":0.41399999999999998,"5min":0.25,"15min":0.25},"max":{"1min":0.80800000000000005,"5min":0.874,"15min":0.874},"last":0.60499999999999998}]},{"osd":1,"last update":"Fri Sep 13 16:29:21 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.36199999999999999,"15min":0.33600000000000002},"min":{"1min":0.29799999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.63400000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.36099999999999999,"15min":0.33800000000000002},"min":{"1min":0.32400000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52300000000000002,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.35799999999999998}]},{"osd":2,"last update":"Fri Sep 13 16:29:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.50600000000000001,"15min":0.48999999999999999},"min":{"1min":0.33800000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73599999999999999,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.501,"15min":0.46400000000000002},"min":{"1min":0.35799999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.53300000000000003}]},{"osd":3,"last update":"Fri Sep 13 16:29:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.51900000000000002,"15min":0.49099999999999999},"min":{"1min":0.435,"5min":0.308,"15min":0.308},"max":{"1min":0.71499999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53400000000000003,"15min":0.5},"min":{"1min":0.377,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.83399999999999996,"5min":0.875,"15min":0.875},"last":0.40100000000000002}]},{"osd":5,"last update":"Fri Sep 13 16:29:38 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.57599999999999996,"15min":0.55400000000000005},"min":{"1min":0.39500000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.57299999999999995,"15min":0.54000000000000004},"min":{"1min":0.50900000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80700000000000005,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.61599999999999999}]},{"osd":6,"last update":"Fri Sep 13 16:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56699999999999995,"15min":0.54100000000000004},"min":{"1min":0.39200000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77100000000000002,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58499999999999996,"15min":0.56899999999999995},"min":{"1min":0.40500000000000003,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71599999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.46100000000000002}]},{"osd":7,"last update":"Fri Sep 13 16:29:26 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.503,"15min":0.48399999999999999},"min":{"1min":0.379,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.83399999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.503,"15min":0.49299999999999999},"min":{"1min":0.37,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76600000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.44800000000000001}]},{"osd":8,"last update":"Fri Sep 13 16:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.59399999999999997,"15min":0.57599999999999996},"min":{"1min":0.47899999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84599999999999997,"5min":0.872,"15min":0.872},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.628,"15min":0.61399999999999999},"min":{"1min":0.49199999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.79500000000000004,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.58899999999999997}]},{"osd":9,"last update":"Fri Sep 13 16:30:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.64200000000000002,"15min":0.628},"min":{"1min":0.51300000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84699999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67600000000000005,"15min":0.67200000000000004},"min":{"1min":0.53500000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.83299999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.501}]},{"osd":10,"last update":"Fri Sep 13 16:29:34 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.48599999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.77900000000000003,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.63300000000000001,"15min":0.63900000000000001},"min":{"1min":0.45500000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.435}]},{"osd":11,"last update":"Fri Sep 13 16:29:38 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.63},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64500000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36228,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708924,"statfs":{"total":95995035648,"available":95957938176,"internally_reserved":0,"allocated":1036288,"data_stored":514070,"data_compressed":23704,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:29:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.371,"15min":0.33200000000000002},"min":{"1min":0.26500000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.38200000000000001,"15min":0.33800000000000002},"min":{"1min":0.23000000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.28799999999999998}]},{"osd":1,"last update":"Fri Sep 13 16:29:57 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57899999999999996,"15min":0.49299999999999999},"min":{"1min":0.39900000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.91800000000000004,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55400000000000005,"15min":0.49299999999999999},"min":{"1min":0.39200000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.46999999999999997}]},{"osd":2,"last update":"Fri Sep 13 16:29:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52900000000000003,"15min":0.51600000000000001},"min":{"1min":0.35799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.57799999999999996,"15min":0.53400000000000003},"min":{"1min":0.42099999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.94599999999999995,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.63400000000000001}]},{"osd":4,"last update":"Fri Sep 13 16:30:12 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57099999999999995,"15min":0.50900000000000001},"min":{"1min":0.42199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.61899999999999999,"15min":0.55700000000000005},"min":{"1min":0.505,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.96799999999999997,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.48899999999999999}]},{"osd":5,"last update":"Fri Sep 13 16:29:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59299999999999997,"15min":0.57499999999999996},"min":{"1min":0.45500000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81599999999999995,"5min":1.038,"15min":1.038},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60899999999999999,"15min":0.59999999999999998},"min":{"1min":0.41599999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.872,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.61599999999999999}]},{"osd":6,"last update":"Fri Sep 13 16:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.52600000000000002,"15min":0.503},"min":{"1min":0.34200000000000003,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.503,"15min":0.48599999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.36299999999999999}]},{"osd":7,"last update":"Fri Sep 13 16:29:23 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.68300000000000005,"15min":0.63300000000000001},"min":{"1min":0.48799999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":5.5099999999999998,"5min":5.5099999999999998,"15min":5.5099999999999998},"last":0.498},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61599999999999999,"15min":0.59599999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.78600000000000003,"5min":1.014,"15min":1.014},"last":0.44800000000000001}]},{"osd":8,"last update":"Fri Sep 13 16:27:45 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.71499999999999997}]},{"osd":9,"last update":"Fri Sep 13 16:30:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65000000000000002,"15min":0.66000000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.68000000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.436}]},{"osd":10,"last update":"Fri Sep 13 16:29:40 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.51300000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79000000000000004,"15min":0.78800000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1140000000000001,"5min":1.157,"15min":1.157},"last":0.74299999999999999}]},{"osd":11,"last update":"Fri Sep 13 16:29:40 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.77000000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":1093632,"data_stored":579096,"data_compressed":25296,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:29:30 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.61699999999999999,"15min":0.502},"min":{"1min":0.35999999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70399999999999996,"5min":16.027999999999999,"15min":16.027999999999999},"last":0.629},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.49299999999999999,"15min":0.47199999999999998},"min":{"1min":0.38300000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.66200000000000003}]},{"osd":1,"last update":"Fri Sep 13 16:29:30 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.52100000000000002,"15min":0.47699999999999998},"min":{"1min":0.41099999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.55400000000000005,"15min":0.50600000000000001},"min":{"1min":0.49199999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.83699999999999997,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.73199999999999998}]},{"osd":3,"last update":"Fri Sep 13 16:29:54 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56100000000000005,"15min":0.52100000000000002},"min":{"1min":0.436,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77100000000000002,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55700000000000005,"15min":0.52100000000000002},"min":{"1min":0.37,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.70899999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.68999999999999995}]},{"osd":4,"last update":"Fri Sep 13 16:29:39 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.59399999999999997}]},{"osd":5,"last update":"Fri Sep 13 16:29:40 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.432,"15min":0.40899999999999997},"min":{"1min":0.32700000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.76900000000000002,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.41699999999999998,"15min":0.39700000000000002},"min":{"1min":0.33300000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.38300000000000001}]},{"osd":6,"last update":"Fri Sep 13 16:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59299999999999997,"15min":0.55700000000000005},"min":{"1min":0.43099999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73299999999999998,"5min":1.585,"15min":1.585},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61899999999999999,"15min":0.58799999999999997},"min":{"1min":0.503,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79400000000000004,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.71899999999999997}]},{"osd":7,"last update":"Fri Sep 13 16:29:43 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.53000000000000003}]},{"osd":8,"last update":"Fri Sep 13 16:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.53700000000000003,"15min":0.52100000000000002},"min":{"1min":0.38300000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.95899999999999996,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52300000000000002,"15min":0.51000000000000001},"min":{"1min":0.40100000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76600000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.47699999999999998}]},{"osd":9,"last update":"Fri Sep 13 16:30:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68300000000000005,"15min":0.67100000000000004},"min":{"1min":0.49299999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88900000000000001,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68799999999999994,"15min":0.67300000000000004},"min":{"1min":0.54500000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.83299999999999996,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.74399999999999999}]},{"osd":10,"last update":"Fri Sep 13 16:29:39 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.68600000000000005,"15min":0.67300000000000004},"min":{"1min":0.504,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.754},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.74399999999999999,"15min":0.73599999999999999},"min":{"1min":0.56000000000000005,"5min":0.498,"15min":0.498},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.57599999999999996}]},{"osd":11,"last update":"Fri Sep 13 16:29:39 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.64200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":580049,"data_compressed":25296,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 16:30:15 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.503,"15min":0.48499999999999999},"min":{"1min":0.30299999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.80100000000000005,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.497,"15min":0.48499999999999999},"min":{"1min":0.29799999999999999,"5min":0.29399999999999998,"15min":0.26900000000000002},"max":{"1min":0.69099999999999995,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.36299999999999999}]},{"osd":2,"last update":"Fri Sep 13 16:29:29 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.68700000000000006,"15min":0.54600000000000004},"min":{"1min":0.30399999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.75700000000000001,"5min":17.655000000000001,"15min":17.655000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.50700000000000001,"15min":0.48499999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.76200000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.50600000000000001}]},{"osd":3,"last update":"Fri Sep 13 16:29:55 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53900000000000003,"15min":0.51200000000000001},"min":{"1min":0.38100000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57299999999999995,"15min":0.52700000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.60499999999999998}]},{"osd":4,"last update":"Fri Sep 13 16:28:13 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.40400000000000003,"15min":0.38},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.433,"5min":0.40899999999999997,"15min":0.39100000000000001},"min":{"1min":0.28299999999999997,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.57899999999999996}]},{"osd":5,"last update":"Fri Sep 13 16:29:37 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.89800000000000002,"15min":1.2290000000000001},"min":{"1min":0.441,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.80800000000000005,"5min":17.640000000000001,"15min":17.640000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.56399999999999995,"15min":0.54900000000000004},"min":{"1min":0.42799999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82899999999999996,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.48299999999999998}]},{"osd":6,"last update":"Fri Sep 13 16:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.69899999999999995,"5min":0.872,"15min":0.872},"last":0.52200000000000002}]},{"osd":7,"last update":"Fri Sep 13 16:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.55300000000000005,"15min":0.55400000000000005},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.77400000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.623},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.36899999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78900000000000003,"5min":1.022,"15min":1.022},"last":0.63600000000000001}]},{"osd":8,"last update":"Fri Sep 13 16:29:43 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.48699999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83699999999999997,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.65500000000000003,"15min":0.64500000000000002},"min":{"1min":0.55400000000000005,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84799999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.46700000000000003}]},{"osd":9,"last update":"Fri Sep 13 16:30:15 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.64600000000000002,"15min":0.65000000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.75700000000000001,"5min":1.008,"15min":1.008},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65500000000000003,"15min":0.65600000000000003},"min":{"1min":0.42699999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.85399999999999998,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.59099999999999997}]},{"osd":10,"last update":"Fri Sep 13 16:29:37 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65500000000000003,"15min":0.65700000000000003},"min":{"1min":0.47599999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85599999999999998,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.82199999999999995,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.57699999999999996}]},{"osd":11,"last update":"Fri Sep 13 16:29:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.67100000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":516101,"data_compressed":23704,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 16:30:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.50900000000000001,"15min":0.48199999999999998},"min":{"1min":0.32600000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.81799999999999995,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.53600000000000003,"15min":0.505},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.84399999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.51200000000000001}]},{"osd":2,"last update":"Fri Sep 13 16:29:37 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56499999999999995,"15min":0.53100000000000003},"min":{"1min":0.36499999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.95399999999999996,"5min":1.111,"15min":1.111},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.55300000000000005,"15min":0.56299999999999994},"min":{"1min":0.34499999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77600000000000002,"5min":1.698,"15min":1.698},"last":0.46899999999999997}]},{"osd":3,"last update":"Fri Sep 13 16:29:39 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.40200000000000002}]},{"osd":4,"last update":"Fri Sep 13 16:30:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.54700000000000004,"15min":0.47599999999999998},"min":{"1min":0.39700000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.84199999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.438},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56299999999999994,"15min":0.5},"min":{"1min":0.35499999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.61899999999999999}]},{"osd":5,"last update":"Fri Sep 13 16:29:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55800000000000005,"15min":0.54300000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.91300000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.58499999999999996,"15min":0.56200000000000006},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.88200000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.55100000000000005}]},{"osd":6,"last update":"Fri Sep 13 16:29:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.52800000000000002,"15min":0.48999999999999999},"min":{"1min":0.34799999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56699999999999995,"15min":0.52600000000000002},"min":{"1min":0.32200000000000001,"5min":0.309,"15min":0.309},"max":{"1min":1.2230000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.45100000000000001}]},{"osd":7,"last update":"Fri Sep 13 16:29:46 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.5}]},{"osd":8,"last update":"Fri Sep 13 16:29:48 2024","interfaces":[{"interface":"back","average":{"1min":1.5880000000000001,"5min":0.83999999999999997,"15min":0.71499999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":19.501999999999999,"5min":19.501999999999999,"15min":19.501999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":1.665,"5min":0.89100000000000001,"15min":0.76700000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":19.488,"5min":19.488,"15min":19.488},"last":0.57599999999999996}]},{"osd":9,"last update":"Fri Sep 13 16:30:14 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.68600000000000005,"15min":0.72199999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.97799999999999998,"5min":1.01,"15min":1.01},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.68000000000000005,"15min":0.71699999999999997},"min":{"1min":0.39300000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.1930000000000001,"5min":1.236,"15min":1.236},"last":0.59899999999999998}]},{"osd":10,"last update":"Fri Sep 13 16:29:37 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.70599999999999996,"15min":0.70399999999999996},"min":{"1min":0.48599999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.248,"5min":1.284,"15min":1.284},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.75600000000000001,"15min":0.76200000000000001},"min":{"1min":0.56200000000000006,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.032,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.65000000000000002}]},{"osd":11,"last update":"Fri Sep 13 16:29:59 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.69999999999999996,"15min":0.69599999999999995},"min":{"1min":0.56399999999999995,"5min":0.502,"15min":0.502},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.73299999999999998,"15min":0.72499999999999998},"min":{"1min":0.60399999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.63100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34652,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36289,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":66078,"data_compressed":1592,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65398,"data_compressed":1592,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":65390,"data_compressed":1592,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T16:30:27.150 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-13T16:30:27.150 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-13T16:30:27.151 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-13T16:30:27.151 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-13T16:30:27.360 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:30:28.935 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:28 smithi016 ceph-mon[31187]: pgmap v335: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-13T16:30:28.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:28 smithi111 ceph-mon[34600]: pgmap v335: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-13T16:30:29.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:28 smithi107 ceph-mon[34139]: pgmap v335: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-13T16:30:29.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.166+0000 7fdc2d3bb640 1 -- 172.21.15.16:0/3691073876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc28104030 msgr2=0x7fdc2810bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:29.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.166+0000 7fdc2d3bb640 1 --2- 172.21.15.16:0/3691073876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc28104030 0x7fdc2810bbb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdc20009cd0 tx=0x7fdc2002a8e0 comp rx=0 tx=0).stop 2024-09-13T16:30:29.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.166+0000 7fdc2d3bb640 1 -- 172.21.15.16:0/3691073876 shutdown_connections 2024-09-13T16:30:29.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.166+0000 7fdc2d3bb640 1 --2- 172.21.15.16:0/3691073876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc28104030 0x7fdc2810bbb0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:29.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.166+0000 7fdc2d3bb640 1 --2- 172.21.15.16:0/3691073876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc280ff8b0 0x7fdc28103900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:29.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.166+0000 7fdc2d3bb640 1 --2- 172.21.15.16:0/3691073876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc280fef00 0x7fdc280ff2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:29.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.166+0000 7fdc2d3bb640 1 -- 172.21.15.16:0/3691073876 >> 172.21.15.16:0/3691073876 conn(0x7fdc280fa9a0 msgr2=0x7fdc280fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:29.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.166+0000 7fdc2d3bb640 1 -- 172.21.15.16:0/3691073876 shutdown_connections 2024-09-13T16:30:29.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.166+0000 7fdc2d3bb640 1 -- 172.21.15.16:0/3691073876 wait complete. 2024-09-13T16:30:29.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.167+0000 7fdc2d3bb640 1 Processor -- start 2024-09-13T16:30:29.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.167+0000 7fdc2d3bb640 1 -- start start 2024-09-13T16:30:29.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.167+0000 7fdc2d3bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc280fef00 0x7fdc281961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:29.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.167+0000 7fdc2d3bb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc280ff8b0 0x7fdc28196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:29.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.167+0000 7fdc2d3bb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc28104030 0x7fdc2819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:29.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.167+0000 7fdc2d3bb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2819b120 con 0x7fdc280fef00 2024-09-13T16:30:29.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.167+0000 7fdc2d3bb640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2819b290 con 0x7fdc28104030 2024-09-13T16:30:29.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.167+0000 7fdc2d3bb640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2819b400 con 0x7fdc280ff8b0 2024-09-13T16:30:29.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.167+0000 7fdc26ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc280fef00 0x7fdc281961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:29.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.168+0000 7fdc26ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc280fef00 0x7fdc281961e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57724/0 (socket says 172.21.15.16:57724) 2024-09-13T16:30:29.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.168+0000 7fdc26ffd640 1 -- 172.21.15.16:0/301309229 learned_addr learned my addr 172.21.15.16:0/301309229 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.168+0000 7fdc277fe640 1 --2- 172.21.15.16:0/301309229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc28104030 0x7fdc2819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.168+0000 7fdc267fc640 1 --2- 172.21.15.16:0/301309229 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc280ff8b0 0x7fdc28196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.168+0000 7fdc26ffd640 1 -- 172.21.15.16:0/301309229 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc280ff8b0 msgr2=0x7fdc28196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.168+0000 7fdc26ffd640 1 --2- 172.21.15.16:0/301309229 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc280ff8b0 0x7fdc28196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.168+0000 7fdc26ffd640 1 -- 172.21.15.16:0/301309229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc28104030 msgr2=0x7fdc2819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.168+0000 7fdc26ffd640 1 --2- 172.21.15.16:0/301309229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc28104030 0x7fdc2819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.168+0000 7fdc26ffd640 1 -- 172.21.15.16:0/301309229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc20009900 con 0x7fdc280fef00 2024-09-13T16:30:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.168+0000 7fdc26ffd640 1 --2- 172.21.15.16:0/301309229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc280fef00 0x7fdc281961e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800d950 tx=0x7fdc1800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.168+0000 7fdc0ffff640 1 -- 172.21.15.16:0/301309229 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc18009c30 con 0x7fdc280fef00 2024-09-13T16:30:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.168+0000 7fdc0ffff640 1 -- 172.21.15.16:0/301309229 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdc18009d90 con 0x7fdc280fef00 2024-09-13T16:30:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.168+0000 7fdc2d3bb640 1 -- 172.21.15.16:0/301309229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc2819b6e0 con 0x7fdc280fef00 2024-09-13T16:30:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.168+0000 7fdc0ffff640 1 -- 172.21.15.16:0/301309229 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1800f620 con 0x7fdc280fef00 2024-09-13T16:30:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.168+0000 7fdc2d3bb640 1 -- 172.21.15.16:0/301309229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc281a2fa0 con 0x7fdc280fef00 2024-09-13T16:30:29.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.169+0000 7fdc2d3bb640 1 -- 172.21.15.16:0/301309229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc281003e0 con 0x7fdc280fef00 2024-09-13T16:30:29.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.170+0000 7fdc0ffff640 1 -- 172.21.15.16:0/301309229 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fdc180109f0 con 0x7fdc280fef00 2024-09-13T16:30:29.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.171+0000 7fdc0ffff640 1 --2- 172.21.15.16:0/301309229 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdc100a1110 0x7fdc100a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:29.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.171+0000 7fdc0ffff640 1 -- 172.21.15.16:0/301309229 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fdc180cc9b0 con 0x7fdc280fef00 2024-09-13T16:30:29.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.171+0000 7fdc267fc640 1 --2- 172.21.15.16:0/301309229 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdc100a1110 0x7fdc100a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:29.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.172+0000 7fdc267fc640 1 --2- 172.21.15.16:0/301309229 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdc100a1110 0x7fdc100a35d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdc28197790 tx=0x7fdc1c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:29.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.175+0000 7fdc0ffff640 1 -- 172.21.15.16:0/301309229 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fdc18095c90 con 0x7fdc280fef00 2024-09-13T16:30:29.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.385+0000 7fdc2d3bb640 1 -- 172.21.15.16:0/301309229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fdc280ff2e0 con 0x7fdc280fef00 2024-09-13T16:30:29.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.386+0000 7fdc0ffff640 1 -- 172.21.15.16:0/301309229 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fdc180992b0 con 0x7fdc280fef00 2024-09-13T16:30:29.387 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:30:29.387 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-13T16:30:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.388+0000 7fdc2d3bb640 1 -- 172.21.15.16:0/301309229 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdc100a1110 msgr2=0x7fdc100a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.388+0000 7fdc2d3bb640 1 --2- 172.21.15.16:0/301309229 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdc100a1110 0x7fdc100a35d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdc28197790 tx=0x7fdc1c005f90 comp rx=0 tx=0).stop 2024-09-13T16:30:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.388+0000 7fdc2d3bb640 1 -- 172.21.15.16:0/301309229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc280fef00 msgr2=0x7fdc281961e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.388+0000 7fdc2d3bb640 1 --2- 172.21.15.16:0/301309229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc280fef00 0x7fdc281961e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800d950 tx=0x7fdc1800de10 comp rx=0 tx=0).stop 2024-09-13T16:30:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.388+0000 7fdc2d3bb640 1 -- 172.21.15.16:0/301309229 shutdown_connections 2024-09-13T16:30:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.388+0000 7fdc2d3bb640 1 --2- 172.21.15.16:0/301309229 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdc100a1110 0x7fdc100a35d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.388+0000 7fdc2d3bb640 1 --2- 172.21.15.16:0/301309229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc28104030 0x7fdc2819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.388+0000 7fdc2d3bb640 1 --2- 172.21.15.16:0/301309229 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc280ff8b0 0x7fdc28196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.388+0000 7fdc2d3bb640 1 --2- 172.21.15.16:0/301309229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc280fef00 0x7fdc281961e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.388+0000 7fdc2d3bb640 1 -- 172.21.15.16:0/301309229 >> 172.21.15.16:0/301309229 conn(0x7fdc280fa9a0 msgr2=0x7fdc28068b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.388+0000 7fdc2d3bb640 1 -- 172.21.15.16:0/301309229 shutdown_connections 2024-09-13T16:30:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:29.388+0000 7fdc2d3bb640 1 -- 172.21.15.16:0/301309229 wait complete. 2024-09-13T16:30:29.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:29 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/301309229' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-13T16:30:29.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:29 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/301309229' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-13T16:30:30.005 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-13T16:30:30.005 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:30:30.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:29 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/301309229' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-13T16:30:30.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.112+0000 7f0a617bc640 1 -- 172.21.15.16:0/2998332983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a5c100880 msgr2=0x7f0a5c107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:30.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.112+0000 7f0a617bc640 1 --2- 172.21.15.16:0/2998332983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a5c100880 0x7f0a5c107110 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009ca0 tx=0x7f0a5402a8c0 comp rx=0 tx=0).stop 2024-09-13T16:30:30.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.112+0000 7f0a617bc640 1 -- 172.21.15.16:0/2998332983 shutdown_connections 2024-09-13T16:30:30.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.112+0000 7f0a617bc640 1 --2- 172.21.15.16:0/2998332983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a5c100880 0x7f0a5c107110 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:30.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.112+0000 7f0a617bc640 1 --2- 172.21.15.16:0/2998332983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a5c0ffec0 0x7f0a5c100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:30.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.112+0000 7f0a617bc640 1 --2- 172.21.15.16:0/2998332983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a5c0fecd0 0x7f0a5c0ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:30.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.112+0000 7f0a617bc640 1 -- 172.21.15.16:0/2998332983 >> 172.21.15.16:0/2998332983 conn(0x7f0a5c0fa430 msgr2=0x7f0a5c0fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:30.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.112+0000 7f0a617bc640 1 -- 172.21.15.16:0/2998332983 shutdown_connections 2024-09-13T16:30:30.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.112+0000 7f0a617bc640 1 -- 172.21.15.16:0/2998332983 wait complete. 2024-09-13T16:30:30.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.113+0000 7f0a617bc640 1 Processor -- start 2024-09-13T16:30:30.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.113+0000 7f0a617bc640 1 -- start start 2024-09-13T16:30:30.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.113+0000 7f0a617bc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a5c0fecd0 0x7f0a5c197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:30.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.113+0000 7f0a617bc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a5c0ffec0 0x7f0a5c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:30.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.113+0000 7f0a617bc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a5c100880 0x7f0a5c19f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:30.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.113+0000 7f0a617bc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c19f9b0 con 0x7f0a5c100880 2024-09-13T16:30:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.113+0000 7f0a617bc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c19fb20 con 0x7f0a5c0ffec0 2024-09-13T16:30:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.113+0000 7f0a617bc640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c19fce0 con 0x7f0a5c0fecd0 2024-09-13T16:30:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.113+0000 7f0a5b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a5c100880 0x7f0a5c19f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.113+0000 7f0a5b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a5c100880 0x7f0a5c19f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57732/0 (socket says 172.21.15.16:57732) 2024-09-13T16:30:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.113+0000 7f0a5b7fe640 1 -- 172.21.15.16:0/239781287 learned_addr learned my addr 172.21.15.16:0/239781287 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.114+0000 7f0a5b7fe640 1 -- 172.21.15.16:0/239781287 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a5c0fecd0 msgr2=0x7f0a5c197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.114+0000 7f0a5b7fe640 1 --2- 172.21.15.16:0/239781287 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a5c0fecd0 0x7f0a5c197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.114+0000 7f0a5b7fe640 1 -- 172.21.15.16:0/239781287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a5c0ffec0 msgr2=0x7f0a5c198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.114+0000 7f0a5b7fe640 1 --2- 172.21.15.16:0/239781287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a5c0ffec0 0x7f0a5c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.114+0000 7f0a5b7fe640 1 -- 172.21.15.16:0/239781287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a54009900 con 0x7f0a5c100880 2024-09-13T16:30:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.114+0000 7f0a5b7fe640 1 --2- 172.21.15.16:0/239781287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a5c100880 0x7f0a5c19f470 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009dc0 tx=0x7f0a54030f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.114+0000 7f0a53fff640 1 -- 172.21.15.16:0/239781287 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a54003b70 con 0x7f0a5c100880 2024-09-13T16:30:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.114+0000 7f0a53fff640 1 -- 172.21.15.16:0/239781287 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0a54040070 con 0x7f0a5c100880 2024-09-13T16:30:30.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.114+0000 7f0a617bc640 1 -- 172.21.15.16:0/239781287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a5c19ff60 con 0x7f0a5c100880 2024-09-13T16:30:30.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.114+0000 7f0a53fff640 1 -- 172.21.15.16:0/239781287 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a54004370 con 0x7f0a5c100880 2024-09-13T16:30:30.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.114+0000 7f0a617bc640 1 -- 172.21.15.16:0/239781287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a5c1a0440 con 0x7f0a5c100880 2024-09-13T16:30:30.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.115+0000 7f0a617bc640 1 -- 172.21.15.16:0/239781287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a5c10b5c0 con 0x7f0a5c100880 2024-09-13T16:30:30.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.116+0000 7f0a53fff640 1 -- 172.21.15.16:0/239781287 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f0a54042030 con 0x7f0a5c100880 2024-09-13T16:30:30.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.117+0000 7f0a53fff640 1 --2- 172.21.15.16:0/239781287 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f0a480a1120 0x7f0a480a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:30.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.117+0000 7f0a53fff640 1 -- 172.21.15.16:0/239781287 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f0a540ee530 con 0x7f0a5c100880 2024-09-13T16:30:30.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.117+0000 7f0a5affd640 1 --2- 172.21.15.16:0/239781287 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f0a480a1120 0x7f0a480a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:30.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.117+0000 7f0a5affd640 1 --2- 172.21.15.16:0/239781287 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f0a480a1120 0x7f0a480a35e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c0ffd20 tx=0x7f0a4c009630 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:30.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.121+0000 7f0a53fff640 1 -- 172.21.15.16:0/239781287 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0a540b78c0 con 0x7f0a5c100880 2024-09-13T16:30:30.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.295+0000 7f0a617bc640 1 -- 172.21.15.16:0/239781287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a5c1a0810 con 0x7f0a5c100880 2024-09-13T16:30:30.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.297+0000 7f0a53fff640 1 -- 172.21.15.16:0/239781287 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24593 (secure 0 0 0) 0x7f0a540baee0 con 0x7f0a5c100880 2024-09-13T16:30:30.298 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:30:30.298 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":93,"fsid":"0d241f6a-71ec-11ef-bceb-c7b262605968","created":"2024-09-13T16:21:17.971738+0000","modified":"2024-09-13T16:30:05.681521+0000","last_up_change":"2024-09-13T16:27:54.673413+0000","last_in_change":"2024-09-13T16:27:35.718390+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:24:22.981019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:28:37.746027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:28:38.602731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:28:41.633322+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e11ef8f-08d6-4e74-afd6-f49f4bd40032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6803","nonce":2079394186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6805","nonce":2079394186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6809","nonce":2079394186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6807","nonce":2079394186}]},"public_addr":"172.21.15.16:6803/2079394186","cluster_addr":"172.21.15.16:6805/2079394186","heartbeat_back_addr":"172.21.15.16:6809/2079394186","heartbeat_front_addr":"172.21.15.16:6807/2079394186","state":["exists","up"]},{"osd":1,"uuid":"f2cd3694-9c9e-4da7-976e-f80a8df120f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6801","nonce":4021360958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6803","nonce":4021360958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6807","nonce":4021360958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6805","nonce":4021360958}]},"public_addr":"172.21.15.107:6801/4021360958","cluster_addr":"172.21.15.107:6803/4021360958","heartbeat_back_addr":"172.21.15.107:6807/4021360958","heartbeat_front_addr":"172.21.15.107:6805/4021360958","state":["exists","up"]},{"osd":2,"uuid":"a5bbf2eb-6599-44d5-ad96-68bb89ee19f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6801","nonce":3227179825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6803","nonce":3227179825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6807","nonce":3227179825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6805","nonce":3227179825}]},"public_addr":"172.21.15.111:6801/3227179825","cluster_addr":"172.21.15.111:6803/3227179825","heartbeat_back_addr":"172.21.15.111:6807/3227179825","heartbeat_front_addr":"172.21.15.111:6805/3227179825","state":["exists","up"]},{"osd":3,"uuid":"d6295ffe-11bc-4964-a250-9fecca870b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6811","nonce":2390375937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6813","nonce":2390375937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6817","nonce":2390375937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6815","nonce":2390375937}]},"public_addr":"172.21.15.16:6811/2390375937","cluster_addr":"172.21.15.16:6813/2390375937","heartbeat_back_addr":"172.21.15.16:6817/2390375937","heartbeat_front_addr":"172.21.15.16:6815/2390375937","state":["exists","up"]},{"osd":4,"uuid":"0a8446e0-fc86-4646-beb6-a7b9d433093d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6809","nonce":1936582048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6811","nonce":1936582048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6815","nonce":1936582048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6813","nonce":1936582048}]},"public_addr":"172.21.15.107:6809/1936582048","cluster_addr":"172.21.15.107:6811/1936582048","heartbeat_back_addr":"172.21.15.107:6815/1936582048","heartbeat_front_addr":"172.21.15.107:6813/1936582048","state":["exists","up"]},{"osd":5,"uuid":"2c97a446-357e-4975-9e51-9c95fe84a663","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6809","nonce":483858448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6811","nonce":483858448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6815","nonce":483858448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6813","nonce":483858448}]},"public_addr":"172.21.15.111:6809/483858448","cluster_addr":"172.21.15.111:6811/483858448","heartbeat_back_addr":"172.21.15.111:6815/483858448","heartbeat_front_addr":"172.21.15.111:6813/483858448","state":["exists","up"]},{"osd":6,"uuid":"890239e6-e6fb-4f9c-b973-05cc66d199bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6819","nonce":1330214141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6821","nonce":1330214141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6825","nonce":1330214141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6823","nonce":1330214141}]},"public_addr":"172.21.15.16:6819/1330214141","cluster_addr":"172.21.15.16:6821/1330214141","heartbeat_back_addr":"172.21.15.16:6825/1330214141","heartbeat_front_addr":"172.21.15.16:6823/1330214141","state":["exists","up"]},{"osd":7,"uuid":"ee826a9c-f441-4903-9b1b-66b627f7b712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6817","nonce":1133876754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6819","nonce":1133876754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6823","nonce":1133876754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6821","nonce":1133876754}]},"public_addr":"172.21.15.107:6817/1133876754","cluster_addr":"172.21.15.107:6819/1133876754","heartbeat_back_addr":"172.21.15.107:6823/1133876754","heartbeat_front_addr":"172.21.15.107:6821/1133876754","state":["exists","up"]},{"osd":8,"uuid":"e19773f2-dec8-44e0-9d33-0252e07d6415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6817","nonce":837548190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6819","nonce":837548190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6823","nonce":837548190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6821","nonce":837548190}]},"public_addr":"172.21.15.111:6817/837548190","cluster_addr":"172.21.15.111:6819/837548190","heartbeat_back_addr":"172.21.15.111:6823/837548190","heartbeat_front_addr":"172.21.15.111:6821/837548190","state":["exists","up"]},{"osd":9,"uuid":"bae4d6db-8b90-4d02-8e83-7d8f2a6f8fdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6827","nonce":4175383686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6829","nonce":4175383686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6833","nonce":4175383686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6831","nonce":4175383686}]},"public_addr":"172.21.15.16:6827/4175383686","cluster_addr":"172.21.15.16:6829/4175383686","heartbeat_back_addr":"172.21.15.16:6833/4175383686","heartbeat_front_addr":"172.21.15.16:6831/4175383686","state":["exists","up"]},{"osd":10,"uuid":"fd0e7740-76c8-4c99-a349-7077b4905143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6825","nonce":2659550426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6827","nonce":2659550426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6831","nonce":2659550426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6829","nonce":2659550426}]},"public_addr":"172.21.15.107:6825/2659550426","cluster_addr":"172.21.15.107:6827/2659550426","heartbeat_back_addr":"172.21.15.107:6831/2659550426","heartbeat_front_addr":"172.21.15.107:6829/2659550426","state":["exists","up"]},{"osd":11,"uuid":"94a1751d-b00c-4e23-815d-3a8bc3dea146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6825","nonce":1633548683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6827","nonce":1633548683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6831","nonce":1633548683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6829","nonce":1633548683}]},"public_addr":"172.21.15.111:6825/1633548683","cluster_addr":"172.21.15.111:6827/1633548683","heartbeat_back_addr":"172.21.15.111:6831/1633548683","heartbeat_front_addr":"172.21.15.111:6829/1633548683","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:36.359852+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:58.014202+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:19.753672+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:43.404817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:06.001677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:29.361587+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:52.554847+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:15.957553+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:39.581093+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:04.911389+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:28.971280+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:53.473561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/2098378667":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/3536229016":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6801/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/2684687547":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6800/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/368410612":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/12158160":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6800/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6801/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/3754423379":"2024-09-14T16:21:40.126459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:30:30.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.300+0000 7f0a617bc640 1 -- 172.21.15.16:0/239781287 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f0a480a1120 msgr2=0x7f0a480a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:30.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.300+0000 7f0a617bc640 1 --2- 172.21.15.16:0/239781287 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f0a480a1120 0x7f0a480a35e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c0ffd20 tx=0x7f0a4c009630 comp rx=0 tx=0).stop 2024-09-13T16:30:30.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.300+0000 7f0a617bc640 1 -- 172.21.15.16:0/239781287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a5c100880 msgr2=0x7f0a5c19f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:30.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.300+0000 7f0a617bc640 1 --2- 172.21.15.16:0/239781287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a5c100880 0x7f0a5c19f470 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009dc0 tx=0x7f0a54030f20 comp rx=0 tx=0).stop 2024-09-13T16:30:30.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.300+0000 7f0a617bc640 1 -- 172.21.15.16:0/239781287 shutdown_connections 2024-09-13T16:30:30.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.300+0000 7f0a617bc640 1 --2- 172.21.15.16:0/239781287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a5c100880 0x7f0a5c19f470 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:30.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.300+0000 7f0a617bc640 1 --2- 172.21.15.16:0/239781287 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f0a480a1120 0x7f0a480a35e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:30.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.300+0000 7f0a617bc640 1 --2- 172.21.15.16:0/239781287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a5c0ffec0 0x7f0a5c198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:30.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.300+0000 7f0a617bc640 1 --2- 172.21.15.16:0/239781287 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a5c0fecd0 0x7f0a5c197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:30.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.300+0000 7f0a617bc640 1 -- 172.21.15.16:0/239781287 >> 172.21.15.16:0/239781287 conn(0x7f0a5c0fa430 msgr2=0x7f0a5c1056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:30.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.300+0000 7f0a617bc640 1 -- 172.21.15.16:0/239781287 shutdown_connections 2024-09-13T16:30:30.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.300+0000 7f0a617bc640 1 -- 172.21.15.16:0/239781287 wait complete. 2024-09-13T16:30:30.315 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:24:22.981019+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T16:28:37.746027+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T16:28:38.602731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T16:28:41.633322+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T16:30:30.316 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:30:30.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.421+0000 7f6773733640 1 -- 172.21.15.16:0/255881216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c0ffe50 msgr2=0x7f676c1002d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:30.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.421+0000 7f6773733640 1 --2- 172.21.15.16:0/255881216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c0ffe50 0x7f676c1002d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f675c009900 tx=0x7f675c02a410 comp rx=0 tx=0).stop 2024-09-13T16:30:30.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.421+0000 7f6773733640 1 -- 172.21.15.16:0/255881216 shutdown_connections 2024-09-13T16:30:30.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.421+0000 7f6773733640 1 --2- 172.21.15.16:0/255881216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f676c100810 0x7f676c1070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:30.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.421+0000 7f6773733640 1 --2- 172.21.15.16:0/255881216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c0ffe50 0x7f676c1002d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:30.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.421+0000 7f6773733640 1 --2- 172.21.15.16:0/255881216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f676c0fec60 0x7f676c0ff060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:30.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.421+0000 7f6773733640 1 -- 172.21.15.16:0/255881216 >> 172.21.15.16:0/255881216 conn(0x7f676c0fa3e0 msgr2=0x7f676c0fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:30.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.421+0000 7f6773733640 1 -- 172.21.15.16:0/255881216 shutdown_connections 2024-09-13T16:30:30.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.421+0000 7f6773733640 1 -- 172.21.15.16:0/255881216 wait complete. 2024-09-13T16:30:30.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.422+0000 7f6773733640 1 Processor -- start 2024-09-13T16:30:30.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.422+0000 7f6773733640 1 -- start start 2024-09-13T16:30:30.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.422+0000 7f6773733640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f676c0fec60 0x7f676c195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:30.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.422+0000 7f6773733640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c0ffe50 0x7f676c1962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:30.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.422+0000 7f6773733640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f676c100810 0x7f676c19d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:30.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.422+0000 7f6773733640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c19d850 con 0x7f676c0ffe50 2024-09-13T16:30:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.422+0000 7f6773733640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c19d9c0 con 0x7f676c100810 2024-09-13T16:30:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.422+0000 7f6773733640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c19db80 con 0x7f676c0fec60 2024-09-13T16:30:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.422+0000 7f6770cce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c0ffe50 0x7f676c1962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.422+0000 7f6770cce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c0ffe50 0x7f676c1962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57752/0 (socket says 172.21.15.16:57752) 2024-09-13T16:30:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.422+0000 7f6770cce640 1 -- 172.21.15.16:0/2894117226 learned_addr learned my addr 172.21.15.16:0/2894117226 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.422+0000 7f67714cf640 1 --2- 172.21.15.16:0/2894117226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f676c0fec60 0x7f676c195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.422+0000 7f6771cd0640 1 --2- 172.21.15.16:0/2894117226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f676c100810 0x7f676c19d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.423+0000 7f6770cce640 1 -- 172.21.15.16:0/2894117226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f676c0fec60 msgr2=0x7f676c195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.423+0000 7f6770cce640 1 --2- 172.21.15.16:0/2894117226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f676c0fec60 0x7f676c195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.423+0000 7f6770cce640 1 -- 172.21.15.16:0/2894117226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f676c100810 msgr2=0x7f676c19d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.423+0000 7f6770cce640 1 --2- 172.21.15.16:0/2894117226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f676c100810 0x7f676c19d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.423+0000 7f6770cce640 1 -- 172.21.15.16:0/2894117226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f675c009560 con 0x7f676c0ffe50 2024-09-13T16:30:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.423+0000 7f6770cce640 1 --2- 172.21.15.16:0/2894117226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c0ffe50 0x7f676c1962a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f675c005ab0 tx=0x7f675c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.423+0000 7f67627fc640 1 -- 172.21.15.16:0/2894117226 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c031d70 con 0x7f676c0ffe50 2024-09-13T16:30:30.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.423+0000 7f67627fc640 1 -- 172.21.15.16:0/2894117226 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f675c005470 con 0x7f676c0ffe50 2024-09-13T16:30:30.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.423+0000 7f6773733640 1 -- 172.21.15.16:0/2894117226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f676c19de00 con 0x7f676c0ffe50 2024-09-13T16:30:30.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.423+0000 7f67627fc640 1 -- 172.21.15.16:0/2894117226 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c003a00 con 0x7f676c0ffe50 2024-09-13T16:30:30.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.423+0000 7f6773733640 1 -- 172.21.15.16:0/2894117226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f676c19e270 con 0x7f676c0ffe50 2024-09-13T16:30:30.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.424+0000 7f6773733640 1 -- 172.21.15.16:0/2894117226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f676c10a3d0 con 0x7f676c0ffe50 2024-09-13T16:30:30.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.426+0000 7f67627fc640 1 -- 172.21.15.16:0/2894117226 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f675c003b60 con 0x7f676c0ffe50 2024-09-13T16:30:30.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.427+0000 7f67627fc640 1 --2- 172.21.15.16:0/2894117226 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f67580a11a0 0x7f67580a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:30.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.427+0000 7f67627fc640 1 -- 172.21.15.16:0/2894117226 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f675c0ee130 con 0x7f676c0ffe50 2024-09-13T16:30:30.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.427+0000 7f67714cf640 1 --2- 172.21.15.16:0/2894117226 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f67580a11a0 0x7f67580a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:30.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.428+0000 7f67714cf640 1 --2- 172.21.15.16:0/2894117226 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f67580a11a0 0x7f67580a3660 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f676c0ffcb0 tx=0x7f6768005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:30.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.430+0000 7f67627fc640 1 -- 172.21.15.16:0/2894117226 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f675c0b7460 con 0x7f676c0ffe50 2024-09-13T16:30:30.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.602+0000 7f6773733640 1 -- 172.21.15.16:0/2894117226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f676c19e690 con 0x7f676c0ffe50 2024-09-13T16:30:30.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.603+0000 7f67627fc640 1 -- 172.21.15.16:0/2894117226 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f675c04a0a0 con 0x7f676c0ffe50 2024-09-13T16:30:30.604 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-13T16:30:30.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.605+0000 7f6773733640 1 -- 172.21.15.16:0/2894117226 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f67580a11a0 msgr2=0x7f67580a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:30.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.605+0000 7f6773733640 1 --2- 172.21.15.16:0/2894117226 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f67580a11a0 0x7f67580a3660 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f676c0ffcb0 tx=0x7f6768005f90 comp rx=0 tx=0).stop 2024-09-13T16:30:30.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.605+0000 7f6773733640 1 -- 172.21.15.16:0/2894117226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c0ffe50 msgr2=0x7f676c1962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:30.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.605+0000 7f6773733640 1 --2- 172.21.15.16:0/2894117226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c0ffe50 0x7f676c1962a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f675c005ab0 tx=0x7f675c0042f0 comp rx=0 tx=0).stop 2024-09-13T16:30:30.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.605+0000 7f6773733640 1 -- 172.21.15.16:0/2894117226 shutdown_connections 2024-09-13T16:30:30.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.605+0000 7f6773733640 1 --2- 172.21.15.16:0/2894117226 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f67580a11a0 0x7f67580a3660 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:30.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.605+0000 7f6773733640 1 --2- 172.21.15.16:0/2894117226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f676c100810 0x7f676c19d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:30.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.605+0000 7f6773733640 1 --2- 172.21.15.16:0/2894117226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f676c0ffe50 0x7f676c1962a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:30.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.605+0000 7f6773733640 1 --2- 172.21.15.16:0/2894117226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f676c0fec60 0x7f676c195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:30.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.605+0000 7f6773733640 1 -- 172.21.15.16:0/2894117226 >> 172.21.15.16:0/2894117226 conn(0x7f676c0fa3e0 msgr2=0x7f676c067630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:30.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.605+0000 7f6773733640 1 -- 172.21.15.16:0/2894117226 shutdown_connections 2024-09-13T16:30:30.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.605+0000 7f6773733640 1 -- 172.21.15.16:0/2894117226 wait complete. 2024-09-13T16:30:30.617 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T16:30:30.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.723+0000 7f21d5b83640 1 -- 172.21.15.16:0/3569247606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d00ffe50 msgr2=0x7f21d01002d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:30.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.723+0000 7f21d5b83640 1 --2- 172.21.15.16:0/3569247606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d00ffe50 0x7f21d01002d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f21c4009900 tx=0x7f21c402a410 comp rx=0 tx=0).stop 2024-09-13T16:30:30.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.723+0000 7f21d5b83640 1 -- 172.21.15.16:0/3569247606 shutdown_connections 2024-09-13T16:30:30.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.723+0000 7f21d5b83640 1 --2- 172.21.15.16:0/3569247606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21d0100810 0x7f21d01070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:30.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.723+0000 7f21d5b83640 1 --2- 172.21.15.16:0/3569247606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d00ffe50 0x7f21d01002d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:30.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.723+0000 7f21d5b83640 1 --2- 172.21.15.16:0/3569247606 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21d00fec60 0x7f21d00ff060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:30.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.723+0000 7f21d5b83640 1 -- 172.21.15.16:0/3569247606 >> 172.21.15.16:0/3569247606 conn(0x7f21d00fa3e0 msgr2=0x7f21d00fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:30.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.723+0000 7f21d5b83640 1 -- 172.21.15.16:0/3569247606 shutdown_connections 2024-09-13T16:30:30.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.723+0000 7f21d5b83640 1 -- 172.21.15.16:0/3569247606 wait complete. 2024-09-13T16:30:30.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.724+0000 7f21d5b83640 1 Processor -- start 2024-09-13T16:30:30.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.724+0000 7f21d5b83640 1 -- start start 2024-09-13T16:30:30.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.724+0000 7f21d5b83640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21d00fec60 0x7f21d0195bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:30.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.724+0000 7f21d5b83640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21d00ffe50 0x7f21d0196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:30.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.724+0000 7f21d5b83640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d0100810 0x7f21d019d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:30.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.724+0000 7f21d5b83640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d019d6e0 con 0x7f21d0100810 2024-09-13T16:30:30.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.724+0000 7f21d5b83640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d019d850 con 0x7f21d00ffe50 2024-09-13T16:30:30.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.724+0000 7f21d5b83640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d019da10 con 0x7f21d00fec60 2024-09-13T16:30:30.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.724+0000 7f21cffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d0100810 0x7f21d019d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:30.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.724+0000 7f21cffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d0100810 0x7f21d019d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57782/0 (socket says 172.21.15.16:57782) 2024-09-13T16:30:30.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.724+0000 7f21cffff640 1 -- 172.21.15.16:0/352923544 learned_addr learned my addr 172.21.15.16:0/352923544 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:30.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.724+0000 7f21ceffd640 1 --2- 172.21.15.16:0/352923544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21d00ffe50 0x7f21d0196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:30.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.725+0000 7f21cf7fe640 1 --2- 172.21.15.16:0/352923544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21d00fec60 0x7f21d0195bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:30.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.725+0000 7f21cffff640 1 -- 172.21.15.16:0/352923544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21d00fec60 msgr2=0x7f21d0195bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:30.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.725+0000 7f21cffff640 1 --2- 172.21.15.16:0/352923544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21d00fec60 0x7f21d0195bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:30.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.725+0000 7f21cffff640 1 -- 172.21.15.16:0/352923544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21d00ffe50 msgr2=0x7f21d0196130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:30.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.725+0000 7f21cffff640 1 --2- 172.21.15.16:0/352923544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21d00ffe50 0x7f21d0196130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:30.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.725+0000 7f21cffff640 1 -- 172.21.15.16:0/352923544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c4009560 con 0x7f21d0100810 2024-09-13T16:30:30.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.725+0000 7f21cffff640 1 --2- 172.21.15.16:0/352923544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d0100810 0x7f21d019d1a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f21c800eaa0 tx=0x7f21c800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:30.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.725+0000 7f21ccff9640 1 -- 172.21.15.16:0/352923544 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c800cdb0 con 0x7f21d0100810 2024-09-13T16:30:30.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.725+0000 7f21ccff9640 1 -- 172.21.15.16:0/352923544 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f21c8005020 con 0x7f21d0100810 2024-09-13T16:30:30.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.725+0000 7f21d5b83640 1 -- 172.21.15.16:0/352923544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d019dcf0 con 0x7f21d0100810 2024-09-13T16:30:30.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.725+0000 7f21ccff9640 1 -- 172.21.15.16:0/352923544 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c80116d0 con 0x7f21d0100810 2024-09-13T16:30:30.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.725+0000 7f21d5b83640 1 -- 172.21.15.16:0/352923544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d019e180 con 0x7f21d0100810 2024-09-13T16:30:30.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.726+0000 7f21d5b83640 1 -- 172.21.15.16:0/352923544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d0106190 con 0x7f21d0100810 2024-09-13T16:30:30.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.727+0000 7f21ccff9640 1 -- 172.21.15.16:0/352923544 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f21c8005190 con 0x7f21d0100810 2024-09-13T16:30:30.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.728+0000 7f21ccff9640 1 --2- 172.21.15.16:0/352923544 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f21b80a1120 0x7f21b80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:30.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.728+0000 7f21ccff9640 1 -- 172.21.15.16:0/352923544 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f21c80ccfb0 con 0x7f21d0100810 2024-09-13T16:30:30.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.728+0000 7f21cf7fe640 1 --2- 172.21.15.16:0/352923544 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f21b80a1120 0x7f21b80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:30.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.728+0000 7f21cf7fe640 1 --2- 172.21.15.16:0/352923544 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f21b80a1120 0x7f21b80a35e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f21c00058c0 tx=0x7f21c0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:30.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.732+0000 7f21ccff9640 1 -- 172.21.15.16:0/352923544 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f21c8100e20 con 0x7f21d0100810 2024-09-13T16:30:30.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.903+0000 7f21d5b83640 1 -- 172.21.15.16:0/352923544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f21d002c410 con 0x7f21d0100810 2024-09-13T16:30:30.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.904+0000 7f21ccff9640 1 -- 172.21.15.16:0/352923544 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f21c801f0a0 con 0x7f21d0100810 2024-09-13T16:30:30.905 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-13T16:30:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.906+0000 7f21d5b83640 1 -- 172.21.15.16:0/352923544 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f21b80a1120 msgr2=0x7f21b80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.906+0000 7f21d5b83640 1 --2- 172.21.15.16:0/352923544 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f21b80a1120 0x7f21b80a35e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f21c00058c0 tx=0x7f21c0005850 comp rx=0 tx=0).stop 2024-09-13T16:30:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.906+0000 7f21d5b83640 1 -- 172.21.15.16:0/352923544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d0100810 msgr2=0x7f21d019d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.906+0000 7f21d5b83640 1 --2- 172.21.15.16:0/352923544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d0100810 0x7f21d019d1a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f21c800eaa0 tx=0x7f21c800ef60 comp rx=0 tx=0).stop 2024-09-13T16:30:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.906+0000 7f21d5b83640 1 -- 172.21.15.16:0/352923544 shutdown_connections 2024-09-13T16:30:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.906+0000 7f21d5b83640 1 --2- 172.21.15.16:0/352923544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d0100810 0x7f21d019d1a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.906+0000 7f21d5b83640 1 --2- 172.21.15.16:0/352923544 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f21b80a1120 0x7f21b80a35e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.906+0000 7f21d5b83640 1 --2- 172.21.15.16:0/352923544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21d00ffe50 0x7f21d0196130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.906+0000 7f21d5b83640 1 --2- 172.21.15.16:0/352923544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21d00fec60 0x7f21d0195bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.906+0000 7f21d5b83640 1 -- 172.21.15.16:0/352923544 >> 172.21.15.16:0/352923544 conn(0x7f21d00fa3e0 msgr2=0x7f21d0103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.906+0000 7f21d5b83640 1 -- 172.21.15.16:0/352923544 shutdown_connections 2024-09-13T16:30:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:30.906+0000 7f21d5b83640 1 -- 172.21.15.16:0/352923544 wait complete. 2024-09-13T16:30:30.917 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:30 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/239781287' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:30:30.917 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:30 smithi016 ceph-mon[31187]: pgmap v336: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T16:30:30.917 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:30 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2894117226' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:30:30.918 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T16:30:30.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:30 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/239781287' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:30:30.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:30 smithi111 ceph-mon[34600]: pgmap v336: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T16:30:30.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:30 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/2894117226' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:30:31.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.023+0000 7fc8b1da3640 1 -- 172.21.15.16:0/629389149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8ac0ffda0 msgr2=0x7fc8ac1001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:31.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.023+0000 7fc8b1da3640 1 --2- 172.21.15.16:0/629389149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8ac0ffda0 0x7fc8ac1001a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc89c009900 tx=0x7fc89c02a410 comp rx=0 tx=0).stop 2024-09-13T16:30:31.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.023+0000 7fc8b1da3640 1 -- 172.21.15.16:0/629389149 shutdown_connections 2024-09-13T16:30:31.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.023+0000 7fc8b1da3640 1 --2- 172.21.15.16:0/629389149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8ac101950 0x7fc8ac1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.023+0000 7fc8b1da3640 1 --2- 172.21.15.16:0/629389149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8ac100f90 0x7fc8ac101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.023+0000 7fc8b1da3640 1 --2- 172.21.15.16:0/629389149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8ac0ffda0 0x7fc8ac1001a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.023+0000 7fc8b1da3640 1 -- 172.21.15.16:0/629389149 >> 172.21.15.16:0/629389149 conn(0x7fc8ac0fb520 msgr2=0x7fc8ac0fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:31.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.023+0000 7fc8b1da3640 1 -- 172.21.15.16:0/629389149 shutdown_connections 2024-09-13T16:30:31.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.023+0000 7fc8b1da3640 1 -- 172.21.15.16:0/629389149 wait complete. 2024-09-13T16:30:31.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.023+0000 7fc8b1da3640 1 Processor -- start 2024-09-13T16:30:31.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.023+0000 7fc8b1da3640 1 -- start start 2024-09-13T16:30:31.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.024+0000 7fc8b1da3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8ac0ffda0 0x7fc8ac195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:31.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.024+0000 7fc8b1da3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8ac100f90 0x7fc8ac196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:31.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.024+0000 7fc8b1da3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8ac101950 0x7fc8ac19d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:31.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.024+0000 7fc8b1da3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac19d730 con 0x7fc8ac0ffda0 2024-09-13T16:30:31.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.024+0000 7fc8b1da3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac19d8a0 con 0x7fc8ac100f90 2024-09-13T16:30:31.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.024+0000 7fc8b1da3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac19da60 con 0x7fc8ac101950 2024-09-13T16:30:31.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.024+0000 7fc8ab7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8ac0ffda0 0x7fc8ac195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:31.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.024+0000 7fc8ab7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8ac0ffda0 0x7fc8ac195c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57788/0 (socket says 172.21.15.16:57788) 2024-09-13T16:30:31.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.024+0000 7fc8ab7fe640 1 -- 172.21.15.16:0/2587700902 learned_addr learned my addr 172.21.15.16:0/2587700902 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:31.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.024+0000 7fc8aaffd640 1 --2- 172.21.15.16:0/2587700902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8ac100f90 0x7fc8ac196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:31.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.024+0000 7fc8abfff640 1 --2- 172.21.15.16:0/2587700902 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8ac101950 0x7fc8ac19d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:31.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.024+0000 7fc8ab7fe640 1 -- 172.21.15.16:0/2587700902 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8ac101950 msgr2=0x7fc8ac19d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:31.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.024+0000 7fc8ab7fe640 1 --2- 172.21.15.16:0/2587700902 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8ac101950 0x7fc8ac19d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.024+0000 7fc8ab7fe640 1 -- 172.21.15.16:0/2587700902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8ac100f90 msgr2=0x7fc8ac196190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:31.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.024+0000 7fc8ab7fe640 1 --2- 172.21.15.16:0/2587700902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8ac100f90 0x7fc8ac196190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.024+0000 7fc8ab7fe640 1 -- 172.21.15.16:0/2587700902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89c009560 con 0x7fc8ac0ffda0 2024-09-13T16:30:31.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.024+0000 7fc8ab7fe640 1 --2- 172.21.15.16:0/2587700902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8ac0ffda0 0x7fc8ac195c50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc89c005ab0 tx=0x7fc89c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:31.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.025+0000 7fc8a8ff9640 1 -- 172.21.15.16:0/2587700902 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c031d70 con 0x7fc8ac0ffda0 2024-09-13T16:30:31.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.025+0000 7fc8a8ff9640 1 -- 172.21.15.16:0/2587700902 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc89c0044e0 con 0x7fc8ac0ffda0 2024-09-13T16:30:31.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.025+0000 7fc8b1da3640 1 -- 172.21.15.16:0/2587700902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8ac19dce0 con 0x7fc8ac0ffda0 2024-09-13T16:30:31.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.025+0000 7fc8a8ff9640 1 -- 172.21.15.16:0/2587700902 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c003a00 con 0x7fc8ac0ffda0 2024-09-13T16:30:31.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.025+0000 7fc8b1da3640 1 -- 172.21.15.16:0/2587700902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8ac19e190 con 0x7fc8ac0ffda0 2024-09-13T16:30:31.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.026+0000 7fc8b1da3640 1 -- 172.21.15.16:0/2587700902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8ac1072d0 con 0x7fc8ac0ffda0 2024-09-13T16:30:31.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.027+0000 7fc8a8ff9640 1 -- 172.21.15.16:0/2587700902 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc89c003b60 con 0x7fc8ac0ffda0 2024-09-13T16:30:31.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.027+0000 7fc8a8ff9640 1 --2- 172.21.15.16:0/2587700902 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc8940a1120 0x7fc8940a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:31.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.028+0000 7fc8a8ff9640 1 -- 172.21.15.16:0/2587700902 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc89c0ee380 con 0x7fc8ac0ffda0 2024-09-13T16:30:31.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.028+0000 7fc8aaffd640 1 --2- 172.21.15.16:0/2587700902 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc8940a1120 0x7fc8940a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:31.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.028+0000 7fc8aaffd640 1 --2- 172.21.15.16:0/2587700902 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc8940a1120 0x7fc8940a35e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0005d00 tx=0x7fc8a000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:31.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.031+0000 7fc8a8ff9640 1 -- 172.21.15.16:0/2587700902 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc89c0b7570 con 0x7fc8ac0ffda0 2024-09-13T16:30:31.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:30 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/239781287' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:30:31.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:30 smithi107 ceph-mon[34139]: pgmap v336: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T16:30:31.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:30 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/2894117226' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:30:31.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.201+0000 7fc8b1da3640 1 -- 172.21.15.16:0/2587700902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc8ac02c3e0 con 0x7fc8ac0ffda0 2024-09-13T16:30:31.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.202+0000 7fc8a8ff9640 1 -- 172.21.15.16:0/2587700902 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc89c03f170 con 0x7fc8ac0ffda0 2024-09-13T16:30:31.203 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-09-13T16:30:31.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.204+0000 7fc8b1da3640 1 -- 172.21.15.16:0/2587700902 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc8940a1120 msgr2=0x7fc8940a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:31.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.204+0000 7fc8b1da3640 1 --2- 172.21.15.16:0/2587700902 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc8940a1120 0x7fc8940a35e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0005d00 tx=0x7fc8a000b410 comp rx=0 tx=0).stop 2024-09-13T16:30:31.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.204+0000 7fc8b1da3640 1 -- 172.21.15.16:0/2587700902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8ac0ffda0 msgr2=0x7fc8ac195c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:31.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.204+0000 7fc8b1da3640 1 --2- 172.21.15.16:0/2587700902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8ac0ffda0 0x7fc8ac195c50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc89c005ab0 tx=0x7fc89c0042f0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.204+0000 7fc8b1da3640 1 -- 172.21.15.16:0/2587700902 shutdown_connections 2024-09-13T16:30:31.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.204+0000 7fc8b1da3640 1 --2- 172.21.15.16:0/2587700902 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8ac101950 0x7fc8ac19d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.204+0000 7fc8b1da3640 1 --2- 172.21.15.16:0/2587700902 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc8940a1120 0x7fc8940a35e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.204+0000 7fc8b1da3640 1 --2- 172.21.15.16:0/2587700902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8ac100f90 0x7fc8ac196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.204+0000 7fc8b1da3640 1 --2- 172.21.15.16:0/2587700902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8ac0ffda0 0x7fc8ac195c50 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.204+0000 7fc8b1da3640 1 -- 172.21.15.16:0/2587700902 >> 172.21.15.16:0/2587700902 conn(0x7fc8ac0fb520 msgr2=0x7fc8ac1044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:31.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.204+0000 7fc8b1da3640 1 -- 172.21.15.16:0/2587700902 shutdown_connections 2024-09-13T16:30:31.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.204+0000 7fc8b1da3640 1 -- 172.21.15.16:0/2587700902 wait complete. 2024-09-13T16:30:31.217 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T16:30:31.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.321+0000 7f4366d83640 1 -- 172.21.15.16:0/1491823807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43600fef90 msgr2=0x7f43600ff3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:31.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.321+0000 7f4366d83640 1 --2- 172.21.15.16:0/1491823807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43600fef90 0x7f43600ff3f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f434c009900 tx=0x7f434c02a480 comp rx=0 tx=0).stop 2024-09-13T16:30:31.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.321+0000 7f4366d83640 1 -- 172.21.15.16:0/1491823807 shutdown_connections 2024-09-13T16:30:31.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.321+0000 7f4366d83640 1 --2- 172.21.15.16:0/1491823807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43600ff930 0x7f43600ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.321+0000 7f4366d83640 1 --2- 172.21.15.16:0/1491823807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43600fef90 0x7f43600ff3f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.321+0000 7f4366d83640 1 --2- 172.21.15.16:0/1491823807 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43600fe650 0x7f43600fea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.321+0000 7f4366d83640 1 -- 172.21.15.16:0/1491823807 >> 172.21.15.16:0/1491823807 conn(0x7f43600fa3e0 msgr2=0x7f43600fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:31.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.321+0000 7f4366d83640 1 -- 172.21.15.16:0/1491823807 shutdown_connections 2024-09-13T16:30:31.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.321+0000 7f4366d83640 1 -- 172.21.15.16:0/1491823807 wait complete. 2024-09-13T16:30:31.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.321+0000 7f4366d83640 1 Processor -- start 2024-09-13T16:30:31.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.321+0000 7f4366d83640 1 -- start start 2024-09-13T16:30:31.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.321+0000 7f4366d83640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43600fe650 0x7f4360197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:31.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.321+0000 7f4366d83640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43600fef90 0x7f4360198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:31.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.321+0000 7f4366d83640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43600ff930 0x7f436019f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:31.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.321+0000 7f4366d83640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436019f8e0 con 0x7f43600ff930 2024-09-13T16:30:31.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.321+0000 7f4366d83640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436019fa50 con 0x7f43600fef90 2024-09-13T16:30:31.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.321+0000 7f4366d83640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436019fc10 con 0x7f43600fe650 2024-09-13T16:30:31.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.322+0000 7f4365320640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43600ff930 0x7f436019f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:31.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.322+0000 7f4365320640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43600ff930 0x7f436019f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57794/0 (socket says 172.21.15.16:57794) 2024-09-13T16:30:31.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.322+0000 7f4365320640 1 -- 172.21.15.16:0/1649227844 learned_addr learned my addr 172.21.15.16:0/1649227844 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:31.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.322+0000 7f4364b1f640 1 --2- 172.21.15.16:0/1649227844 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43600fe650 0x7f4360197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:31.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.322+0000 7f435ffff640 1 --2- 172.21.15.16:0/1649227844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43600fef90 0x7f4360198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:31.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.322+0000 7f4365320640 1 -- 172.21.15.16:0/1649227844 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43600fe650 msgr2=0x7f4360197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:31.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.322+0000 7f4365320640 1 --2- 172.21.15.16:0/1649227844 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43600fe650 0x7f4360197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.322+0000 7f4365320640 1 -- 172.21.15.16:0/1649227844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43600fef90 msgr2=0x7f4360198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:31.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.322+0000 7f4365320640 1 --2- 172.21.15.16:0/1649227844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43600fef90 0x7f4360198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.322+0000 7f4365320640 1 -- 172.21.15.16:0/1649227844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f434c009560 con 0x7f43600ff930 2024-09-13T16:30:31.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.322+0000 7f4364b1f640 1 --2- 172.21.15.16:0/1649227844 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43600fe650 0x7f4360197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:30:31.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.322+0000 7f4365320640 1 --2- 172.21.15.16:0/1649227844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43600ff930 0x7f436019f3a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f435800cce0 tx=0x7f435800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:31.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.322+0000 7f435dffb640 1 -- 172.21.15.16:0/1649227844 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435800f450 con 0x7f43600ff930 2024-09-13T16:30:31.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.322+0000 7f435dffb640 1 -- 172.21.15.16:0/1649227844 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f435800fa40 con 0x7f43600ff930 2024-09-13T16:30:31.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.322+0000 7f4366d83640 1 -- 172.21.15.16:0/1649227844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f436019fef0 con 0x7f43600ff930 2024-09-13T16:30:31.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.322+0000 7f435dffb640 1 -- 172.21.15.16:0/1649227844 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43580189a0 con 0x7f43600ff930 2024-09-13T16:30:31.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.322+0000 7f4366d83640 1 -- 172.21.15.16:0/1649227844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43601a0320 con 0x7f43600ff930 2024-09-13T16:30:31.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.323+0000 7f4366d83640 1 -- 172.21.15.16:0/1649227844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43601093a0 con 0x7f43600ff930 2024-09-13T16:30:31.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.324+0000 7f435dffb640 1 -- 172.21.15.16:0/1649227844 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f435800fbb0 con 0x7f43600ff930 2024-09-13T16:30:31.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.325+0000 7f435dffb640 1 --2- 172.21.15.16:0/1649227844 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f43500a1120 0x7f43500a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:31.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.325+0000 7f435dffb640 1 -- 172.21.15.16:0/1649227844 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f43580cd2e0 con 0x7f43600ff930 2024-09-13T16:30:31.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.325+0000 7f4364b1f640 1 --2- 172.21.15.16:0/1649227844 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f43500a1120 0x7f43500a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:31.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.326+0000 7f4364b1f640 1 --2- 172.21.15.16:0/1649227844 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f43500a1120 0x7f43500a35e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f435400abe0 tx=0x7f435400a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:31.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.329+0000 7f435dffb640 1 -- 172.21.15.16:0/1649227844 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f43580965c0 con 0x7f43600ff930 2024-09-13T16:30:31.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.501+0000 7f4366d83640 1 -- 172.21.15.16:0/1649227844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f436002c410 con 0x7f43600ff930 2024-09-13T16:30:31.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.501+0000 7f435dffb640 1 -- 172.21.15.16:0/1649227844 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4358099be0 con 0x7f43600ff930 2024-09-13T16:30:31.502 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-13T16:30:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.503+0000 7f4366d83640 1 -- 172.21.15.16:0/1649227844 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f43500a1120 msgr2=0x7f43500a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.503+0000 7f4366d83640 1 --2- 172.21.15.16:0/1649227844 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f43500a1120 0x7f43500a35e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f435400abe0 tx=0x7f435400a5c0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.503+0000 7f4366d83640 1 -- 172.21.15.16:0/1649227844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43600ff930 msgr2=0x7f436019f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.503+0000 7f4366d83640 1 --2- 172.21.15.16:0/1649227844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43600ff930 0x7f436019f3a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f435800cce0 tx=0x7f435800a620 comp rx=0 tx=0).stop 2024-09-13T16:30:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.503+0000 7f4366d83640 1 -- 172.21.15.16:0/1649227844 shutdown_connections 2024-09-13T16:30:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.503+0000 7f4366d83640 1 --2- 172.21.15.16:0/1649227844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43600ff930 0x7f436019f3a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.503+0000 7f4366d83640 1 --2- 172.21.15.16:0/1649227844 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f43500a1120 0x7f43500a35e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.503+0000 7f4366d83640 1 --2- 172.21.15.16:0/1649227844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43600fef90 0x7f4360198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.503+0000 7f4366d83640 1 --2- 172.21.15.16:0/1649227844 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43600fe650 0x7f4360197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.503+0000 7f4366d83640 1 -- 172.21.15.16:0/1649227844 >> 172.21.15.16:0/1649227844 conn(0x7f43600fa3e0 msgr2=0x7f43600fbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.503+0000 7f4366d83640 1 -- 172.21.15.16:0/1649227844 shutdown_connections 2024-09-13T16:30:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.503+0000 7f4366d83640 1 -- 172.21.15.16:0/1649227844 wait complete. 2024-09-13T16:30:31.515 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:30:31.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.616+0000 7faac411f640 1 -- 172.21.15.16:0/2053278992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faabc103240 msgr2=0x7faabc105630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:31.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.616+0000 7faac411f640 1 --2- 172.21.15.16:0/2053278992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faabc103240 0x7faabc105630 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7faaac009900 tx=0x7faaac02a410 comp rx=0 tx=0).stop 2024-09-13T16:30:31.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.616+0000 7faac411f640 1 -- 172.21.15.16:0/2053278992 shutdown_connections 2024-09-13T16:30:31.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.616+0000 7faac411f640 1 --2- 172.21.15.16:0/2053278992 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faabc105b70 0x7faabc107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.616+0000 7faac411f640 1 --2- 172.21.15.16:0/2053278992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faabc103240 0x7faabc105630 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.616+0000 7faac411f640 1 --2- 172.21.15.16:0/2053278992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faabc100910 0x7faabc102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.616+0000 7faac411f640 1 -- 172.21.15.16:0/2053278992 >> 172.21.15.16:0/2053278992 conn(0x7faabc0fa430 msgr2=0x7faabc0fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:31.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.616+0000 7faac411f640 1 -- 172.21.15.16:0/2053278992 shutdown_connections 2024-09-13T16:30:31.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.616+0000 7faac411f640 1 -- 172.21.15.16:0/2053278992 wait complete. 2024-09-13T16:30:31.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.617+0000 7faac411f640 1 Processor -- start 2024-09-13T16:30:31.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.617+0000 7faac411f640 1 -- start start 2024-09-13T16:30:31.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.617+0000 7faac411f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faabc100910 0x7faabc100280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:31.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.617+0000 7faac411f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faabc103240 0x7faabc0fe8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:31.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.617+0000 7faac411f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faabc105b70 0x7faabc0fee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:31.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.617+0000 7faac411f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faabc0ff560 con 0x7faabc100910 2024-09-13T16:30:31.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.617+0000 7faac411f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faabc0ff6d0 con 0x7faabc103240 2024-09-13T16:30:31.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.617+0000 7faac411f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faabc19fdd0 con 0x7faabc105b70 2024-09-13T16:30:31.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.617+0000 7faac1ebb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faabc100910 0x7faabc100280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.617+0000 7faac1ebb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faabc100910 0x7faabc100280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57814/0 (socket says 172.21.15.16:57814) 2024-09-13T16:30:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.617+0000 7faac1ebb640 1 -- 172.21.15.16:0/2088155589 learned_addr learned my addr 172.21.15.16:0/2088155589 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.617+0000 7faac16ba640 1 --2- 172.21.15.16:0/2088155589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faabc103240 0x7faabc0fe8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.617+0000 7faac26bc640 1 --2- 172.21.15.16:0/2088155589 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faabc105b70 0x7faabc0fee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.618+0000 7faac1ebb640 1 -- 172.21.15.16:0/2088155589 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faabc105b70 msgr2=0x7faabc0fee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.618+0000 7faac1ebb640 1 --2- 172.21.15.16:0/2088155589 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faabc105b70 0x7faabc0fee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.618+0000 7faac1ebb640 1 -- 172.21.15.16:0/2088155589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faabc103240 msgr2=0x7faabc0fe8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.618+0000 7faac1ebb640 1 --2- 172.21.15.16:0/2088155589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faabc103240 0x7faabc0fe8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.618+0000 7faac1ebb640 1 -- 172.21.15.16:0/2088155589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaac009560 con 0x7faabc100910 2024-09-13T16:30:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.618+0000 7faac26bc640 1 --2- 172.21.15.16:0/2088155589 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faabc105b70 0x7faabc0fee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:30:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.618+0000 7faac1ebb640 1 --2- 172.21.15.16:0/2088155589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faabc100910 0x7faabc100280 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7faab800c9a0 tx=0x7faab800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.618+0000 7faab2ffd640 1 -- 172.21.15.16:0/2088155589 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab8007bb0 con 0x7faabc100910 2024-09-13T16:30:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.618+0000 7faac411f640 1 -- 172.21.15.16:0/2088155589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faabc1a0080 con 0x7faabc100910 2024-09-13T16:30:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.618+0000 7faab2ffd640 1 -- 172.21.15.16:0/2088155589 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faab8007d10 con 0x7faabc100910 2024-09-13T16:30:31.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.618+0000 7faab2ffd640 1 -- 172.21.15.16:0/2088155589 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab8004de0 con 0x7faabc100910 2024-09-13T16:30:31.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.618+0000 7faac411f640 1 -- 172.21.15.16:0/2088155589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faabc1a0510 con 0x7faabc100910 2024-09-13T16:30:31.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.619+0000 7faac411f640 1 -- 172.21.15.16:0/2088155589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faabc0fbf20 con 0x7faabc100910 2024-09-13T16:30:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.620+0000 7faab2ffd640 1 -- 172.21.15.16:0/2088155589 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7faab8004f80 con 0x7faabc100910 2024-09-13T16:30:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.621+0000 7faab2ffd640 1 --2- 172.21.15.16:0/2088155589 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7faaa80a1120 0x7faaa80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.621+0000 7faac16ba640 1 --2- 172.21.15.16:0/2088155589 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7faaa80a1120 0x7faaa80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.621+0000 7faab2ffd640 1 -- 172.21.15.16:0/2088155589 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7faab80ccbb0 con 0x7faabc100910 2024-09-13T16:30:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.621+0000 7faac16ba640 1 --2- 172.21.15.16:0/2088155589 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7faaa80a1120 0x7faaa80a35e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7faaac0057c0 tx=0x7faaac005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:31.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.625+0000 7faab2ffd640 1 -- 172.21.15.16:0/2088155589 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7faab8095e30 con 0x7faabc100910 2024-09-13T16:30:31.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.834+0000 7faac411f640 1 -- 172.21.15.16:0/2088155589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faabc10b540 con 0x7faabc100910 2024-09-13T16:30:31.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.834+0000 7faab2ffd640 1 -- 172.21.15.16:0/2088155589 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10924 (secure 0 0 0) 0x7faab801d020 con 0x7faabc100910 2024-09-13T16:30:31.836 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:30:31.836 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3292781664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3292781664},{"type":"v1","addr":"172.21.15.16:6835","nonce":3292781664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4109039027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4109039027},{"type":"v1","addr":"172.21.15.16:6837","nonce":4109039027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T16:28:39.184734+0000","modified":"2024-09-13T16:29:06.636937+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24518,"mds_1":24490,"mds_2":24485,"mds_3":24422,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/2391618693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2391618693},{"type":"v1","addr":"172.21.15.16:6839","nonce":2391618693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/521042681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":521042681},{"type":"v1","addr":"172.21.15.16:6841","nonce":521042681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24418":{"gid":24418,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.111:6833/1984275936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1984275936},{"type":"v1","addr":"172.21.15.111:6833","nonce":1984275936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.107:6833/3797712690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3797712690},{"type":"v1","addr":"172.21.15.107:6833","nonce":3797712690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/338548468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":338548468},{"type":"v1","addr":"172.21.15.107:6835","nonce":338548468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/662156111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":662156111},{"type":"v1","addr":"172.21.15.111:6835","nonce":662156111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/1877128453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1877128453},{"type":"v1","addr":"172.21.15.111:6837","nonce":1877128453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.107:6837/2838457611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2838457611},{"type":"v1","addr":"172.21.15.107:6837","nonce":2838457611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.111:6839/1423842285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":1423842285},{"type":"v1","addr":"172.21.15.111:6839","nonce":1423842285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6839/1914689806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1914689806},{"type":"v1","addr":"172.21.15.107:6839","nonce":1914689806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T16:30:31.836 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-13T16:30:31.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.836+0000 7faac411f640 1 -- 172.21.15.16:0/2088155589 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7faaa80a1120 msgr2=0x7faaa80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:31.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.836+0000 7faac411f640 1 --2- 172.21.15.16:0/2088155589 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7faaa80a1120 0x7faaa80a35e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7faaac0057c0 tx=0x7faaac005a20 comp rx=0 tx=0).stop 2024-09-13T16:30:31.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.836+0000 7faac411f640 1 -- 172.21.15.16:0/2088155589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faabc100910 msgr2=0x7faabc100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:31.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.836+0000 7faac411f640 1 --2- 172.21.15.16:0/2088155589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faabc100910 0x7faabc100280 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7faab800c9a0 tx=0x7faab800ce60 comp rx=0 tx=0).stop 2024-09-13T16:30:31.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.836+0000 7faac411f640 1 -- 172.21.15.16:0/2088155589 shutdown_connections 2024-09-13T16:30:31.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.836+0000 7faac411f640 1 --2- 172.21.15.16:0/2088155589 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faabc105b70 0x7faabc0fee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.836+0000 7faac411f640 1 --2- 172.21.15.16:0/2088155589 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7faaa80a1120 0x7faaa80a35e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.836+0000 7faac411f640 1 --2- 172.21.15.16:0/2088155589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faabc103240 0x7faabc0fe8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.836+0000 7faac411f640 1 --2- 172.21.15.16:0/2088155589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faabc100910 0x7faabc100280 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.836+0000 7faac411f640 1 -- 172.21.15.16:0/2088155589 >> 172.21.15.16:0/2088155589 conn(0x7faabc0fa430 msgr2=0x7faabc101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:31.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.836+0000 7faac411f640 1 -- 172.21.15.16:0/2088155589 shutdown_connections 2024-09-13T16:30:31.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.837+0000 7faac411f640 1 -- 172.21.15.16:0/2088155589 wait complete. 2024-09-13T16:30:31.847 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:31 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/352923544' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T16:30:31.847 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:31 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2587700902' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T16:30:31.847 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:31 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1649227844' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T16:30:31.849 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:30:31.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:31 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/352923544' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T16:30:31.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:31 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/2587700902' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T16:30:31.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:31 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1649227844' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T16:30:31.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.949+0000 7fb92f8f7640 1 -- 172.21.15.16:0/4245150570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb928100fb0 msgr2=0x7fb928101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:31.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.949+0000 7fb92f8f7640 1 --2- 172.21.15.16:0/4245150570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb928100fb0 0x7fb928101430 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb918009900 tx=0x7fb91802a410 comp rx=0 tx=0).stop 2024-09-13T16:30:31.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.949+0000 7fb92f8f7640 1 -- 172.21.15.16:0/4245150570 shutdown_connections 2024-09-13T16:30:31.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.949+0000 7fb92f8f7640 1 --2- 172.21.15.16:0/4245150570 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb928101970 0x7fb928108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.949+0000 7fb92f8f7640 1 --2- 172.21.15.16:0/4245150570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb928100fb0 0x7fb928101430 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.949+0000 7fb92f8f7640 1 --2- 172.21.15.16:0/4245150570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9280ffdc0 0x7fb9281001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.949+0000 7fb92f8f7640 1 -- 172.21.15.16:0/4245150570 >> 172.21.15.16:0/4245150570 conn(0x7fb9280fb540 msgr2=0x7fb9280fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:31.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.949+0000 7fb92f8f7640 1 -- 172.21.15.16:0/4245150570 shutdown_connections 2024-09-13T16:30:31.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.949+0000 7fb92f8f7640 1 -- 172.21.15.16:0/4245150570 wait complete. 2024-09-13T16:30:31.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.950+0000 7fb92f8f7640 1 Processor -- start 2024-09-13T16:30:31.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.950+0000 7fb92f8f7640 1 -- start start 2024-09-13T16:30:31.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.950+0000 7fb92f8f7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9280ffdc0 0x7fb928074260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:31.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.950+0000 7fb92f8f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb928100fb0 0x7fb9280728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:31.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.950+0000 7fb92f8f7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb928101970 0x7fb928072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:31.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.950+0000 7fb92f8f7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb928073540 con 0x7fb928100fb0 2024-09-13T16:30:31.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.950+0000 7fb92f8f7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9280736b0 con 0x7fb9280ffdc0 2024-09-13T16:30:31.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.950+0000 7fb92f8f7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92819fd30 con 0x7fb928101970 2024-09-13T16:30:31.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.950+0000 7fb92ce92640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb928100fb0 0x7fb9280728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:31.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.950+0000 7fb92ce92640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb928100fb0 0x7fb9280728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57826/0 (socket says 172.21.15.16:57826) 2024-09-13T16:30:31.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.950+0000 7fb92ce92640 1 -- 172.21.15.16:0/2504903054 learned_addr learned my addr 172.21.15.16:0/2504903054 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:31.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.950+0000 7fb92d693640 1 --2- 172.21.15.16:0/2504903054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9280ffdc0 0x7fb928074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:31.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.950+0000 7fb92ce92640 1 -- 172.21.15.16:0/2504903054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb928101970 msgr2=0x7fb928072de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:31.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.950+0000 7fb92de94640 1 --2- 172.21.15.16:0/2504903054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb928101970 0x7fb928072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:31.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.950+0000 7fb92ce92640 1 --2- 172.21.15.16:0/2504903054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb928101970 0x7fb928072de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.950+0000 7fb92ce92640 1 -- 172.21.15.16:0/2504903054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9280ffdc0 msgr2=0x7fb928074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:31.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.950+0000 7fb92ce92640 1 --2- 172.21.15.16:0/2504903054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9280ffdc0 0x7fb928074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:31.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.950+0000 7fb92ce92640 1 -- 172.21.15.16:0/2504903054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb918009560 con 0x7fb928100fb0 2024-09-13T16:30:31.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.950+0000 7fb92d693640 1 --2- 172.21.15.16:0/2504903054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9280ffdc0 0x7fb928074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:30:31.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.951+0000 7fb92ce92640 1 --2- 172.21.15.16:0/2504903054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb928100fb0 0x7fb9280728a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb918005ab0 tx=0x7fb9180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:31.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.951+0000 7fb91e7fc640 1 -- 172.21.15.16:0/2504903054 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb918031d70 con 0x7fb928100fb0 2024-09-13T16:30:31.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.951+0000 7fb91e7fc640 1 -- 172.21.15.16:0/2504903054 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb9180044e0 con 0x7fb928100fb0 2024-09-13T16:30:31.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.951+0000 7fb92f8f7640 1 -- 172.21.15.16:0/2504903054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb92819ff80 con 0x7fb928100fb0 2024-09-13T16:30:31.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.951+0000 7fb91e7fc640 1 -- 172.21.15.16:0/2504903054 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb918003b80 con 0x7fb928100fb0 2024-09-13T16:30:31.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.951+0000 7fb92f8f7640 1 -- 172.21.15.16:0/2504903054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9281a03b0 con 0x7fb928100fb0 2024-09-13T16:30:31.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.952+0000 7fb92f8f7640 1 -- 172.21.15.16:0/2504903054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9281072f0 con 0x7fb928100fb0 2024-09-13T16:30:31.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.953+0000 7fb91e7fc640 1 -- 172.21.15.16:0/2504903054 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fb918003ce0 con 0x7fb928100fb0 2024-09-13T16:30:31.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.954+0000 7fb91e7fc640 1 --2- 172.21.15.16:0/2504903054 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb9140a11a0 0x7fb9140a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:31.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.954+0000 7fb92d693640 1 --2- 172.21.15.16:0/2504903054 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb9140a11a0 0x7fb9140a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:31.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.954+0000 7fb91e7fc640 1 -- 172.21.15.16:0/2504903054 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb9180ee210 con 0x7fb928100fb0 2024-09-13T16:30:31.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.954+0000 7fb92d693640 1 --2- 172.21.15.16:0/2504903054 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb9140a11a0 0x7fb9140a3660 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb928100e10 tx=0x7fb924005e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:31.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:31.958+0000 7fb91e7fc640 1 -- 172.21.15.16:0/2504903054 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb9180b74f0 con 0x7fb928100fb0 2024-09-13T16:30:32.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:31 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/352923544' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T16:30:32.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:31 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/2587700902' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T16:30:32.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:31 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1649227844' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T16:30:32.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.130+0000 7fb92f8f7640 1 -- 172.21.15.16:0/2504903054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb92802c480 con 0x7fb928100fb0 2024-09-13T16:30:32.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.131+0000 7fb91e7fc640 1 -- 172.21.15.16:0/2504903054 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24593 (secure 0 0 0) 0x7fb9180bab10 con 0x7fb928100fb0 2024-09-13T16:30:32.132 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:30:32.133 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":93,"fsid":"0d241f6a-71ec-11ef-bceb-c7b262605968","created":"2024-09-13T16:21:17.971738+0000","modified":"2024-09-13T16:30:05.681521+0000","last_up_change":"2024-09-13T16:27:54.673413+0000","last_in_change":"2024-09-13T16:27:35.718390+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:24:22.981019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:28:37.746027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:28:38.602731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:28:41.633322+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e11ef8f-08d6-4e74-afd6-f49f4bd40032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6803","nonce":2079394186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6805","nonce":2079394186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6809","nonce":2079394186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6807","nonce":2079394186}]},"public_addr":"172.21.15.16:6803/2079394186","cluster_addr":"172.21.15.16:6805/2079394186","heartbeat_back_addr":"172.21.15.16:6809/2079394186","heartbeat_front_addr":"172.21.15.16:6807/2079394186","state":["exists","up"]},{"osd":1,"uuid":"f2cd3694-9c9e-4da7-976e-f80a8df120f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6801","nonce":4021360958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6803","nonce":4021360958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6807","nonce":4021360958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6805","nonce":4021360958}]},"public_addr":"172.21.15.107:6801/4021360958","cluster_addr":"172.21.15.107:6803/4021360958","heartbeat_back_addr":"172.21.15.107:6807/4021360958","heartbeat_front_addr":"172.21.15.107:6805/4021360958","state":["exists","up"]},{"osd":2,"uuid":"a5bbf2eb-6599-44d5-ad96-68bb89ee19f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6801","nonce":3227179825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6803","nonce":3227179825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6807","nonce":3227179825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6805","nonce":3227179825}]},"public_addr":"172.21.15.111:6801/3227179825","cluster_addr":"172.21.15.111:6803/3227179825","heartbeat_back_addr":"172.21.15.111:6807/3227179825","heartbeat_front_addr":"172.21.15.111:6805/3227179825","state":["exists","up"]},{"osd":3,"uuid":"d6295ffe-11bc-4964-a250-9fecca870b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6811","nonce":2390375937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6813","nonce":2390375937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6817","nonce":2390375937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6815","nonce":2390375937}]},"public_addr":"172.21.15.16:6811/2390375937","cluster_addr":"172.21.15.16:6813/2390375937","heartbeat_back_addr":"172.21.15.16:6817/2390375937","heartbeat_front_addr":"172.21.15.16:6815/2390375937","state":["exists","up"]},{"osd":4,"uuid":"0a8446e0-fc86-4646-beb6-a7b9d433093d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6809","nonce":1936582048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6811","nonce":1936582048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6815","nonce":1936582048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6813","nonce":1936582048}]},"public_addr":"172.21.15.107:6809/1936582048","cluster_addr":"172.21.15.107:6811/1936582048","heartbeat_back_addr":"172.21.15.107:6815/1936582048","heartbeat_front_addr":"172.21.15.107:6813/1936582048","state":["exists","up"]},{"osd":5,"uuid":"2c97a446-357e-4975-9e51-9c95fe84a663","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6809","nonce":483858448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6811","nonce":483858448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6815","nonce":483858448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6813","nonce":483858448}]},"public_addr":"172.21.15.111:6809/483858448","cluster_addr":"172.21.15.111:6811/483858448","heartbeat_back_addr":"172.21.15.111:6815/483858448","heartbeat_front_addr":"172.21.15.111:6813/483858448","state":["exists","up"]},{"osd":6,"uuid":"890239e6-e6fb-4f9c-b973-05cc66d199bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6819","nonce":1330214141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6821","nonce":1330214141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6825","nonce":1330214141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6823","nonce":1330214141}]},"public_addr":"172.21.15.16:6819/1330214141","cluster_addr":"172.21.15.16:6821/1330214141","heartbeat_back_addr":"172.21.15.16:6825/1330214141","heartbeat_front_addr":"172.21.15.16:6823/1330214141","state":["exists","up"]},{"osd":7,"uuid":"ee826a9c-f441-4903-9b1b-66b627f7b712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6817","nonce":1133876754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6819","nonce":1133876754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6823","nonce":1133876754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6821","nonce":1133876754}]},"public_addr":"172.21.15.107:6817/1133876754","cluster_addr":"172.21.15.107:6819/1133876754","heartbeat_back_addr":"172.21.15.107:6823/1133876754","heartbeat_front_addr":"172.21.15.107:6821/1133876754","state":["exists","up"]},{"osd":8,"uuid":"e19773f2-dec8-44e0-9d33-0252e07d6415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6817","nonce":837548190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6819","nonce":837548190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6823","nonce":837548190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6821","nonce":837548190}]},"public_addr":"172.21.15.111:6817/837548190","cluster_addr":"172.21.15.111:6819/837548190","heartbeat_back_addr":"172.21.15.111:6823/837548190","heartbeat_front_addr":"172.21.15.111:6821/837548190","state":["exists","up"]},{"osd":9,"uuid":"bae4d6db-8b90-4d02-8e83-7d8f2a6f8fdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6827","nonce":4175383686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6829","nonce":4175383686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6833","nonce":4175383686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6831","nonce":4175383686}]},"public_addr":"172.21.15.16:6827/4175383686","cluster_addr":"172.21.15.16:6829/4175383686","heartbeat_back_addr":"172.21.15.16:6833/4175383686","heartbeat_front_addr":"172.21.15.16:6831/4175383686","state":["exists","up"]},{"osd":10,"uuid":"fd0e7740-76c8-4c99-a349-7077b4905143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6825","nonce":2659550426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6827","nonce":2659550426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6831","nonce":2659550426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6829","nonce":2659550426}]},"public_addr":"172.21.15.107:6825/2659550426","cluster_addr":"172.21.15.107:6827/2659550426","heartbeat_back_addr":"172.21.15.107:6831/2659550426","heartbeat_front_addr":"172.21.15.107:6829/2659550426","state":["exists","up"]},{"osd":11,"uuid":"94a1751d-b00c-4e23-815d-3a8bc3dea146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6825","nonce":1633548683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6827","nonce":1633548683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6831","nonce":1633548683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6829","nonce":1633548683}]},"public_addr":"172.21.15.111:6825/1633548683","cluster_addr":"172.21.15.111:6827/1633548683","heartbeat_back_addr":"172.21.15.111:6831/1633548683","heartbeat_front_addr":"172.21.15.111:6829/1633548683","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:36.359852+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:58.014202+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:19.753672+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:43.404817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:06.001677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:29.361587+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:52.554847+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:15.957553+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:39.581093+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:04.911389+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:28.971280+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:53.473561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/2098378667":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/3536229016":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6801/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/2684687547":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6800/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/368410612":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/12158160":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6800/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6801/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/3754423379":"2024-09-14T16:21:40.126459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:30:32.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.133+0000 7fb92f8f7640 1 -- 172.21.15.16:0/2504903054 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb9140a11a0 msgr2=0x7fb9140a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:32.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.133+0000 7fb92f8f7640 1 --2- 172.21.15.16:0/2504903054 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb9140a11a0 0x7fb9140a3660 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb928100e10 tx=0x7fb924005e70 comp rx=0 tx=0).stop 2024-09-13T16:30:32.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.133+0000 7fb92f8f7640 1 -- 172.21.15.16:0/2504903054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb928100fb0 msgr2=0x7fb9280728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:32.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.133+0000 7fb92f8f7640 1 --2- 172.21.15.16:0/2504903054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb928100fb0 0x7fb9280728a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb918005ab0 tx=0x7fb9180042f0 comp rx=0 tx=0).stop 2024-09-13T16:30:32.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.133+0000 7fb92f8f7640 1 -- 172.21.15.16:0/2504903054 shutdown_connections 2024-09-13T16:30:32.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.133+0000 7fb92f8f7640 1 --2- 172.21.15.16:0/2504903054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb928101970 0x7fb928072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:32.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.133+0000 7fb92f8f7640 1 --2- 172.21.15.16:0/2504903054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb928100fb0 0x7fb9280728a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:32.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.133+0000 7fb92f8f7640 1 --2- 172.21.15.16:0/2504903054 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb9140a11a0 0x7fb9140a3660 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:32.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.133+0000 7fb92f8f7640 1 --2- 172.21.15.16:0/2504903054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9280ffdc0 0x7fb928074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:32.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.133+0000 7fb92f8f7640 1 -- 172.21.15.16:0/2504903054 >> 172.21.15.16:0/2504903054 conn(0x7fb9280fb540 msgr2=0x7fb9281044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:32.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.133+0000 7fb92f8f7640 1 -- 172.21.15.16:0/2504903054 shutdown_connections 2024-09-13T16:30:32.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.133+0000 7fb92f8f7640 1 -- 172.21.15.16:0/2504903054 wait complete. 2024-09-13T16:30:32.145 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:24:22.981019+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T16:28:37.746027+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T16:28:38.602731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T16:28:41.633322+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T16:30:32.145 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:30:32.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.250+0000 7f537e733640 1 -- 172.21.15.16:0/1132681896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53780fec90 msgr2=0x7f53780ff090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:32.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.250+0000 7f537e733640 1 --2- 172.21.15.16:0/1132681896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53780fec90 0x7f53780ff090 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5368009900 tx=0x7f536802a410 comp rx=0 tx=0).stop 2024-09-13T16:30:32.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.250+0000 7f537e733640 1 -- 172.21.15.16:0/1132681896 shutdown_connections 2024-09-13T16:30:32.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.250+0000 7f537e733640 1 --2- 172.21.15.16:0/1132681896 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5378100840 0x7f53781070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:32.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.250+0000 7f537e733640 1 --2- 172.21.15.16:0/1132681896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53780ffe80 0x7f5378100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:32.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.250+0000 7f537e733640 1 --2- 172.21.15.16:0/1132681896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53780fec90 0x7f53780ff090 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:32.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.250+0000 7f537e733640 1 -- 172.21.15.16:0/1132681896 >> 172.21.15.16:0/1132681896 conn(0x7f53780fa3f0 msgr2=0x7f53780fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:32.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.250+0000 7f537e733640 1 -- 172.21.15.16:0/1132681896 shutdown_connections 2024-09-13T16:30:32.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.250+0000 7f537e733640 1 -- 172.21.15.16:0/1132681896 wait complete. 2024-09-13T16:30:32.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.251+0000 7f537e733640 1 Processor -- start 2024-09-13T16:30:32.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.251+0000 7f537e733640 1 -- start start 2024-09-13T16:30:32.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.251+0000 7f537e733640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53780fec90 0x7f5378102c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:32.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.251+0000 7f537e733640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53780ffe80 0x7f5378101260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:32.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.251+0000 7f537e733640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5378100840 0x7f53781017a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:32.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.251+0000 7f537e733640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5378101f00 con 0x7f53780ffe80 2024-09-13T16:30:32.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.251+0000 7f537e733640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5378102070 con 0x7f53780fec90 2024-09-13T16:30:32.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.251+0000 7f537e733640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537819fcc0 con 0x7f5378100840 2024-09-13T16:30:32.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.251+0000 7f53777fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53780ffe80 0x7f5378101260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:32.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.251+0000 7f53777fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53780ffe80 0x7f5378101260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57844/0 (socket says 172.21.15.16:57844) 2024-09-13T16:30:32.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.251+0000 7f53777fe640 1 -- 172.21.15.16:0/1465279339 learned_addr learned my addr 172.21.15.16:0/1465279339 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:32.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.251+0000 7f5377fff640 1 --2- 172.21.15.16:0/1465279339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53780fec90 0x7f5378102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:32.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.252+0000 7f537ccd0640 1 --2- 172.21.15.16:0/1465279339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5378100840 0x7f53781017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:32.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.252+0000 7f53777fe640 1 -- 172.21.15.16:0/1465279339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5378100840 msgr2=0x7f53781017a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:32.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.252+0000 7f53777fe640 1 --2- 172.21.15.16:0/1465279339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5378100840 0x7f53781017a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:32.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.252+0000 7f53777fe640 1 -- 172.21.15.16:0/1465279339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53780fec90 msgr2=0x7f5378102c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:32.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.252+0000 7f53777fe640 1 --2- 172.21.15.16:0/1465279339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53780fec90 0x7f5378102c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:32.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.252+0000 7f53777fe640 1 -- 172.21.15.16:0/1465279339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5368009560 con 0x7f53780ffe80 2024-09-13T16:30:32.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.252+0000 7f53777fe640 1 --2- 172.21.15.16:0/1465279339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53780ffe80 0x7f5378101260 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f536c00d980 tx=0x7f536c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:32.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.252+0000 7f53757fa640 1 -- 172.21.15.16:0/1465279339 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c009a30 con 0x7f53780ffe80 2024-09-13T16:30:32.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.252+0000 7f53757fa640 1 -- 172.21.15.16:0/1465279339 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f536c0044b0 con 0x7f53780ffe80 2024-09-13T16:30:32.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.252+0000 7f537e733640 1 -- 172.21.15.16:0/1465279339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f537819ff70 con 0x7f53780ffe80 2024-09-13T16:30:32.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.252+0000 7f53757fa640 1 -- 172.21.15.16:0/1465279339 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c005690 con 0x7f53780ffe80 2024-09-13T16:30:32.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.252+0000 7f537e733640 1 -- 172.21.15.16:0/1465279339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53781a0440 con 0x7f53780ffe80 2024-09-13T16:30:32.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.253+0000 7f537e733640 1 -- 172.21.15.16:0/1465279339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f537810a3e0 con 0x7f53780ffe80 2024-09-13T16:30:32.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.254+0000 7f53757fa640 1 -- 172.21.15.16:0/1465279339 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f536c004020 con 0x7f53780ffe80 2024-09-13T16:30:32.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.255+0000 7f53757fa640 1 --2- 172.21.15.16:0/1465279339 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f53600a11a0 0x7f53600a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:32.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.255+0000 7f5377fff640 1 --2- 172.21.15.16:0/1465279339 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f53600a11a0 0x7f53600a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:32.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.255+0000 7f53757fa640 1 -- 172.21.15.16:0/1465279339 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f536c0cc810 con 0x7f53780ffe80 2024-09-13T16:30:32.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.255+0000 7f5377fff640 1 --2- 172.21.15.16:0/1465279339 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f53600a11a0 0x7f53600a3660 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f53680057c0 tx=0x7f5368005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:32.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.259+0000 7f53757fa640 1 -- 172.21.15.16:0/1465279339 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f536c095a90 con 0x7f53780ffe80 2024-09-13T16:30:32.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.430+0000 7f537e733640 1 -- 172.21.15.16:0/1465279339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f53781a08c0 con 0x7f53780ffe80 2024-09-13T16:30:32.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.431+0000 7f53757fa640 1 -- 172.21.15.16:0/1465279339 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f536c0990b0 con 0x7f53780ffe80 2024-09-13T16:30:32.432 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-13T16:30:32.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.433+0000 7f537e733640 1 -- 172.21.15.16:0/1465279339 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f53600a11a0 msgr2=0x7f53600a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:32.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.433+0000 7f537e733640 1 --2- 172.21.15.16:0/1465279339 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f53600a11a0 0x7f53600a3660 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f53680057c0 tx=0x7f5368005a20 comp rx=0 tx=0).stop 2024-09-13T16:30:32.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.433+0000 7f537e733640 1 -- 172.21.15.16:0/1465279339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53780ffe80 msgr2=0x7f5378101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:32.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.433+0000 7f537e733640 1 --2- 172.21.15.16:0/1465279339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53780ffe80 0x7f5378101260 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f536c00d980 tx=0x7f536c00de40 comp rx=0 tx=0).stop 2024-09-13T16:30:32.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.433+0000 7f537e733640 1 -- 172.21.15.16:0/1465279339 shutdown_connections 2024-09-13T16:30:32.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.433+0000 7f537e733640 1 --2- 172.21.15.16:0/1465279339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5378100840 0x7f53781017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:32.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.433+0000 7f537e733640 1 --2- 172.21.15.16:0/1465279339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53780ffe80 0x7f5378101260 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:32.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.433+0000 7f537e733640 1 --2- 172.21.15.16:0/1465279339 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f53600a11a0 0x7f53600a3660 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:32.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.433+0000 7f537e733640 1 --2- 172.21.15.16:0/1465279339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53780fec90 0x7f5378102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:32.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.433+0000 7f537e733640 1 -- 172.21.15.16:0/1465279339 >> 172.21.15.16:0/1465279339 conn(0x7f53780fa3f0 msgr2=0x7f5378068620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:32.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.433+0000 7f537e733640 1 -- 172.21.15.16:0/1465279339 shutdown_connections 2024-09-13T16:30:32.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.433+0000 7f537e733640 1 -- 172.21.15.16:0/1465279339 wait complete. 2024-09-13T16:30:32.444 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T16:30:32.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.548+0000 7f15a1126640 1 -- 172.21.15.16:0/1776490887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f159c100840 msgr2=0x7f159c1070d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:32.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.548+0000 7f15a1126640 1 --2- 172.21.15.16:0/1776490887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f159c100840 0x7f159c1070d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1594009ca0 tx=0x7f159402a8c0 comp rx=0 tx=0).stop 2024-09-13T16:30:32.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.548+0000 7f15a1126640 1 -- 172.21.15.16:0/1776490887 shutdown_connections 2024-09-13T16:30:32.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.548+0000 7f15a1126640 1 --2- 172.21.15.16:0/1776490887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f159c100840 0x7f159c1070d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:32.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.548+0000 7f15a1126640 1 --2- 172.21.15.16:0/1776490887 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f159c0ffe80 0x7f159c100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:32.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.548+0000 7f15a1126640 1 --2- 172.21.15.16:0/1776490887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f159c0fec90 0x7f159c0ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:32.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.548+0000 7f15a1126640 1 -- 172.21.15.16:0/1776490887 >> 172.21.15.16:0/1776490887 conn(0x7f159c0fa3f0 msgr2=0x7f159c0fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:32.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.548+0000 7f15a1126640 1 -- 172.21.15.16:0/1776490887 shutdown_connections 2024-09-13T16:30:32.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.548+0000 7f15a1126640 1 -- 172.21.15.16:0/1776490887 wait complete. 2024-09-13T16:30:32.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.549+0000 7f15a1126640 1 Processor -- start 2024-09-13T16:30:32.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.549+0000 7f15a1126640 1 -- start start 2024-09-13T16:30:32.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.549+0000 7f15a1126640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f159c0fec90 0x7f159c109550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:32.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.549+0000 7f15a1126640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f159c0ffe80 0x7f159c107b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:32.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.549+0000 7f15a1126640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f159c100840 0x7f159c1080d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:32.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.549+0000 7f15a1126640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159c108830 con 0x7f159c0fec90 2024-09-13T16:30:32.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.549+0000 7f15a1126640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159c1089a0 con 0x7f159c0ffe80 2024-09-13T16:30:32.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.549+0000 7f15a1126640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159c19fca0 con 0x7f159c100840 2024-09-13T16:30:32.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.549+0000 7f159ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f159c0fec90 0x7f159c109550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:32.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.549+0000 7f159ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f159c0fec90 0x7f159c109550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57856/0 (socket says 172.21.15.16:57856) 2024-09-13T16:30:32.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.549+0000 7f159ad9d640 1 -- 172.21.15.16:0/1581657843 learned_addr learned my addr 172.21.15.16:0/1581657843 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:32.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.550+0000 7f159b59e640 1 --2- 172.21.15.16:0/1581657843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f159c100840 0x7f159c1080d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:32.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.550+0000 7f159a59c640 1 --2- 172.21.15.16:0/1581657843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f159c0ffe80 0x7f159c107b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:32.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.550+0000 7f159ad9d640 1 -- 172.21.15.16:0/1581657843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f159c100840 msgr2=0x7f159c1080d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:32.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.550+0000 7f159ad9d640 1 --2- 172.21.15.16:0/1581657843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f159c100840 0x7f159c1080d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:32.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.550+0000 7f159ad9d640 1 -- 172.21.15.16:0/1581657843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f159c0ffe80 msgr2=0x7f159c107b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:32.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.550+0000 7f159ad9d640 1 --2- 172.21.15.16:0/1581657843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f159c0ffe80 0x7f159c107b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:32.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.550+0000 7f159ad9d640 1 -- 172.21.15.16:0/1581657843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1594009900 con 0x7f159c0fec90 2024-09-13T16:30:32.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.550+0000 7f159b59e640 1 --2- 172.21.15.16:0/1581657843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f159c100840 0x7f159c1080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:30:32.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.550+0000 7f159a59c640 1 --2- 172.21.15.16:0/1581657843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f159c0ffe80 0x7f159c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:30:32.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.550+0000 7f159ad9d640 1 --2- 172.21.15.16:0/1581657843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f159c0fec90 0x7f159c109550 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f158c00e9f0 tx=0x7f158c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:32.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.550+0000 7f1593fff640 1 -- 172.21.15.16:0/1581657843 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c009700 con 0x7f159c0fec90 2024-09-13T16:30:32.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.550+0000 7f1593fff640 1 -- 172.21.15.16:0/1581657843 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f158c0044b0 con 0x7f159c0fec90 2024-09-13T16:30:32.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.550+0000 7f15a1126640 1 -- 172.21.15.16:0/1581657843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f159c19ff50 con 0x7f159c0fec90 2024-09-13T16:30:32.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.550+0000 7f1593fff640 1 -- 172.21.15.16:0/1581657843 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c010430 con 0x7f159c0fec90 2024-09-13T16:30:32.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.550+0000 7f15a1126640 1 -- 172.21.15.16:0/1581657843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f159c1a03d0 con 0x7f159c0fec90 2024-09-13T16:30:32.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.551+0000 7f15a1126640 1 -- 172.21.15.16:0/1581657843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f159c106120 con 0x7f159c0fec90 2024-09-13T16:30:32.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.552+0000 7f1593fff640 1 -- 172.21.15.16:0/1581657843 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f158c00cd80 con 0x7f159c0fec90 2024-09-13T16:30:32.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.553+0000 7f1593fff640 1 --2- 172.21.15.16:0/1581657843 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f15880a11a0 0x7f15880a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:32.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.553+0000 7f1593fff640 1 -- 172.21.15.16:0/1581657843 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f158c014070 con 0x7f159c0fec90 2024-09-13T16:30:32.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.553+0000 7f159a59c640 1 --2- 172.21.15.16:0/1581657843 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f15880a11a0 0x7f15880a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:32.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.553+0000 7f159a59c640 1 --2- 172.21.15.16:0/1581657843 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f15880a11a0 0x7f15880a3660 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f159c1092c0 tx=0x7f1584005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:32.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.557+0000 7f1593fff640 1 -- 172.21.15.16:0/1581657843 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f158c0964a0 con 0x7f159c0fec90 2024-09-13T16:30:32.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.734+0000 7f15a1126640 1 -- 172.21.15.16:0/1581657843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f159c02c480 con 0x7f159c0fec90 2024-09-13T16:30:32.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.735+0000 7f1593fff640 1 -- 172.21.15.16:0/1581657843 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f158c099ac0 con 0x7f159c0fec90 2024-09-13T16:30:32.736 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-13T16:30:32.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.737+0000 7f15a1126640 1 -- 172.21.15.16:0/1581657843 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f15880a11a0 msgr2=0x7f15880a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:32.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.737+0000 7f15a1126640 1 --2- 172.21.15.16:0/1581657843 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f15880a11a0 0x7f15880a3660 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f159c1092c0 tx=0x7f1584005e50 comp rx=0 tx=0).stop 2024-09-13T16:30:32.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.737+0000 7f15a1126640 1 -- 172.21.15.16:0/1581657843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f159c0fec90 msgr2=0x7f159c109550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:32.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.737+0000 7f15a1126640 1 --2- 172.21.15.16:0/1581657843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f159c0fec90 0x7f159c109550 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f158c00e9f0 tx=0x7f158c00eeb0 comp rx=0 tx=0).stop 2024-09-13T16:30:32.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.737+0000 7f15a1126640 1 -- 172.21.15.16:0/1581657843 shutdown_connections 2024-09-13T16:30:32.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.737+0000 7f15a1126640 1 --2- 172.21.15.16:0/1581657843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f159c100840 0x7f159c1080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:32.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.737+0000 7f15a1126640 1 --2- 172.21.15.16:0/1581657843 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f15880a11a0 0x7f15880a3660 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:32.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.737+0000 7f15a1126640 1 --2- 172.21.15.16:0/1581657843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f159c0ffe80 0x7f159c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:32.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.737+0000 7f15a1126640 1 --2- 172.21.15.16:0/1581657843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f159c0fec90 0x7f159c109550 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:32.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.737+0000 7f15a1126640 1 -- 172.21.15.16:0/1581657843 >> 172.21.15.16:0/1581657843 conn(0x7f159c0fa3f0 msgr2=0x7f159c1032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:32.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.737+0000 7f15a1126640 1 -- 172.21.15.16:0/1581657843 shutdown_connections 2024-09-13T16:30:32.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.737+0000 7f15a1126640 1 -- 172.21.15.16:0/1581657843 wait complete. 2024-09-13T16:30:32.749 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:32 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2088155589' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:30:32.749 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:32 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2504903054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:30:32.749 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:32 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1465279339' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:30:32.750 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:32 smithi016 ceph-mon[31187]: pgmap v337: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-13T16:30:32.751 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T16:30:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.856+0000 7f61dbc68640 1 -- 172.21.15.16:0/416133989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d40fec60 msgr2=0x7f61d40ff060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.856+0000 7f61dbc68640 1 --2- 172.21.15.16:0/416133989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d40fec60 0x7f61d40ff060 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f61d0009ca0 tx=0x7f61d002a8c0 comp rx=0 tx=0).stop 2024-09-13T16:30:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.856+0000 7f61dbc68640 1 -- 172.21.15.16:0/416133989 shutdown_connections 2024-09-13T16:30:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.856+0000 7f61dbc68640 1 --2- 172.21.15.16:0/416133989 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61d4100810 0x7f61d41070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.856+0000 7f61dbc68640 1 --2- 172.21.15.16:0/416133989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61d40ffe50 0x7f61d41002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.856+0000 7f61dbc68640 1 --2- 172.21.15.16:0/416133989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d40fec60 0x7f61d40ff060 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.856+0000 7f61dbc68640 1 -- 172.21.15.16:0/416133989 >> 172.21.15.16:0/416133989 conn(0x7f61d40fa3e0 msgr2=0x7f61d40fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.856+0000 7f61dbc68640 1 -- 172.21.15.16:0/416133989 shutdown_connections 2024-09-13T16:30:32.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.856+0000 7f61dbc68640 1 -- 172.21.15.16:0/416133989 wait complete. 2024-09-13T16:30:32.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.857+0000 7f61dbc68640 1 Processor -- start 2024-09-13T16:30:32.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.857+0000 7f61dbc68640 1 -- start start 2024-09-13T16:30:32.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.857+0000 7f61dbc68640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61d40fec60 0x7f61d4195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:32.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.857+0000 7f61dbc68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d40ffe50 0x7f61d41961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:32.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.857+0000 7f61dbc68640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61d4100810 0x7f61d419d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:32.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.857+0000 7f61dbc68640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d419d790 con 0x7f61d40ffe50 2024-09-13T16:30:32.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.857+0000 7f61dbc68640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d419d900 con 0x7f61d4100810 2024-09-13T16:30:32.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.857+0000 7f61dbc68640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d419dac0 con 0x7f61d40fec60 2024-09-13T16:30:32.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.857+0000 7f61d9203640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d40ffe50 0x7f61d41961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:32.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.857+0000 7f61d9203640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d40ffe50 0x7f61d41961c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57874/0 (socket says 172.21.15.16:57874) 2024-09-13T16:30:32.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.857+0000 7f61d9203640 1 -- 172.21.15.16:0/25297841 learned_addr learned my addr 172.21.15.16:0/25297841 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:32.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.857+0000 7f61d9a04640 1 --2- 172.21.15.16:0/25297841 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61d40fec60 0x7f61d4195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:32.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.857+0000 7f61da205640 1 --2- 172.21.15.16:0/25297841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61d4100810 0x7f61d419d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:32.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.857+0000 7f61d9203640 1 -- 172.21.15.16:0/25297841 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61d40fec60 msgr2=0x7f61d4195c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:32.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.857+0000 7f61d9203640 1 --2- 172.21.15.16:0/25297841 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61d40fec60 0x7f61d4195c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:32.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.857+0000 7f61d9203640 1 -- 172.21.15.16:0/25297841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61d4100810 msgr2=0x7f61d419d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:32.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.857+0000 7f61d9203640 1 --2- 172.21.15.16:0/25297841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61d4100810 0x7f61d419d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:32.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.857+0000 7f61d9203640 1 -- 172.21.15.16:0/25297841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d0009900 con 0x7f61d40ffe50 2024-09-13T16:30:32.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.857+0000 7f61d9203640 1 --2- 172.21.15.16:0/25297841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d40ffe50 0x7f61d41961c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f61c400d950 tx=0x7f61c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:32.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.858+0000 7f61caffd640 1 -- 172.21.15.16:0/25297841 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c4009a30 con 0x7f61d40ffe50 2024-09-13T16:30:32.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.858+0000 7f61caffd640 1 -- 172.21.15.16:0/25297841 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f61c40044b0 con 0x7f61d40ffe50 2024-09-13T16:30:32.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.858+0000 7f61dbc68640 1 -- 172.21.15.16:0/25297841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d419dda0 con 0x7f61d40ffe50 2024-09-13T16:30:32.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.858+0000 7f61caffd640 1 -- 172.21.15.16:0/25297841 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c40047a0 con 0x7f61d40ffe50 2024-09-13T16:30:32.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.858+0000 7f61dbc68640 1 -- 172.21.15.16:0/25297841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d419e220 con 0x7f61d40ffe50 2024-09-13T16:30:32.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.859+0000 7f61dbc68640 1 -- 172.21.15.16:0/25297841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d4106230 con 0x7f61d40ffe50 2024-09-13T16:30:32.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.860+0000 7f61caffd640 1 -- 172.21.15.16:0/25297841 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f61c4004020 con 0x7f61d40ffe50 2024-09-13T16:30:32.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.860+0000 7f61caffd640 1 --2- 172.21.15.16:0/25297841 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f61c00a1120 0x7f61c00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:32.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.860+0000 7f61d9a04640 1 --2- 172.21.15.16:0/25297841 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f61c00a1120 0x7f61c00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:32.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.860+0000 7f61caffd640 1 -- 172.21.15.16:0/25297841 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f61c40cc850 con 0x7f61d40ffe50 2024-09-13T16:30:32.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.861+0000 7f61d9a04640 1 --2- 172.21.15.16:0/25297841 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f61c00a1120 0x7f61c00a35e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f61d0005e30 tx=0x7f61d0009800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:32.864+0000 7f61caffd640 1 -- 172.21.15.16:0/25297841 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f61c4095b30 con 0x7f61d40ffe50 2024-09-13T16:30:32.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:32 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/2088155589' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:30:32.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:32 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/2504903054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:30:32.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:32 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1465279339' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:30:32.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:32 smithi111 ceph-mon[34600]: pgmap v337: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-13T16:30:33.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.035+0000 7f61dbc68640 1 -- 172.21.15.16:0/25297841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f61d402c480 con 0x7f61d40ffe50 2024-09-13T16:30:33.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.035+0000 7f61caffd640 1 -- 172.21.15.16:0/25297841 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f61c4099150 con 0x7f61d40ffe50 2024-09-13T16:30:33.037 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-09-13T16:30:33.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.038+0000 7f61dbc68640 1 -- 172.21.15.16:0/25297841 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f61c00a1120 msgr2=0x7f61c00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:33.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.038+0000 7f61dbc68640 1 --2- 172.21.15.16:0/25297841 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f61c00a1120 0x7f61c00a35e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f61d0005e30 tx=0x7f61d0009800 comp rx=0 tx=0).stop 2024-09-13T16:30:33.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.038+0000 7f61dbc68640 1 -- 172.21.15.16:0/25297841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d40ffe50 msgr2=0x7f61d41961c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:33.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.038+0000 7f61dbc68640 1 --2- 172.21.15.16:0/25297841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d40ffe50 0x7f61d41961c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f61c400d950 tx=0x7f61c400de10 comp rx=0 tx=0).stop 2024-09-13T16:30:33.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.038+0000 7f61dbc68640 1 -- 172.21.15.16:0/25297841 shutdown_connections 2024-09-13T16:30:33.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.038+0000 7f61dbc68640 1 --2- 172.21.15.16:0/25297841 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f61c00a1120 0x7f61c00a35e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:33.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.038+0000 7f61dbc68640 1 --2- 172.21.15.16:0/25297841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61d4100810 0x7f61d419d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:33.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.038+0000 7f61dbc68640 1 --2- 172.21.15.16:0/25297841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d40ffe50 0x7f61d41961c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:33.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.038+0000 7f61dbc68640 1 --2- 172.21.15.16:0/25297841 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61d40fec60 0x7f61d4195c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:33.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.038+0000 7f61dbc68640 1 -- 172.21.15.16:0/25297841 >> 172.21.15.16:0/25297841 conn(0x7f61d40fa3e0 msgr2=0x7f61d4103400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:33.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.038+0000 7f61dbc68640 1 -- 172.21.15.16:0/25297841 shutdown_connections 2024-09-13T16:30:33.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.038+0000 7f61dbc68640 1 -- 172.21.15.16:0/25297841 wait complete. 2024-09-13T16:30:33.049 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T16:30:33.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:32 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/2088155589' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:30:33.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:32 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/2504903054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:30:33.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:32 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1465279339' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:30:33.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:32 smithi107 ceph-mon[34139]: pgmap v337: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-13T16:30:33.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.153+0000 7fc073d18640 1 -- 172.21.15.16:0/2904010487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc06c0fec90 msgr2=0x7fc06c0ff090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:33.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.153+0000 7fc073d18640 1 --2- 172.21.15.16:0/2904010487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc06c0fec90 0x7fc06c0ff090 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc068009900 tx=0x7fc06802a410 comp rx=0 tx=0).stop 2024-09-13T16:30:33.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.153+0000 7fc073d18640 1 -- 172.21.15.16:0/2904010487 shutdown_connections 2024-09-13T16:30:33.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.153+0000 7fc073d18640 1 --2- 172.21.15.16:0/2904010487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc06c100840 0x7fc06c1070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:33.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.153+0000 7fc073d18640 1 --2- 172.21.15.16:0/2904010487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc06c0ffe80 0x7fc06c100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:33.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.153+0000 7fc073d18640 1 --2- 172.21.15.16:0/2904010487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc06c0fec90 0x7fc06c0ff090 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:33.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.153+0000 7fc073d18640 1 -- 172.21.15.16:0/2904010487 >> 172.21.15.16:0/2904010487 conn(0x7fc06c0fa3f0 msgr2=0x7fc06c0fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:33.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.153+0000 7fc073d18640 1 -- 172.21.15.16:0/2904010487 shutdown_connections 2024-09-13T16:30:33.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.153+0000 7fc073d18640 1 -- 172.21.15.16:0/2904010487 wait complete. 2024-09-13T16:30:33.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.154+0000 7fc073d18640 1 Processor -- start 2024-09-13T16:30:33.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.154+0000 7fc073d18640 1 -- start start 2024-09-13T16:30:33.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.154+0000 7fc073d18640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc06c0fec90 0x7fc06c0690e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:33.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.154+0000 7fc073d18640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc06c0ffe80 0x7fc06c19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:33.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.154+0000 7fc073d18640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc06c100840 0x7fc06c19cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:33.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.154+0000 7fc073d18640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c19d790 con 0x7fc06c100840 2024-09-13T16:30:33.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.154+0000 7fc073d18640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c19d900 con 0x7fc06c0ffe80 2024-09-13T16:30:33.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.154+0000 7fc073d18640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c19dac0 con 0x7fc06c0fec90 2024-09-13T16:30:33.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.154+0000 7fc0722b5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc06c100840 0x7fc06c19cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:33.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.154+0000 7fc0722b5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc06c100840 0x7fc06c19cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57890/0 (socket says 172.21.15.16:57890) 2024-09-13T16:30:33.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.154+0000 7fc0722b5640 1 -- 172.21.15.16:0/2477677346 learned_addr learned my addr 172.21.15.16:0/2477677346 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:33.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.154+0000 7fc0712b3640 1 --2- 172.21.15.16:0/2477677346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc06c0ffe80 0x7fc06c19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:33.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.155+0000 7fc071ab4640 1 --2- 172.21.15.16:0/2477677346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc06c0fec90 0x7fc06c0690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:33.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.155+0000 7fc0722b5640 1 -- 172.21.15.16:0/2477677346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc06c0fec90 msgr2=0x7fc06c0690e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:33.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.155+0000 7fc0722b5640 1 --2- 172.21.15.16:0/2477677346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc06c0fec90 0x7fc06c0690e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:33.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.155+0000 7fc0722b5640 1 -- 172.21.15.16:0/2477677346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc06c0ffe80 msgr2=0x7fc06c19a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:33.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.155+0000 7fc0722b5640 1 --2- 172.21.15.16:0/2477677346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc06c0ffe80 0x7fc06c19a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:33.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.155+0000 7fc0722b5640 1 -- 172.21.15.16:0/2477677346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc068009560 con 0x7fc06c100840 2024-09-13T16:30:33.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.155+0000 7fc0722b5640 1 --2- 172.21.15.16:0/2477677346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc06c100840 0x7fc06c19cfb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc06400cce0 tx=0x7fc06400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:33.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.155+0000 7fc062ffd640 1 -- 172.21.15.16:0/2477677346 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06400f450 con 0x7fc06c100840 2024-09-13T16:30:33.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.155+0000 7fc062ffd640 1 -- 172.21.15.16:0/2477677346 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc06400fa40 con 0x7fc06c100840 2024-09-13T16:30:33.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.155+0000 7fc062ffd640 1 -- 172.21.15.16:0/2477677346 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0640189a0 con 0x7fc06c100840 2024-09-13T16:30:33.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.155+0000 7fc073d18640 1 -- 172.21.15.16:0/2477677346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc06c19dd40 con 0x7fc06c100840 2024-09-13T16:30:33.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.155+0000 7fc073d18640 1 -- 172.21.15.16:0/2477677346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc06c19e1a0 con 0x7fc06c100840 2024-09-13T16:30:33.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.156+0000 7fc073d18640 1 -- 172.21.15.16:0/2477677346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc06c1061c0 con 0x7fc06c100840 2024-09-13T16:30:33.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.157+0000 7fc062ffd640 1 -- 172.21.15.16:0/2477677346 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc06400fbb0 con 0x7fc06c100840 2024-09-13T16:30:33.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.158+0000 7fc062ffd640 1 --2- 172.21.15.16:0/2477677346 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc0580a1120 0x7fc0580a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:33.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.158+0000 7fc071ab4640 1 --2- 172.21.15.16:0/2477677346 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc0580a1120 0x7fc0580a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:33.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.158+0000 7fc062ffd640 1 -- 172.21.15.16:0/2477677346 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc0640cd230 con 0x7fc06c100840 2024-09-13T16:30:33.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.158+0000 7fc071ab4640 1 --2- 172.21.15.16:0/2477677346 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc0580a1120 0x7fc0580a35e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc068006010 tx=0x7fc068005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:33.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.161+0000 7fc062ffd640 1 -- 172.21.15.16:0/2477677346 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc064096540 con 0x7fc06c100840 2024-09-13T16:30:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.335+0000 7fc073d18640 1 -- 172.21.15.16:0/2477677346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc06c02c480 con 0x7fc06c100840 2024-09-13T16:30:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.335+0000 7fc062ffd640 1 -- 172.21.15.16:0/2477677346 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc064099b60 con 0x7fc06c100840 2024-09-13T16:30:33.336 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-13T16:30:33.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.337+0000 7fc073d18640 1 -- 172.21.15.16:0/2477677346 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc0580a1120 msgr2=0x7fc0580a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:33.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.337+0000 7fc073d18640 1 --2- 172.21.15.16:0/2477677346 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc0580a1120 0x7fc0580a35e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc068006010 tx=0x7fc068005a20 comp rx=0 tx=0).stop 2024-09-13T16:30:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.337+0000 7fc073d18640 1 -- 172.21.15.16:0/2477677346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc06c100840 msgr2=0x7fc06c19cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.337+0000 7fc073d18640 1 --2- 172.21.15.16:0/2477677346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc06c100840 0x7fc06c19cfb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc06400cce0 tx=0x7fc06400a620 comp rx=0 tx=0).stop 2024-09-13T16:30:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.337+0000 7fc073d18640 1 -- 172.21.15.16:0/2477677346 shutdown_connections 2024-09-13T16:30:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.337+0000 7fc073d18640 1 --2- 172.21.15.16:0/2477677346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc06c100840 0x7fc06c19cfb0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.337+0000 7fc073d18640 1 --2- 172.21.15.16:0/2477677346 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc0580a1120 0x7fc0580a35e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.337+0000 7fc073d18640 1 --2- 172.21.15.16:0/2477677346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc06c0ffe80 0x7fc06c19a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.337+0000 7fc073d18640 1 --2- 172.21.15.16:0/2477677346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc06c0fec90 0x7fc06c0690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.337+0000 7fc073d18640 1 -- 172.21.15.16:0/2477677346 >> 172.21.15.16:0/2477677346 conn(0x7fc06c0fa3f0 msgr2=0x7fc06c103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.337+0000 7fc073d18640 1 -- 172.21.15.16:0/2477677346 shutdown_connections 2024-09-13T16:30:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.337+0000 7fc073d18640 1 -- 172.21.15.16:0/2477677346 wait complete. 2024-09-13T16:30:33.349 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:30:33.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.451+0000 7f4ca9e87640 1 -- 172.21.15.16:0/145507809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ca41008f0 msgr2=0x7f4ca4102ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:33.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.451+0000 7f4ca9e87640 1 --2- 172.21.15.16:0/145507809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ca41008f0 0x7f4ca4102ce0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4c94009900 tx=0x7f4c9402a410 comp rx=0 tx=0).stop 2024-09-13T16:30:33.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.451+0000 7f4ca9e87640 1 -- 172.21.15.16:0/145507809 shutdown_connections 2024-09-13T16:30:33.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.451+0000 7f4ca9e87640 1 --2- 172.21.15.16:0/145507809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ca4105b50 0x7f4ca4107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:33.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.451+0000 7f4ca9e87640 1 --2- 172.21.15.16:0/145507809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ca4103220 0x7f4ca4105610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:33.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.451+0000 7f4ca9e87640 1 --2- 172.21.15.16:0/145507809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ca41008f0 0x7f4ca4102ce0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.451+0000 7f4ca9e87640 1 -- 172.21.15.16:0/145507809 >> 172.21.15.16:0/145507809 conn(0x7f4ca40fa430 msgr2=0x7f4ca40fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.451+0000 7f4ca9e87640 1 -- 172.21.15.16:0/145507809 shutdown_connections 2024-09-13T16:30:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.451+0000 7f4ca9e87640 1 -- 172.21.15.16:0/145507809 wait complete. 2024-09-13T16:30:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.452+0000 7f4ca9e87640 1 Processor -- start 2024-09-13T16:30:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.452+0000 7f4ca9e87640 1 -- start start 2024-09-13T16:30:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.452+0000 7f4ca9e87640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ca41008f0 0x7f4ca4195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.452+0000 7f4ca9e87640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ca4103220 0x7f4ca4196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.452+0000 7f4ca9e87640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ca4105b50 0x7f4ca419d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.452+0000 7f4ca9e87640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca419d940 con 0x7f4ca4105b50 2024-09-13T16:30:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.452+0000 7f4ca9e87640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca419dab0 con 0x7f4ca4103220 2024-09-13T16:30:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.452+0000 7f4ca9e87640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca419dc70 con 0x7f4ca41008f0 2024-09-13T16:30:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.452+0000 7f4ca3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ca4105b50 0x7f4ca419d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.452+0000 7f4ca3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ca4105b50 0x7f4ca419d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52072/0 (socket says 172.21.15.16:52072) 2024-09-13T16:30:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.452+0000 7f4ca3fff640 1 -- 172.21.15.16:0/216660610 learned_addr learned my addr 172.21.15.16:0/216660610 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.452+0000 7f4ca37fe640 1 --2- 172.21.15.16:0/216660610 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ca41008f0 0x7f4ca4195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.452+0000 7f4c9affd640 1 --2- 172.21.15.16:0/216660610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ca4103220 0x7f4ca4196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.452+0000 7f4ca3fff640 1 -- 172.21.15.16:0/216660610 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ca41008f0 msgr2=0x7f4ca4195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.452+0000 7f4ca3fff640 1 --2- 172.21.15.16:0/216660610 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ca41008f0 0x7f4ca4195e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.452+0000 7f4ca3fff640 1 -- 172.21.15.16:0/216660610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ca4103220 msgr2=0x7f4ca4196390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.452+0000 7f4ca3fff640 1 --2- 172.21.15.16:0/216660610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ca4103220 0x7f4ca4196390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.452+0000 7f4ca3fff640 1 -- 172.21.15.16:0/216660610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c94009560 con 0x7f4ca4105b50 2024-09-13T16:30:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.452+0000 7f4c9affd640 1 --2- 172.21.15.16:0/216660610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ca4103220 0x7f4ca4196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:30:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.453+0000 7f4ca3fff640 1 --2- 172.21.15.16:0/216660610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ca4105b50 0x7f4ca419d400 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c00eb40 tx=0x7f4c9c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:33.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.453+0000 7f4ca17fa640 1 -- 172.21.15.16:0/216660610 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c004b20 con 0x7f4ca4105b50 2024-09-13T16:30:33.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.453+0000 7f4ca17fa640 1 -- 172.21.15.16:0/216660610 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c9c004c80 con 0x7f4ca4105b50 2024-09-13T16:30:33.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.453+0000 7f4ca9e87640 1 -- 172.21.15.16:0/216660610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ca419df50 con 0x7f4ca4105b50 2024-09-13T16:30:33.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.453+0000 7f4ca17fa640 1 -- 172.21.15.16:0/216660610 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c010690 con 0x7f4ca4105b50 2024-09-13T16:30:33.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.453+0000 7f4ca9e87640 1 -- 172.21.15.16:0/216660610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca419e420 con 0x7f4ca4105b50 2024-09-13T16:30:33.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.454+0000 7f4ca9e87640 1 -- 172.21.15.16:0/216660610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ca40fbf20 con 0x7f4ca4105b50 2024-09-13T16:30:33.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.455+0000 7f4ca17fa640 1 -- 172.21.15.16:0/216660610 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f4c9c005020 con 0x7f4ca4105b50 2024-09-13T16:30:33.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.456+0000 7f4ca17fa640 1 --2- 172.21.15.16:0/216660610 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f4c900a11a0 0x7f4c900a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:33.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.456+0000 7f4ca37fe640 1 --2- 172.21.15.16:0/216660610 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f4c900a11a0 0x7f4c900a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:33.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.456+0000 7f4ca17fa640 1 -- 172.21.15.16:0/216660610 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f4c9c014070 con 0x7f4ca4105b50 2024-09-13T16:30:33.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.456+0000 7f4ca37fe640 1 --2- 172.21.15.16:0/216660610 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f4c900a11a0 0x7f4c900a3660 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4c94000c00 tx=0x7f4c94004040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:33.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.460+0000 7f4ca17fa640 1 -- 172.21.15.16:0/216660610 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f4c9c0967a0 con 0x7f4ca4105b50 2024-09-13T16:30:33.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.672+0000 7f4ca9e87640 1 -- 172.21.15.16:0/216660610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ca40fb990 con 0x7f4ca4105b50 2024-09-13T16:30:33.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.672+0000 7f4ca17fa640 1 -- 172.21.15.16:0/216660610 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10924 (secure 0 0 0) 0x7f4c9c099dc0 con 0x7f4ca4105b50 2024-09-13T16:30:33.674 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:30:33.674 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3292781664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3292781664},{"type":"v1","addr":"172.21.15.16:6835","nonce":3292781664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4109039027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4109039027},{"type":"v1","addr":"172.21.15.16:6837","nonce":4109039027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T16:28:39.184734+0000","modified":"2024-09-13T16:29:06.636937+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24518,"mds_1":24490,"mds_2":24485,"mds_3":24422,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/2391618693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2391618693},{"type":"v1","addr":"172.21.15.16:6839","nonce":2391618693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/521042681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":521042681},{"type":"v1","addr":"172.21.15.16:6841","nonce":521042681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24418":{"gid":24418,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.111:6833/1984275936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1984275936},{"type":"v1","addr":"172.21.15.111:6833","nonce":1984275936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.107:6833/3797712690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3797712690},{"type":"v1","addr":"172.21.15.107:6833","nonce":3797712690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/338548468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":338548468},{"type":"v1","addr":"172.21.15.107:6835","nonce":338548468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/662156111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":662156111},{"type":"v1","addr":"172.21.15.111:6835","nonce":662156111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/1877128453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1877128453},{"type":"v1","addr":"172.21.15.111:6837","nonce":1877128453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.107:6837/2838457611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2838457611},{"type":"v1","addr":"172.21.15.107:6837","nonce":2838457611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.111:6839/1423842285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":1423842285},{"type":"v1","addr":"172.21.15.111:6839","nonce":1423842285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6839/1914689806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1914689806},{"type":"v1","addr":"172.21.15.107:6839","nonce":1914689806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T16:30:33.674 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-13T16:30:33.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.674+0000 7f4ca9e87640 1 -- 172.21.15.16:0/216660610 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f4c900a11a0 msgr2=0x7f4c900a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:33.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.674+0000 7f4ca9e87640 1 --2- 172.21.15.16:0/216660610 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f4c900a11a0 0x7f4c900a3660 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4c94000c00 tx=0x7f4c94004040 comp rx=0 tx=0).stop 2024-09-13T16:30:33.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.674+0000 7f4ca9e87640 1 -- 172.21.15.16:0/216660610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ca4105b50 msgr2=0x7f4ca419d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:33.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.674+0000 7f4ca9e87640 1 --2- 172.21.15.16:0/216660610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ca4105b50 0x7f4ca419d400 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c00eb40 tx=0x7f4c9c00c2d0 comp rx=0 tx=0).stop 2024-09-13T16:30:33.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.674+0000 7f4ca9e87640 1 -- 172.21.15.16:0/216660610 shutdown_connections 2024-09-13T16:30:33.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.674+0000 7f4ca9e87640 1 --2- 172.21.15.16:0/216660610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ca4105b50 0x7f4ca419d400 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:33.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.674+0000 7f4ca9e87640 1 --2- 172.21.15.16:0/216660610 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f4c900a11a0 0x7f4c900a3660 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:33.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.674+0000 7f4ca9e87640 1 --2- 172.21.15.16:0/216660610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ca4103220 0x7f4ca4196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:33.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.674+0000 7f4ca9e87640 1 --2- 172.21.15.16:0/216660610 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ca41008f0 0x7f4ca4195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:33.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.674+0000 7f4ca9e87640 1 -- 172.21.15.16:0/216660610 >> 172.21.15.16:0/216660610 conn(0x7f4ca40fa430 msgr2=0x7f4ca4101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:33.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.675+0000 7f4ca9e87640 1 -- 172.21.15.16:0/216660610 shutdown_connections 2024-09-13T16:30:33.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.675+0000 7f4ca9e87640 1 -- 172.21.15.16:0/216660610 wait complete. 2024-09-13T16:30:33.687 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:30:33.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.789+0000 7fdd36183640 1 -- 172.21.15.16:0/3624872084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd30100850 msgr2=0x7fdd30102c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:33.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.789+0000 7fdd36183640 1 --2- 172.21.15.16:0/3624872084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd30100850 0x7fdd30102c40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fdd20009900 tx=0x7fdd2002a410 comp rx=0 tx=0).stop 2024-09-13T16:30:33.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.789+0000 7fdd36183640 1 -- 172.21.15.16:0/3624872084 shutdown_connections 2024-09-13T16:30:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.789+0000 7fdd36183640 1 --2- 172.21.15.16:0/3624872084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd30105ab0 0x7fdd30107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.789+0000 7fdd36183640 1 --2- 172.21.15.16:0/3624872084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd30103180 0x7fdd30105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.789+0000 7fdd36183640 1 --2- 172.21.15.16:0/3624872084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd30100850 0x7fdd30102c40 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.789+0000 7fdd36183640 1 -- 172.21.15.16:0/3624872084 >> 172.21.15.16:0/3624872084 conn(0x7fdd300fa430 msgr2=0x7fdd300fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.789+0000 7fdd36183640 1 -- 172.21.15.16:0/3624872084 shutdown_connections 2024-09-13T16:30:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.789+0000 7fdd36183640 1 -- 172.21.15.16:0/3624872084 wait complete. 2024-09-13T16:30:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.790+0000 7fdd36183640 1 Processor -- start 2024-09-13T16:30:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.790+0000 7fdd36183640 1 -- start start 2024-09-13T16:30:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.790+0000 7fdd36183640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd30100850 0x7fdd30195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.790+0000 7fdd36183640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd30103180 0x7fdd301962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.790+0000 7fdd36183640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd30105ab0 0x7fdd3019d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.790+0000 7fdd36183640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3019d890 con 0x7fdd30103180 2024-09-13T16:30:33.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.790+0000 7fdd36183640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3019da00 con 0x7fdd30105ab0 2024-09-13T16:30:33.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.790+0000 7fdd36183640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3019dbc0 con 0x7fdd30100850 2024-09-13T16:30:33.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.790+0000 7fdd2effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd30103180 0x7fdd301962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:33.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.790+0000 7fdd2effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd30103180 0x7fdd301962f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52096/0 (socket says 172.21.15.16:52096) 2024-09-13T16:30:33.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.790+0000 7fdd2effd640 1 -- 172.21.15.16:0/1124412342 learned_addr learned my addr 172.21.15.16:0/1124412342 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:33.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.790+0000 7fdd2ffff640 1 --2- 172.21.15.16:0/1124412342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd30105ab0 0x7fdd3019d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:33.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.790+0000 7fdd2f7fe640 1 --2- 172.21.15.16:0/1124412342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd30100850 0x7fdd30195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:33.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.790+0000 7fdd2effd640 1 -- 172.21.15.16:0/1124412342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd30100850 msgr2=0x7fdd30195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:33.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.790+0000 7fdd2effd640 1 --2- 172.21.15.16:0/1124412342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd30100850 0x7fdd30195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:33.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.790+0000 7fdd2effd640 1 -- 172.21.15.16:0/1124412342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd30105ab0 msgr2=0x7fdd3019d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:33.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.790+0000 7fdd2effd640 1 --2- 172.21.15.16:0/1124412342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd30105ab0 0x7fdd3019d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:33.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.790+0000 7fdd2effd640 1 -- 172.21.15.16:0/1124412342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd20009560 con 0x7fdd30103180 2024-09-13T16:30:33.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.791+0000 7fdd2effd640 1 --2- 172.21.15.16:0/1124412342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd30103180 0x7fdd301962f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdd2400d740 tx=0x7fdd2400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:33.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.791+0000 7fdd2cff9640 1 -- 172.21.15.16:0/1124412342 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd24009c30 con 0x7fdd30103180 2024-09-13T16:30:33.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.791+0000 7fdd2cff9640 1 -- 172.21.15.16:0/1124412342 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd2400de50 con 0x7fdd30103180 2024-09-13T16:30:33.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.791+0000 7fdd36183640 1 -- 172.21.15.16:0/1124412342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd3019dea0 con 0x7fdd30103180 2024-09-13T16:30:33.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.791+0000 7fdd2cff9640 1 -- 172.21.15.16:0/1124412342 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd24004d30 con 0x7fdd30103180 2024-09-13T16:30:33.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.791+0000 7fdd36183640 1 -- 172.21.15.16:0/1124412342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd3019e270 con 0x7fdd30103180 2024-09-13T16:30:33.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.792+0000 7fdd36183640 1 -- 172.21.15.16:0/1124412342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd30103560 con 0x7fdd30103180 2024-09-13T16:30:33.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.793+0000 7fdd2cff9640 1 -- 172.21.15.16:0/1124412342 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fdd24004e90 con 0x7fdd30103180 2024-09-13T16:30:33.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.794+0000 7fdd2cff9640 1 --2- 172.21.15.16:0/1124412342 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdd180a1120 0x7fdd180a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:33.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.794+0000 7fdd2cff9640 1 -- 172.21.15.16:0/1124412342 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fdd240cc920 con 0x7fdd30103180 2024-09-13T16:30:33.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.794+0000 7fdd2f7fe640 1 --2- 172.21.15.16:0/1124412342 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdd180a1120 0x7fdd180a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:33.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.795+0000 7fdd2f7fe640 1 --2- 172.21.15.16:0/1124412342 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdd180a1120 0x7fdd180a35e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdd20006010 tx=0x7fdd20005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:33.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.798+0000 7fdd2cff9640 1 -- 172.21.15.16:0/1124412342 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fdd24095b10 con 0x7fdd30103180 2024-09-13T16:30:33.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:33 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1581657843' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T16:30:33.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:33 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/25297841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T16:30:33.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:33 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/2477677346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T16:30:33.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:33 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/216660610' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:30:33.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.973+0000 7fdd36183640 1 -- 172.21.15.16:0/1124412342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd3010b5f0 con 0x7fdd30103180 2024-09-13T16:30:33.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.974+0000 7fdd2cff9640 1 -- 172.21.15.16:0/1124412342 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24593 (secure 0 0 0) 0x7fdd24020090 con 0x7fdd30103180 2024-09-13T16:30:33.976 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:30:33.976 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":93,"fsid":"0d241f6a-71ec-11ef-bceb-c7b262605968","created":"2024-09-13T16:21:17.971738+0000","modified":"2024-09-13T16:30:05.681521+0000","last_up_change":"2024-09-13T16:27:54.673413+0000","last_in_change":"2024-09-13T16:27:35.718390+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:24:22.981019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:28:37.746027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:28:38.602731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:28:41.633322+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e11ef8f-08d6-4e74-afd6-f49f4bd40032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6803","nonce":2079394186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6805","nonce":2079394186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6809","nonce":2079394186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6807","nonce":2079394186}]},"public_addr":"172.21.15.16:6803/2079394186","cluster_addr":"172.21.15.16:6805/2079394186","heartbeat_back_addr":"172.21.15.16:6809/2079394186","heartbeat_front_addr":"172.21.15.16:6807/2079394186","state":["exists","up"]},{"osd":1,"uuid":"f2cd3694-9c9e-4da7-976e-f80a8df120f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6801","nonce":4021360958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6803","nonce":4021360958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6807","nonce":4021360958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6805","nonce":4021360958}]},"public_addr":"172.21.15.107:6801/4021360958","cluster_addr":"172.21.15.107:6803/4021360958","heartbeat_back_addr":"172.21.15.107:6807/4021360958","heartbeat_front_addr":"172.21.15.107:6805/4021360958","state":["exists","up"]},{"osd":2,"uuid":"a5bbf2eb-6599-44d5-ad96-68bb89ee19f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6801","nonce":3227179825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6803","nonce":3227179825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6807","nonce":3227179825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6805","nonce":3227179825}]},"public_addr":"172.21.15.111:6801/3227179825","cluster_addr":"172.21.15.111:6803/3227179825","heartbeat_back_addr":"172.21.15.111:6807/3227179825","heartbeat_front_addr":"172.21.15.111:6805/3227179825","state":["exists","up"]},{"osd":3,"uuid":"d6295ffe-11bc-4964-a250-9fecca870b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6811","nonce":2390375937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6813","nonce":2390375937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6817","nonce":2390375937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6815","nonce":2390375937}]},"public_addr":"172.21.15.16:6811/2390375937","cluster_addr":"172.21.15.16:6813/2390375937","heartbeat_back_addr":"172.21.15.16:6817/2390375937","heartbeat_front_addr":"172.21.15.16:6815/2390375937","state":["exists","up"]},{"osd":4,"uuid":"0a8446e0-fc86-4646-beb6-a7b9d433093d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6809","nonce":1936582048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6811","nonce":1936582048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6815","nonce":1936582048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6813","nonce":1936582048}]},"public_addr":"172.21.15.107:6809/1936582048","cluster_addr":"172.21.15.107:6811/1936582048","heartbeat_back_addr":"172.21.15.107:6815/1936582048","heartbeat_front_addr":"172.21.15.107:6813/1936582048","state":["exists","up"]},{"osd":5,"uuid":"2c97a446-357e-4975-9e51-9c95fe84a663","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6809","nonce":483858448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6811","nonce":483858448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6815","nonce":483858448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6813","nonce":483858448}]},"public_addr":"172.21.15.111:6809/483858448","cluster_addr":"172.21.15.111:6811/483858448","heartbeat_back_addr":"172.21.15.111:6815/483858448","heartbeat_front_addr":"172.21.15.111:6813/483858448","state":["exists","up"]},{"osd":6,"uuid":"890239e6-e6fb-4f9c-b973-05cc66d199bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6819","nonce":1330214141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6821","nonce":1330214141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6825","nonce":1330214141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6823","nonce":1330214141}]},"public_addr":"172.21.15.16:6819/1330214141","cluster_addr":"172.21.15.16:6821/1330214141","heartbeat_back_addr":"172.21.15.16:6825/1330214141","heartbeat_front_addr":"172.21.15.16:6823/1330214141","state":["exists","up"]},{"osd":7,"uuid":"ee826a9c-f441-4903-9b1b-66b627f7b712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6817","nonce":1133876754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6819","nonce":1133876754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6823","nonce":1133876754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6821","nonce":1133876754}]},"public_addr":"172.21.15.107:6817/1133876754","cluster_addr":"172.21.15.107:6819/1133876754","heartbeat_back_addr":"172.21.15.107:6823/1133876754","heartbeat_front_addr":"172.21.15.107:6821/1133876754","state":["exists","up"]},{"osd":8,"uuid":"e19773f2-dec8-44e0-9d33-0252e07d6415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6817","nonce":837548190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6819","nonce":837548190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6823","nonce":837548190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6821","nonce":837548190}]},"public_addr":"172.21.15.111:6817/837548190","cluster_addr":"172.21.15.111:6819/837548190","heartbeat_back_addr":"172.21.15.111:6823/837548190","heartbeat_front_addr":"172.21.15.111:6821/837548190","state":["exists","up"]},{"osd":9,"uuid":"bae4d6db-8b90-4d02-8e83-7d8f2a6f8fdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6827","nonce":4175383686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6829","nonce":4175383686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6833","nonce":4175383686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6831","nonce":4175383686}]},"public_addr":"172.21.15.16:6827/4175383686","cluster_addr":"172.21.15.16:6829/4175383686","heartbeat_back_addr":"172.21.15.16:6833/4175383686","heartbeat_front_addr":"172.21.15.16:6831/4175383686","state":["exists","up"]},{"osd":10,"uuid":"fd0e7740-76c8-4c99-a349-7077b4905143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6825","nonce":2659550426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6827","nonce":2659550426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6831","nonce":2659550426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6829","nonce":2659550426}]},"public_addr":"172.21.15.107:6825/2659550426","cluster_addr":"172.21.15.107:6827/2659550426","heartbeat_back_addr":"172.21.15.107:6831/2659550426","heartbeat_front_addr":"172.21.15.107:6829/2659550426","state":["exists","up"]},{"osd":11,"uuid":"94a1751d-b00c-4e23-815d-3a8bc3dea146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6825","nonce":1633548683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6827","nonce":1633548683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6831","nonce":1633548683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6829","nonce":1633548683}]},"public_addr":"172.21.15.111:6825/1633548683","cluster_addr":"172.21.15.111:6827/1633548683","heartbeat_back_addr":"172.21.15.111:6831/1633548683","heartbeat_front_addr":"172.21.15.111:6829/1633548683","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:36.359852+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:58.014202+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:19.753672+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:43.404817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:06.001677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:29.361587+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:52.554847+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:15.957553+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:39.581093+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:04.911389+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:28.971280+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:53.473561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/2098378667":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/3536229016":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6801/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/2684687547":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6800/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/368410612":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/12158160":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6800/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6801/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/3754423379":"2024-09-14T16:21:40.126459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:30:33.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.977+0000 7fdd36183640 1 -- 172.21.15.16:0/1124412342 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdd180a1120 msgr2=0x7fdd180a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:33.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.977+0000 7fdd36183640 1 --2- 172.21.15.16:0/1124412342 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdd180a1120 0x7fdd180a35e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdd20006010 tx=0x7fdd20005a20 comp rx=0 tx=0).stop 2024-09-13T16:30:33.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.977+0000 7fdd36183640 1 -- 172.21.15.16:0/1124412342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd30103180 msgr2=0x7fdd301962f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:33.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.977+0000 7fdd36183640 1 --2- 172.21.15.16:0/1124412342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd30103180 0x7fdd301962f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdd2400d740 tx=0x7fdd2400dc00 comp rx=0 tx=0).stop 2024-09-13T16:30:33.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.977+0000 7fdd36183640 1 -- 172.21.15.16:0/1124412342 shutdown_connections 2024-09-13T16:30:33.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.977+0000 7fdd36183640 1 --2- 172.21.15.16:0/1124412342 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdd180a1120 0x7fdd180a35e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:33.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.977+0000 7fdd36183640 1 --2- 172.21.15.16:0/1124412342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd30105ab0 0x7fdd3019d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:33.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.977+0000 7fdd36183640 1 --2- 172.21.15.16:0/1124412342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd30103180 0x7fdd301962f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:33.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.977+0000 7fdd36183640 1 --2- 172.21.15.16:0/1124412342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd30100850 0x7fdd30195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:33.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.977+0000 7fdd36183640 1 -- 172.21.15.16:0/1124412342 >> 172.21.15.16:0/1124412342 conn(0x7fdd300fa430 msgr2=0x7fdd301012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:33.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.977+0000 7fdd36183640 1 -- 172.21.15.16:0/1124412342 shutdown_connections 2024-09-13T16:30:33.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:33.977+0000 7fdd36183640 1 -- 172.21.15.16:0/1124412342 wait complete. 2024-09-13T16:30:33.987 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:33 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1581657843' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T16:30:33.987 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:33 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/25297841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T16:30:33.988 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:33 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2477677346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T16:30:33.988 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:33 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/216660610' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:30:33.989 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:30:34.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.091+0000 7f976688e640 1 -- 172.21.15.16:0/1575873909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9760103180 msgr2=0x7f9760105570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.091+0000 7f976688e640 1 --2- 172.21.15.16:0/1575873909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9760103180 0x7f9760105570 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9748009900 tx=0x7f974802a410 comp rx=0 tx=0).stop 2024-09-13T16:30:34.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.091+0000 7f976688e640 1 -- 172.21.15.16:0/1575873909 shutdown_connections 2024-09-13T16:30:34.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.091+0000 7f976688e640 1 --2- 172.21.15.16:0/1575873909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9760105ab0 0x7f9760107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.091+0000 7f976688e640 1 --2- 172.21.15.16:0/1575873909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9760103180 0x7f9760105570 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.091+0000 7f976688e640 1 --2- 172.21.15.16:0/1575873909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9760100850 0x7f9760102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.091+0000 7f976688e640 1 -- 172.21.15.16:0/1575873909 >> 172.21.15.16:0/1575873909 conn(0x7f97600fa430 msgr2=0x7f97600fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:34.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.091+0000 7f976688e640 1 -- 172.21.15.16:0/1575873909 shutdown_connections 2024-09-13T16:30:34.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.091+0000 7f976688e640 1 -- 172.21.15.16:0/1575873909 wait complete. 2024-09-13T16:30:34.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.092+0000 7f976688e640 1 Processor -- start 2024-09-13T16:30:34.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.092+0000 7f976688e640 1 -- start start 2024-09-13T16:30:34.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.092+0000 7f976688e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9760100850 0x7f97601001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.092+0000 7f976688e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9760103180 0x7f97600fe800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.092+0000 7f976688e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9760105ab0 0x7f97600fed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.092+0000 7f976688e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97600ff4a0 con 0x7f9760105ab0 2024-09-13T16:30:34.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.092+0000 7f976688e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97600ff610 con 0x7f9760103180 2024-09-13T16:30:34.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.092+0000 7f976688e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976019fd50 con 0x7f9760100850 2024-09-13T16:30:34.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.092+0000 7f9764e2b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9760105ab0 0x7f97600fed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.092+0000 7f9764e2b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9760105ab0 0x7f97600fed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52112/0 (socket says 172.21.15.16:52112) 2024-09-13T16:30:34.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.092+0000 7f9764e2b640 1 -- 172.21.15.16:0/1034819777 learned_addr learned my addr 172.21.15.16:0/1034819777 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:34.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.092+0000 7f9757fff640 1 --2- 172.21.15.16:0/1034819777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9760103180 0x7f97600fe800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.092+0000 7f975ffff640 1 --2- 172.21.15.16:0/1034819777 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9760100850 0x7f97601001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.092+0000 7f9764e2b640 1 -- 172.21.15.16:0/1034819777 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9760100850 msgr2=0x7f97601001c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.092+0000 7f9764e2b640 1 --2- 172.21.15.16:0/1034819777 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9760100850 0x7f97601001c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.092+0000 7f9764e2b640 1 -- 172.21.15.16:0/1034819777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9760103180 msgr2=0x7f97600fe800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.092+0000 7f9764e2b640 1 --2- 172.21.15.16:0/1034819777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9760103180 0x7f97600fe800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.092+0000 7f9764e2b640 1 -- 172.21.15.16:0/1034819777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9748009560 con 0x7f9760105ab0 2024-09-13T16:30:34.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.092+0000 7f9764e2b640 1 --2- 172.21.15.16:0/1034819777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9760105ab0 0x7f97600fed40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f975800cce0 tx=0x7f975800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:34.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.093+0000 7f975dffb640 1 -- 172.21.15.16:0/1034819777 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975800f450 con 0x7f9760105ab0 2024-09-13T16:30:34.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.093+0000 7f975dffb640 1 -- 172.21.15.16:0/1034819777 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f975800fa40 con 0x7f9760105ab0 2024-09-13T16:30:34.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.093+0000 7f976688e640 1 -- 172.21.15.16:0/1034819777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f976019ffa0 con 0x7f9760105ab0 2024-09-13T16:30:34.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.093+0000 7f975dffb640 1 -- 172.21.15.16:0/1034819777 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97580189d0 con 0x7f9760105ab0 2024-09-13T16:30:34.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.093+0000 7f976688e640 1 -- 172.21.15.16:0/1034819777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97601a0480 con 0x7f9760105ab0 2024-09-13T16:30:34.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.094+0000 7f976688e640 1 -- 172.21.15.16:0/1034819777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97600fbf20 con 0x7f9760105ab0 2024-09-13T16:30:34.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.095+0000 7f975dffb640 1 -- 172.21.15.16:0/1034819777 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f975800fbb0 con 0x7f9760105ab0 2024-09-13T16:30:34.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.095+0000 7f975dffb640 1 --2- 172.21.15.16:0/1034819777 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f974c0a11a0 0x7f974c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.096+0000 7f975ffff640 1 --2- 172.21.15.16:0/1034819777 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f974c0a11a0 0x7f974c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.096+0000 7f975dffb640 1 -- 172.21.15.16:0/1034819777 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f97580cd220 con 0x7f9760105ab0 2024-09-13T16:30:34.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.096+0000 7f975ffff640 1 --2- 172.21.15.16:0/1034819777 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f974c0a11a0 0x7f974c0a3660 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9750005fd0 tx=0x7f9750005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:34.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:33 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1581657843' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T16:30:34.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:33 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/25297841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T16:30:34.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:33 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/2477677346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T16:30:34.100 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:33 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/216660610' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:30:34.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.100+0000 7f975dffb640 1 -- 172.21.15.16:0/1034819777 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f9758096500 con 0x7f9760105ab0 2024-09-13T16:30:34.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.305+0000 7f976688e640 1 -- 172.21.15.16:0/1034819777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f976002c410 con 0x7f9760105ab0 2024-09-13T16:30:34.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.305+0000 7f975dffb640 1 -- 172.21.15.16:0/1034819777 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10924 (secure 0 0 0) 0x7f9758099b20 con 0x7f9760105ab0 2024-09-13T16:30:34.307 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:30:34.307 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3292781664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3292781664},{"type":"v1","addr":"172.21.15.16:6835","nonce":3292781664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4109039027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4109039027},{"type":"v1","addr":"172.21.15.16:6837","nonce":4109039027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T16:28:39.184734+0000","modified":"2024-09-13T16:29:06.636937+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24518,"mds_1":24490,"mds_2":24485,"mds_3":24422,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/2391618693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2391618693},{"type":"v1","addr":"172.21.15.16:6839","nonce":2391618693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/521042681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":521042681},{"type":"v1","addr":"172.21.15.16:6841","nonce":521042681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24418":{"gid":24418,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.111:6833/1984275936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1984275936},{"type":"v1","addr":"172.21.15.111:6833","nonce":1984275936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.107:6833/3797712690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3797712690},{"type":"v1","addr":"172.21.15.107:6833","nonce":3797712690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/338548468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":338548468},{"type":"v1","addr":"172.21.15.107:6835","nonce":338548468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/662156111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":662156111},{"type":"v1","addr":"172.21.15.111:6835","nonce":662156111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/1877128453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1877128453},{"type":"v1","addr":"172.21.15.111:6837","nonce":1877128453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.107:6837/2838457611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2838457611},{"type":"v1","addr":"172.21.15.107:6837","nonce":2838457611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.111:6839/1423842285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":1423842285},{"type":"v1","addr":"172.21.15.111:6839","nonce":1423842285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6839/1914689806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1914689806},{"type":"v1","addr":"172.21.15.107:6839","nonce":1914689806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T16:30:34.307 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-13T16:30:34.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.308+0000 7f976688e640 1 -- 172.21.15.16:0/1034819777 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f974c0a11a0 msgr2=0x7f974c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.308+0000 7f976688e640 1 --2- 172.21.15.16:0/1034819777 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f974c0a11a0 0x7f974c0a3660 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9750005fd0 tx=0x7f9750005850 comp rx=0 tx=0).stop 2024-09-13T16:30:34.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.308+0000 7f976688e640 1 -- 172.21.15.16:0/1034819777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9760105ab0 msgr2=0x7f97600fed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.308+0000 7f976688e640 1 --2- 172.21.15.16:0/1034819777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9760105ab0 0x7f97600fed40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f975800cce0 tx=0x7f975800a620 comp rx=0 tx=0).stop 2024-09-13T16:30:34.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.308+0000 7f976688e640 1 -- 172.21.15.16:0/1034819777 shutdown_connections 2024-09-13T16:30:34.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.308+0000 7f976688e640 1 --2- 172.21.15.16:0/1034819777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9760105ab0 0x7f97600fed40 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.308+0000 7f976688e640 1 --2- 172.21.15.16:0/1034819777 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f974c0a11a0 0x7f974c0a3660 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.308+0000 7f976688e640 1 --2- 172.21.15.16:0/1034819777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9760103180 0x7f97600fe800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.308+0000 7f976688e640 1 --2- 172.21.15.16:0/1034819777 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9760100850 0x7f97601001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.308+0000 7f976688e640 1 -- 172.21.15.16:0/1034819777 >> 172.21.15.16:0/1034819777 conn(0x7f97600fa430 msgr2=0x7f97601012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:34.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.308+0000 7f976688e640 1 -- 172.21.15.16:0/1034819777 shutdown_connections 2024-09-13T16:30:34.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.308+0000 7f976688e640 1 -- 172.21.15.16:0/1034819777 wait complete. 2024-09-13T16:30:34.320 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-13T16:28:39.184734+0000', 'modified': '2024-09-13T16:29:06.636937+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24518, 'mds_1': 24490, 'mds_2': 24485, 'mds_3': 24422, 'mds_4': 24418}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14586': {'gid': 14586, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6839/2391618693', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6838', 'nonce': 2391618693}, {'type': 'v1', 'addr': '172.21.15.16:6839', 'nonce': 2391618693}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14598': {'gid': 14598, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6841/521042681', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6840', 'nonce': 521042681}, {'type': 'v1', 'addr': '172.21.15.16:6841', 'nonce': 521042681}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24418': {'gid': 24418, 'name': 'l', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.111:6833/1984275936', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6832', 'nonce': 1984275936}, {'type': 'v1', 'addr': '172.21.15.111:6833', 'nonce': 1984275936}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24422': {'gid': 24422, 'name': 'k', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.107:6833/3797712690', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6832', 'nonce': 3797712690}, {'type': 'v1', 'addr': '172.21.15.107:6833', 'nonce': 3797712690}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24446': {'gid': 24446, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.107:6835/338548468', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6834', 'nonce': 338548468}, {'type': 'v1', 'addr': '172.21.15.107:6835', 'nonce': 338548468}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.111:6835/662156111', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6834', 'nonce': 662156111}, {'type': 'v1', 'addr': '172.21.15.111:6835', 'nonce': 662156111}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24472': {'gid': 24472, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.111:6837/1877128453', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6836', 'nonce': 1877128453}, {'type': 'v1', 'addr': '172.21.15.111:6837', 'nonce': 1877128453}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24485': {'gid': 24485, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.107:6837/2838457611', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6836', 'nonce': 2838457611}, {'type': 'v1', 'addr': '172.21.15.107:6837', 'nonce': 2838457611}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24490': {'gid': 24490, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.111:6839/1423842285', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6838', 'nonce': 1423842285}, {'type': 'v1', 'addr': '172.21.15.111:6839', 'nonce': 1423842285}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24518': {'gid': 24518, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.107:6839/1914689806', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6838', 'nonce': 1914689806}, {'type': 'v1', 'addr': '172.21.15.107:6839', 'nonce': 1914689806}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T16:30:34.320 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-13T16:30:34.321 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-09-13T16:30:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.422+0000 7f5afbf18640 1 -- 172.21.15.16:0/4220341260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5af40fef20 msgr2=0x7f5af40ff380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.422+0000 7f5afbf18640 1 --2- 172.21.15.16:0/4220341260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5af40fef20 0x7f5af40ff380 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4009900 tx=0x7f5ae402a410 comp rx=0 tx=0).stop 2024-09-13T16:30:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.423+0000 7f5afbf18640 1 -- 172.21.15.16:0/4220341260 shutdown_connections 2024-09-13T16:30:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.423+0000 7f5afbf18640 1 --2- 172.21.15.16:0/4220341260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5af40ff8c0 0x7f5af40ffd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.423+0000 7f5afbf18640 1 --2- 172.21.15.16:0/4220341260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5af40fef20 0x7f5af40ff380 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.423+0000 7f5afbf18640 1 --2- 172.21.15.16:0/4220341260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5af40fe5e0 0x7f5af40fe9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.423+0000 7f5afbf18640 1 -- 172.21.15.16:0/4220341260 >> 172.21.15.16:0/4220341260 conn(0x7f5af40fa410 msgr2=0x7f5af40fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.423+0000 7f5afbf18640 1 -- 172.21.15.16:0/4220341260 shutdown_connections 2024-09-13T16:30:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.423+0000 7f5afbf18640 1 -- 172.21.15.16:0/4220341260 wait complete. 2024-09-13T16:30:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.423+0000 7f5afbf18640 1 Processor -- start 2024-09-13T16:30:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.423+0000 7f5afbf18640 1 -- start start 2024-09-13T16:30:34.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.423+0000 7f5afbf18640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5af40fe5e0 0x7f5af4197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.423+0000 7f5afbf18640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5af40fef20 0x7f5af41983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.423+0000 7f5afbf18640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5af40ff8c0 0x7f5af419f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.423+0000 7f5afbf18640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af419f9a0 con 0x7f5af40fe5e0 2024-09-13T16:30:34.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.423+0000 7f5afbf18640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af419fb10 con 0x7f5af40fef20 2024-09-13T16:30:34.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.423+0000 7f5afbf18640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af419fcd0 con 0x7f5af40ff8c0 2024-09-13T16:30:34.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.423+0000 7f5af9cb4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5af40fe5e0 0x7f5af4197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.424+0000 7f5af9cb4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5af40fe5e0 0x7f5af4197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52138/0 (socket says 172.21.15.16:52138) 2024-09-13T16:30:34.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.424+0000 7f5af9cb4640 1 -- 172.21.15.16:0/3152567478 learned_addr learned my addr 172.21.15.16:0/3152567478 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:34.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.424+0000 7f5af94b3640 1 --2- 172.21.15.16:0/3152567478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5af40fef20 0x7f5af41983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.424+0000 7f5af9cb4640 1 -- 172.21.15.16:0/3152567478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5af40ff8c0 msgr2=0x7f5af419f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.424+0000 7f5af9cb4640 1 --2- 172.21.15.16:0/3152567478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5af40ff8c0 0x7f5af419f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.424+0000 7f5af9cb4640 1 -- 172.21.15.16:0/3152567478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5af40fef20 msgr2=0x7f5af41983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.424+0000 7f5af9cb4640 1 --2- 172.21.15.16:0/3152567478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5af40fef20 0x7f5af41983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.424+0000 7f5af9cb4640 1 -- 172.21.15.16:0/3152567478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae4009560 con 0x7f5af40fe5e0 2024-09-13T16:30:34.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.424+0000 7f5af94b3640 1 --2- 172.21.15.16:0/3152567478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5af40fef20 0x7f5af41983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:30:34.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.424+0000 7f5af9cb4640 1 --2- 172.21.15.16:0/3152567478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5af40fe5e0 0x7f5af4197eb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5af000b4a0 tx=0x7f5af000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:34.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.424+0000 7f5aeaffd640 1 -- 172.21.15.16:0/3152567478 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af0010610 con 0x7f5af40fe5e0 2024-09-13T16:30:34.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.424+0000 7f5aeaffd640 1 -- 172.21.15.16:0/3152567478 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5af0010c00 con 0x7f5af40fe5e0 2024-09-13T16:30:34.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.424+0000 7f5afbf18640 1 -- 172.21.15.16:0/3152567478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af419ffb0 con 0x7f5af40fe5e0 2024-09-13T16:30:34.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.424+0000 7f5aeaffd640 1 -- 172.21.15.16:0/3152567478 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af0017ca0 con 0x7f5af40fe5e0 2024-09-13T16:30:34.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.424+0000 7f5afbf18640 1 -- 172.21.15.16:0/3152567478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af41a03e0 con 0x7f5af40fe5e0 2024-09-13T16:30:34.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.426+0000 7f5afec3a740 1 Processor -- start 2024-09-13T16:30:34.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.427+0000 7f5aeaffd640 1 -- 172.21.15.16:0/3152567478 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f5af0010770 con 0x7f5af40fe5e0 2024-09-13T16:30:34.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.427+0000 7f5afec3a740 1 -- start start 2024-09-13T16:30:34.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.427+0000 7f5afec3a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561311226500 0x561311226900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.427+0000 7f5afec3a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561311226e40 0x56131122b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.427+0000 7f5afec3a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56131122bed0 0x56131122e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.427+0000 7f5afec3a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613111817c0 con 0x561311226500 2024-09-13T16:30:34.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.427+0000 7f5afec3a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x561311190b70 con 0x561311226e40 2024-09-13T16:30:34.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.427+0000 7f5afec3a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x56131118e300 con 0x56131122bed0 2024-09-13T16:30:34.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.427+0000 7f5afa4b5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561311226500 0x561311226900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.427+0000 7f5afa4b5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561311226500 0x561311226900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52148/0 (socket says 172.21.15.16:52148) 2024-09-13T16:30:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.427+0000 7f5afa4b5640 1 -- 172.21.15.16:0/751926593 learned_addr learned my addr 172.21.15.16:0/751926593 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.427+0000 7f5af9cb4640 1 --2- 172.21.15.16:0/751926593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561311226e40 0x56131122b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.427+0000 7f5af94b3640 1 --2- 172.21.15.16:0/751926593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56131122bed0 0x56131122e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.427+0000 7f5aeaffd640 1 --2- 172.21.15.16:0/3152567478 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f5ae00a1120 0x7f5ae00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.427+0000 7f5afa4b5640 1 -- 172.21.15.16:0/751926593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56131122bed0 msgr2=0x56131122e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.427+0000 7f5afa4b5640 1 --2- 172.21.15.16:0/751926593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56131122bed0 0x56131122e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.427+0000 7f5afa4b5640 1 -- 172.21.15.16:0/751926593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561311226e40 msgr2=0x56131122b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.427+0000 7f5afa4b5640 1 --2- 172.21.15.16:0/751926593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561311226e40 0x56131122b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.427+0000 7f5afa4b5640 1 -- 172.21.15.16:0/751926593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613111164b0 con 0x561311226500 2024-09-13T16:30:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.428+0000 7f5aeaffd640 1 -- 172.21.15.16:0/3152567478 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f5af00cc750 con 0x7f5af40fe5e0 2024-09-13T16:30:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.428+0000 7f5afa4b5640 1 --2- 172.21.15.16:0/3152567478 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f5ae00a1120 0x7f5ae00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.428+0000 7f5afa4b5640 1 --2- 172.21.15.16:0/751926593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561311226500 0x561311226900 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5aec007cf0 tx=0x7f5aec00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb56ad8a01e6051 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:34.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.428+0000 7f5adffff640 1 -- 172.21.15.16:0/751926593 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec00fa90 con 0x561311226500 2024-09-13T16:30:34.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.428+0000 7f5adffff640 1 -- 172.21.15.16:0/751926593 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5aec0077a0 con 0x561311226500 2024-09-13T16:30:34.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.428+0000 7f5adffff640 1 -- 172.21.15.16:0/751926593 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec00e5b0 con 0x561311226500 2024-09-13T16:30:34.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.429+0000 7f5afec3a740 1 -- 172.21.15.16:0/751926593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561311226500 msgr2=0x561311226900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.429+0000 7f5afa4b5640 1 --2- 172.21.15.16:0/3152567478 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f5ae00a1120 0x7f5ae00a35e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5aec01b000 tx=0x7f5aec01e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:34.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.429+0000 7f5afec3a740 1 --2- 172.21.15.16:0/751926593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561311226500 0x561311226900 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5aec007cf0 tx=0x7f5aec00c6a0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.429+0000 7f5afec3a740 1 -- 172.21.15.16:0/751926593 shutdown_connections 2024-09-13T16:30:34.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.429+0000 7f5afec3a740 1 --2- 172.21.15.16:0/751926593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56131122bed0 0x56131122e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.429+0000 7f5afec3a740 1 --2- 172.21.15.16:0/751926593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561311226e40 0x56131122b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.429+0000 7f5afec3a740 1 --2- 172.21.15.16:0/751926593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561311226500 0x561311226900 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.429+0000 7f5afec3a740 1 -- 172.21.15.16:0/751926593 >> 172.21.15.16:0/751926593 conn(0x561311221b70 msgr2=0x561311221f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:34.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.429+0000 7f5afec3a740 1 -- 172.21.15.16:0/751926593 shutdown_connections 2024-09-13T16:30:34.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.429+0000 7f5afec3a740 1 -- 172.21.15.16:0/751926593 wait complete. 2024-09-13T16:30:34.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.429+0000 7f5afec3a740 1 -- start start 2024-09-13T16:30:34.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.429+0000 7f5afec3a740 1 Processor -- start 2024-09-13T16:30:34.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.430+0000 7f5afec3a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561311226500 0x561311230100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.430+0000 7f5afec3a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561311226e40 0x561311230640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.430+0000 7f5afec3a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56131122bed0 0x561311237680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.430+0000 7f5afec3a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561311189bf0 con 0x561311226500 2024-09-13T16:30:34.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.430+0000 7f5afec3a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x561311148250 con 0x56131122bed0 2024-09-13T16:30:34.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.430+0000 7f5afec3a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x561311147ef0 con 0x561311226e40 2024-09-13T16:30:34.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.430+0000 7f5afec3a740 10 client.0 _ll_register_callbacks cb 0x56131117a740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:30:34.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.430+0000 7f5af9cb4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561311226500 0x561311230100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.430+0000 7f5af9cb4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561311226500 0x561311230100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52158/0 (socket says 172.21.15.16:52158) 2024-09-13T16:30:34.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.430+0000 7f5af9cb4640 1 -- 172.21.15.16:0/2642721762 learned_addr learned my addr 172.21.15.16:0/2642721762 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:34.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.430+0000 7f5af94b3640 1 --2- 172.21.15.16:0/2642721762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561311226e40 0x561311230640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.430+0000 7f5afa4b5640 1 --2- 172.21.15.16:0/2642721762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56131122bed0 0x561311237680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.430+0000 7f5af9cb4640 1 -- 172.21.15.16:0/2642721762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561311226e40 msgr2=0x561311230640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:34.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.430+0000 7f5af9cb4640 1 --2- 172.21.15.16:0/2642721762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561311226e40 0x561311230640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.430+0000 7f5af9cb4640 1 -- 172.21.15.16:0/2642721762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56131122bed0 msgr2=0x561311237680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:34.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.430+0000 7f5af9cb4640 1 --2- 172.21.15.16:0/2642721762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56131122bed0 0x561311237680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.430+0000 7f5af9cb4640 1 -- 172.21.15.16:0/2642721762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aec007920 con 0x561311226500 2024-09-13T16:30:34.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.430+0000 7f5afa4b5640 1 --2- 172.21.15.16:0/2642721762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56131122bed0 0x561311237680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:30:34.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.430+0000 7f5af9cb4640 1 --2- 172.21.15.16:0/2642721762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561311226500 0x561311230100 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5af000be60 tx=0x7f5af0099400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a5ccbe57b2bba4f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:34.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.430+0000 7f5add7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:30:34.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.430+0000 7f5add7fa640 1 -- 172.21.15.16:0/2642721762 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af0099f00 con 0x561311226500 2024-09-13T16:30:34.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.431+0000 7f5add7fa640 1 -- 172.21.15.16:0/2642721762 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f5af0099800 con 0x561311226500 2024-09-13T16:30:34.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.431+0000 7f5afec3a740 1 -- 172.21.15.16:0/2642721762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561310e8e6c0 con 0x561311226500 2024-09-13T16:30:34.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.431+0000 7f5add7fa640 1 -- 172.21.15.16:0/2642721762 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5af009d630 con 0x561311226500 2024-09-13T16:30:34.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.431+0000 7f5afec3a740 10 client.15210 fetch_fsmap learned FSMap version 28 2024-09-13T16:30:34.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.431+0000 7f5afec3a740 1 -- 172.21.15.16:0/2642721762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561311188270 con 0x561311226500 2024-09-13T16:30:34.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.431+0000 7f5add7fa640 1 -- 172.21.15.16:0/2642721762 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5af00b1020 con 0x561311226500 2024-09-13T16:30:34.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.431+0000 7f5add7fa640 1 -- 172.21.15.16:0/2642721762 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5af00ced20 con 0x561311226500 2024-09-13T16:30:34.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.431+0000 7f5afec3a740 10 client.15210 fetch_fsmap finished waiting for FSMap version 28 2024-09-13T16:30:34.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.431+0000 7f5afec3a740 10 client.15210 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-13T16:30:34.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.431+0000 7f5afec3a740 20 client.15210 populate_metadata read hostname 'smithi016' 2024-09-13T16:30:34.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.431+0000 7f5afec3a740 1 --2- 172.21.15.16:0/2642721762 >> [v2:172.21.15.111:6832/1984275936,v1:172.21.15.111:6833/1984275936] conn(0x56131123bec0 0x561311240630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.431+0000 7f5afec3a740 4 client.15210 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T16:30:34.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.431+0000 7f5afec3a740 1 -- 172.21.15.16:0/2642721762 --> [v2:172.21.15.111:6832/1984275936,v1:172.21.15.111:6833/1984275936] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561310ee1d10 con 0x56131123bec0 2024-09-13T16:30:34.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.431+0000 7f5af94b3640 1 --2- 172.21.15.16:0/2642721762 >> [v2:172.21.15.111:6832/1984275936,v1:172.21.15.111:6833/1984275936] conn(0x56131123bec0 0x561311240630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.432+0000 7f5af94b3640 1 --2- 172.21.15.16:0/2642721762 >> [v2:172.21.15.111:6832/1984275936,v1:172.21.15.111:6833/1984275936] conn(0x56131123bec0 0x561311240630 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e8352a97d42d969d server_cookie=41104473a1bc2d39 in_seq=0 out_seq=0 2024-09-13T16:30:34.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.432+0000 7f5add7fa640 10 client.15210 ms_handle_connect on v2:172.21.15.111:6832/1984275936 2024-09-13T16:30:34.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.432+0000 7f5add7fa640 1 -- 172.21.15.16:0/2642721762 <== mds.4 v2:172.21.15.111:6832/1984275936 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561310ee1d10 con 0x56131123bec0 2024-09-13T16:30:34.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.432+0000 7f5add7fa640 10 client.15210 handle_command_reply: tid=0 2024-09-13T16:30:34.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.433+0000 7f5afec3a740 1 client.15210 shutdown 2024-09-13T16:30:34.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.433+0000 7f5afec3a740 1 -- 172.21.15.16:0/2642721762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561311226500 msgr2=0x561311230100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.433+0000 7f5afec3a740 1 --2- 172.21.15.16:0/2642721762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561311226500 0x561311230100 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5af000be60 tx=0x7f5af0099400 comp rx=0 tx=0).stop 2024-09-13T16:30:34.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.433+0000 7f5afec3a740 1 -- 172.21.15.16:0/2642721762 shutdown_connections 2024-09-13T16:30:34.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.433+0000 7f5afec3a740 1 --2- 172.21.15.16:0/2642721762 >> [v2:172.21.15.111:6832/1984275936,v1:172.21.15.111:6833/1984275936] conn(0x56131123bec0 0x561311240630 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.433+0000 7f5afec3a740 1 --2- 172.21.15.16:0/2642721762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56131122bed0 0x561311237680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.433+0000 7f5afec3a740 1 --2- 172.21.15.16:0/2642721762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561311226e40 0x561311230640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.433+0000 7f5add7fa640 0 client.15210 ms_handle_reset on v2:172.21.15.111:6832/1984275936 2024-09-13T16:30:34.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.433+0000 7f5afec3a740 1 --2- 172.21.15.16:0/2642721762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561311226500 0x561311230100 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.433+0000 7f5afec3a740 1 -- 172.21.15.16:0/2642721762 >> 172.21.15.16:0/2642721762 conn(0x561311221b70 msgr2=0x561311223380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:34.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.433+0000 7f5afec3a740 1 -- 172.21.15.16:0/2642721762 shutdown_connections 2024-09-13T16:30:34.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.433+0000 7f5afec3a740 1 -- 172.21.15.16:0/2642721762 wait complete. 2024-09-13T16:30:34.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.433+0000 7f5afec3a740 20 client.15210 trim_cache size 0 max 16384 2024-09-13T16:30:34.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.448+0000 7f5afec3a740 1 Processor -- start 2024-09-13T16:30:34.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.448+0000 7f5afec3a740 1 -- start start 2024-09-13T16:30:34.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.448+0000 7f5afec3a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561311226500 0x5613112305c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.448+0000 7f5afec3a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561311226e40 0x5613112375a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.448+0000 7f5afec3a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56131122bed0 0x561311244650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.448+0000 7f5afec3a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56131110d460 con 0x561311226500 2024-09-13T16:30:34.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.448+0000 7f5afec3a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56131114cdb0 con 0x56131122bed0 2024-09-13T16:30:34.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.448+0000 7f5afec3a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x561311186850 con 0x561311226e40 2024-09-13T16:30:34.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.448+0000 7f5af9cb4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561311226500 0x5613112305c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.448+0000 7f5af9cb4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561311226500 0x5613112305c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52168/0 (socket says 172.21.15.16:52168) 2024-09-13T16:30:34.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.448+0000 7f5af9cb4640 1 -- 172.21.15.16:0/2488071684 learned_addr learned my addr 172.21.15.16:0/2488071684 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:34.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.448+0000 7f5af94b3640 1 --2- 172.21.15.16:0/2488071684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561311226e40 0x5613112375a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.448+0000 7f5afa4b5640 1 --2- 172.21.15.16:0/2488071684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56131122bed0 0x561311244650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.448+0000 7f5af9cb4640 1 -- 172.21.15.16:0/2488071684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561311226e40 msgr2=0x5613112375a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:34.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.448+0000 7f5af9cb4640 1 --2- 172.21.15.16:0/2488071684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561311226e40 0x5613112375a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.449+0000 7f5af9cb4640 1 -- 172.21.15.16:0/2488071684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56131122bed0 msgr2=0x561311244650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:34.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.449+0000 7f5af9cb4640 1 --2- 172.21.15.16:0/2488071684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56131122bed0 0x561311244650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.449+0000 7f5af9cb4640 1 -- 172.21.15.16:0/2488071684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56131112a5f0 con 0x561311226500 2024-09-13T16:30:34.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.449+0000 7f5afa4b5640 1 --2- 172.21.15.16:0/2488071684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56131122bed0 0x561311244650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:30:34.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.449+0000 7f5af9cb4640 1 --2- 172.21.15.16:0/2488071684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561311226500 0x5613112305c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5af0099790 tx=0x7f5af00d0000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe56d775f57967a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:34.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.449+0000 7f5ae8ff9640 1 -- 172.21.15.16:0/2488071684 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af00092b0 con 0x561311226500 2024-09-13T16:30:34.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.449+0000 7f5ae8ff9640 1 -- 172.21.15.16:0/2488071684 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5af00b1070 con 0x561311226500 2024-09-13T16:30:34.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.449+0000 7f5ae8ff9640 1 -- 172.21.15.16:0/2488071684 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af0099800 con 0x561311226500 2024-09-13T16:30:34.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.449+0000 7f5afec3a740 1 -- 172.21.15.16:0/2488071684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561311226500 msgr2=0x5613112305c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.449+0000 7f5afec3a740 1 --2- 172.21.15.16:0/2488071684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561311226500 0x5613112305c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5af0099790 tx=0x7f5af00d0000 comp rx=0 tx=0).stop 2024-09-13T16:30:34.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.449+0000 7f5afec3a740 1 -- 172.21.15.16:0/2488071684 shutdown_connections 2024-09-13T16:30:34.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.449+0000 7f5afec3a740 1 --2- 172.21.15.16:0/2488071684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56131122bed0 0x561311244650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.449+0000 7f5afec3a740 1 --2- 172.21.15.16:0/2488071684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561311226e40 0x5613112375a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.449+0000 7f5afec3a740 1 --2- 172.21.15.16:0/2488071684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561311226500 0x5613112305c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.449+0000 7f5afec3a740 1 -- 172.21.15.16:0/2488071684 >> 172.21.15.16:0/2488071684 conn(0x561311221b70 msgr2=0x56131122feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.449+0000 7f5afec3a740 1 -- 172.21.15.16:0/2488071684 shutdown_connections 2024-09-13T16:30:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.450+0000 7f5afec3a740 1 -- 172.21.15.16:0/2488071684 wait complete. 2024-09-13T16:30:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.450+0000 7f5afec3a740 1 -- start start 2024-09-13T16:30:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.450+0000 7f5afec3a740 1 Processor -- start 2024-09-13T16:30:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.450+0000 7f5afec3a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561311226500 0x5613112324c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.450+0000 7f5afec3a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561311226e40 0x561311232a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.450+0000 7f5afec3a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56131122bed0 0x5613112922f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.450+0000 7f5afec3a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561311232f70 con 0x561311226500 2024-09-13T16:30:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.450+0000 7f5afec3a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613112330e0 con 0x561311226e40 2024-09-13T16:30:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.450+0000 7f5afec3a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x561311292860 con 0x56131122bed0 2024-09-13T16:30:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.450+0000 7f5afec3a740 10 client.0 _ll_register_callbacks cb 0x7f5aec007480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:30:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.450+0000 7f5af9cb4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561311226500 0x5613112324c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.450+0000 7f5af9cb4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561311226500 0x5613112324c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52170/0 (socket says 172.21.15.16:52170) 2024-09-13T16:30:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.450+0000 7f5af9cb4640 1 -- 172.21.15.16:0/4230160456 learned_addr learned my addr 172.21.15.16:0/4230160456 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:34.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.450+0000 7f5af94b3640 1 --2- 172.21.15.16:0/4230160456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561311226e40 0x561311232a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.450+0000 7f5afa4b5640 1 --2- 172.21.15.16:0/4230160456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56131122bed0 0x5613112922f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.451+0000 7f5af9cb4640 1 -- 172.21.15.16:0/4230160456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56131122bed0 msgr2=0x5613112922f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:34.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.451+0000 7f5af9cb4640 1 --2- 172.21.15.16:0/4230160456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56131122bed0 0x5613112922f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.451+0000 7f5af9cb4640 1 -- 172.21.15.16:0/4230160456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561311226e40 msgr2=0x561311232a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:34.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.451+0000 7f5af9cb4640 1 --2- 172.21.15.16:0/4230160456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561311226e40 0x561311232a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.451+0000 7f5af9cb4640 1 -- 172.21.15.16:0/4230160456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561311188270 con 0x561311226500 2024-09-13T16:30:34.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.451+0000 7f5af9cb4640 1 --2- 172.21.15.16:0/4230160456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561311226500 0x5613112324c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5af00a1750 tx=0x7f5af00b9380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f54749c2616e110 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:34.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.451+0000 7f5add7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:30:34.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.451+0000 7f5add7fa640 1 -- 172.21.15.16:0/4230160456 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af0099f70 con 0x561311226500 2024-09-13T16:30:34.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.451+0000 7f5add7fa640 1 -- 172.21.15.16:0/4230160456 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f5af00b2080 con 0x561311226500 2024-09-13T16:30:34.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.451+0000 7f5afec3a740 1 -- 172.21.15.16:0/4230160456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561310e7d3c0 con 0x561311226500 2024-09-13T16:30:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.451+0000 7f5add7fa640 1 -- 172.21.15.16:0/4230160456 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5af00092b0 con 0x561311226500 2024-09-13T16:30:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.451+0000 7f5afec3a740 10 client.15216 fetch_fsmap learned FSMap version 28 2024-09-13T16:30:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.451+0000 7f5afec3a740 1 -- 172.21.15.16:0/4230160456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56131114fd00 con 0x561311226500 2024-09-13T16:30:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.451+0000 7f5add7fa640 1 -- 172.21.15.16:0/4230160456 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561310e7d3c0 con 0x561311226500 2024-09-13T16:30:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.451+0000 7f5add7fa640 1 -- 172.21.15.16:0/4230160456 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5af0021070 con 0x561311226500 2024-09-13T16:30:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.451+0000 7f5afec3a740 10 client.15216 fetch_fsmap finished waiting for FSMap version 28 2024-09-13T16:30:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.451+0000 7f5afec3a740 10 client.15216 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-13T16:30:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.451+0000 7f5afec3a740 20 client.15216 populate_metadata read hostname 'smithi016' 2024-09-13T16:30:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.451+0000 7f5afec3a740 1 --2- 172.21.15.16:0/4230160456 >> [v2:172.21.15.111:6832/1984275936,v1:172.21.15.111:6833/1984275936] conn(0x561311296f10 0x561311293ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.451+0000 7f5afec3a740 4 client.15216 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T16:30:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.451+0000 7f5afec3a740 1 -- 172.21.15.16:0/4230160456 --> [v2:172.21.15.111:6832/1984275936,v1:172.21.15.111:6833/1984275936] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56131123c8c0 con 0x561311296f10 2024-09-13T16:30:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.452+0000 7f5af94b3640 1 --2- 172.21.15.16:0/4230160456 >> [v2:172.21.15.111:6832/1984275936,v1:172.21.15.111:6833/1984275936] conn(0x561311296f10 0x561311293ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.452+0000 7f5add7fa640 10 client.15216 ms_handle_connect on v2:172.21.15.111:6832/1984275936 2024-09-13T16:30:34.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.452+0000 7f5af94b3640 1 --2- 172.21.15.16:0/4230160456 >> [v2:172.21.15.111:6832/1984275936,v1:172.21.15.111:6833/1984275936] conn(0x561311296f10 0x561311293ac0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4d473ec1cbb6bff2 server_cookie=5f10a0f8fd31eba in_seq=0 out_seq=0 2024-09-13T16:30:34.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.452+0000 7f5add7fa640 1 -- 172.21.15.16:0/4230160456 <== mds.4 v2:172.21.15.111:6832/1984275936 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x7f5ae40053e0 con 0x561311296f10 2024-09-13T16:30:34.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.452+0000 7f5add7fa640 10 client.15216 handle_command_reply: tid=0 2024-09-13T16:30:34.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.452+0000 7f5afec3a740 1 client.15216 shutdown 2024-09-13T16:30:34.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.452+0000 7f5afec3a740 1 -- 172.21.15.16:0/4230160456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561311226500 msgr2=0x5613112324c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.453+0000 7f5afec3a740 1 --2- 172.21.15.16:0/4230160456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561311226500 0x5613112324c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5af00a1750 tx=0x7f5af00b9380 comp rx=0 tx=0).stop 2024-09-13T16:30:34.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.453+0000 7f5afec3a740 1 -- 172.21.15.16:0/4230160456 shutdown_connections 2024-09-13T16:30:34.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.453+0000 7f5afec3a740 1 --2- 172.21.15.16:0/4230160456 >> [v2:172.21.15.111:6832/1984275936,v1:172.21.15.111:6833/1984275936] conn(0x561311296f10 0x561311293ac0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.453+0000 7f5afec3a740 1 --2- 172.21.15.16:0/4230160456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56131122bed0 0x5613112922f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.453+0000 7f5afec3a740 1 --2- 172.21.15.16:0/4230160456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561311226e40 0x561311232a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.453+0000 7f5afec3a740 1 --2- 172.21.15.16:0/4230160456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561311226500 0x5613112324c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.453+0000 7f5add7fa640 0 client.15216 ms_handle_reset on v2:172.21.15.111:6832/1984275936 2024-09-13T16:30:34.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.453+0000 7f5afec3a740 1 -- 172.21.15.16:0/4230160456 >> 172.21.15.16:0/4230160456 conn(0x561311221b70 msgr2=0x561311244650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:34.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.453+0000 7f5afec3a740 1 -- 172.21.15.16:0/4230160456 shutdown_connections 2024-09-13T16:30:34.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.453+0000 7f5afec3a740 1 -- 172.21.15.16:0/4230160456 wait complete. 2024-09-13T16:30:34.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.453+0000 7f5afec3a740 20 client.15216 trim_cache size 0 max 16384 2024-09-13T16:30:34.461 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-13T16:30:34.461 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "0d241f6a-71ec-11ef-bceb-c7b262605968", 2024-09-13T16:30:34.461 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 4, 2024-09-13T16:30:34.461 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24418, 2024-09-13T16:30:34.461 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-13T16:30:34.461 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-13T16:30:34.462 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-13T16:30:34.462 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 89.825055771999999, 2024-09-13T16:30:34.462 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 28, 2024-09-13T16:30:34.462 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 93, 2024-09-13T16:30:34.462 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 91, 2024-09-13T16:30:34.462 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 148.06175026299999 2024-09-13T16:30:34.462 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-13T16:30:34.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.453+0000 7f5afbf18640 1 -- 172.21.15.16:0/3152567478 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f5ae00a1120 msgr2=0x7f5ae00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.453+0000 7f5afbf18640 1 --2- 172.21.15.16:0/3152567478 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f5ae00a1120 0x7f5ae00a35e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5aec01b000 tx=0x7f5aec01e000 comp rx=0 tx=0).stop 2024-09-13T16:30:34.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.453+0000 7f5afbf18640 1 -- 172.21.15.16:0/3152567478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5af40fe5e0 msgr2=0x7f5af4197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.453+0000 7f5afbf18640 1 --2- 172.21.15.16:0/3152567478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5af40fe5e0 0x7f5af4197eb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5af000b4a0 tx=0x7f5af000b960 comp rx=0 tx=0).stop 2024-09-13T16:30:34.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.453+0000 7f5afbf18640 1 -- 172.21.15.16:0/3152567478 shutdown_connections 2024-09-13T16:30:34.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.453+0000 7f5afbf18640 1 --2- 172.21.15.16:0/3152567478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5af40ff8c0 0x7f5af419f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.453+0000 7f5afbf18640 1 --2- 172.21.15.16:0/3152567478 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f5ae00a1120 0x7f5ae00a35e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.453+0000 7f5afbf18640 1 --2- 172.21.15.16:0/3152567478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5af40fef20 0x7f5af41983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.453+0000 7f5afbf18640 1 --2- 172.21.15.16:0/3152567478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5af40fe5e0 0x7f5af4197eb0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.453+0000 7f5afbf18640 1 -- 172.21.15.16:0/3152567478 >> 172.21.15.16:0/3152567478 conn(0x7f5af40fa410 msgr2=0x7f5af40fbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:34.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.453+0000 7f5afbf18640 1 -- 172.21.15.16:0/3152567478 shutdown_connections 2024-09-13T16:30:34.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.453+0000 7f5afbf18640 1 -- 172.21.15.16:0/3152567478 wait complete. 2024-09-13T16:30:34.465 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-09-13T16:30:34.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.589+0000 7fe8b26aa640 1 -- 172.21.15.16:0/2261590057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8ac105ab0 msgr2=0x7fe8ac107ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.589+0000 7fe8b26aa640 1 --2- 172.21.15.16:0/2261590057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8ac105ab0 0x7fe8ac107ec0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4009ca0 tx=0x7fe8a402a8c0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.589+0000 7fe8b26aa640 1 -- 172.21.15.16:0/2261590057 shutdown_connections 2024-09-13T16:30:34.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.589+0000 7fe8b26aa640 1 --2- 172.21.15.16:0/2261590057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8ac105ab0 0x7fe8ac107ec0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.589+0000 7fe8b26aa640 1 --2- 172.21.15.16:0/2261590057 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8ac103180 0x7fe8ac105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.589+0000 7fe8b26aa640 1 --2- 172.21.15.16:0/2261590057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8ac100850 0x7fe8ac102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.589+0000 7fe8b26aa640 1 -- 172.21.15.16:0/2261590057 >> 172.21.15.16:0/2261590057 conn(0x7fe8ac0fa410 msgr2=0x7fe8ac0fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:34.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.589+0000 7fe8b26aa640 1 -- 172.21.15.16:0/2261590057 shutdown_connections 2024-09-13T16:30:34.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.589+0000 7fe8b26aa640 1 -- 172.21.15.16:0/2261590057 wait complete. 2024-09-13T16:30:34.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.589+0000 7fe8b26aa640 1 Processor -- start 2024-09-13T16:30:34.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.589+0000 7fe8b26aa640 1 -- start start 2024-09-13T16:30:34.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.590+0000 7fe8b26aa640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8ac100850 0x7fe8ac195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.590+0000 7fe8b26aa640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8ac103180 0x7fe8ac196230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.590+0000 7fe8b26aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8ac105ab0 0x7fe8ac19d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.590+0000 7fe8b26aa640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac19d7b0 con 0x7fe8ac105ab0 2024-09-13T16:30:34.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.590+0000 7fe8b26aa640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac19d920 con 0x7fe8ac100850 2024-09-13T16:30:34.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.590+0000 7fe8b26aa640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac19dae0 con 0x7fe8ac103180 2024-09-13T16:30:34.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.590+0000 7fe8b0c47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8ac105ab0 0x7fe8ac19d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.590+0000 7fe8b0c47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8ac105ab0 0x7fe8ac19d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52186/0 (socket says 172.21.15.16:52186) 2024-09-13T16:30:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.590+0000 7fe8b0c47640 1 -- 172.21.15.16:0/1071697313 learned_addr learned my addr 172.21.15.16:0/1071697313 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.590+0000 7fe8ab7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8ac103180 0x7fe8ac196230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.590+0000 7fe8abfff640 1 --2- 172.21.15.16:0/1071697313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8ac100850 0x7fe8ac195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.590+0000 7fe8b0c47640 1 -- 172.21.15.16:0/1071697313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8ac103180 msgr2=0x7fe8ac196230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.590+0000 7fe8b0c47640 1 --2- 172.21.15.16:0/1071697313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8ac103180 0x7fe8ac196230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.590+0000 7fe8b0c47640 1 -- 172.21.15.16:0/1071697313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8ac100850 msgr2=0x7fe8ac195cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.590+0000 7fe8b0c47640 1 --2- 172.21.15.16:0/1071697313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8ac100850 0x7fe8ac195cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.590+0000 7fe8b0c47640 1 -- 172.21.15.16:0/1071697313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a4009900 con 0x7fe8ac105ab0 2024-09-13T16:30:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.590+0000 7fe8abfff640 1 --2- 172.21.15.16:0/1071697313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8ac100850 0x7fe8ac195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:30:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.590+0000 7fe8b0c47640 1 --2- 172.21.15.16:0/1071697313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8ac105ab0 0x7fe8ac19d240 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe8a402aeb0 tx=0x7fe8a4003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.591+0000 7fe8a97fa640 1 -- 172.21.15.16:0/1071697313 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a40307c0 con 0x7fe8ac105ab0 2024-09-13T16:30:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.591+0000 7fe8a97fa640 1 -- 172.21.15.16:0/1071697313 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe8a4040070 con 0x7fe8ac105ab0 2024-09-13T16:30:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.591+0000 7fe8b26aa640 1 -- 172.21.15.16:0/1071697313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8ac19dd60 con 0x7fe8ac105ab0 2024-09-13T16:30:34.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.591+0000 7fe8a97fa640 1 -- 172.21.15.16:0/1071697313 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a40043a0 con 0x7fe8ac105ab0 2024-09-13T16:30:34.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.591+0000 7fe8b26aa640 1 -- 172.21.15.16:0/1071697313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ac19e210 con 0x7fe8ac105ab0 2024-09-13T16:30:34.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.593+0000 7fe8b53d0740 1 Processor -- start 2024-09-13T16:30:34.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.593+0000 7fe8a97fa640 1 -- 172.21.15.16:0/1071697313 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fe8a4042030 con 0x7fe8ac105ab0 2024-09-13T16:30:34.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.593+0000 7fe8b53d0740 1 -- start start 2024-09-13T16:30:34.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.593+0000 7fe8b53d0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b724b35500 0x55b724b35900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.593+0000 7fe8b53d0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b724b35e40 0x55b724b3a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.593+0000 7fe8b53d0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b724b3aed0 0x55b724b3d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.593+0000 7fe8b53d0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b724a907c0 con 0x55b724b35e40 2024-09-13T16:30:34.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.593+0000 7fe8b53d0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b724a9fb70 con 0x55b724b35500 2024-09-13T16:30:34.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.593+0000 7fe8b53d0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b724a9d300 con 0x55b724b3aed0 2024-09-13T16:30:34.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.593+0000 7fe8abfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b724b35e40 0x55b724b3a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.594+0000 7fe8abfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b724b35e40 0x55b724b3a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52200/0 (socket says 172.21.15.16:52200) 2024-09-13T16:30:34.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.594+0000 7fe8abfff640 1 -- 172.21.15.16:0/3519107829 learned_addr learned my addr 172.21.15.16:0/3519107829 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:34.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.594+0000 7fe8ab7fe640 1 --2- 172.21.15.16:0/3519107829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b724b35500 0x55b724b35900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.594+0000 7fe8ab7fe640 1 --2- 172.21.15.16:0/3519107829 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b724b3aed0 0x55b724b3d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.594+0000 7fe8abfff640 1 -- 172.21.15.16:0/3519107829 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b724b3aed0 msgr2=0x55b724b3d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:34.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.594+0000 7fe8abfff640 1 --2- 172.21.15.16:0/3519107829 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b724b3aed0 0x55b724b3d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.594+0000 7fe8abfff640 1 -- 172.21.15.16:0/3519107829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b724b35500 msgr2=0x55b724b35900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:34.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.594+0000 7fe8abfff640 1 --2- 172.21.15.16:0/3519107829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b724b35500 0x55b724b35900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.594+0000 7fe8abfff640 1 -- 172.21.15.16:0/3519107829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b724a254b0 con 0x55b724b35e40 2024-09-13T16:30:34.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.594+0000 7fe8a97fa640 1 --2- 172.21.15.16:0/1071697313 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe8940a1120 0x7fe8940a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.594+0000 7fe8abfff640 1 --2- 172.21.15.16:0/3519107829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b724b35e40 0x55b724b3a900 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe89c009cd0 tx=0x7fe89c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5375fce200841b89 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:34.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.594+0000 7fe8ab7fe640 1 --2- 172.21.15.16:0/1071697313 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe8940a1120 0x7fe8940a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.594+0000 7fe8a97fa640 1 -- 172.21.15.16:0/1071697313 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fe8a40ee260 con 0x7fe8ac105ab0 2024-09-13T16:30:34.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.594+0000 7fe8927fc640 1 -- 172.21.15.16:0/3519107829 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c012450 con 0x55b724b35e40 2024-09-13T16:30:34.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.594+0000 7fe8927fc640 1 -- 172.21.15.16:0/3519107829 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe89c012a40 con 0x55b724b35e40 2024-09-13T16:30:34.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.594+0000 7fe8927fc640 1 -- 172.21.15.16:0/3519107829 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c00fa10 con 0x55b724b35e40 2024-09-13T16:30:34.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.595+0000 7fe8b53d0740 1 -- 172.21.15.16:0/3519107829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b724b35e40 msgr2=0x55b724b3a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.595+0000 7fe8b53d0740 1 --2- 172.21.15.16:0/3519107829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b724b35e40 0x55b724b3a900 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe89c009cd0 tx=0x7fe89c0117b0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.595+0000 7fe8ab7fe640 1 --2- 172.21.15.16:0/1071697313 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe8940a1120 0x7fe8940a35e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0009730 tx=0x7fe8a00096c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:34.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.595+0000 7fe8b53d0740 1 -- 172.21.15.16:0/3519107829 shutdown_connections 2024-09-13T16:30:34.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.595+0000 7fe8b53d0740 1 --2- 172.21.15.16:0/3519107829 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b724b3aed0 0x55b724b3d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.595+0000 7fe8b53d0740 1 --2- 172.21.15.16:0/3519107829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b724b35e40 0x55b724b3a900 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.595+0000 7fe8b53d0740 1 --2- 172.21.15.16:0/3519107829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b724b35500 0x55b724b35900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.595+0000 7fe8b53d0740 1 -- 172.21.15.16:0/3519107829 >> 172.21.15.16:0/3519107829 conn(0x55b724b30b70 msgr2=0x55b724b30f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:34.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.595+0000 7fe8b53d0740 1 -- 172.21.15.16:0/3519107829 shutdown_connections 2024-09-13T16:30:34.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.595+0000 7fe8b53d0740 1 -- 172.21.15.16:0/3519107829 wait complete. 2024-09-13T16:30:34.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.595+0000 7fe8b53d0740 1 -- start start 2024-09-13T16:30:34.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.596+0000 7fe8b53d0740 1 Processor -- start 2024-09-13T16:30:34.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.596+0000 7fe8b53d0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b724b35500 0x55b724b3f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.596+0000 7fe8b53d0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b724b35e40 0x55b724b3f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.596+0000 7fe8b53d0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b724b3aed0 0x55b724b46660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.596+0000 7fe8b53d0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b724a98bf0 con 0x55b724b3aed0 2024-09-13T16:30:34.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.596+0000 7fe8b53d0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b724a57250 con 0x55b724b35500 2024-09-13T16:30:34.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.596+0000 7fe8b53d0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b724a56ef0 con 0x55b724b35e40 2024-09-13T16:30:34.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.596+0000 7fe8b53d0740 10 client.0 _ll_register_callbacks cb 0x55b724a89740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:30:34.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.596+0000 7fe8b0c47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b724b3aed0 0x55b724b46660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.596+0000 7fe8abfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b724b35500 0x55b724b3f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.596+0000 7fe8b0c47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b724b3aed0 0x55b724b46660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52208/0 (socket says 172.21.15.16:52208) 2024-09-13T16:30:34.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.596+0000 7fe8b0c47640 1 -- 172.21.15.16:0/3514419410 learned_addr learned my addr 172.21.15.16:0/3514419410 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:34.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.596+0000 7fe8ab7fe640 1 --2- 172.21.15.16:0/3514419410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b724b35e40 0x55b724b3f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.596+0000 7fe8b0c47640 1 -- 172.21.15.16:0/3514419410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b724b35e40 msgr2=0x55b724b3f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:34.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.596+0000 7fe8b0c47640 1 --2- 172.21.15.16:0/3514419410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b724b35e40 0x55b724b3f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.596+0000 7fe8b0c47640 1 -- 172.21.15.16:0/3514419410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b724b35500 msgr2=0x55b724b3f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:34.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.596+0000 7fe8b0c47640 1 --2- 172.21.15.16:0/3514419410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b724b35500 0x55b724b3f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.596+0000 7fe8b0c47640 1 -- 172.21.15.16:0/3514419410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe89c009900 con 0x55b724b3aed0 2024-09-13T16:30:34.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.596+0000 7fe8abfff640 1 --2- 172.21.15.16:0/3514419410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b724b35500 0x55b724b3f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:30:34.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.596+0000 7fe8ab7fe640 1 --2- 172.21.15.16:0/3514419410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b724b35e40 0x55b724b3f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:30:34.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.596+0000 7fe8b0c47640 1 --2- 172.21.15.16:0/3514419410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b724b3aed0 0x55b724b46660 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe8a402f510 tx=0x7fe8a40bb2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386a10b41535ad32 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:34.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.596+0000 7fe88b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:30:34.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.597+0000 7fe88b7fe640 1 -- 172.21.15.16:0/3514419410 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a40bbb30 con 0x55b724b3aed0 2024-09-13T16:30:34.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.597+0000 7fe88b7fe640 1 -- 172.21.15.16:0/3514419410 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fe8a40bb430 con 0x55b724b3aed0 2024-09-13T16:30:34.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.597+0000 7fe8b53d0740 1 -- 172.21.15.16:0/3514419410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b72478c3c0 con 0x55b724b3aed0 2024-09-13T16:30:34.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.597+0000 7fe88b7fe640 1 -- 172.21.15.16:0/3514419410 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8a40bf260 con 0x55b724b3aed0 2024-09-13T16:30:34.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.597+0000 7fe8b53d0740 10 client.15228 fetch_fsmap learned FSMap version 28 2024-09-13T16:30:34.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.597+0000 7fe8b53d0740 1 -- 172.21.15.16:0/3514419410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b724a97270 con 0x55b724b3aed0 2024-09-13T16:30:34.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.597+0000 7fe88b7fe640 1 -- 172.21.15.16:0/3514419410 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b72478c3c0 con 0x55b724b3aed0 2024-09-13T16:30:34.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.597+0000 7fe88b7fe640 1 -- 172.21.15.16:0/3514419410 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe8a404c060 con 0x55b724b3aed0 2024-09-13T16:30:34.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.597+0000 7fe8b53d0740 10 client.15228 fetch_fsmap finished waiting for FSMap version 28 2024-09-13T16:30:34.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.597+0000 7fe8b53d0740 10 client.15228 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-13T16:30:34.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.597+0000 7fe8b53d0740 20 client.15228 populate_metadata read hostname 'smithi016' 2024-09-13T16:30:34.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.597+0000 7fe8b53d0740 1 --2- 172.21.15.16:0/3514419410 >> [v2:172.21.15.107:6832/3797712690,v1:172.21.15.107:6833/3797712690] conn(0x55b724b4f120 0x55b724b53890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.597+0000 7fe8b53d0740 4 client.15228 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T16:30:34.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.598+0000 7fe8b53d0740 1 -- 172.21.15.16:0/3514419410 --> [v2:172.21.15.107:6832/3797712690,v1:172.21.15.107:6833/3797712690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b72479d6c0 con 0x55b724b4f120 2024-09-13T16:30:34.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.598+0000 7fe8abfff640 1 --2- 172.21.15.16:0/3514419410 >> [v2:172.21.15.107:6832/3797712690,v1:172.21.15.107:6833/3797712690] conn(0x55b724b4f120 0x55b724b53890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.598+0000 7fe88b7fe640 10 client.15228 ms_handle_connect on v2:172.21.15.107:6832/3797712690 2024-09-13T16:30:34.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.598+0000 7fe8abfff640 1 --2- 172.21.15.16:0/3514419410 >> [v2:172.21.15.107:6832/3797712690,v1:172.21.15.107:6833/3797712690] conn(0x55b724b4f120 0x55b724b53890 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fd918bb4469395e9 server_cookie=77d635bf416e4b06 in_seq=0 out_seq=0 2024-09-13T16:30:34.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.599+0000 7fe88b7fe640 1 -- 172.21.15.16:0/3514419410 <== mds.3 v2:172.21.15.107:6832/3797712690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b72479d6c0 con 0x55b724b4f120 2024-09-13T16:30:34.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.599+0000 7fe88b7fe640 10 client.15228 handle_command_reply: tid=0 2024-09-13T16:30:34.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.599+0000 7fe8b53d0740 1 client.15228 shutdown 2024-09-13T16:30:34.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.599+0000 7fe8b53d0740 1 -- 172.21.15.16:0/3514419410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b724b3aed0 msgr2=0x55b724b46660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.599+0000 7fe8b53d0740 1 --2- 172.21.15.16:0/3514419410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b724b3aed0 0x55b724b46660 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe8a402f510 tx=0x7fe8a40bb2c0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.599+0000 7fe8b53d0740 1 -- 172.21.15.16:0/3514419410 shutdown_connections 2024-09-13T16:30:34.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.599+0000 7fe8b53d0740 1 --2- 172.21.15.16:0/3514419410 >> [v2:172.21.15.107:6832/3797712690,v1:172.21.15.107:6833/3797712690] conn(0x55b724b4f120 0x55b724b53890 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.599+0000 7fe8b53d0740 1 --2- 172.21.15.16:0/3514419410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b724b3aed0 0x55b724b46660 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.599+0000 7fe8b53d0740 1 --2- 172.21.15.16:0/3514419410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b724b35e40 0x55b724b3f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.599+0000 7fe8b53d0740 1 --2- 172.21.15.16:0/3514419410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b724b35500 0x55b724b3f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.599+0000 7fe88b7fe640 0 client.15228 ms_handle_reset on v2:172.21.15.107:6832/3797712690 2024-09-13T16:30:34.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.599+0000 7fe8b53d0740 1 -- 172.21.15.16:0/3514419410 >> 172.21.15.16:0/3514419410 conn(0x55b724b30b70 msgr2=0x55b724b32380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:34.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.600+0000 7fe8b53d0740 1 -- 172.21.15.16:0/3514419410 shutdown_connections 2024-09-13T16:30:34.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.600+0000 7fe8b53d0740 1 -- 172.21.15.16:0/3514419410 wait complete. 2024-09-13T16:30:34.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.600+0000 7fe8b53d0740 20 client.15228 trim_cache size 0 max 16384 2024-09-13T16:30:34.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.614+0000 7fe8b53d0740 1 Processor -- start 2024-09-13T16:30:34.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.615+0000 7fe8b53d0740 1 -- start start 2024-09-13T16:30:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.615+0000 7fe8b53d0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b724b35500 0x55b724b3a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.615+0000 7fe8b53d0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b724b35e40 0x55b724b46620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.615+0000 7fe8b53d0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b724b3aed0 0x55b724b3a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.615+0000 7fe8b53d0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b724a1c460 con 0x55b724b3aed0 2024-09-13T16:30:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.615+0000 7fe8b53d0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b724b40150 con 0x55b724b35500 2024-09-13T16:30:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.615+0000 7fe8b53d0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b72479d6c0 con 0x55b724b35e40 2024-09-13T16:30:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.615+0000 7fe8b0c47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b724b3aed0 0x55b724b3a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.615+0000 7fe8b0c47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b724b3aed0 0x55b724b3a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52210/0 (socket says 172.21.15.16:52210) 2024-09-13T16:30:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.615+0000 7fe8b0c47640 1 -- 172.21.15.16:0/1190274416 learned_addr learned my addr 172.21.15.16:0/1190274416 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.615+0000 7fe8abfff640 1 --2- 172.21.15.16:0/1190274416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b724b35500 0x55b724b3a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.615+0000 7fe8ab7fe640 1 --2- 172.21.15.16:0/1190274416 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b724b35e40 0x55b724b46620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.615+0000 7fe8b0c47640 1 -- 172.21.15.16:0/1190274416 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b724b35e40 msgr2=0x55b724b46620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.615+0000 7fe8b0c47640 1 --2- 172.21.15.16:0/1190274416 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b724b35e40 0x55b724b46620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.615+0000 7fe8b0c47640 1 -- 172.21.15.16:0/1190274416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b724b35500 msgr2=0x55b724b3a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:34.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.615+0000 7fe8b0c47640 1 --2- 172.21.15.16:0/1190274416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b724b35500 0x55b724b3a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.615+0000 7fe8b0c47640 1 -- 172.21.15.16:0/1190274416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b724a395f0 con 0x55b724b3aed0 2024-09-13T16:30:34.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.615+0000 7fe8b0c47640 1 --2- 172.21.15.16:0/1190274416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b724b3aed0 0x55b724b3a770 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4006010 tx=0x7fe8a40ebd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1af686fff28336a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:34.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.615+0000 7fe892ffd640 1 -- 172.21.15.16:0/1190274416 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4003d90 con 0x55b724b3aed0 2024-09-13T16:30:34.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.615+0000 7fe892ffd640 1 -- 172.21.15.16:0/1190274416 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe8a40317c0 con 0x55b724b3aed0 2024-09-13T16:30:34.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.615+0000 7fe892ffd640 1 -- 172.21.15.16:0/1190274416 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a40bbb30 con 0x55b724b3aed0 2024-09-13T16:30:34.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.616+0000 7fe8b53d0740 1 -- 172.21.15.16:0/1190274416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b724b3aed0 msgr2=0x55b724b3a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.616+0000 7fe8b53d0740 1 --2- 172.21.15.16:0/1190274416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b724b3aed0 0x55b724b3a770 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4006010 tx=0x7fe8a40ebd20 comp rx=0 tx=0).stop 2024-09-13T16:30:34.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.616+0000 7fe8b53d0740 1 -- 172.21.15.16:0/1190274416 shutdown_connections 2024-09-13T16:30:34.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.616+0000 7fe8b53d0740 1 --2- 172.21.15.16:0/1190274416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b724b3aed0 0x55b724b3a770 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.616+0000 7fe8b53d0740 1 --2- 172.21.15.16:0/1190274416 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b724b35e40 0x55b724b46620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.616+0000 7fe8b53d0740 1 --2- 172.21.15.16:0/1190274416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b724b35500 0x55b724b3a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.616+0000 7fe8b53d0740 1 -- 172.21.15.16:0/1190274416 >> 172.21.15.16:0/1190274416 conn(0x55b724b30b70 msgr2=0x55b724b36c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:34.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.616+0000 7fe8b53d0740 1 -- 172.21.15.16:0/1190274416 shutdown_connections 2024-09-13T16:30:34.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.616+0000 7fe8b53d0740 1 -- 172.21.15.16:0/1190274416 wait complete. 2024-09-13T16:30:34.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.616+0000 7fe8b53d0740 1 -- start start 2024-09-13T16:30:34.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.617+0000 7fe8b53d0740 1 Processor -- start 2024-09-13T16:30:34.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.617+0000 7fe8b53d0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b724b35500 0x55b724b40840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.617+0000 7fe8b53d0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b724b35e40 0x55b724b40da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.617+0000 7fe8b53d0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b724b3aed0 0x55b724b45b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.617+0000 7fe8b53d0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b724b46060 con 0x55b724b35e40 2024-09-13T16:30:34.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.617+0000 7fe8b53d0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b724b461d0 con 0x55b724b3aed0 2024-09-13T16:30:34.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.617+0000 7fe8b53d0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b724b46340 con 0x55b724b35500 2024-09-13T16:30:34.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.617+0000 7fe8b53d0740 10 client.0 _ll_register_callbacks cb 0x7fe89c009690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:30:34.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.617+0000 7fe8ab7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b724b35e40 0x55b724b40da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.617+0000 7fe8abfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b724b35500 0x55b724b40840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.617+0000 7fe8ab7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b724b35e40 0x55b724b40da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52220/0 (socket says 172.21.15.16:52220) 2024-09-13T16:30:34.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.617+0000 7fe8ab7fe640 1 -- 172.21.15.16:0/3869201001 learned_addr learned my addr 172.21.15.16:0/3869201001 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:34.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.617+0000 7fe8b0c47640 1 --2- 172.21.15.16:0/3869201001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b724b3aed0 0x55b724b45b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.617+0000 7fe8ab7fe640 1 -- 172.21.15.16:0/3869201001 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b724b35500 msgr2=0x55b724b40840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:34.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.617+0000 7fe8ab7fe640 1 --2- 172.21.15.16:0/3869201001 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b724b35500 0x55b724b40840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.617+0000 7fe8ab7fe640 1 -- 172.21.15.16:0/3869201001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b724b3aed0 msgr2=0x55b724b45b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:34.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.617+0000 7fe8ab7fe640 1 --2- 172.21.15.16:0/3869201001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b724b3aed0 0x55b724b45b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.617+0000 7fe8ab7fe640 1 -- 172.21.15.16:0/3869201001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a40d7030 con 0x55b724b35e40 2024-09-13T16:30:34.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.617+0000 7fe8abfff640 1 --2- 172.21.15.16:0/3869201001 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b724b35500 0x55b724b40840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:30:34.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.617+0000 7fe8ab7fe640 1 --2- 172.21.15.16:0/3869201001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b724b35e40 0x55b724b40da0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0003870 tx=0x7fe8a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cebe70d436cc7dd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:34.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.617+0000 7fe88b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:30:34.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.618+0000 7fe88b7fe640 1 -- 172.21.15.16:0/3869201001 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0016040 con 0x55b724b35e40 2024-09-13T16:30:34.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.618+0000 7fe88b7fe640 1 -- 172.21.15.16:0/3869201001 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fe8a001e8d0 con 0x55b724b35e40 2024-09-13T16:30:34.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.618+0000 7fe8b53d0740 1 -- 172.21.15.16:0/3869201001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe89c012a40 con 0x55b724b35e40 2024-09-13T16:30:34.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.618+0000 7fe88b7fe640 1 -- 172.21.15.16:0/3869201001 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8a001ea90 con 0x55b724b35e40 2024-09-13T16:30:34.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.618+0000 7fe8b53d0740 10 client.15234 fetch_fsmap learned FSMap version 28 2024-09-13T16:30:34.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.618+0000 7fe8b53d0740 1 -- 172.21.15.16:0/3869201001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b724a5ed00 con 0x55b724b35e40 2024-09-13T16:30:34.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.618+0000 7fe88b7fe640 1 -- 172.21.15.16:0/3869201001 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe89c012a40 con 0x55b724b35e40 2024-09-13T16:30:34.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.618+0000 7fe88b7fe640 1 -- 172.21.15.16:0/3869201001 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe8a0010ca0 con 0x55b724b35e40 2024-09-13T16:30:34.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.618+0000 7fe8b53d0740 10 client.15234 fetch_fsmap finished waiting for FSMap version 28 2024-09-13T16:30:34.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.618+0000 7fe8b53d0740 10 client.15234 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-13T16:30:34.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.618+0000 7fe8b53d0740 20 client.15234 populate_metadata read hostname 'smithi016' 2024-09-13T16:30:34.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.618+0000 7fe8b53d0740 1 --2- 172.21.15.16:0/3869201001 >> [v2:172.21.15.107:6832/3797712690,v1:172.21.15.107:6833/3797712690] conn(0x55b724ba9c30 0x55b724ba65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.618+0000 7fe8b53d0740 4 client.15234 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T16:30:34.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.618+0000 7fe8b53d0740 1 -- 172.21.15.16:0/3869201001 --> [v2:172.21.15.107:6832/3797712690,v1:172.21.15.107:6833/3797712690] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b7247f0d10 con 0x55b724ba9c30 2024-09-13T16:30:34.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.618+0000 7fe8abfff640 1 --2- 172.21.15.16:0/3869201001 >> [v2:172.21.15.107:6832/3797712690,v1:172.21.15.107:6833/3797712690] conn(0x55b724ba9c30 0x55b724ba65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.619+0000 7fe88b7fe640 10 client.15234 ms_handle_connect on v2:172.21.15.107:6832/3797712690 2024-09-13T16:30:34.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.619+0000 7fe8abfff640 1 --2- 172.21.15.16:0/3869201001 >> [v2:172.21.15.107:6832/3797712690,v1:172.21.15.107:6833/3797712690] conn(0x55b724ba9c30 0x55b724ba65a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f69be1348d50c5c5 server_cookie=8d2f52dd8717a8e4 in_seq=0 out_seq=0 2024-09-13T16:30:34.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.619+0000 7fe88b7fe640 1 -- 172.21.15.16:0/3869201001 <== mds.3 v2:172.21.15.107:6832/3797712690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55b7247f0d10 con 0x55b724ba9c30 2024-09-13T16:30:34.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.619+0000 7fe88b7fe640 10 client.15234 handle_command_reply: tid=0 2024-09-13T16:30:34.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.619+0000 7fe8b53d0740 1 client.15234 shutdown 2024-09-13T16:30:34.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.619+0000 7fe8b53d0740 1 -- 172.21.15.16:0/3869201001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b724b35e40 msgr2=0x55b724b40da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.619+0000 7fe8b53d0740 1 --2- 172.21.15.16:0/3869201001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b724b35e40 0x55b724b40da0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0003870 tx=0x7fe8a0004730 comp rx=0 tx=0).stop 2024-09-13T16:30:34.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.619+0000 7fe8b53d0740 1 -- 172.21.15.16:0/3869201001 shutdown_connections 2024-09-13T16:30:34.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.619+0000 7fe8b53d0740 1 --2- 172.21.15.16:0/3869201001 >> [v2:172.21.15.107:6832/3797712690,v1:172.21.15.107:6833/3797712690] conn(0x55b724ba9c30 0x55b724ba65a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.619+0000 7fe8b53d0740 1 --2- 172.21.15.16:0/3869201001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b724b3aed0 0x55b724b45b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.619+0000 7fe88b7fe640 0 client.15234 ms_handle_reset on v2:172.21.15.107:6832/3797712690 2024-09-13T16:30:34.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.619+0000 7fe8b53d0740 1 --2- 172.21.15.16:0/3869201001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b724b35e40 0x55b724b40da0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.619+0000 7fe8b53d0740 1 --2- 172.21.15.16:0/3869201001 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b724b35500 0x55b724b40840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.619+0000 7fe8b53d0740 1 -- 172.21.15.16:0/3869201001 >> 172.21.15.16:0/3869201001 conn(0x55b724b30b70 msgr2=0x55b724b3ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:34.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.619+0000 7fe8b53d0740 1 -- 172.21.15.16:0/3869201001 shutdown_connections 2024-09-13T16:30:34.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.619+0000 7fe8b53d0740 1 -- 172.21.15.16:0/3869201001 wait complete. 2024-09-13T16:30:34.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.619+0000 7fe8b53d0740 20 client.15234 trim_cache size 0 max 16384 2024-09-13T16:30:34.627 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-13T16:30:34.627 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "0d241f6a-71ec-11ef-bceb-c7b262605968", 2024-09-13T16:30:34.627 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 3, 2024-09-13T16:30:34.627 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24422, 2024-09-13T16:30:34.627 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-13T16:30:34.627 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-13T16:30:34.628 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-13T16:30:34.628 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 91.002817944, 2024-09-13T16:30:34.628 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 28, 2024-09-13T16:30:34.628 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 93, 2024-09-13T16:30:34.628 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 91, 2024-09-13T16:30:34.628 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 145.64557134099999 2024-09-13T16:30:34.628 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-13T16:30:34.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.620+0000 7fe8b26aa640 1 -- 172.21.15.16:0/1071697313 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe8940a1120 msgr2=0x7fe8940a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.620+0000 7fe8b26aa640 1 --2- 172.21.15.16:0/1071697313 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe8940a1120 0x7fe8940a35e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0009730 tx=0x7fe8a00096c0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.620+0000 7fe8b26aa640 1 -- 172.21.15.16:0/1071697313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8ac105ab0 msgr2=0x7fe8ac19d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.620+0000 7fe8b26aa640 1 --2- 172.21.15.16:0/1071697313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8ac105ab0 0x7fe8ac19d240 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe8a402aeb0 tx=0x7fe8a4003c20 comp rx=0 tx=0).stop 2024-09-13T16:30:34.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.620+0000 7fe8b26aa640 1 -- 172.21.15.16:0/1071697313 shutdown_connections 2024-09-13T16:30:34.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.620+0000 7fe8b26aa640 1 --2- 172.21.15.16:0/1071697313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8ac105ab0 0x7fe8ac19d240 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.620+0000 7fe8b26aa640 1 --2- 172.21.15.16:0/1071697313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8ac103180 0x7fe8ac196230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.620+0000 7fe8b26aa640 1 --2- 172.21.15.16:0/1071697313 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe8940a1120 0x7fe8940a35e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.620+0000 7fe8b26aa640 1 --2- 172.21.15.16:0/1071697313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8ac100850 0x7fe8ac195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.620+0000 7fe8b26aa640 1 -- 172.21.15.16:0/1071697313 >> 172.21.15.16:0/1071697313 conn(0x7fe8ac0fa410 msgr2=0x7fe8ac101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:34.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.620+0000 7fe8b26aa640 1 -- 172.21.15.16:0/1071697313 shutdown_connections 2024-09-13T16:30:34.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.620+0000 7fe8b26aa640 1 -- 172.21.15.16:0/1071697313 wait complete. 2024-09-13T16:30:34.631 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-09-13T16:30:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.738+0000 7fd99b7e9640 1 -- 172.21.15.16:0/72672104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd994100f80 msgr2=0x7fd994101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.738+0000 7fd99b7e9640 1 --2- 172.21.15.16:0/72672104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd994100f80 0x7fd994101400 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd988009900 tx=0x7fd98802a410 comp rx=0 tx=0).stop 2024-09-13T16:30:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.739+0000 7fd99b7e9640 1 -- 172.21.15.16:0/72672104 shutdown_connections 2024-09-13T16:30:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.739+0000 7fd99b7e9640 1 --2- 172.21.15.16:0/72672104 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd994101940 0x7fd9941081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.739+0000 7fd99b7e9640 1 --2- 172.21.15.16:0/72672104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd994100f80 0x7fd994101400 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.739+0000 7fd99b7e9640 1 --2- 172.21.15.16:0/72672104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9940ffd90 0x7fd994100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.739+0000 7fd99b7e9640 1 -- 172.21.15.16:0/72672104 >> 172.21.15.16:0/72672104 conn(0x7fd9940fb4d0 msgr2=0x7fd9940fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:34.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.739+0000 7fd99b7e9640 1 -- 172.21.15.16:0/72672104 shutdown_connections 2024-09-13T16:30:34.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.739+0000 7fd99b7e9640 1 -- 172.21.15.16:0/72672104 wait complete. 2024-09-13T16:30:34.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.739+0000 7fd99b7e9640 1 Processor -- start 2024-09-13T16:30:34.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.739+0000 7fd99b7e9640 1 -- start start 2024-09-13T16:30:34.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.739+0000 7fd99b7e9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9940ffd90 0x7fd994195c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.740+0000 7fd99b7e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd994100f80 0x7fd9941961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.740+0000 7fd99b7e9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd994101940 0x7fd99419d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.740+0000 7fd99b7e9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99419d740 con 0x7fd994100f80 2024-09-13T16:30:34.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.740+0000 7fd99b7e9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99419d8b0 con 0x7fd9940ffd90 2024-09-13T16:30:34.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.740+0000 7fd99b7e9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99419da70 con 0x7fd994101940 2024-09-13T16:30:34.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.740+0000 7fd998d84640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd994100f80 0x7fd9941961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.740+0000 7fd999585640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9940ffd90 0x7fd994195c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.740+0000 7fd998d84640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd994100f80 0x7fd9941961a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52228/0 (socket says 172.21.15.16:52228) 2024-09-13T16:30:34.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.740+0000 7fd998d84640 1 -- 172.21.15.16:0/2198763487 learned_addr learned my addr 172.21.15.16:0/2198763487 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:34.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.740+0000 7fd999d86640 1 --2- 172.21.15.16:0/2198763487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd994101940 0x7fd99419d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.740+0000 7fd998d84640 1 -- 172.21.15.16:0/2198763487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd994101940 msgr2=0x7fd99419d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.740+0000 7fd998d84640 1 --2- 172.21.15.16:0/2198763487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd994101940 0x7fd99419d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.740+0000 7fd998d84640 1 -- 172.21.15.16:0/2198763487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9940ffd90 msgr2=0x7fd994195c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.740+0000 7fd998d84640 1 --2- 172.21.15.16:0/2198763487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9940ffd90 0x7fd994195c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.740+0000 7fd998d84640 1 -- 172.21.15.16:0/2198763487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd988009560 con 0x7fd994100f80 2024-09-13T16:30:34.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.740+0000 7fd999d86640 1 --2- 172.21.15.16:0/2198763487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd994101940 0x7fd99419d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:30:34.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.740+0000 7fd998d84640 1 --2- 172.21.15.16:0/2198763487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd994100f80 0x7fd9941961a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd988006010 tx=0x7fd988004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:34.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.740+0000 7fd9927fc640 1 -- 172.21.15.16:0/2198763487 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9880037f0 con 0x7fd994100f80 2024-09-13T16:30:34.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.740+0000 7fd9927fc640 1 -- 172.21.15.16:0/2198763487 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd9880044f0 con 0x7fd994100f80 2024-09-13T16:30:34.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.740+0000 7fd9927fc640 1 -- 172.21.15.16:0/2198763487 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98803ca80 con 0x7fd994100f80 2024-09-13T16:30:34.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.740+0000 7fd99b7e9640 1 -- 172.21.15.16:0/2198763487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd99419dcf0 con 0x7fd994100f80 2024-09-13T16:30:34.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.740+0000 7fd99b7e9640 1 -- 172.21.15.16:0/2198763487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd99419e150 con 0x7fd994100f80 2024-09-13T16:30:34.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.743+0000 7fd9927fc640 1 -- 172.21.15.16:0/2198763487 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd988003950 con 0x7fd994100f80 2024-09-13T16:30:34.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.743+0000 7fd99e5e6740 1 Processor -- start 2024-09-13T16:30:34.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.743+0000 7fd99e5e6740 1 -- start start 2024-09-13T16:30:34.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.743+0000 7fd99e5e6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c617ec9500 0x55c617ec9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.743+0000 7fd99e5e6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c617ec9e40 0x55c617ece900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.743+0000 7fd99e5e6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c617eceed0 0x55c617ed1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.743+0000 7fd99e5e6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c617e247c0 con 0x55c617ec9e40 2024-09-13T16:30:34.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.743+0000 7fd99e5e6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c617e33b70 con 0x55c617eceed0 2024-09-13T16:30:34.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.743+0000 7fd99e5e6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c617e31300 con 0x55c617ec9500 2024-09-13T16:30:34.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.743+0000 7fd9927fc640 1 --2- 172.21.15.16:0/2198763487 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd97c0a1110 0x7fd97c0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.743+0000 7fd999585640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c617ec9e40 0x55c617ece900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.743+0000 7fd999585640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c617ec9e40 0x55c617ece900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52236/0 (socket says 172.21.15.16:52236) 2024-09-13T16:30:34.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.743+0000 7fd999585640 1 -- 172.21.15.16:0/600428499 learned_addr learned my addr 172.21.15.16:0/600428499 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:34.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.743+0000 7fd998d84640 1 --2- 172.21.15.16:0/600428499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c617eceed0 0x55c617ed1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.744+0000 7fd999d86640 1 --2- 172.21.15.16:0/2198763487 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd97c0a1110 0x7fd97c0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.744+0000 7fd9927fc640 1 -- 172.21.15.16:0/2198763487 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd9880edc90 con 0x7fd994100f80 2024-09-13T16:30:34.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.744+0000 7fd999d86640 1 --2- 172.21.15.16:0/600428499 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c617ec9500 0x55c617ec9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.744+0000 7fd999585640 1 -- 172.21.15.16:0/600428499 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c617ec9500 msgr2=0x55c617ec9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:34.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.744+0000 7fd999585640 1 --2- 172.21.15.16:0/600428499 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c617ec9500 0x55c617ec9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.744+0000 7fd999585640 1 -- 172.21.15.16:0/600428499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c617eceed0 msgr2=0x55c617ed1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:34.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.744+0000 7fd999585640 1 --2- 172.21.15.16:0/600428499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c617eceed0 0x55c617ed1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.744+0000 7fd999585640 1 -- 172.21.15.16:0/600428499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c617db94b0 con 0x55c617ec9e40 2024-09-13T16:30:34.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.744+0000 7fd999585640 1 --2- 172.21.15.16:0/600428499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c617ec9e40 0x55c617ece900 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00b750 tx=0x7fd98c00bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf830bf90b054652 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:34.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.744+0000 7fd999d86640 1 --2- 172.21.15.16:0/2198763487 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd97c0a1110 0x7fd97c0a35d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd984007310 tx=0x7fd984007460 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:34.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.744+0000 7fd97b7fe640 1 -- 172.21.15.16:0/600428499 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c003dd0 con 0x55c617ec9e40 2024-09-13T16:30:34.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.744+0000 7fd97b7fe640 1 -- 172.21.15.16:0/600428499 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd98c00be60 con 0x55c617ec9e40 2024-09-13T16:30:34.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.744+0000 7fd97b7fe640 1 -- 172.21.15.16:0/600428499 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c010680 con 0x55c617ec9e40 2024-09-13T16:30:34.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.745+0000 7fd99e5e6740 1 -- 172.21.15.16:0/600428499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c617ec9e40 msgr2=0x55c617ece900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.745+0000 7fd99e5e6740 1 --2- 172.21.15.16:0/600428499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c617ec9e40 0x55c617ece900 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00b750 tx=0x7fd98c00bc10 comp rx=0 tx=0).stop 2024-09-13T16:30:34.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.745+0000 7fd99e5e6740 1 -- 172.21.15.16:0/600428499 shutdown_connections 2024-09-13T16:30:34.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.745+0000 7fd99e5e6740 1 --2- 172.21.15.16:0/600428499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c617eceed0 0x55c617ed1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.745+0000 7fd99e5e6740 1 --2- 172.21.15.16:0/600428499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c617ec9e40 0x55c617ece900 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.745+0000 7fd99e5e6740 1 --2- 172.21.15.16:0/600428499 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c617ec9500 0x55c617ec9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.745+0000 7fd99e5e6740 1 -- 172.21.15.16:0/600428499 >> 172.21.15.16:0/600428499 conn(0x55c617ec4b70 msgr2=0x55c617ec4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:34.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.745+0000 7fd99e5e6740 1 -- 172.21.15.16:0/600428499 shutdown_connections 2024-09-13T16:30:34.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.745+0000 7fd99e5e6740 1 -- 172.21.15.16:0/600428499 wait complete. 2024-09-13T16:30:34.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.745+0000 7fd99e5e6740 1 -- start start 2024-09-13T16:30:34.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.745+0000 7fd99e5e6740 1 Processor -- start 2024-09-13T16:30:34.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.746+0000 7fd99e5e6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c617ec9500 0x55c617ed3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.746+0000 7fd99e5e6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c617ec9e40 0x55c617ed36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.746+0000 7fd99e5e6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c617eceed0 0x55c617eda710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.746+0000 7fd99e5e6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c617e2cbf0 con 0x55c617ec9500 2024-09-13T16:30:34.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.746+0000 7fd99e5e6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c617deb250 con 0x55c617eceed0 2024-09-13T16:30:34.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.746+0000 7fd99e5e6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c617deaef0 con 0x55c617ec9e40 2024-09-13T16:30:34.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.746+0000 7fd99e5e6740 10 client.0 _ll_register_callbacks cb 0x55c617e1d740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:30:34.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.746+0000 7fd999585640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c617ec9500 0x55c617ed3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.746+0000 7fd999585640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c617ec9500 0x55c617ed3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52246/0 (socket says 172.21.15.16:52246) 2024-09-13T16:30:34.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.746+0000 7fd999585640 1 -- 172.21.15.16:0/1998801642 learned_addr learned my addr 172.21.15.16:0/1998801642 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:34.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.746+0000 7fd998d84640 1 --2- 172.21.15.16:0/1998801642 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c617ec9e40 0x55c617ed36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.746+0000 7fd999585640 1 -- 172.21.15.16:0/1998801642 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c617ec9e40 msgr2=0x55c617ed36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:34.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.746+0000 7fd999585640 1 --2- 172.21.15.16:0/1998801642 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c617ec9e40 0x55c617ed36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.746+0000 7fd999d86640 1 --2- 172.21.15.16:0/1998801642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c617eceed0 0x55c617eda710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.746+0000 7fd999585640 1 -- 172.21.15.16:0/1998801642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c617eceed0 msgr2=0x55c617eda710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.746+0000 7fd999585640 1 --2- 172.21.15.16:0/1998801642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c617eceed0 0x55c617eda710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.746+0000 7fd999585640 1 -- 172.21.15.16:0/1998801642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd98c004290 con 0x55c617ec9500 2024-09-13T16:30:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.746+0000 7fd999d86640 1 --2- 172.21.15.16:0/1998801642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c617eceed0 0x55c617eda710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:30:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.746+0000 7fd999585640 1 --2- 172.21.15.16:0/1998801642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c617ec9500 0x55c617ed3190 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd98c004650 tx=0x7fd98c003e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33fb71e231f68b98 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.746+0000 7fd978ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:30:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.747+0000 7fd978ff9640 1 -- 172.21.15.16:0/1998801642 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c01e040 con 0x55c617ec9500 2024-09-13T16:30:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.747+0000 7fd978ff9640 1 -- 172.21.15.16:0/1998801642 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd98c011850 con 0x55c617ec9500 2024-09-13T16:30:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.747+0000 7fd99e5e6740 1 -- 172.21.15.16:0/1998801642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c617b203c0 con 0x55c617ec9500 2024-09-13T16:30:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.747+0000 7fd978ff9640 1 -- 172.21.15.16:0/1998801642 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd98c011a10 con 0x55c617ec9500 2024-09-13T16:30:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.747+0000 7fd99e5e6740 10 client.15246 fetch_fsmap learned FSMap version 28 2024-09-13T16:30:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.747+0000 7fd99e5e6740 1 -- 172.21.15.16:0/1998801642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c617e2b270 con 0x55c617ec9500 2024-09-13T16:30:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.747+0000 7fd978ff9640 1 -- 172.21.15.16:0/1998801642 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c617b203c0 con 0x55c617ec9500 2024-09-13T16:30:34.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.747+0000 7fd978ff9640 1 -- 172.21.15.16:0/1998801642 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd98c021050 con 0x55c617ec9500 2024-09-13T16:30:34.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.747+0000 7fd99e5e6740 10 client.15246 fetch_fsmap finished waiting for FSMap version 28 2024-09-13T16:30:34.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.747+0000 7fd99e5e6740 10 client.15246 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-13T16:30:34.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.747+0000 7fd99e5e6740 20 client.15246 populate_metadata read hostname 'smithi016' 2024-09-13T16:30:34.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.747+0000 7fd99e5e6740 1 --2- 172.21.15.16:0/1998801642 >> [v2:172.21.15.107:6836/2838457611,v1:172.21.15.107:6837/2838457611] conn(0x55c617edefb0 0x55c617ee3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.747+0000 7fd99e5e6740 4 client.15246 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T16:30:34.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.747+0000 7fd99e5e6740 1 -- 172.21.15.16:0/1998801642 --> [v2:172.21.15.107:6836/2838457611,v1:172.21.15.107:6837/2838457611] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c617b316c0 con 0x55c617edefb0 2024-09-13T16:30:34.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.748+0000 7fd998d84640 1 --2- 172.21.15.16:0/1998801642 >> [v2:172.21.15.107:6836/2838457611,v1:172.21.15.107:6837/2838457611] conn(0x55c617edefb0 0x55c617ee3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.748+0000 7fd978ff9640 10 client.15246 ms_handle_connect on v2:172.21.15.107:6836/2838457611 2024-09-13T16:30:34.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.748+0000 7fd998d84640 1 --2- 172.21.15.16:0/1998801642 >> [v2:172.21.15.107:6836/2838457611,v1:172.21.15.107:6837/2838457611] conn(0x55c617edefb0 0x55c617ee3720 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3b317c08eec7359e server_cookie=41f647231798c360 in_seq=0 out_seq=0 2024-09-13T16:30:34.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.749+0000 7fd978ff9640 1 -- 172.21.15.16:0/1998801642 <== mds.2 v2:172.21.15.107:6836/2838457611 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c617b316c0 con 0x55c617edefb0 2024-09-13T16:30:34.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.749+0000 7fd978ff9640 10 client.15246 handle_command_reply: tid=0 2024-09-13T16:30:34.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.749+0000 7fd99e5e6740 1 client.15246 shutdown 2024-09-13T16:30:34.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.749+0000 7fd99e5e6740 1 -- 172.21.15.16:0/1998801642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c617ec9500 msgr2=0x55c617ed3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.749+0000 7fd99e5e6740 1 --2- 172.21.15.16:0/1998801642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c617ec9500 0x55c617ed3190 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd98c004650 tx=0x7fd98c003e70 comp rx=0 tx=0).stop 2024-09-13T16:30:34.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.749+0000 7fd99e5e6740 1 -- 172.21.15.16:0/1998801642 shutdown_connections 2024-09-13T16:30:34.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.749+0000 7fd99e5e6740 1 --2- 172.21.15.16:0/1998801642 >> [v2:172.21.15.107:6836/2838457611,v1:172.21.15.107:6837/2838457611] conn(0x55c617edefb0 0x55c617ee3720 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.749+0000 7fd99e5e6740 1 --2- 172.21.15.16:0/1998801642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c617eceed0 0x55c617eda710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.749+0000 7fd99e5e6740 1 --2- 172.21.15.16:0/1998801642 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c617ec9e40 0x55c617ed36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.749+0000 7fd99e5e6740 1 --2- 172.21.15.16:0/1998801642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c617ec9500 0x55c617ed3190 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.749+0000 7fd99e5e6740 1 -- 172.21.15.16:0/1998801642 >> 172.21.15.16:0/1998801642 conn(0x55c617ec4b70 msgr2=0x55c617ec6380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:34.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.749+0000 7fd978ff9640 0 client.15246 ms_handle_reset on v2:172.21.15.107:6836/2838457611 2024-09-13T16:30:34.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.749+0000 7fd99e5e6740 1 -- 172.21.15.16:0/1998801642 shutdown_connections 2024-09-13T16:30:34.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.749+0000 7fd99e5e6740 1 -- 172.21.15.16:0/1998801642 wait complete. 2024-09-13T16:30:34.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.750+0000 7fd99e5e6740 20 client.15246 trim_cache size 0 max 16384 2024-09-13T16:30:34.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.765+0000 7fd99e5e6740 1 Processor -- start 2024-09-13T16:30:34.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.765+0000 7fd99e5e6740 1 -- start start 2024-09-13T16:30:34.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.765+0000 7fd99e5e6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c617ec9500 0x55c617ec4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.765+0000 7fd99e5e6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c617ec9e40 0x55c617eda630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.765+0000 7fd99e5e6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c617eceed0 0x55c617ed2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.765+0000 7fd99e5e6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c617db0460 con 0x55c617ec9500 2024-09-13T16:30:34.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.765+0000 7fd99e5e6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c617e25990 con 0x55c617eceed0 2024-09-13T16:30:34.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.765+0000 7fd99e5e6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c617e29850 con 0x55c617ec9e40 2024-09-13T16:30:34.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.765+0000 7fd999585640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c617ec9500 0x55c617ec4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.765+0000 7fd999585640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c617ec9500 0x55c617ec4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52248/0 (socket says 172.21.15.16:52248) 2024-09-13T16:30:34.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.765+0000 7fd999585640 1 -- 172.21.15.16:0/324953449 learned_addr learned my addr 172.21.15.16:0/324953449 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:34.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.765+0000 7fd999d86640 1 --2- 172.21.15.16:0/324953449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c617eceed0 0x55c617ed2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.765+0000 7fd998d84640 1 --2- 172.21.15.16:0/324953449 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c617ec9e40 0x55c617eda630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.765+0000 7fd999585640 1 -- 172.21.15.16:0/324953449 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c617ec9e40 msgr2=0x55c617eda630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:34.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.765+0000 7fd999585640 1 --2- 172.21.15.16:0/324953449 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c617ec9e40 0x55c617eda630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.765+0000 7fd999585640 1 -- 172.21.15.16:0/324953449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c617eceed0 msgr2=0x55c617ed2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:34.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.765+0000 7fd999585640 1 --2- 172.21.15.16:0/324953449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c617eceed0 0x55c617ed2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.765+0000 7fd999585640 1 -- 172.21.15.16:0/324953449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c617dcd5f0 con 0x55c617ec9500 2024-09-13T16:30:34.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.766+0000 7fd999585640 1 --2- 172.21.15.16:0/324953449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c617ec9500 0x55c617ec4470 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00b750 tx=0x7fd98c004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4773a1f2d5bda6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:34.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.766+0000 7fd97bfff640 1 -- 172.21.15.16:0/324953449 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c01e040 con 0x55c617ec9500 2024-09-13T16:30:34.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.766+0000 7fd97bfff640 1 -- 172.21.15.16:0/324953449 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd98c005ed0 con 0x55c617ec9500 2024-09-13T16:30:34.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.766+0000 7fd97bfff640 1 -- 172.21.15.16:0/324953449 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c0118b0 con 0x55c617ec9500 2024-09-13T16:30:34.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.766+0000 7fd99e5e6740 1 -- 172.21.15.16:0/324953449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c617ec9500 msgr2=0x55c617ec4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.766+0000 7fd99e5e6740 1 --2- 172.21.15.16:0/324953449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c617ec9500 0x55c617ec4470 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00b750 tx=0x7fd98c004530 comp rx=0 tx=0).stop 2024-09-13T16:30:34.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.766+0000 7fd99e5e6740 1 -- 172.21.15.16:0/324953449 shutdown_connections 2024-09-13T16:30:34.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.766+0000 7fd99e5e6740 1 --2- 172.21.15.16:0/324953449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c617eceed0 0x55c617ed2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.766+0000 7fd99e5e6740 1 --2- 172.21.15.16:0/324953449 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c617ec9e40 0x55c617eda630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.766+0000 7fd99e5e6740 1 --2- 172.21.15.16:0/324953449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c617ec9500 0x55c617ec4470 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.766+0000 7fd99e5e6740 1 -- 172.21.15.16:0/324953449 >> 172.21.15.16:0/324953449 conn(0x55c617ec4b70 msgr2=0x55c617ed2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:34.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.766+0000 7fd99e5e6740 1 -- 172.21.15.16:0/324953449 shutdown_connections 2024-09-13T16:30:34.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.767+0000 7fd99e5e6740 1 -- 172.21.15.16:0/324953449 wait complete. 2024-09-13T16:30:34.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.767+0000 7fd99e5e6740 1 -- start start 2024-09-13T16:30:34.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.767+0000 7fd99e5e6740 1 Processor -- start 2024-09-13T16:30:34.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.767+0000 7fd99e5e6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c617ec9500 0x55c617ed4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.767+0000 7fd99e5e6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c617ec9e40 0x55c617ed51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.767+0000 7fd99e5e6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c617eceed0 0x55c617eda020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.767+0000 7fd99e5e6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c617eda560 con 0x55c617ec9e40 2024-09-13T16:30:34.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.767+0000 7fd99e5e6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c617eda6d0 con 0x55c617ec9500 2024-09-13T16:30:34.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.767+0000 7fd99e5e6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c617eda890 con 0x55c617eceed0 2024-09-13T16:30:34.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.767+0000 7fd99e5e6740 10 client.0 _ll_register_callbacks cb 0x7fd98c004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:30:34.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.767+0000 7fd998d84640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c617ec9e40 0x55c617ed51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.767+0000 7fd999585640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c617ec9500 0x55c617ed4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.767+0000 7fd998d84640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c617ec9e40 0x55c617ed51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52252/0 (socket says 172.21.15.16:52252) 2024-09-13T16:30:34.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.767+0000 7fd998d84640 1 -- 172.21.15.16:0/4239087800 learned_addr learned my addr 172.21.15.16:0/4239087800 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:34.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.767+0000 7fd999d86640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c617eceed0 0x55c617eda020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.768+0000 7fd998d84640 1 -- 172.21.15.16:0/4239087800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c617eceed0 msgr2=0x55c617eda020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:34.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.768+0000 7fd998d84640 1 --2- 172.21.15.16:0/4239087800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c617eceed0 0x55c617eda020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.768+0000 7fd998d84640 1 -- 172.21.15.16:0/4239087800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c617ec9500 msgr2=0x55c617ed4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:34.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.768+0000 7fd998d84640 1 --2- 172.21.15.16:0/4239087800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c617ec9500 0x55c617ed4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.768+0000 7fd998d84640 1 -- 172.21.15.16:0/4239087800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c617e2b270 con 0x55c617ec9e40 2024-09-13T16:30:34.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.768+0000 7fd998d84640 1 --2- 172.21.15.16:0/4239087800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c617ec9e40 0x55c617ed51b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd9880b7770 tx=0x7fd98808dee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71969666647fa9e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:34.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.768+0000 7fd978ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:30:34.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.768+0000 7fd978ff9640 1 -- 172.21.15.16:0/4239087800 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd988004da0 con 0x55c617ec9e40 2024-09-13T16:30:34.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.768+0000 7fd978ff9640 1 -- 172.21.15.16:0/4239087800 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd9880bb490 con 0x55c617ec9e40 2024-09-13T16:30:34.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.768+0000 7fd99e5e6740 1 -- 172.21.15.16:0/4239087800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd98c00be60 con 0x55c617ec9e40 2024-09-13T16:30:34.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.768+0000 7fd978ff9640 1 -- 172.21.15.16:0/4239087800 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9880bad90 con 0x55c617ec9e40 2024-09-13T16:30:34.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.768+0000 7fd99e5e6740 10 client.15252 fetch_fsmap learned FSMap version 28 2024-09-13T16:30:34.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.768+0000 7fd99e5e6740 1 -- 172.21.15.16:0/4239087800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c617df2d00 con 0x55c617ec9e40 2024-09-13T16:30:34.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.768+0000 7fd978ff9640 1 -- 172.21.15.16:0/4239087800 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd98c00be60 con 0x55c617ec9e40 2024-09-13T16:30:34.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.768+0000 7fd978ff9640 1 -- 172.21.15.16:0/4239087800 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd9880baff0 con 0x55c617ec9e40 2024-09-13T16:30:34.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.768+0000 7fd99e5e6740 10 client.15252 fetch_fsmap finished waiting for FSMap version 28 2024-09-13T16:30:34.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.768+0000 7fd99e5e6740 10 client.15252 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-13T16:30:34.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.768+0000 7fd99e5e6740 20 client.15252 populate_metadata read hostname 'smithi016' 2024-09-13T16:30:34.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.768+0000 7fd99e5e6740 1 --2- 172.21.15.16:0/4239087800 >> [v2:172.21.15.107:6836/2838457611,v1:172.21.15.107:6837/2838457611] conn(0x55c617f39960 0x55c617f362d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.768+0000 7fd99e5e6740 4 client.15252 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T16:30:34.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.768+0000 7fd99e5e6740 1 -- 172.21.15.16:0/4239087800 --> [v2:172.21.15.107:6836/2838457611,v1:172.21.15.107:6837/2838457611] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c617b84d10 con 0x55c617f39960 2024-09-13T16:30:34.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.769+0000 7fd999585640 1 --2- 172.21.15.16:0/4239087800 >> [v2:172.21.15.107:6836/2838457611,v1:172.21.15.107:6837/2838457611] conn(0x55c617f39960 0x55c617f362d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.769+0000 7fd999585640 1 --2- 172.21.15.16:0/4239087800 >> [v2:172.21.15.107:6836/2838457611,v1:172.21.15.107:6837/2838457611] conn(0x55c617f39960 0x55c617f362d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=496aeb5a6f58a991 server_cookie=e12508670804dbda in_seq=0 out_seq=0 2024-09-13T16:30:34.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.769+0000 7fd978ff9640 10 client.15252 ms_handle_connect on v2:172.21.15.107:6836/2838457611 2024-09-13T16:30:34.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.769+0000 7fd978ff9640 1 -- 172.21.15.16:0/4239087800 <== mds.2 v2:172.21.15.107:6836/2838457611 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55c617b84d10 con 0x55c617f39960 2024-09-13T16:30:34.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.769+0000 7fd978ff9640 10 client.15252 handle_command_reply: tid=0 2024-09-13T16:30:34.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.769+0000 7fd99e5e6740 1 client.15252 shutdown 2024-09-13T16:30:34.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.769+0000 7fd99e5e6740 1 -- 172.21.15.16:0/4239087800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c617ec9e40 msgr2=0x55c617ed51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.769+0000 7fd99e5e6740 1 --2- 172.21.15.16:0/4239087800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c617ec9e40 0x55c617ed51b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd9880b7770 tx=0x7fd98808dee0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.769+0000 7fd99e5e6740 1 -- 172.21.15.16:0/4239087800 shutdown_connections 2024-09-13T16:30:34.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.769+0000 7fd99e5e6740 1 --2- 172.21.15.16:0/4239087800 >> [v2:172.21.15.107:6836/2838457611,v1:172.21.15.107:6837/2838457611] conn(0x55c617f39960 0x55c617f362d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.769+0000 7fd99e5e6740 1 --2- 172.21.15.16:0/4239087800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c617eceed0 0x55c617eda020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.769+0000 7fd99e5e6740 1 --2- 172.21.15.16:0/4239087800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c617ec9e40 0x55c617ed51b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.769+0000 7fd99e5e6740 1 --2- 172.21.15.16:0/4239087800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c617ec9500 0x55c617ed4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.769+0000 7fd978ff9640 0 client.15252 ms_handle_reset on v2:172.21.15.107:6836/2838457611 2024-09-13T16:30:34.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.769+0000 7fd99e5e6740 1 -- 172.21.15.16:0/4239087800 >> 172.21.15.16:0/4239087800 conn(0x55c617ec4b70 msgr2=0x55c617ecdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:34.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.769+0000 7fd99e5e6740 1 -- 172.21.15.16:0/4239087800 shutdown_connections 2024-09-13T16:30:34.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.769+0000 7fd99e5e6740 1 -- 172.21.15.16:0/4239087800 wait complete. 2024-09-13T16:30:34.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.770+0000 7fd99e5e6740 20 client.15252 trim_cache size 0 max 16384 2024-09-13T16:30:34.778 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-13T16:30:34.778 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "0d241f6a-71ec-11ef-bceb-c7b262605968", 2024-09-13T16:30:34.778 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 2, 2024-09-13T16:30:34.778 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24485, 2024-09-13T16:30:34.778 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-13T16:30:34.778 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-13T16:30:34.779 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-13T16:30:34.779 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 91.187382798000002, 2024-09-13T16:30:34.779 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 28, 2024-09-13T16:30:34.779 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 93, 2024-09-13T16:30:34.779 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 91, 2024-09-13T16:30:34.779 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 130.25248192999999 2024-09-13T16:30:34.779 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-13T16:30:34.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.770+0000 7fd99b7e9640 1 -- 172.21.15.16:0/2198763487 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd97c0a1110 msgr2=0x7fd97c0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.770+0000 7fd99b7e9640 1 --2- 172.21.15.16:0/2198763487 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd97c0a1110 0x7fd97c0a35d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd984007310 tx=0x7fd984007460 comp rx=0 tx=0).stop 2024-09-13T16:30:34.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.770+0000 7fd99b7e9640 1 -- 172.21.15.16:0/2198763487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd994100f80 msgr2=0x7fd9941961a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.770+0000 7fd99b7e9640 1 --2- 172.21.15.16:0/2198763487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd994100f80 0x7fd9941961a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd988006010 tx=0x7fd988004970 comp rx=0 tx=0).stop 2024-09-13T16:30:34.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.770+0000 7fd99b7e9640 1 -- 172.21.15.16:0/2198763487 shutdown_connections 2024-09-13T16:30:34.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.770+0000 7fd99b7e9640 1 --2- 172.21.15.16:0/2198763487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd994101940 0x7fd99419d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.770+0000 7fd99b7e9640 1 --2- 172.21.15.16:0/2198763487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd994100f80 0x7fd9941961a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.770+0000 7fd99b7e9640 1 --2- 172.21.15.16:0/2198763487 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd97c0a1110 0x7fd97c0a35d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.770+0000 7fd99b7e9640 1 --2- 172.21.15.16:0/2198763487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9940ffd90 0x7fd994195c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.770+0000 7fd99b7e9640 1 -- 172.21.15.16:0/2198763487 >> 172.21.15.16:0/2198763487 conn(0x7fd9940fb4d0 msgr2=0x7fd994104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:34.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.770+0000 7fd99b7e9640 1 -- 172.21.15.16:0/2198763487 shutdown_connections 2024-09-13T16:30:34.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.770+0000 7fd99b7e9640 1 -- 172.21.15.16:0/2198763487 wait complete. 2024-09-13T16:30:34.782 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-09-13T16:30:34.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.884+0000 7fdbdc219640 1 -- 172.21.15.16:0/577298620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd40ffe70 msgr2=0x7fdbd41002f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.884+0000 7fdbdc219640 1 --2- 172.21.15.16:0/577298620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd40ffe70 0x7fdbd41002f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4009900 tx=0x7fdbc402a410 comp rx=0 tx=0).stop 2024-09-13T16:30:34.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.884+0000 7fdbdc219640 1 -- 172.21.15.16:0/577298620 shutdown_connections 2024-09-13T16:30:34.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.884+0000 7fdbdc219640 1 --2- 172.21.15.16:0/577298620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbd4100830 0x7fdbd41070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.884+0000 7fdbdc219640 1 --2- 172.21.15.16:0/577298620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd40ffe70 0x7fdbd41002f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.884+0000 7fdbdc219640 1 --2- 172.21.15.16:0/577298620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbd40fec80 0x7fdbd40ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.884+0000 7fdbdc219640 1 -- 172.21.15.16:0/577298620 >> 172.21.15.16:0/577298620 conn(0x7fdbd40fa3e0 msgr2=0x7fdbd40fc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:34.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.884+0000 7fdbdc219640 1 -- 172.21.15.16:0/577298620 shutdown_connections 2024-09-13T16:30:34.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.884+0000 7fdbdc219640 1 -- 172.21.15.16:0/577298620 wait complete. 2024-09-13T16:30:34.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.884+0000 7fdbdc219640 1 Processor -- start 2024-09-13T16:30:34.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.885+0000 7fdbdc219640 1 -- start start 2024-09-13T16:30:34.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.885+0000 7fdbdc219640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd40fec80 0x7fdbd4195c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.885+0000 7fdbdc219640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbd40ffe70 0x7fdbd41961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.885+0000 7fdbdc219640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbd4100830 0x7fdbd419d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.885+0000 7fdbdc219640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd419d740 con 0x7fdbd40fec80 2024-09-13T16:30:34.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.885+0000 7fdbdc219640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd419d8b0 con 0x7fdbd40ffe70 2024-09-13T16:30:34.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.885+0000 7fdbdc219640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd419da70 con 0x7fdbd4100830 2024-09-13T16:30:34.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.885+0000 7fdbd9fb5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd40fec80 0x7fdbd4195c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.885+0000 7fdbd9fb5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd40fec80 0x7fdbd4195c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52280/0 (socket says 172.21.15.16:52280) 2024-09-13T16:30:34.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.885+0000 7fdbd9fb5640 1 -- 172.21.15.16:0/3078993817 learned_addr learned my addr 172.21.15.16:0/3078993817 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.885+0000 7fdbd97b4640 1 --2- 172.21.15.16:0/3078993817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbd40ffe70 0x7fdbd41961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.885+0000 7fdbda7b6640 1 --2- 172.21.15.16:0/3078993817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbd4100830 0x7fdbd419d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.885+0000 7fdbd9fb5640 1 -- 172.21.15.16:0/3078993817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbd4100830 msgr2=0x7fdbd419d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.885+0000 7fdbd9fb5640 1 --2- 172.21.15.16:0/3078993817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbd4100830 0x7fdbd419d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.885+0000 7fdbd9fb5640 1 -- 172.21.15.16:0/3078993817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbd40ffe70 msgr2=0x7fdbd41961a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.885+0000 7fdbd9fb5640 1 --2- 172.21.15.16:0/3078993817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbd40ffe70 0x7fdbd41961a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.885+0000 7fdbd9fb5640 1 -- 172.21.15.16:0/3078993817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc4009560 con 0x7fdbd40fec80 2024-09-13T16:30:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.885+0000 7fdbda7b6640 1 --2- 172.21.15.16:0/3078993817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbd4100830 0x7fdbd419d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:30:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.885+0000 7fdbd9fb5640 1 --2- 172.21.15.16:0/3078993817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd40fec80 0x7fdbd4195c60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000e9c0 tx=0x7fdbd000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.886+0000 7fdbcaffd640 1 -- 172.21.15.16:0/3078993817 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0009700 con 0x7fdbd40fec80 2024-09-13T16:30:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.886+0000 7fdbcaffd640 1 -- 172.21.15.16:0/3078993817 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdbd00044b0 con 0x7fdbd40fec80 2024-09-13T16:30:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.886+0000 7fdbcaffd640 1 -- 172.21.15.16:0/3078993817 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0003e40 con 0x7fdbd40fec80 2024-09-13T16:30:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.886+0000 7fdbdc219640 1 -- 172.21.15.16:0/3078993817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd419dd50 con 0x7fdbd40fec80 2024-09-13T16:30:34.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.886+0000 7fdbdc219640 1 -- 172.21.15.16:0/3078993817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd419e1b0 con 0x7fdbd40fec80 2024-09-13T16:30:34.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.888+0000 7fdbcaffd640 1 -- 172.21.15.16:0/3078993817 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fdbd0004000 con 0x7fdbd40fec80 2024-09-13T16:30:34.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.888+0000 7fdbdf018740 1 Processor -- start 2024-09-13T16:30:34.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.888+0000 7fdbdf018740 1 -- start start 2024-09-13T16:30:34.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.889+0000 7fdbdf018740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646919d6500 0x5646919d6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.889+0000 7fdbdf018740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646919d6e40 0x5646919db900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.889+0000 7fdbdf018740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646919dbed0 0x5646919de390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.889+0000 7fdbdf018740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646919317c0 con 0x5646919dbed0 2024-09-13T16:30:34.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.889+0000 7fdbdf018740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x564691940b70 con 0x5646919d6e40 2024-09-13T16:30:34.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.889+0000 7fdbdf018740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x56469193e300 con 0x5646919d6500 2024-09-13T16:30:34.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.889+0000 7fdbd97b4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646919dbed0 0x5646919de390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.889+0000 7fdbd97b4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646919dbed0 0x5646919de390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52286/0 (socket says 172.21.15.16:52286) 2024-09-13T16:30:34.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.889+0000 7fdbda7b6640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646919d6500 0x5646919d6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.889+0000 7fdbd97b4640 1 -- 172.21.15.16:0/2271634857 learned_addr learned my addr 172.21.15.16:0/2271634857 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:34.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.889+0000 7fdbcaffd640 1 --2- 172.21.15.16:0/3078993817 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdbc00a1090 0x7fdbc00a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.889+0000 7fdbd9fb5640 1 --2- 172.21.15.16:0/2271634857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646919d6e40 0x5646919db900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.889+0000 7fdbda7b6640 1 --2- 172.21.15.16:0/3078993817 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdbc00a1090 0x7fdbc00a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.889+0000 7fdbd97b4640 1 -- 172.21.15.16:0/2271634857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646919d6500 msgr2=0x5646919d6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:34.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.889+0000 7fdbd97b4640 1 --2- 172.21.15.16:0/2271634857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646919d6500 0x5646919d6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.889+0000 7fdbd97b4640 1 -- 172.21.15.16:0/2271634857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646919d6e40 msgr2=0x5646919db900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:34.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.889+0000 7fdbd97b4640 1 --2- 172.21.15.16:0/2271634857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646919d6e40 0x5646919db900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.889+0000 7fdbd97b4640 1 -- 172.21.15.16:0/2271634857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646918c64b0 con 0x5646919dbed0 2024-09-13T16:30:34.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.889+0000 7fdbd9fb5640 1 --2- 172.21.15.16:0/2271634857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646919d6e40 0x5646919db900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:30:34.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.889+0000 7fdbcaffd640 1 -- 172.21.15.16:0/3078993817 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fdbd0014070 con 0x7fdbd40fec80 2024-09-13T16:30:34.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.889+0000 7fdbd97b4640 1 --2- 172.21.15.16:0/2271634857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646919dbed0 0x5646919de390 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fdbc402a930 tx=0x7fdbc40314f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92fd2bf58aafb7c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:34.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.890+0000 7fdbda7b6640 1 --2- 172.21.15.16:0/3078993817 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdbc00a1090 0x7fdbc00a3550 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009fd0 tx=0x7fdbcc004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:34.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.890+0000 7fdbbffff640 1 -- 172.21.15.16:0/2271634857 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4004230 con 0x5646919dbed0 2024-09-13T16:30:34.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.890+0000 7fdbbffff640 1 -- 172.21.15.16:0/2271634857 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdbc40317a0 con 0x5646919dbed0 2024-09-13T16:30:34.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.890+0000 7fdbbffff640 1 -- 172.21.15.16:0/2271634857 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4003bc0 con 0x5646919dbed0 2024-09-13T16:30:34.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.890+0000 7fdbdf018740 1 -- 172.21.15.16:0/2271634857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646919dbed0 msgr2=0x5646919de390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.890+0000 7fdbdf018740 1 --2- 172.21.15.16:0/2271634857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646919dbed0 0x5646919de390 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fdbc402a930 tx=0x7fdbc40314f0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.890+0000 7fdbdf018740 1 -- 172.21.15.16:0/2271634857 shutdown_connections 2024-09-13T16:30:34.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.890+0000 7fdbdf018740 1 --2- 172.21.15.16:0/2271634857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646919dbed0 0x5646919de390 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.890+0000 7fdbdf018740 1 --2- 172.21.15.16:0/2271634857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646919d6e40 0x5646919db900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.890+0000 7fdbdf018740 1 --2- 172.21.15.16:0/2271634857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646919d6500 0x5646919d6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.890+0000 7fdbdf018740 1 -- 172.21.15.16:0/2271634857 >> 172.21.15.16:0/2271634857 conn(0x5646919d1b70 msgr2=0x5646919d1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:34.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.890+0000 7fdbdf018740 1 -- 172.21.15.16:0/2271634857 shutdown_connections 2024-09-13T16:30:34.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.890+0000 7fdbdf018740 1 -- 172.21.15.16:0/2271634857 wait complete. 2024-09-13T16:30:34.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.891+0000 7fdbdf018740 1 -- start start 2024-09-13T16:30:34.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.891+0000 7fdbdf018740 1 Processor -- start 2024-09-13T16:30:34.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.891+0000 7fdbdf018740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646919d6500 0x5646919e0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.891+0000 7fdbdf018740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646919d6e40 0x5646919e05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.891+0000 7fdbdf018740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646919dbed0 0x5646919e75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.891+0000 7fdbdf018740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564691939bf0 con 0x5646919dbed0 2024-09-13T16:30:34.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.891+0000 7fdbdf018740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646918f8250 con 0x5646919d6e40 2024-09-13T16:30:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.891+0000 7fdbdf018740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646918f7ef0 con 0x5646919d6500 2024-09-13T16:30:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.891+0000 7fdbdf018740 10 client.0 _ll_register_callbacks cb 0x56469192a740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:30:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.891+0000 7fdbda7b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646919dbed0 0x5646919e75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.891+0000 7fdbda7b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646919dbed0 0x5646919e75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52294/0 (socket says 172.21.15.16:52294) 2024-09-13T16:30:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.891+0000 7fdbda7b6640 1 -- 172.21.15.16:0/3381806508 learned_addr learned my addr 172.21.15.16:0/3381806508 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.891+0000 7fdbd97b4640 1 --2- 172.21.15.16:0/3381806508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646919d6e40 0x5646919e05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.892+0000 7fdbd9fb5640 1 --2- 172.21.15.16:0/3381806508 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646919d6500 0x5646919e0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.892+0000 7fdbda7b6640 1 -- 172.21.15.16:0/3381806508 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646919d6500 msgr2=0x5646919e0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.892+0000 7fdbda7b6640 1 --2- 172.21.15.16:0/3381806508 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646919d6500 0x5646919e0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.892+0000 7fdbda7b6640 1 -- 172.21.15.16:0/3381806508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646919d6e40 msgr2=0x5646919e05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.892+0000 7fdbda7b6640 1 --2- 172.21.15.16:0/3381806508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646919d6e40 0x5646919e05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.892+0000 7fdbda7b6640 1 -- 172.21.15.16:0/3381806508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd40685f0 con 0x5646919dbed0 2024-09-13T16:30:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.892+0000 7fdbd9fb5640 1 --2- 172.21.15.16:0/3381806508 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646919d6500 0x5646919e0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:30:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.892+0000 7fdbda7b6640 1 --2- 172.21.15.16:0/3381806508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646919dbed0 0x5646919e75f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00ec70 tx=0x7fdbcc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10a506df429a21 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.892+0000 7fdbbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:30:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.892+0000 7fdbbd7fa640 1 -- 172.21.15.16:0/3381806508 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc004d20 con 0x5646919dbed0 2024-09-13T16:30:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.892+0000 7fdbbd7fa640 1 -- 172.21.15.16:0/3381806508 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fdbcc005020 con 0x5646919dbed0 2024-09-13T16:30:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.892+0000 7fdbdf018740 1 -- 172.21.15.16:0/3381806508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56469162d3c0 con 0x5646919dbed0 2024-09-13T16:30:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.892+0000 7fdbbd7fa640 1 -- 172.21.15.16:0/3381806508 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbcc004e80 con 0x5646919dbed0 2024-09-13T16:30:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.892+0000 7fdbdf018740 10 client.15264 fetch_fsmap learned FSMap version 28 2024-09-13T16:30:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.892+0000 7fdbdf018740 1 -- 172.21.15.16:0/3381806508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564691938270 con 0x5646919dbed0 2024-09-13T16:30:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.892+0000 7fdbbd7fa640 1 -- 172.21.15.16:0/3381806508 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56469162d3c0 con 0x5646919dbed0 2024-09-13T16:30:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.893+0000 7fdbbd7fa640 1 -- 172.21.15.16:0/3381806508 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdbcc020590 con 0x5646919dbed0 2024-09-13T16:30:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.893+0000 7fdbdf018740 10 client.15264 fetch_fsmap finished waiting for FSMap version 28 2024-09-13T16:30:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.893+0000 7fdbdf018740 10 client.15264 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-13T16:30:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.893+0000 7fdbdf018740 20 client.15264 populate_metadata read hostname 'smithi016' 2024-09-13T16:30:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.893+0000 7fdbdf018740 1 --2- 172.21.15.16:0/3381806508 >> [v2:172.21.15.111:6838/1423842285,v1:172.21.15.111:6839/1423842285] conn(0x5646919ebad0 0x5646919f07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.893+0000 7fdbdf018740 4 client.15264 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T16:30:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.893+0000 7fdbdf018740 1 -- 172.21.15.16:0/3381806508 --> [v2:172.21.15.111:6838/1423842285,v1:172.21.15.111:6839/1423842285] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56469163e6c0 con 0x5646919ebad0 2024-09-13T16:30:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.893+0000 7fdbd9fb5640 1 --2- 172.21.15.16:0/3381806508 >> [v2:172.21.15.111:6838/1423842285,v1:172.21.15.111:6839/1423842285] conn(0x5646919ebad0 0x5646919f07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.893+0000 7fdbd9fb5640 1 --2- 172.21.15.16:0/3381806508 >> [v2:172.21.15.111:6838/1423842285,v1:172.21.15.111:6839/1423842285] conn(0x5646919ebad0 0x5646919f07d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9c913cd0941cf38f server_cookie=ecc2de5ca821613b in_seq=0 out_seq=0 2024-09-13T16:30:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.893+0000 7fdbbd7fa640 10 client.15264 ms_handle_connect on v2:172.21.15.111:6838/1423842285 2024-09-13T16:30:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.894+0000 7fdbbd7fa640 1 -- 172.21.15.16:0/3381806508 <== mds.1 v2:172.21.15.111:6838/1423842285 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56469163e6c0 con 0x5646919ebad0 2024-09-13T16:30:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.894+0000 7fdbbd7fa640 10 client.15264 handle_command_reply: tid=0 2024-09-13T16:30:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.894+0000 7fdbdf018740 1 client.15264 shutdown 2024-09-13T16:30:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.894+0000 7fdbdf018740 1 -- 172.21.15.16:0/3381806508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646919dbed0 msgr2=0x5646919e75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.894+0000 7fdbdf018740 1 --2- 172.21.15.16:0/3381806508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646919dbed0 0x5646919e75f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00ec70 tx=0x7fdbcc004a40 comp rx=0 tx=0).stop 2024-09-13T16:30:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.894+0000 7fdbdf018740 1 -- 172.21.15.16:0/3381806508 shutdown_connections 2024-09-13T16:30:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.894+0000 7fdbdf018740 1 --2- 172.21.15.16:0/3381806508 >> [v2:172.21.15.111:6838/1423842285,v1:172.21.15.111:6839/1423842285] conn(0x5646919ebad0 0x5646919f07d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.894+0000 7fdbbd7fa640 0 client.15264 ms_handle_reset on v2:172.21.15.111:6838/1423842285 2024-09-13T16:30:34.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.894+0000 7fdbdf018740 1 --2- 172.21.15.16:0/3381806508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646919dbed0 0x5646919e75f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.894+0000 7fdbdf018740 1 --2- 172.21.15.16:0/3381806508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646919d6e40 0x5646919e05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.894+0000 7fdbdf018740 1 --2- 172.21.15.16:0/3381806508 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646919d6500 0x5646919e0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.894+0000 7fdbdf018740 1 -- 172.21.15.16:0/3381806508 >> 172.21.15.16:0/3381806508 conn(0x5646919d1b70 msgr2=0x5646919d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:34.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.894+0000 7fdbdf018740 1 -- 172.21.15.16:0/3381806508 shutdown_connections 2024-09-13T16:30:34.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.894+0000 7fdbdf018740 1 -- 172.21.15.16:0/3381806508 wait complete. 2024-09-13T16:30:34.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.894+0000 7fdbdf018740 20 client.15264 trim_cache size 0 max 16384 2024-09-13T16:30:34.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.909+0000 7fdbdf018740 1 Processor -- start 2024-09-13T16:30:34.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.909+0000 7fdbdf018740 1 -- start start 2024-09-13T16:30:34.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.909+0000 7fdbdf018740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646919d6500 0x5646919db270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.909+0000 7fdbdf018740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646919d6e40 0x5646919db7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.909+0000 7fdbdf018740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646919dbed0 0x5646919e7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.909+0000 7fdbdf018740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646918bd460 con 0x5646919d6500 2024-09-13T16:30:34.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.909+0000 7fdbdf018740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56469163e6f0 con 0x5646919d6e40 2024-09-13T16:30:34.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.909+0000 7fdbdf018740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x564691932990 con 0x5646919dbed0 2024-09-13T16:30:34.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.909+0000 7fdbd9fb5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646919d6500 0x5646919db270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.909+0000 7fdbd9fb5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646919d6500 0x5646919db270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52308/0 (socket says 172.21.15.16:52308) 2024-09-13T16:30:34.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.909+0000 7fdbd9fb5640 1 -- 172.21.15.16:0/2123678921 learned_addr learned my addr 172.21.15.16:0/2123678921 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:34.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.909+0000 7fdbd97b4640 1 --2- 172.21.15.16:0/2123678921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646919d6e40 0x5646919db7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.910+0000 7fdbda7b6640 1 --2- 172.21.15.16:0/2123678921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646919dbed0 0x5646919e7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.910+0000 7fdbd9fb5640 1 -- 172.21.15.16:0/2123678921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646919dbed0 msgr2=0x5646919e7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:34.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.910+0000 7fdbd9fb5640 1 --2- 172.21.15.16:0/2123678921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646919dbed0 0x5646919e7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.910+0000 7fdbd9fb5640 1 -- 172.21.15.16:0/2123678921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646919d6e40 msgr2=0x5646919db7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:34.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.910+0000 7fdbd9fb5640 1 --2- 172.21.15.16:0/2123678921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646919d6e40 0x5646919db7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.910+0000 7fdbd9fb5640 1 -- 172.21.15.16:0/2123678921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646918da5f0 con 0x5646919d6500 2024-09-13T16:30:34.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.910+0000 7fdbda7b6640 1 --2- 172.21.15.16:0/2123678921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646919dbed0 0x5646919e7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:30:34.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.910+0000 7fdbd9fb5640 1 --2- 172.21.15.16:0/2123678921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646919d6500 0x5646919db270 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000c6a0 tx=0x7fdbd0099d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9e10413404b1fd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:34.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.910+0000 7fdbc8ff9640 1 -- 172.21.15.16:0/2123678921 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0065ea0 con 0x5646919d6500 2024-09-13T16:30:34.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.910+0000 7fdbc8ff9640 1 -- 172.21.15.16:0/2123678921 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdbd009a2b0 con 0x5646919d6500 2024-09-13T16:30:34.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.910+0000 7fdbc8ff9640 1 -- 172.21.15.16:0/2123678921 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd00caa70 con 0x5646919d6500 2024-09-13T16:30:34.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.910+0000 7fdbdf018740 1 -- 172.21.15.16:0/2123678921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646919d6500 msgr2=0x5646919db270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.910+0000 7fdbdf018740 1 --2- 172.21.15.16:0/2123678921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646919d6500 0x5646919db270 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000c6a0 tx=0x7fdbd0099d40 comp rx=0 tx=0).stop 2024-09-13T16:30:34.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.910+0000 7fdbdf018740 1 -- 172.21.15.16:0/2123678921 shutdown_connections 2024-09-13T16:30:34.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.910+0000 7fdbdf018740 1 --2- 172.21.15.16:0/2123678921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646919dbed0 0x5646919e7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.910+0000 7fdbdf018740 1 --2- 172.21.15.16:0/2123678921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646919d6e40 0x5646919db7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.910+0000 7fdbdf018740 1 --2- 172.21.15.16:0/2123678921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646919d6500 0x5646919db270 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.911+0000 7fdbdf018740 1 -- 172.21.15.16:0/2123678921 >> 172.21.15.16:0/2123678921 conn(0x5646919d1b70 msgr2=0x5646919d7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.911+0000 7fdbdf018740 1 -- 172.21.15.16:0/2123678921 shutdown_connections 2024-09-13T16:30:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.911+0000 7fdbdf018740 1 -- 172.21.15.16:0/2123678921 wait complete. 2024-09-13T16:30:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.911+0000 7fdbdf018740 1 -- start start 2024-09-13T16:30:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.911+0000 7fdbdf018740 1 Processor -- start 2024-09-13T16:30:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.911+0000 7fdbdf018740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646919d6500 0x5646919e1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.911+0000 7fdbdf018740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646919d6e40 0x5646919e2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.911+0000 7fdbdf018740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646919dbed0 0x5646919e72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.911+0000 7fdbdf018740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646919e7800 con 0x5646919d6e40 2024-09-13T16:30:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.911+0000 7fdbdf018740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646919e7970 con 0x5646919d6500 2024-09-13T16:30:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.911+0000 7fdbdf018740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x564691a42490 con 0x5646919dbed0 2024-09-13T16:30:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.911+0000 7fdbdf018740 10 client.0 _ll_register_callbacks cb 0x7fdbcc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:30:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.911+0000 7fdbd97b4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646919d6e40 0x5646919e2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.911+0000 7fdbd97b4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646919d6e40 0x5646919e2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52310/0 (socket says 172.21.15.16:52310) 2024-09-13T16:30:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.911+0000 7fdbd97b4640 1 -- 172.21.15.16:0/1733821567 learned_addr learned my addr 172.21.15.16:0/1733821567 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.911+0000 7fdbd9fb5640 1 --2- 172.21.15.16:0/1733821567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646919d6500 0x5646919e1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.912+0000 7fdbda7b6640 1 --2- 172.21.15.16:0/1733821567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646919dbed0 0x5646919e72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.912+0000 7fdbd97b4640 1 -- 172.21.15.16:0/1733821567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646919dbed0 msgr2=0x5646919e72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.912+0000 7fdbd97b4640 1 --2- 172.21.15.16:0/1733821567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646919dbed0 0x5646919e72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.912+0000 7fdbd97b4640 1 -- 172.21.15.16:0/1733821567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646919d6500 msgr2=0x5646919e1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.912+0000 7fdbd97b4640 1 --2- 172.21.15.16:0/1733821567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646919d6500 0x5646919e1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.912+0000 7fdbd97b4640 1 -- 172.21.15.16:0/1733821567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd419e1b0 con 0x5646919d6e40 2024-09-13T16:30:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.912+0000 7fdbd97b4640 1 --2- 172.21.15.16:0/1733821567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646919d6e40 0x5646919e2290 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdbc402aa00 tx=0x7fdbc4031b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8132d16859db1021 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.912+0000 7fdbbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:30:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.912+0000 7fdbbd7fa640 1 -- 172.21.15.16:0/1733821567 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc404c050 con 0x5646919d6e40 2024-09-13T16:30:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.912+0000 7fdbbd7fa640 1 -- 172.21.15.16:0/1733821567 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fdbc403d650 con 0x5646919d6e40 2024-09-13T16:30:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.912+0000 7fdbdf018740 1 -- 172.21.15.16:0/1733821567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdbc40317a0 con 0x5646919d6e40 2024-09-13T16:30:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.912+0000 7fdbbd7fa640 1 -- 172.21.15.16:0/1733821567 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbc403d810 con 0x5646919d6e40 2024-09-13T16:30:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.912+0000 7fdbdf018740 10 client.15270 fetch_fsmap learned FSMap version 28 2024-09-13T16:30:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.912+0000 7fdbdf018740 1 -- 172.21.15.16:0/1733821567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5646918ffd00 con 0x5646919d6e40 2024-09-13T16:30:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.912+0000 7fdbbd7fa640 1 -- 172.21.15.16:0/1733821567 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdbc40317a0 con 0x5646919d6e40 2024-09-13T16:30:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.912+0000 7fdbbd7fa640 1 -- 172.21.15.16:0/1733821567 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdbc40458b0 con 0x5646919d6e40 2024-09-13T16:30:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.912+0000 7fdbdf018740 10 client.15270 fetch_fsmap finished waiting for FSMap version 28 2024-09-13T16:30:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.912+0000 7fdbdf018740 10 client.15270 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-13T16:30:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.912+0000 7fdbdf018740 20 client.15270 populate_metadata read hostname 'smithi016' 2024-09-13T16:30:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.912+0000 7fdbdf018740 1 --2- 172.21.15.16:0/1733821567 >> [v2:172.21.15.111:6838/1423842285,v1:172.21.15.111:6839/1423842285] conn(0x564691a46e70 0x564691a437e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.913+0000 7fdbdf018740 4 client.15270 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T16:30:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.913+0000 7fdbdf018740 1 -- 172.21.15.16:0/1733821567 --> [v2:172.21.15.111:6838/1423842285,v1:172.21.15.111:6839/1423842285] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564691691d10 con 0x564691a46e70 2024-09-13T16:30:34.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.913+0000 7fdbd9fb5640 1 --2- 172.21.15.16:0/1733821567 >> [v2:172.21.15.111:6838/1423842285,v1:172.21.15.111:6839/1423842285] conn(0x564691a46e70 0x564691a437e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:34.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.913+0000 7fdbd9fb5640 1 --2- 172.21.15.16:0/1733821567 >> [v2:172.21.15.111:6838/1423842285,v1:172.21.15.111:6839/1423842285] conn(0x564691a46e70 0x564691a437e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f8512e785242e92e server_cookie=c98c3e27865b8d25 in_seq=0 out_seq=0 2024-09-13T16:30:34.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.913+0000 7fdbbd7fa640 10 client.15270 ms_handle_connect on v2:172.21.15.111:6838/1423842285 2024-09-13T16:30:34.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.913+0000 7fdbbd7fa640 1 -- 172.21.15.16:0/1733821567 <== mds.1 v2:172.21.15.111:6838/1423842285 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x564691691d10 con 0x564691a46e70 2024-09-13T16:30:34.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.913+0000 7fdbbd7fa640 10 client.15270 handle_command_reply: tid=0 2024-09-13T16:30:34.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.913+0000 7fdbdf018740 1 client.15270 shutdown 2024-09-13T16:30:34.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.913+0000 7fdbdf018740 1 -- 172.21.15.16:0/1733821567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646919d6e40 msgr2=0x5646919e2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.913+0000 7fdbdf018740 1 --2- 172.21.15.16:0/1733821567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646919d6e40 0x5646919e2290 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdbc402aa00 tx=0x7fdbc4031b90 comp rx=0 tx=0).stop 2024-09-13T16:30:34.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.914+0000 7fdbdf018740 1 -- 172.21.15.16:0/1733821567 shutdown_connections 2024-09-13T16:30:34.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.914+0000 7fdbdf018740 1 --2- 172.21.15.16:0/1733821567 >> [v2:172.21.15.111:6838/1423842285,v1:172.21.15.111:6839/1423842285] conn(0x564691a46e70 0x564691a437e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.914+0000 7fdbdf018740 1 --2- 172.21.15.16:0/1733821567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646919dbed0 0x5646919e72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.914+0000 7fdbdf018740 1 --2- 172.21.15.16:0/1733821567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646919d6e40 0x5646919e2290 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.914+0000 7fdbdf018740 1 --2- 172.21.15.16:0/1733821567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646919d6500 0x5646919e1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.914+0000 7fdbbd7fa640 0 client.15270 ms_handle_reset on v2:172.21.15.111:6838/1423842285 2024-09-13T16:30:34.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.914+0000 7fdbdf018740 1 -- 172.21.15.16:0/1733821567 >> 172.21.15.16:0/1733821567 conn(0x5646919d1b70 msgr2=0x5646919d7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:34.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.914+0000 7fdbdf018740 1 -- 172.21.15.16:0/1733821567 shutdown_connections 2024-09-13T16:30:34.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.914+0000 7fdbdf018740 1 -- 172.21.15.16:0/1733821567 wait complete. 2024-09-13T16:30:34.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.914+0000 7fdbdf018740 20 client.15270 trim_cache size 0 max 16384 2024-09-13T16:30:34.922 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-13T16:30:34.922 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "0d241f6a-71ec-11ef-bceb-c7b262605968", 2024-09-13T16:30:34.922 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 1, 2024-09-13T16:30:34.922 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24490, 2024-09-13T16:30:34.923 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-13T16:30:34.923 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-13T16:30:34.923 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-13T16:30:34.923 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 92.343166347999997, 2024-09-13T16:30:34.923 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 28, 2024-09-13T16:30:34.923 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 93, 2024-09-13T16:30:34.923 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 91, 2024-09-13T16:30:34.923 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 127.86724911899999 2024-09-13T16:30:34.923 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-13T16:30:34.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.914+0000 7fdbdc219640 1 -- 172.21.15.16:0/3078993817 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdbc00a1090 msgr2=0x7fdbc00a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.914+0000 7fdbdc219640 1 --2- 172.21.15.16:0/3078993817 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdbc00a1090 0x7fdbc00a3550 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009fd0 tx=0x7fdbcc004ab0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.914+0000 7fdbdc219640 1 -- 172.21.15.16:0/3078993817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd40fec80 msgr2=0x7fdbd4195c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:34.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.914+0000 7fdbdc219640 1 --2- 172.21.15.16:0/3078993817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd40fec80 0x7fdbd4195c60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000e9c0 tx=0x7fdbd000ee80 comp rx=0 tx=0).stop 2024-09-13T16:30:34.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.914+0000 7fdbdc219640 1 -- 172.21.15.16:0/3078993817 shutdown_connections 2024-09-13T16:30:34.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.914+0000 7fdbdc219640 1 --2- 172.21.15.16:0/3078993817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbd4100830 0x7fdbd419d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.914+0000 7fdbdc219640 1 --2- 172.21.15.16:0/3078993817 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdbc00a1090 0x7fdbc00a3550 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.914+0000 7fdbdc219640 1 --2- 172.21.15.16:0/3078993817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbd40ffe70 0x7fdbd41961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.914+0000 7fdbdc219640 1 --2- 172.21.15.16:0/3078993817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbd40fec80 0x7fdbd4195c60 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:34.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.914+0000 7fdbdc219640 1 -- 172.21.15.16:0/3078993817 >> 172.21.15.16:0/3078993817 conn(0x7fdbd40fa3e0 msgr2=0x7fdbd4103330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:34.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.914+0000 7fdbdc219640 1 -- 172.21.15.16:0/3078993817 shutdown_connections 2024-09-13T16:30:34.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:34.914+0000 7fdbdc219640 1 -- 172.21.15.16:0/3078993817 wait complete. 2024-09-13T16:30:34.925 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:34 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1124412342' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:30:34.925 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:34 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1034819777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:30:34.925 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:34 smithi016 ceph-mon[31187]: pgmap v338: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T16:30:34.926 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-13T16:30:34.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:34 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1124412342' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:30:34.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:34 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1034819777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:30:34.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:34 smithi111 ceph-mon[34600]: pgmap v338: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T16:30:35.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.028+0000 7fa9836c9640 1 -- 172.21.15.16:0/3879927739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa97c0ffe60 msgr2=0x7fa97c1002e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:35.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.028+0000 7fa9836c9640 1 --2- 172.21.15.16:0/3879927739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa97c0ffe60 0x7fa97c1002e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa96c009900 tx=0x7fa96c02a410 comp rx=0 tx=0).stop 2024-09-13T16:30:35.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.028+0000 7fa9836c9640 1 -- 172.21.15.16:0/3879927739 shutdown_connections 2024-09-13T16:30:35.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.028+0000 7fa9836c9640 1 --2- 172.21.15.16:0/3879927739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa97c100820 0x7fa97c1070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:35.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.028+0000 7fa9836c9640 1 --2- 172.21.15.16:0/3879927739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa97c0ffe60 0x7fa97c1002e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:35.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.028+0000 7fa9836c9640 1 --2- 172.21.15.16:0/3879927739 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa97c0fec70 0x7fa97c0ff070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:35.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.029+0000 7fa9836c9640 1 -- 172.21.15.16:0/3879927739 >> 172.21.15.16:0/3879927739 conn(0x7fa97c0fa410 msgr2=0x7fa97c0fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:35.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.029+0000 7fa9836c9640 1 -- 172.21.15.16:0/3879927739 shutdown_connections 2024-09-13T16:30:35.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.029+0000 7fa9836c9640 1 -- 172.21.15.16:0/3879927739 wait complete. 2024-09-13T16:30:35.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.029+0000 7fa9836c9640 1 Processor -- start 2024-09-13T16:30:35.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.029+0000 7fa9836c9640 1 -- start start 2024-09-13T16:30:35.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.029+0000 7fa9836c9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa97c0fec70 0x7fa97c068150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:35.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.029+0000 7fa9836c9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa97c0ffe60 0x7fa97c19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:35.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.029+0000 7fa9836c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa97c100820 0x7fa97c19cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:35.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.029+0000 7fa9836c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97c19d760 con 0x7fa97c100820 2024-09-13T16:30:35.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.029+0000 7fa9836c9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97c19d8d0 con 0x7fa97c0fec70 2024-09-13T16:30:35.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.029+0000 7fa9836c9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97c19da90 con 0x7fa97c0ffe60 2024-09-13T16:30:35.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.029+0000 7fa981c66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa97c100820 0x7fa97c19cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:35.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.030+0000 7fa981465640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa97c0fec70 0x7fa97c068150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:35.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.030+0000 7fa981465640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa97c0fec70 0x7fa97c068150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.16:50390/0 (socket says 172.21.15.16:50390) 2024-09-13T16:30:35.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.030+0000 7fa981c66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa97c100820 0x7fa97c19cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52332/0 (socket says 172.21.15.16:52332) 2024-09-13T16:30:35.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.030+0000 7fa981465640 1 -- 172.21.15.16:0/2691393853 learned_addr learned my addr 172.21.15.16:0/2691393853 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:35.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.030+0000 7fa980c64640 1 --2- 172.21.15.16:0/2691393853 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa97c0ffe60 0x7fa97c19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:35.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.030+0000 7fa981465640 1 -- 172.21.15.16:0/2691393853 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa97c0ffe60 msgr2=0x7fa97c19a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:35.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.030+0000 7fa981465640 1 --2- 172.21.15.16:0/2691393853 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa97c0ffe60 0x7fa97c19a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:35.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.030+0000 7fa981465640 1 -- 172.21.15.16:0/2691393853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa97c100820 msgr2=0x7fa97c19cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:35.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.030+0000 7fa981465640 1 --2- 172.21.15.16:0/2691393853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa97c100820 0x7fa97c19cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:35.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.030+0000 7fa981465640 1 -- 172.21.15.16:0/2691393853 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96c009560 con 0x7fa97c0fec70 2024-09-13T16:30:35.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.030+0000 7fa981c66640 1 --2- 172.21.15.16:0/2691393853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa97c100820 0x7fa97c19cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:30:35.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.030+0000 7fa981465640 1 --2- 172.21.15.16:0/2691393853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa97c0fec70 0x7fa97c068150 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa97800eb70 tx=0x7fa97800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:35.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.030+0000 7fa9727fc640 1 -- 172.21.15.16:0/2691393853 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa978003b70 con 0x7fa97c0fec70 2024-09-13T16:30:35.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.030+0000 7fa9836c9640 1 -- 172.21.15.16:0/2691393853 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa97c19dd70 con 0x7fa97c0fec70 2024-09-13T16:30:35.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.030+0000 7fa9727fc640 1 -- 172.21.15.16:0/2691393853 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa97800ee70 con 0x7fa97c0fec70 2024-09-13T16:30:35.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.030+0000 7fa9727fc640 1 -- 172.21.15.16:0/2691393853 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa978010690 con 0x7fa97c0fec70 2024-09-13T16:30:35.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.030+0000 7fa9836c9640 1 -- 172.21.15.16:0/2691393853 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa97c19e1a0 con 0x7fa97c0fec70 2024-09-13T16:30:35.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.033+0000 7fa9727fc640 1 -- 172.21.15.16:0/2691393853 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa978004020 con 0x7fa97c0fec70 2024-09-13T16:30:35.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.033+0000 7fa9863eb740 1 Processor -- start 2024-09-13T16:30:35.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.033+0000 7fa9863eb740 1 -- start start 2024-09-13T16:30:35.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.033+0000 7fa9863eb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1daa40500 0x55e1daa40900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:35.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.033+0000 7fa9863eb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1daa40e40 0x55e1daa45900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:35.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.033+0000 7fa9863eb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1daa45ed0 0x55e1daa48390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:35.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.033+0000 7fa9863eb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1da99b7c0 con 0x55e1daa40e40 2024-09-13T16:30:35.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.033+0000 7fa9863eb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1da9aab70 con 0x55e1daa40500 2024-09-13T16:30:35.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.033+0000 7fa9863eb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1da9a8300 con 0x55e1daa45ed0 2024-09-13T16:30:35.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.033+0000 7fa981465640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1daa40e40 0x55e1daa45900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:35.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.033+0000 7fa981465640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1daa40e40 0x55e1daa45900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52336/0 (socket says 172.21.15.16:52336) 2024-09-13T16:30:35.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.033+0000 7fa981465640 1 -- 172.21.15.16:0/786922660 learned_addr learned my addr 172.21.15.16:0/786922660 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:35.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.033+0000 7fa981c66640 1 --2- 172.21.15.16:0/786922660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1daa40500 0x55e1daa40900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:35.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.033+0000 7fa980c64640 1 --2- 172.21.15.16:0/786922660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1daa45ed0 0x55e1daa48390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:35.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.033+0000 7fa981465640 1 -- 172.21.15.16:0/786922660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1daa45ed0 msgr2=0x55e1daa48390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:35.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.033+0000 7fa981465640 1 --2- 172.21.15.16:0/786922660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1daa45ed0 0x55e1daa48390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:35.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.033+0000 7fa981465640 1 -- 172.21.15.16:0/786922660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1daa40500 msgr2=0x55e1daa40900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:35.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.033+0000 7fa981465640 1 --2- 172.21.15.16:0/786922660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1daa40500 0x55e1daa40900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:35.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.033+0000 7fa981465640 1 -- 172.21.15.16:0/786922660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1da9304b0 con 0x55e1daa40e40 2024-09-13T16:30:35.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.033+0000 7fa980c64640 1 --2- 172.21.15.16:0/786922660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1daa45ed0 0x55e1daa48390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:30:35.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.033+0000 7fa9727fc640 1 --2- 172.21.15.16:0/2691393853 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa9680a1090 0x7fa9680a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:35.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.034+0000 7fa981465640 1 --2- 172.21.15.16:0/786922660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1daa40e40 0x55e1daa45900 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa9780d5160 tx=0x7fa9780d5250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e4c98bd144e9704 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:35.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.034+0000 7fa981c66640 1 --2- 172.21.15.16:0/2691393853 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa9680a1090 0x7fa9680a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:35.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.034+0000 7fa9727fc640 1 -- 172.21.15.16:0/2691393853 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fa978014070 con 0x7fa97c0fec70 2024-09-13T16:30:35.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.034+0000 7fa9677fe640 1 -- 172.21.15.16:0/786922660 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9780d0cf0 con 0x55e1daa40e40 2024-09-13T16:30:35.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.034+0000 7fa9677fe640 1 -- 172.21.15.16:0/786922660 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa978096730 con 0x55e1daa40e40 2024-09-13T16:30:35.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.034+0000 7fa9677fe640 1 -- 172.21.15.16:0/786922660 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97809a1c0 con 0x55e1daa40e40 2024-09-13T16:30:35.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.035+0000 7fa9863eb740 1 -- 172.21.15.16:0/786922660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1daa40e40 msgr2=0x55e1daa45900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:35.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.035+0000 7fa981c66640 1 --2- 172.21.15.16:0/2691393853 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa9680a1090 0x7fa9680a3550 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa974012640 tx=0x7fa974005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:35.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.035+0000 7fa9863eb740 1 --2- 172.21.15.16:0/786922660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1daa40e40 0x55e1daa45900 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa9780d5160 tx=0x7fa9780d5250 comp rx=0 tx=0).stop 2024-09-13T16:30:35.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.035+0000 7fa9863eb740 1 -- 172.21.15.16:0/786922660 shutdown_connections 2024-09-13T16:30:35.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.035+0000 7fa9863eb740 1 --2- 172.21.15.16:0/786922660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1daa45ed0 0x55e1daa48390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:35.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.035+0000 7fa9863eb740 1 --2- 172.21.15.16:0/786922660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1daa40e40 0x55e1daa45900 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:35.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.035+0000 7fa9863eb740 1 --2- 172.21.15.16:0/786922660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1daa40500 0x55e1daa40900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:35.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.035+0000 7fa9863eb740 1 -- 172.21.15.16:0/786922660 >> 172.21.15.16:0/786922660 conn(0x55e1daa3bb70 msgr2=0x55e1daa3bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:35.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.035+0000 7fa9863eb740 1 -- 172.21.15.16:0/786922660 shutdown_connections 2024-09-13T16:30:35.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.035+0000 7fa9863eb740 1 -- 172.21.15.16:0/786922660 wait complete. 2024-09-13T16:30:35.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.035+0000 7fa9863eb740 1 -- start start 2024-09-13T16:30:35.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.035+0000 7fa9863eb740 1 Processor -- start 2024-09-13T16:30:35.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.035+0000 7fa9863eb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1daa40500 0x55e1daa4a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:35.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.036+0000 7fa9863eb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1daa40e40 0x55e1daa4a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:35.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.036+0000 7fa9863eb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1daa45ed0 0x55e1daa516b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:35.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.036+0000 7fa9863eb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1da9a3bf0 con 0x55e1daa45ed0 2024-09-13T16:30:35.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.036+0000 7fa9863eb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1da962250 con 0x55e1daa40e40 2024-09-13T16:30:35.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.036+0000 7fa9863eb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1da961ef0 con 0x55e1daa40500 2024-09-13T16:30:35.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.036+0000 7fa9863eb740 10 client.0 _ll_register_callbacks cb 0x55e1da994740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:30:35.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.036+0000 7fa981c66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1daa45ed0 0x55e1daa516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:35.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.036+0000 7fa981c66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1daa45ed0 0x55e1daa516b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52342/0 (socket says 172.21.15.16:52342) 2024-09-13T16:30:35.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.036+0000 7fa981c66640 1 -- 172.21.15.16:0/2687414064 learned_addr learned my addr 172.21.15.16:0/2687414064 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:35.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.036+0000 7fa981465640 1 --2- 172.21.15.16:0/2687414064 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1daa40500 0x55e1daa4a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:35.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.036+0000 7fa980c64640 1 --2- 172.21.15.16:0/2687414064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1daa40e40 0x55e1daa4a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:35.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.036+0000 7fa981c66640 1 -- 172.21.15.16:0/2687414064 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1daa40500 msgr2=0x55e1daa4a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:35.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.036+0000 7fa981c66640 1 --2- 172.21.15.16:0/2687414064 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1daa40500 0x55e1daa4a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:35.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.036+0000 7fa981c66640 1 -- 172.21.15.16:0/2687414064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1daa40e40 msgr2=0x55e1daa4a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:35.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.036+0000 7fa981c66640 1 --2- 172.21.15.16:0/2687414064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1daa40e40 0x55e1daa4a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:35.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.036+0000 7fa981c66640 1 -- 172.21.15.16:0/2687414064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa97c19e1a0 con 0x55e1daa45ed0 2024-09-13T16:30:35.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.036+0000 7fa981c66640 1 --2- 172.21.15.16:0/2687414064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1daa45ed0 0x55e1daa516b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa974009710 tx=0x7fa974004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da26d6b86f1deb17 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:35.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.036+0000 7fa964ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:30:35.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.036+0000 7fa964ff9640 1 -- 172.21.15.16:0/2687414064 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa974017070 con 0x55e1daa45ed0 2024-09-13T16:30:35.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.036+0000 7fa964ff9640 1 -- 172.21.15.16:0/2687414064 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fa97400f8d0 con 0x55e1daa45ed0 2024-09-13T16:30:35.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.036+0000 7fa9863eb740 1 -- 172.21.15.16:0/2687414064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e1da6973c0 con 0x55e1daa45ed0 2024-09-13T16:30:35.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.036+0000 7fa964ff9640 1 -- 172.21.15.16:0/2687414064 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa97400fa90 con 0x55e1daa45ed0 2024-09-13T16:30:35.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.036+0000 7fa9863eb740 10 client.15282 fetch_fsmap learned FSMap version 28 2024-09-13T16:30:35.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.036+0000 7fa9863eb740 1 -- 172.21.15.16:0/2687414064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e1da9a2270 con 0x55e1daa45ed0 2024-09-13T16:30:35.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.037+0000 7fa964ff9640 1 -- 172.21.15.16:0/2687414064 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e1da6973c0 con 0x55e1daa45ed0 2024-09-13T16:30:35.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.037+0000 7fa964ff9640 1 -- 172.21.15.16:0/2687414064 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa97401ece0 con 0x55e1daa45ed0 2024-09-13T16:30:35.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.037+0000 7fa9863eb740 10 client.15282 fetch_fsmap finished waiting for FSMap version 28 2024-09-13T16:30:35.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.037+0000 7fa9863eb740 10 client.15282 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-13T16:30:35.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.037+0000 7fa9863eb740 20 client.15282 populate_metadata read hostname 'smithi016' 2024-09-13T16:30:35.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.037+0000 7fa9863eb740 1 --2- 172.21.15.16:0/2687414064 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55e1daa55c30 0x55e1daa5a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:35.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.037+0000 7fa9863eb740 4 client.15282 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T16:30:35.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.037+0000 7fa9863eb740 1 -- 172.21.15.16:0/2687414064 --> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e1da6a86c0 con 0x55e1daa55c30 2024-09-13T16:30:35.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.037+0000 7fa981465640 1 --2- 172.21.15.16:0/2687414064 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55e1daa55c30 0x55e1daa5a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:35.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.037+0000 7fa981465640 1 --2- 172.21.15.16:0/2687414064 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55e1daa55c30 0x55e1daa5a910 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9ca5d5162430d89 server_cookie=43f23d41995f85ba in_seq=0 out_seq=0 2024-09-13T16:30:35.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.037+0000 7fa964ff9640 10 client.15282 ms_handle_connect on v2:172.21.15.107:6838/1914689806 2024-09-13T16:30:35.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.038+0000 7fa964ff9640 1 -- 172.21.15.16:0/2687414064 <== mds.0 v2:172.21.15.107:6838/1914689806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e1da6a86c0 con 0x55e1daa55c30 2024-09-13T16:30:35.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.038+0000 7fa964ff9640 10 client.15282 handle_command_reply: tid=0 2024-09-13T16:30:35.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.038+0000 7fa9863eb740 1 client.15282 shutdown 2024-09-13T16:30:35.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.038+0000 7fa9863eb740 1 -- 172.21.15.16:0/2687414064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1daa45ed0 msgr2=0x55e1daa516b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:35.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.038+0000 7fa9863eb740 1 --2- 172.21.15.16:0/2687414064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1daa45ed0 0x55e1daa516b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa974009710 tx=0x7fa974004730 comp rx=0 tx=0).stop 2024-09-13T16:30:35.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.038+0000 7fa9863eb740 1 -- 172.21.15.16:0/2687414064 shutdown_connections 2024-09-13T16:30:35.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.038+0000 7fa9863eb740 1 --2- 172.21.15.16:0/2687414064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1daa45ed0 0x55e1daa516b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:35.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.038+0000 7fa9863eb740 1 --2- 172.21.15.16:0/2687414064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1daa40e40 0x55e1daa4a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:35.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.038+0000 7fa9863eb740 1 --2- 172.21.15.16:0/2687414064 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55e1daa55c30 0x55e1daa5a910 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:35.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.038+0000 7fa9863eb740 1 --2- 172.21.15.16:0/2687414064 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1daa40500 0x55e1daa4a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:35.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.038+0000 7fa964ff9640 0 client.15282 ms_handle_reset on v2:172.21.15.107:6838/1914689806 2024-09-13T16:30:35.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.038+0000 7fa9863eb740 1 -- 172.21.15.16:0/2687414064 >> 172.21.15.16:0/2687414064 conn(0x55e1daa3bb70 msgr2=0x55e1daa3d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:35.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.038+0000 7fa9863eb740 1 -- 172.21.15.16:0/2687414064 shutdown_connections 2024-09-13T16:30:35.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.038+0000 7fa9863eb740 1 -- 172.21.15.16:0/2687414064 wait complete. 2024-09-13T16:30:35.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.039+0000 7fa9863eb740 20 client.15282 trim_cache size 0 max 16384 2024-09-13T16:30:35.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.053+0000 7fa9863eb740 1 Processor -- start 2024-09-13T16:30:35.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.053+0000 7fa9863eb740 1 -- start start 2024-09-13T16:30:35.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.053+0000 7fa9863eb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1daa40500 0x55e1daa4a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:35.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.053+0000 7fa9863eb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1daa40e40 0x55e1daa515d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:35.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.053+0000 7fa9863eb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1daa45ed0 0x55e1daa5f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:35.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.053+0000 7fa9863eb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1daa49d00 con 0x55e1daa40500 2024-09-13T16:30:35.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.053+0000 7fa9863eb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1da927460 con 0x55e1daa45ed0 2024-09-13T16:30:35.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.053+0000 7fa9863eb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1daa484f0 con 0x55e1daa40e40 2024-09-13T16:30:35.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.054+0000 7fa981465640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1daa40500 0x55e1daa4a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:35.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.054+0000 7fa981465640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1daa40500 0x55e1daa4a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52356/0 (socket says 172.21.15.16:52356) 2024-09-13T16:30:35.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.054+0000 7fa981465640 1 -- 172.21.15.16:0/2933685612 learned_addr learned my addr 172.21.15.16:0/2933685612 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:35.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.054+0000 7fa981c66640 1 --2- 172.21.15.16:0/2933685612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1daa45ed0 0x55e1daa5f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:35.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.054+0000 7fa980c64640 1 --2- 172.21.15.16:0/2933685612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1daa40e40 0x55e1daa515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:35.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.054+0000 7fa981465640 1 -- 172.21.15.16:0/2933685612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1daa40e40 msgr2=0x55e1daa515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:35.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.054+0000 7fa981465640 1 --2- 172.21.15.16:0/2933685612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1daa40e40 0x55e1daa515d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:35.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.054+0000 7fa981465640 1 -- 172.21.15.16:0/2933685612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1daa45ed0 msgr2=0x55e1daa5f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:35.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.054+0000 7fa981465640 1 --2- 172.21.15.16:0/2933685612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1daa45ed0 0x55e1daa5f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:35.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.054+0000 7fa981465640 1 -- 172.21.15.16:0/2933685612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1da9445f0 con 0x55e1daa40500 2024-09-13T16:30:35.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.054+0000 7fa981465640 1 --2- 172.21.15.16:0/2933685612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1daa40500 0x55e1daa4a700 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa9780d5750 tx=0x7fa97800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bcaa75f1c3c5965 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:35.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.054+0000 7fa967fff640 1 -- 172.21.15.16:0/2933685612 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9780d0cf0 con 0x55e1daa40500 2024-09-13T16:30:35.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.054+0000 7fa967fff640 1 -- 172.21.15.16:0/2933685612 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa978099f30 con 0x55e1daa40500 2024-09-13T16:30:35.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.054+0000 7fa967fff640 1 -- 172.21.15.16:0/2933685612 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97809aaf0 con 0x55e1daa40500 2024-09-13T16:30:35.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.055+0000 7fa9863eb740 1 -- 172.21.15.16:0/2933685612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1daa40500 msgr2=0x55e1daa4a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:35.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.055+0000 7fa9863eb740 1 --2- 172.21.15.16:0/2933685612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1daa40500 0x55e1daa4a700 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa9780d5750 tx=0x7fa97800cf90 comp rx=0 tx=0).stop 2024-09-13T16:30:35.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.055+0000 7fa9863eb740 1 -- 172.21.15.16:0/2933685612 shutdown_connections 2024-09-13T16:30:35.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.055+0000 7fa9863eb740 1 --2- 172.21.15.16:0/2933685612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1daa45ed0 0x55e1daa5f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:35.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.055+0000 7fa9863eb740 1 --2- 172.21.15.16:0/2933685612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1daa40e40 0x55e1daa515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:35.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.055+0000 7fa9863eb740 1 --2- 172.21.15.16:0/2933685612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1daa40500 0x55e1daa4a700 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:35.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.055+0000 7fa9863eb740 1 -- 172.21.15.16:0/2933685612 >> 172.21.15.16:0/2933685612 conn(0x55e1daa3bb70 msgr2=0x55e1daa5e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:35.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.055+0000 7fa9863eb740 1 -- 172.21.15.16:0/2933685612 shutdown_connections 2024-09-13T16:30:35.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.055+0000 7fa9863eb740 1 -- 172.21.15.16:0/2933685612 wait complete. 2024-09-13T16:30:35.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.055+0000 7fa9863eb740 1 -- start start 2024-09-13T16:30:35.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.055+0000 7fa9863eb740 1 Processor -- start 2024-09-13T16:30:35.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.055+0000 7fa9863eb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1daa40500 0x55e1daa4e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:35.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.056+0000 7fa9863eb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1daa40e40 0x55e1daa4eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:35.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.056+0000 7fa9863eb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1daa45ed0 0x55e1daaac5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:35.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.056+0000 7fa9863eb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1daa4f270 con 0x55e1daa45ed0 2024-09-13T16:30:35.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.056+0000 7fa9863eb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1daaacb10 con 0x55e1daa40500 2024-09-13T16:30:35.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.056+0000 7fa9863eb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1daaaccd0 con 0x55e1daa40e40 2024-09-13T16:30:35.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.056+0000 7fa9863eb740 10 client.0 _ll_register_callbacks cb 0x7fa9780d0a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:30:35.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.056+0000 7fa981c66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1daa45ed0 0x55e1daaac5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:35.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.056+0000 7fa981c66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1daa45ed0 0x55e1daaac5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52360/0 (socket says 172.21.15.16:52360) 2024-09-13T16:30:35.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.056+0000 7fa981c66640 1 -- 172.21.15.16:0/3212031508 learned_addr learned my addr 172.21.15.16:0/3212031508 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:35.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.056+0000 7fa980c64640 1 --2- 172.21.15.16:0/3212031508 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1daa40e40 0x55e1daa4eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:35.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.056+0000 7fa981465640 1 --2- 172.21.15.16:0/3212031508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1daa40500 0x55e1daa4e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:35.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.056+0000 7fa981c66640 1 -- 172.21.15.16:0/3212031508 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1daa40e40 msgr2=0x55e1daa4eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:35.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.056+0000 7fa981c66640 1 --2- 172.21.15.16:0/3212031508 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1daa40e40 0x55e1daa4eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:35.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.056+0000 7fa981c66640 1 -- 172.21.15.16:0/3212031508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1daa40500 msgr2=0x55e1daa4e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:30:35.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.056+0000 7fa981c66640 1 --2- 172.21.15.16:0/3212031508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1daa40500 0x55e1daa4e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:35.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.056+0000 7fa981c66640 1 -- 172.21.15.16:0/3212031508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1da9304b0 con 0x55e1daa45ed0 2024-09-13T16:30:35.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.056+0000 7fa980c64640 1 --2- 172.21.15.16:0/3212031508 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1daa40e40 0x55e1daa4eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:30:35.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.056+0000 7fa981c66640 1 --2- 172.21.15.16:0/3212031508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1daa45ed0 0x55e1daaac5d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa9740096e0 tx=0x7fa97400fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca80ca4a70e30df server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:35.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.056+0000 7fa964ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:30:35.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.056+0000 7fa964ff9640 1 -- 172.21.15.16:0/3212031508 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa974017040 con 0x55e1daa45ed0 2024-09-13T16:30:35.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.056+0000 7fa964ff9640 1 -- 172.21.15.16:0/3212031508 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fa974005220 con 0x55e1daa45ed0 2024-09-13T16:30:35.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.056+0000 7fa9863eb740 1 -- 172.21.15.16:0/3212031508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa978096730 con 0x55e1daa45ed0 2024-09-13T16:30:35.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.056+0000 7fa964ff9640 1 -- 172.21.15.16:0/3212031508 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9740053e0 con 0x55e1daa45ed0 2024-09-13T16:30:35.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.056+0000 7fa9863eb740 10 client.15288 fetch_fsmap learned FSMap version 28 2024-09-13T16:30:35.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.056+0000 7fa9863eb740 1 -- 172.21.15.16:0/3212031508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e1da969d00 con 0x55e1daa45ed0 2024-09-13T16:30:35.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.057+0000 7fa964ff9640 1 -- 172.21.15.16:0/3212031508 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa978096730 con 0x55e1daa45ed0 2024-09-13T16:30:35.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.057+0000 7fa964ff9640 1 -- 172.21.15.16:0/3212031508 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa974029730 con 0x55e1daa45ed0 2024-09-13T16:30:35.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.057+0000 7fa9863eb740 10 client.15288 fetch_fsmap finished waiting for FSMap version 28 2024-09-13T16:30:35.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.057+0000 7fa9863eb740 10 client.15288 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-13T16:30:35.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.057+0000 7fa9863eb740 20 client.15288 populate_metadata read hostname 'smithi016' 2024-09-13T16:30:35.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.057+0000 7fa9863eb740 1 --2- 172.21.15.16:0/3212031508 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55e1daab1b40 0x55e1daaae460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.057+0000 7fa9863eb740 4 client.15288 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T16:30:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.057+0000 7fa9863eb740 1 -- 172.21.15.16:0/3212031508 --> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e1da6fbd10 con 0x55e1daab1b40 2024-09-13T16:30:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.057+0000 7fa981465640 1 --2- 172.21.15.16:0/3212031508 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55e1daab1b40 0x55e1daaae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.058+0000 7fa964ff9640 10 client.15288 ms_handle_connect on v2:172.21.15.107:6838/1914689806 2024-09-13T16:30:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.058+0000 7fa981465640 1 --2- 172.21.15.16:0/3212031508 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55e1daab1b40 0x55e1daaae460 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e84826829093a99f server_cookie=6816f63bd6a552b6 in_seq=0 out_seq=0 2024-09-13T16:30:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.058+0000 7fa964ff9640 1 -- 172.21.15.16:0/3212031508 <== mds.0 v2:172.21.15.107:6838/1914689806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55e1da6fbd10 con 0x55e1daab1b40 2024-09-13T16:30:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.058+0000 7fa964ff9640 10 client.15288 handle_command_reply: tid=0 2024-09-13T16:30:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.058+0000 7fa9863eb740 1 client.15288 shutdown 2024-09-13T16:30:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.058+0000 7fa9863eb740 1 -- 172.21.15.16:0/3212031508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1daa45ed0 msgr2=0x55e1daaac5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.058+0000 7fa9863eb740 1 --2- 172.21.15.16:0/3212031508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1daa45ed0 0x55e1daaac5d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa9740096e0 tx=0x7fa97400fb00 comp rx=0 tx=0).stop 2024-09-13T16:30:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.058+0000 7fa9863eb740 1 -- 172.21.15.16:0/3212031508 shutdown_connections 2024-09-13T16:30:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.058+0000 7fa9863eb740 1 --2- 172.21.15.16:0/3212031508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1daa45ed0 0x55e1daaac5d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.058+0000 7fa9863eb740 1 --2- 172.21.15.16:0/3212031508 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55e1daab1b40 0x55e1daaae460 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.058+0000 7fa9863eb740 1 --2- 172.21.15.16:0/3212031508 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1daa40e40 0x55e1daa4eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:35.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.058+0000 7fa9863eb740 1 --2- 172.21.15.16:0/3212031508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1daa40500 0x55e1daa4e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:35.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.058+0000 7fa964ff9640 0 client.15288 ms_handle_reset on v2:172.21.15.107:6838/1914689806 2024-09-13T16:30:35.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.058+0000 7fa9863eb740 1 -- 172.21.15.16:0/3212031508 >> 172.21.15.16:0/3212031508 conn(0x55e1daa3bb70 msgr2=0x55e1daa5f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:35.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.058+0000 7fa9863eb740 1 -- 172.21.15.16:0/3212031508 shutdown_connections 2024-09-13T16:30:35.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.058+0000 7fa9863eb740 1 -- 172.21.15.16:0/3212031508 wait complete. 2024-09-13T16:30:35.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.058+0000 7fa9863eb740 20 client.15288 trim_cache size 0 max 16384 2024-09-13T16:30:35.066 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-13T16:30:35.066 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "0d241f6a-71ec-11ef-bceb-c7b262605968", 2024-09-13T16:30:35.066 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-13T16:30:35.066 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24518, 2024-09-13T16:30:35.066 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-13T16:30:35.067 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-13T16:30:35.067 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-13T16:30:35.067 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 115.871101404, 2024-09-13T16:30:35.067 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 28, 2024-09-13T16:30:35.067 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 93, 2024-09-13T16:30:35.067 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 91, 2024-09-13T16:30:35.067 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 120.27188363499999 2024-09-13T16:30:35.067 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-13T16:30:35.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.059+0000 7fa9836c9640 1 -- 172.21.15.16:0/2691393853 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa9680a1090 msgr2=0x7fa9680a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:35.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.059+0000 7fa9836c9640 1 --2- 172.21.15.16:0/2691393853 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa9680a1090 0x7fa9680a3550 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa974012640 tx=0x7fa974005f90 comp rx=0 tx=0).stop 2024-09-13T16:30:35.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.059+0000 7fa9836c9640 1 -- 172.21.15.16:0/2691393853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa97c0fec70 msgr2=0x7fa97c068150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:35.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.059+0000 7fa9836c9640 1 --2- 172.21.15.16:0/2691393853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa97c0fec70 0x7fa97c068150 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa97800eb70 tx=0x7fa97800c6a0 comp rx=0 tx=0).stop 2024-09-13T16:30:35.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.059+0000 7fa9836c9640 1 -- 172.21.15.16:0/2691393853 shutdown_connections 2024-09-13T16:30:35.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.059+0000 7fa9836c9640 1 --2- 172.21.15.16:0/2691393853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa97c100820 0x7fa97c19cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:35.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.059+0000 7fa9836c9640 1 --2- 172.21.15.16:0/2691393853 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa97c0ffe60 0x7fa97c19a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:35.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.059+0000 7fa9836c9640 1 --2- 172.21.15.16:0/2691393853 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa9680a1090 0x7fa9680a3550 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:35.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.059+0000 7fa9836c9640 1 --2- 172.21.15.16:0/2691393853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa97c0fec70 0x7fa97c068150 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:35.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.059+0000 7fa9836c9640 1 -- 172.21.15.16:0/2691393853 >> 172.21.15.16:0/2691393853 conn(0x7fa97c0fa410 msgr2=0x7fa97c103340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:35.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.059+0000 7fa9836c9640 1 -- 172.21.15.16:0/2691393853 shutdown_connections 2024-09-13T16:30:35.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:35.059+0000 7fa9836c9640 1 -- 172.21.15.16:0/2691393853 wait complete. 2024-09-13T16:30:35.071 INFO:tasks.cephadm:Setup complete, yielding 2024-09-13T16:30:35.071 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-13T16:30:35.080 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-09-13T16:30:35.081 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-13T16:30:35.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:34 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1124412342' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:30:35.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:34 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1034819777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:30:35.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:34 smithi107 ceph-mon[34139]: pgmap v338: 129 pgs: 129 active+clean; 646 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T16:30:35.287 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:30:35.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:35 smithi111 ceph-mon[34600]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-13T16:30:36.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:35 smithi107 ceph-mon[34139]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-13T16:30:36.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:35 smithi016 ceph-mon[31187]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-13T16:30:36.997 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:36 smithi016 ceph-mon[31187]: pgmap v339: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-13T16:30:37.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:36 smithi107 ceph-mon[34139]: pgmap v339: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-13T16:30:37.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.083+0000 7fdf9aef5640 1 -- 172.21.15.16:0/3219300788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf940ffee0 msgr2=0x7fdf94100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:37.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.083+0000 7fdf9aef5640 1 --2- 172.21.15.16:0/3219300788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf940ffee0 0x7fdf94100360 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fdf84009ca0 tx=0x7fdf8402a8c0 comp rx=0 tx=0).stop 2024-09-13T16:30:37.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.083+0000 7fdf9aef5640 1 -- 172.21.15.16:0/3219300788 shutdown_connections 2024-09-13T16:30:37.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.083+0000 7fdf9aef5640 1 --2- 172.21.15.16:0/3219300788 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf941008a0 0x7fdf94107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:37.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.083+0000 7fdf9aef5640 1 --2- 172.21.15.16:0/3219300788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf940ffee0 0x7fdf94100360 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:37.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.083+0000 7fdf9aef5640 1 --2- 172.21.15.16:0/3219300788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf940fece0 0x7fdf940ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:37.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.083+0000 7fdf9aef5640 1 -- 172.21.15.16:0/3219300788 >> 172.21.15.16:0/3219300788 conn(0x7fdf940fa490 msgr2=0x7fdf940fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:37.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.083+0000 7fdf9aef5640 1 -- 172.21.15.16:0/3219300788 shutdown_connections 2024-09-13T16:30:37.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.083+0000 7fdf9aef5640 1 -- 172.21.15.16:0/3219300788 wait complete. 2024-09-13T16:30:37.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.084+0000 7fdf9aef5640 1 Processor -- start 2024-09-13T16:30:37.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.084+0000 7fdf9aef5640 1 -- start start 2024-09-13T16:30:37.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.084+0000 7fdf9aef5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf940fece0 0x7fdf94194bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:37.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.084+0000 7fdf9aef5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf940ffee0 0x7fdf941950f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:37.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.084+0000 7fdf9aef5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf941008a0 0x7fdf9419c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:37.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.084+0000 7fdf9aef5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9419c6b0 con 0x7fdf941008a0 2024-09-13T16:30:37.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.084+0000 7fdf9aef5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9419c820 con 0x7fdf940fece0 2024-09-13T16:30:37.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.084+0000 7fdf9aef5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9419c9e0 con 0x7fdf940ffee0 2024-09-13T16:30:37.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.084+0000 7fdf99492640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf941008a0 0x7fdf9419c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:37.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.084+0000 7fdf99492640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf941008a0 0x7fdf9419c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52380/0 (socket says 172.21.15.16:52380) 2024-09-13T16:30:37.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.084+0000 7fdf99492640 1 -- 172.21.15.16:0/1868798479 learned_addr learned my addr 172.21.15.16:0/1868798479 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:37.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.085+0000 7fdf98c91640 1 --2- 172.21.15.16:0/1868798479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf940fece0 0x7fdf94194bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:37.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.085+0000 7fdf93fff640 1 --2- 172.21.15.16:0/1868798479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf940ffee0 0x7fdf941950f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:37.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.085+0000 7fdf99492640 1 -- 172.21.15.16:0/1868798479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf940ffee0 msgr2=0x7fdf941950f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:37.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.085+0000 7fdf99492640 1 --2- 172.21.15.16:0/1868798479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf940ffee0 0x7fdf941950f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:37.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.085+0000 7fdf99492640 1 -- 172.21.15.16:0/1868798479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf940fece0 msgr2=0x7fdf94194bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:37.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.085+0000 7fdf99492640 1 --2- 172.21.15.16:0/1868798479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf940fece0 0x7fdf94194bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:37.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.085+0000 7fdf99492640 1 -- 172.21.15.16:0/1868798479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf84009900 con 0x7fdf941008a0 2024-09-13T16:30:37.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.085+0000 7fdf98c91640 1 --2- 172.21.15.16:0/1868798479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf940fece0 0x7fdf94194bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:30:37.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.085+0000 7fdf99492640 1 --2- 172.21.15.16:0/1868798479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf941008a0 0x7fdf9419c170 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00d750 tx=0x7fdf8c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:37.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.085+0000 7fdf91ffb640 1 -- 172.21.15.16:0/1868798479 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c009bd0 con 0x7fdf941008a0 2024-09-13T16:30:37.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.085+0000 7fdf91ffb640 1 -- 172.21.15.16:0/1868798479 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdf8c00de60 con 0x7fdf941008a0 2024-09-13T16:30:37.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.085+0000 7fdf9aef5640 1 -- 172.21.15.16:0/1868798479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf9419ccc0 con 0x7fdf941008a0 2024-09-13T16:30:37.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.085+0000 7fdf91ffb640 1 -- 172.21.15.16:0/1868798479 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c004cf0 con 0x7fdf941008a0 2024-09-13T16:30:37.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.085+0000 7fdf9aef5640 1 -- 172.21.15.16:0/1868798479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf9419d1c0 con 0x7fdf941008a0 2024-09-13T16:30:37.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.086+0000 7fdf9aef5640 1 -- 172.21.15.16:0/1868798479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf94106220 con 0x7fdf941008a0 2024-09-13T16:30:37.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.087+0000 7fdf91ffb640 1 -- 172.21.15.16:0/1868798479 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fdf8c004e50 con 0x7fdf941008a0 2024-09-13T16:30:37.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.088+0000 7fdf91ffb640 1 --2- 172.21.15.16:0/1868798479 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdf800a1130 0x7fdf800a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:37.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.088+0000 7fdf91ffb640 1 -- 172.21.15.16:0/1868798479 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fdf8c0cc5d0 con 0x7fdf941008a0 2024-09-13T16:30:37.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.088+0000 7fdf98c91640 1 --2- 172.21.15.16:0/1868798479 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdf800a1130 0x7fdf800a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:37.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.089+0000 7fdf98c91640 1 --2- 172.21.15.16:0/1868798479 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdf800a1130 0x7fdf800a35f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0058c0 tx=0x7fdf7c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:37.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.092+0000 7fdf91ffb640 1 -- 172.21.15.16:0/1868798479 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fdf8c095a10 con 0x7fdf941008a0 2024-09-13T16:30:37.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:36 smithi111 ceph-mon[34600]: pgmap v339: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-13T16:30:37.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.283+0000 7fdf9aef5640 1 -- 172.21.15.16:0/1868798479 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf9402c4f0 con 0x7fdf800a1130 2024-09-13T16:30:37.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.284+0000 7fdf91ffb640 1 -- 172.21.15.16:0/1868798479 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fdf9402c4f0 con 0x7fdf800a1130 2024-09-13T16:30:37.285 INFO:teuthology.orchestra.run.smithi016.stdout:Backend: cephadm 2024-09-13T16:30:37.286 INFO:teuthology.orchestra.run.smithi016.stdout:Available: Yes 2024-09-13T16:30:37.286 INFO:teuthology.orchestra.run.smithi016.stdout:Paused: No 2024-09-13T16:30:37.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.287+0000 7fdf9aef5640 1 -- 172.21.15.16:0/1868798479 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdf800a1130 msgr2=0x7fdf800a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:37.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.287+0000 7fdf9aef5640 1 --2- 172.21.15.16:0/1868798479 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdf800a1130 0x7fdf800a35f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0058c0 tx=0x7fdf7c005850 comp rx=0 tx=0).stop 2024-09-13T16:30:37.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.287+0000 7fdf9aef5640 1 -- 172.21.15.16:0/1868798479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf941008a0 msgr2=0x7fdf9419c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:37.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.287+0000 7fdf9aef5640 1 --2- 172.21.15.16:0/1868798479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf941008a0 0x7fdf9419c170 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00d750 tx=0x7fdf8c00dc10 comp rx=0 tx=0).stop 2024-09-13T16:30:37.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.287+0000 7fdf9aef5640 1 -- 172.21.15.16:0/1868798479 shutdown_connections 2024-09-13T16:30:37.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.287+0000 7fdf9aef5640 1 --2- 172.21.15.16:0/1868798479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf941008a0 0x7fdf9419c170 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:37.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.287+0000 7fdf9aef5640 1 --2- 172.21.15.16:0/1868798479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf940ffee0 0x7fdf941950f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:37.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.287+0000 7fdf9aef5640 1 --2- 172.21.15.16:0/1868798479 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fdf800a1130 0x7fdf800a35f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:37.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.287+0000 7fdf9aef5640 1 --2- 172.21.15.16:0/1868798479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf940fece0 0x7fdf94194bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:37.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.287+0000 7fdf9aef5640 1 -- 172.21.15.16:0/1868798479 >> 172.21.15.16:0/1868798479 conn(0x7fdf940fa490 msgr2=0x7fdf941033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:37.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.287+0000 7fdf9aef5640 1 -- 172.21.15.16:0/1868798479 shutdown_connections 2024-09-13T16:30:37.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:37.287+0000 7fdf9aef5640 1 -- 172.21.15.16:0/1868798479 wait complete. 2024-09-13T16:30:37.944 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:37 smithi016 ceph-mon[31187]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:30:37.969 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-13T16:30:38.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:37 smithi107 ceph-mon[34139]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:30:38.180 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:30:38.186 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:37 smithi111 ceph-mon[34600]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:30:38.985 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:38 smithi016 ceph-mon[31187]: pgmap v340: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 13 op/s 2024-09-13T16:30:39.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:38 smithi107 ceph-mon[34139]: pgmap v340: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 13 op/s 2024-09-13T16:30:39.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:38 smithi111 ceph-mon[34600]: pgmap v340: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 13 op/s 2024-09-13T16:30:39.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.904+0000 7f8a796ad640 1 -- 172.21.15.16:0/3593108499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a740fecf0 msgr2=0x7f8a740ff0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:39.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.904+0000 7f8a796ad640 1 --2- 172.21.15.16:0/3593108499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a740fecf0 0x7f8a740ff0f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8a64009900 tx=0x7f8a6402a410 comp rx=0 tx=0).stop 2024-09-13T16:30:39.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.904+0000 7f8a796ad640 1 -- 172.21.15.16:0/3593108499 shutdown_connections 2024-09-13T16:30:39.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.904+0000 7f8a796ad640 1 --2- 172.21.15.16:0/3593108499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a741008b0 0x7f8a74107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:39.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.904+0000 7f8a796ad640 1 --2- 172.21.15.16:0/3593108499 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a740ffef0 0x7f8a74100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:39.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.904+0000 7f8a796ad640 1 --2- 172.21.15.16:0/3593108499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a740fecf0 0x7f8a740ff0f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:39.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.904+0000 7f8a796ad640 1 -- 172.21.15.16:0/3593108499 >> 172.21.15.16:0/3593108499 conn(0x7f8a740fa480 msgr2=0x7f8a740fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:39.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.904+0000 7f8a796ad640 1 -- 172.21.15.16:0/3593108499 shutdown_connections 2024-09-13T16:30:39.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.904+0000 7f8a796ad640 1 -- 172.21.15.16:0/3593108499 wait complete. 2024-09-13T16:30:39.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.905+0000 7f8a796ad640 1 Processor -- start 2024-09-13T16:30:39.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.905+0000 7f8a796ad640 1 -- start start 2024-09-13T16:30:39.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.905+0000 7f8a796ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a740fecf0 0x7f8a74195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:39.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.905+0000 7f8a796ad640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a740ffef0 0x7f8a74196230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:39.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.905+0000 7f8a796ad640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a741008b0 0x7f8a7419d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:39.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.905+0000 7f8a796ad640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7419d7e0 con 0x7f8a740fecf0 2024-09-13T16:30:39.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.905+0000 7f8a796ad640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7419d950 con 0x7f8a741008b0 2024-09-13T16:30:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.905+0000 7f8a796ad640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7419db10 con 0x7f8a740ffef0 2024-09-13T16:30:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.905+0000 7f8a72ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a740fecf0 0x7f8a74195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.905+0000 7f8a72ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a740fecf0 0x7f8a74195cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52394/0 (socket says 172.21.15.16:52394) 2024-09-13T16:30:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.905+0000 7f8a72ffd640 1 -- 172.21.15.16:0/1577385576 learned_addr learned my addr 172.21.15.16:0/1577385576 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.905+0000 7f8a6bfff640 1 --2- 172.21.15.16:0/1577385576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a740ffef0 0x7f8a74196230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.905+0000 7f8a72ffd640 1 -- 172.21.15.16:0/1577385576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a740ffef0 msgr2=0x7f8a74196230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.905+0000 7f8a737fe640 1 --2- 172.21.15.16:0/1577385576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a741008b0 0x7f8a7419d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.906+0000 7f8a72ffd640 1 --2- 172.21.15.16:0/1577385576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a740ffef0 0x7f8a74196230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.906+0000 7f8a72ffd640 1 -- 172.21.15.16:0/1577385576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a741008b0 msgr2=0x7f8a7419d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.906+0000 7f8a72ffd640 1 --2- 172.21.15.16:0/1577385576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a741008b0 0x7f8a7419d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.906+0000 7f8a72ffd640 1 -- 172.21.15.16:0/1577385576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a64009560 con 0x7f8a740fecf0 2024-09-13T16:30:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.906+0000 7f8a6bfff640 1 --2- 172.21.15.16:0/1577385576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a740ffef0 0x7f8a74196230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:30:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.906+0000 7f8a72ffd640 1 --2- 172.21.15.16:0/1577385576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a740fecf0 0x7f8a74195cf0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8a64005ab0 tx=0x7f8a640052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:39.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.906+0000 7f8a70ff9640 1 -- 172.21.15.16:0/1577385576 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a64031d70 con 0x7f8a740fecf0 2024-09-13T16:30:39.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.906+0000 7f8a70ff9640 1 -- 172.21.15.16:0/1577385576 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8a640054a0 con 0x7f8a740fecf0 2024-09-13T16:30:39.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.906+0000 7f8a796ad640 1 -- 172.21.15.16:0/1577385576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a7419dd90 con 0x7f8a740fecf0 2024-09-13T16:30:39.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.906+0000 7f8a70ff9640 1 -- 172.21.15.16:0/1577385576 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a64003a90 con 0x7f8a740fecf0 2024-09-13T16:30:39.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.906+0000 7f8a796ad640 1 -- 172.21.15.16:0/1577385576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a7419e1c0 con 0x7f8a740fecf0 2024-09-13T16:30:39.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.907+0000 7f8a796ad640 1 -- 172.21.15.16:0/1577385576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a7410a440 con 0x7f8a740fecf0 2024-09-13T16:30:39.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.908+0000 7f8a70ff9640 1 -- 172.21.15.16:0/1577385576 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8a64004be0 con 0x7f8a740fecf0 2024-09-13T16:30:39.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.909+0000 7f8a70ff9640 1 --2- 172.21.15.16:0/1577385576 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8a600a1100 0x7f8a600a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:39.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.909+0000 7f8a6bfff640 1 --2- 172.21.15.16:0/1577385576 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8a600a1100 0x7f8a600a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:39.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.909+0000 7f8a70ff9640 1 -- 172.21.15.16:0/1577385576 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f8a640edcf0 con 0x7f8a740fecf0 2024-09-13T16:30:39.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.910+0000 7f8a6bfff640 1 --2- 172.21.15.16:0/1577385576 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8a600a1100 0x7f8a600a35c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c00baa0 tx=0x7f8a5c005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:39.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:39.913+0000 7f8a70ff9640 1 -- 172.21.15.16:0/1577385576 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f8a640b7130 con 0x7f8a740fecf0 2024-09-13T16:30:40.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:40.094+0000 7f8a796ad640 1 -- 172.21.15.16:0/1577385576 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a74068f00 con 0x7f8a600a1100 2024-09-13T16:30:40.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:40.106+0000 7f8a70ff9640 1 -- 172.21.15.16:0/1577385576 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4061 (secure 0 0 0) 0x7f8a74068f00 con 0x7f8a600a1100 2024-09-13T16:30:40.107 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-13T16:30:40.108 INFO:teuthology.orchestra.run.smithi016.stdout:mds.a smithi016 running (2m) 27s ago 2m 16.4M - 18.2.4-850-g50bbe7f3 152d8bc6b19e 25504be1b45d 2024-09-13T16:30:40.108 INFO:teuthology.orchestra.run.smithi016.stdout:mds.b smithi107 running (2m) 59s ago 2m 25.9M - 18.2.4-850-g50bbe7f3 152d8bc6b19e 5a0a934d5a9e 2024-09-13T16:30:40.108 INFO:teuthology.orchestra.run.smithi016.stdout:mds.c smithi111 running (2m) 59s ago 2m 17.5M - 18.2.4-850-g50bbe7f3 152d8bc6b19e 1ae73ad6ef96 2024-09-13T16:30:40.108 INFO:teuthology.orchestra.run.smithi016.stdout:mds.d smithi016 running (2m) 27s ago 2m 16.5M - 18.2.4-850-g50bbe7f3 152d8bc6b19e 47749514ffd2 2024-09-13T16:30:40.108 INFO:teuthology.orchestra.run.smithi016.stdout:mds.e smithi107 running (2m) 59s ago 2m 16.4M - 18.2.4-850-g50bbe7f3 152d8bc6b19e b50793f01f19 2024-09-13T16:30:40.108 INFO:teuthology.orchestra.run.smithi016.stdout:mds.f smithi111 running (2m) 59s ago 2m 15.4M - 18.2.4-850-g50bbe7f3 152d8bc6b19e 19749af66c83 2024-09-13T16:30:40.109 INFO:teuthology.orchestra.run.smithi016.stdout:mds.g smithi016 running (2m) 27s ago 2m 18.1M - 18.2.4-850-g50bbe7f3 152d8bc6b19e 8d4e0e08d9fa 2024-09-13T16:30:40.109 INFO:teuthology.orchestra.run.smithi016.stdout:mds.h smithi107 running (2m) 59s ago 2m 16.2M - 18.2.4-850-g50bbe7f3 152d8bc6b19e db4523075397 2024-09-13T16:30:40.109 INFO:teuthology.orchestra.run.smithi016.stdout:mds.i smithi111 running (2m) 59s ago 2m 17.3M - 18.2.4-850-g50bbe7f3 152d8bc6b19e d958eadab886 2024-09-13T16:30:40.109 INFO:teuthology.orchestra.run.smithi016.stdout:mds.j smithi016 running (2m) 27s ago 2m 15.8M - 18.2.4-850-g50bbe7f3 152d8bc6b19e 216caf59eb38 2024-09-13T16:30:40.109 INFO:teuthology.orchestra.run.smithi016.stdout:mds.k smithi107 running (2m) 59s ago 2m 19.2M - 18.2.4-850-g50bbe7f3 152d8bc6b19e 9fb9b76f26ad 2024-09-13T16:30:40.109 INFO:teuthology.orchestra.run.smithi016.stdout:mds.l smithi111 running (2m) 59s ago 2m 16.3M - 18.2.4-850-g50bbe7f3 152d8bc6b19e aad66e58c1dc 2024-09-13T16:30:40.109 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.x smithi016 *:9283,8765 running (9m) 27s ago 9m 497M - 18.2.4-850-g50bbe7f3 152d8bc6b19e afeb66054b34 2024-09-13T16:30:40.110 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.y smithi107 *:8443,8765 running (7m) 59s ago 7m 440M - 18.2.4-850-g50bbe7f3 152d8bc6b19e 787cdddb9b5c 2024-09-13T16:30:40.110 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.z smithi111 *:8443,8765 running (7m) 59s ago 7m 439M - 18.2.4-850-g50bbe7f3 152d8bc6b19e 4bfb8e2c8cc5 2024-09-13T16:30:40.110 INFO:teuthology.orchestra.run.smithi016.stdout:mon.a smithi016 running (9m) 27s ago 9m 51.1M 2048M 18.2.4-850-g50bbe7f3 152d8bc6b19e 0d55658237b8 2024-09-13T16:30:40.110 INFO:teuthology.orchestra.run.smithi016.stdout:mon.b smithi107 running (7m) 59s ago 7m 40.4M 2048M 18.2.4-850-g50bbe7f3 152d8bc6b19e f046c66a1dec 2024-09-13T16:30:40.110 INFO:teuthology.orchestra.run.smithi016.stdout:mon.c smithi111 running (7m) 59s ago 7m 40.0M 2048M 18.2.4-850-g50bbe7f3 152d8bc6b19e 6ae37addcae4 2024-09-13T16:30:40.110 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (7m) 27s ago 7m 46.4M 4273M 18.2.4-850-g50bbe7f3 152d8bc6b19e c049eb0fd2a2 2024-09-13T16:30:40.110 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi107 running (6m) 59s ago 6m 47.7M 4271M 18.2.4-850-g50bbe7f3 152d8bc6b19e f9afea093395 2024-09-13T16:30:40.110 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi111 running (6m) 59s ago 6m 60.3M 4272M 18.2.4-850-g50bbe7f3 152d8bc6b19e 55a8f88cec68 2024-09-13T16:30:40.111 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi016 running (6m) 27s ago 6m 53.2M 4273M 18.2.4-850-g50bbe7f3 152d8bc6b19e d03fd3938153 2024-09-13T16:30:40.111 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi107 running (5m) 59s ago 5m 43.8M 4271M 18.2.4-850-g50bbe7f3 152d8bc6b19e 3249a8b86719 2024-09-13T16:30:40.111 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi111 running (5m) 59s ago 5m 45.0M 4272M 18.2.4-850-g50bbe7f3 152d8bc6b19e c56eba6e0e1f 2024-09-13T16:30:40.111 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 smithi016 running (4m) 27s ago 4m 52.4M 4273M 18.2.4-850-g50bbe7f3 152d8bc6b19e 2ce0d96ea6e4 2024-09-13T16:30:40.111 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 smithi107 running (4m) 59s ago 4m 42.9M 4271M 18.2.4-850-g50bbe7f3 152d8bc6b19e a4c4d1143f31 2024-09-13T16:30:40.111 INFO:teuthology.orchestra.run.smithi016.stdout:osd.8 smithi111 running (4m) 59s ago 4m 53.9M 4272M 18.2.4-850-g50bbe7f3 152d8bc6b19e e38d24c92ddf 2024-09-13T16:30:40.112 INFO:teuthology.orchestra.run.smithi016.stdout:osd.9 smithi016 running (3m) 27s ago 3m 42.5M 4273M 18.2.4-850-g50bbe7f3 152d8bc6b19e 70cae2916efe 2024-09-13T16:30:40.112 INFO:teuthology.orchestra.run.smithi016.stdout:osd.10 smithi107 running (3m) 59s ago 3m 52.2M 4271M 18.2.4-850-g50bbe7f3 152d8bc6b19e 396277c04f0f 2024-09-13T16:30:40.112 INFO:teuthology.orchestra.run.smithi016.stdout:osd.11 smithi111 running (2m) 59s ago 2m 41.0M 4272M 18.2.4-850-g50bbe7f3 152d8bc6b19e e5bf2a1c5f27 2024-09-13T16:30:40.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:40.108+0000 7f8a796ad640 1 -- 172.21.15.16:0/1577385576 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8a600a1100 msgr2=0x7f8a600a35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:40.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:40.108+0000 7f8a796ad640 1 --2- 172.21.15.16:0/1577385576 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8a600a1100 0x7f8a600a35c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c00baa0 tx=0x7f8a5c005d90 comp rx=0 tx=0).stop 2024-09-13T16:30:40.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:40.108+0000 7f8a796ad640 1 -- 172.21.15.16:0/1577385576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a740fecf0 msgr2=0x7f8a74195cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:40.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:40.108+0000 7f8a796ad640 1 --2- 172.21.15.16:0/1577385576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a740fecf0 0x7f8a74195cf0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8a64005ab0 tx=0x7f8a640052b0 comp rx=0 tx=0).stop 2024-09-13T16:30:40.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:40.108+0000 7f8a796ad640 1 -- 172.21.15.16:0/1577385576 shutdown_connections 2024-09-13T16:30:40.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:40.108+0000 7f8a796ad640 1 --2- 172.21.15.16:0/1577385576 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f8a600a1100 0x7f8a600a35c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:40.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:40.108+0000 7f8a796ad640 1 --2- 172.21.15.16:0/1577385576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a741008b0 0x7f8a7419d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:40.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:40.108+0000 7f8a796ad640 1 --2- 172.21.15.16:0/1577385576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a740ffef0 0x7f8a74196230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:40.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:40.108+0000 7f8a796ad640 1 --2- 172.21.15.16:0/1577385576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a740fecf0 0x7f8a74195cf0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:40.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:40.108+0000 7f8a796ad640 1 -- 172.21.15.16:0/1577385576 >> 172.21.15.16:0/1577385576 conn(0x7f8a740fa480 msgr2=0x7f8a740687f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:40.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:40.108+0000 7f8a796ad640 1 -- 172.21.15.16:0/1577385576 shutdown_connections 2024-09-13T16:30:40.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:40.108+0000 7f8a796ad640 1 -- 172.21.15.16:0/1577385576 wait complete. 2024-09-13T16:30:40.825 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-13T16:30:40.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:40 smithi111 ceph-mon[34600]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:30:40.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:40 smithi111 ceph-mon[34600]: pgmap v341: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-13T16:30:41.028 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:30:41.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:40 smithi107 ceph-mon[34139]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:30:41.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:40 smithi107 ceph-mon[34139]: pgmap v341: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-13T16:30:41.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:40 smithi016 ceph-mon[31187]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:30:41.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:40 smithi016 ceph-mon[31187]: pgmap v341: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-13T16:30:42.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:42 smithi016 ceph-mon[31187]: pgmap v342: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-13T16:30:42.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.862+0000 7f4bc6d21640 1 -- 172.21.15.16:0/1849303635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bc0100f60 msgr2=0x7f4bc01013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:42.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.862+0000 7f4bc6d21640 1 --2- 172.21.15.16:0/1849303635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bc0100f60 0x7f4bc01013e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009900 tx=0x7f4bac02a410 comp rx=0 tx=0).stop 2024-09-13T16:30:42.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.863+0000 7f4bc6d21640 1 -- 172.21.15.16:0/1849303635 shutdown_connections 2024-09-13T16:30:42.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.863+0000 7f4bc6d21640 1 --2- 172.21.15.16:0/1849303635 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc0101920 0x7f4bc0108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:42.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.863+0000 7f4bc6d21640 1 --2- 172.21.15.16:0/1849303635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bc0100f60 0x7f4bc01013e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:42.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.863+0000 7f4bc6d21640 1 --2- 172.21.15.16:0/1849303635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bc00ffe00 0x7f4bc0100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:42.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.863+0000 7f4bc6d21640 1 -- 172.21.15.16:0/1849303635 >> 172.21.15.16:0/1849303635 conn(0x7f4bc00fb5d0 msgr2=0x7f4bc00fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:42.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.863+0000 7f4bc6d21640 1 -- 172.21.15.16:0/1849303635 shutdown_connections 2024-09-13T16:30:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.863+0000 7f4bc6d21640 1 -- 172.21.15.16:0/1849303635 wait complete. 2024-09-13T16:30:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.863+0000 7f4bc6d21640 1 Processor -- start 2024-09-13T16:30:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.863+0000 7f4bc6d21640 1 -- start start 2024-09-13T16:30:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.864+0000 7f4bc6d21640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bc00ffe00 0x7f4bc0074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.864+0000 7f4bc6d21640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bc0100f60 0x7f4bc00728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.864+0000 7f4bc6d21640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc0101920 0x7f4bc0072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.864+0000 7f4bc6d21640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc0073570 con 0x7f4bc0100f60 2024-09-13T16:30:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.864+0000 7f4bc6d21640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc00736e0 con 0x7f4bc00ffe00 2024-09-13T16:30:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.864+0000 7f4bc6d21640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc019fcc0 con 0x7f4bc0101920 2024-09-13T16:30:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.864+0000 7f4bbffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bc0100f60 0x7f4bc00728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.864+0000 7f4bbffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bc0100f60 0x7f4bc00728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52398/0 (socket says 172.21.15.16:52398) 2024-09-13T16:30:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.864+0000 7f4bbffff640 1 -- 172.21.15.16:0/343969389 learned_addr learned my addr 172.21.15.16:0/343969389 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.864+0000 7f4bc4abd640 1 --2- 172.21.15.16:0/343969389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bc00ffe00 0x7f4bc0074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.864+0000 7f4bc52be640 1 --2- 172.21.15.16:0/343969389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc0101920 0x7f4bc0072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.864+0000 7f4bbffff640 1 -- 172.21.15.16:0/343969389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc0101920 msgr2=0x7f4bc0072de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.864+0000 7f4bbffff640 1 --2- 172.21.15.16:0/343969389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc0101920 0x7f4bc0072de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.864+0000 7f4bbffff640 1 -- 172.21.15.16:0/343969389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bc00ffe00 msgr2=0x7f4bc0074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.864+0000 7f4bbffff640 1 --2- 172.21.15.16:0/343969389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bc00ffe00 0x7f4bc0074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.864+0000 7f4bbffff640 1 -- 172.21.15.16:0/343969389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bac009560 con 0x7f4bc0100f60 2024-09-13T16:30:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.864+0000 7f4bc52be640 1 --2- 172.21.15.16:0/343969389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc0101920 0x7f4bc0072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:30:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.864+0000 7f4bbffff640 1 --2- 172.21.15.16:0/343969389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bc0100f60 0x7f4bc00728a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4bac004360 tx=0x7f4bac004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.864+0000 7f4bbdffb640 1 -- 172.21.15.16:0/343969389 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bac031d70 con 0x7f4bc0100f60 2024-09-13T16:30:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.864+0000 7f4bbdffb640 1 -- 172.21.15.16:0/343969389 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4bac004b50 con 0x7f4bc0100f60 2024-09-13T16:30:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.865+0000 7f4bc6d21640 1 -- 172.21.15.16:0/343969389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc019ff10 con 0x7f4bc0100f60 2024-09-13T16:30:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.865+0000 7f4bbdffb640 1 -- 172.21.15.16:0/343969389 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bac003cd0 con 0x7f4bc0100f60 2024-09-13T16:30:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.865+0000 7f4bc6d21640 1 -- 172.21.15.16:0/343969389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc01a02c0 con 0x7f4bc0100f60 2024-09-13T16:30:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.866+0000 7f4bc6d21640 1 -- 172.21.15.16:0/343969389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc0107240 con 0x7f4bc0100f60 2024-09-13T16:30:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.867+0000 7f4bbdffb640 1 -- 172.21.15.16:0/343969389 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f4bac0046c0 con 0x7f4bc0100f60 2024-09-13T16:30:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.868+0000 7f4bbdffb640 1 --2- 172.21.15.16:0/343969389 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f4bb00a1080 0x7f4bb00a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.868+0000 7f4bbdffb640 1 -- 172.21.15.16:0/343969389 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f4bac0eded0 con 0x7f4bc0100f60 2024-09-13T16:30:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.868+0000 7f4bc4abd640 1 --2- 172.21.15.16:0/343969389 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f4bb00a1080 0x7f4bb00a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.868+0000 7f4bc4abd640 1 --2- 172.21.15.16:0/343969389 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f4bb00a1080 0x7f4bb00a3540 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4005e10 tx=0x7f4bb4005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:42.872+0000 7f4bbdffb640 1 -- 172.21.15.16:0/343969389 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f4bac0b7310 con 0x7f4bc0100f60 2024-09-13T16:30:42.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:42 smithi111 ceph-mon[34600]: pgmap v342: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-13T16:30:43.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:43.051+0000 7f4bc6d21640 1 -- 172.21.15.16:0/343969389 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f4bc002c550 con 0x7f4bb00a1080 2024-09-13T16:30:43.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:43.057+0000 7f4bbdffb640 1 -- 172.21.15.16:0/343969389 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f4bc002c550 con 0x7f4bb00a1080 2024-09-13T16:30:43.058 INFO:teuthology.orchestra.run.smithi016.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-13T16:30:43.058 INFO:teuthology.orchestra.run.smithi016.stdout:mds.all 12/12 62s ago 2m smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi111=c;smithi111=f;smithi111=i;smithi111=l;count:12 2024-09-13T16:30:43.058 INFO:teuthology.orchestra.run.smithi016.stdout:mgr 3/3 62s ago 7m smithi016=x;smithi107=y;smithi111=z;count:3 2024-09-13T16:30:43.058 INFO:teuthology.orchestra.run.smithi016.stdout:mon 3/3 62s ago 8m smithi016:172.21.15.16=a;smithi107:172.21.15.107=b;smithi111:172.21.15.111=c;count:3 2024-09-13T16:30:43.058 INFO:teuthology.orchestra.run.smithi016.stdout:osd 12 62s ago - 2024-09-13T16:30:43.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:43.059+0000 7f4bc6d21640 1 -- 172.21.15.16:0/343969389 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f4bb00a1080 msgr2=0x7f4bb00a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:43.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:43.059+0000 7f4bc6d21640 1 --2- 172.21.15.16:0/343969389 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f4bb00a1080 0x7f4bb00a3540 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4005e10 tx=0x7f4bb4005da0 comp rx=0 tx=0).stop 2024-09-13T16:30:43.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:43.059+0000 7f4bc6d21640 1 -- 172.21.15.16:0/343969389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bc0100f60 msgr2=0x7f4bc00728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:43.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:43.059+0000 7f4bc6d21640 1 --2- 172.21.15.16:0/343969389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bc0100f60 0x7f4bc00728a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4bac004360 tx=0x7f4bac004390 comp rx=0 tx=0).stop 2024-09-13T16:30:43.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:43.059+0000 7f4bc6d21640 1 -- 172.21.15.16:0/343969389 shutdown_connections 2024-09-13T16:30:43.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:43.059+0000 7f4bc6d21640 1 --2- 172.21.15.16:0/343969389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bc0101920 0x7f4bc0072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:43.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:43.059+0000 7f4bc6d21640 1 --2- 172.21.15.16:0/343969389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bc0100f60 0x7f4bc00728a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:43.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:43.059+0000 7f4bc6d21640 1 --2- 172.21.15.16:0/343969389 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f4bb00a1080 0x7f4bb00a3540 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:43.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:43.059+0000 7f4bc6d21640 1 --2- 172.21.15.16:0/343969389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bc00ffe00 0x7f4bc0074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:43.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:43.059+0000 7f4bc6d21640 1 -- 172.21.15.16:0/343969389 >> 172.21.15.16:0/343969389 conn(0x7f4bc00fb5d0 msgr2=0x7f4bc0104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:43.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:43.059+0000 7f4bc6d21640 1 -- 172.21.15.16:0/343969389 shutdown_connections 2024-09-13T16:30:43.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:43.059+0000 7f4bc6d21640 1 -- 172.21.15.16:0/343969389 wait complete. 2024-09-13T16:30:43.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:42 smithi107 ceph-mon[34139]: pgmap v342: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-13T16:30:43.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:43 smithi016 ceph-mon[31187]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:30:43.834 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-13T16:30:43.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:43 smithi111 ceph-mon[34600]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:30:44.000 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:43 smithi107 ceph-mon[34139]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:30:44.045 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:30:44.848 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:44 smithi016 ceph-mon[31187]: pgmap v343: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-13T16:30:44.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:44 smithi111 ceph-mon[34600]: pgmap v343: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-13T16:30:45.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:44 smithi107 ceph-mon[34139]: pgmap v343: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-13T16:30:45.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.808+0000 7f2730bde640 1 -- 172.21.15.16:0/3984813859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f272c100f60 msgr2=0x7f272c1013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:45.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.808+0000 7f2730bde640 1 --2- 172.21.15.16:0/3984813859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f272c100f60 0x7f272c1013e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2714009900 tx=0x7f271402a3f0 comp rx=0 tx=0).stop 2024-09-13T16:30:45.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.809+0000 7f2730bde640 1 -- 172.21.15.16:0/3984813859 shutdown_connections 2024-09-13T16:30:45.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.809+0000 7f2730bde640 1 --2- 172.21.15.16:0/3984813859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f272c101920 0x7f272c108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:45.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.809+0000 7f2730bde640 1 --2- 172.21.15.16:0/3984813859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f272c100f60 0x7f272c1013e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:45.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.809+0000 7f2730bde640 1 --2- 172.21.15.16:0/3984813859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f272c0ffe00 0x7f272c100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:45.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.809+0000 7f2730bde640 1 -- 172.21.15.16:0/3984813859 >> 172.21.15.16:0/3984813859 conn(0x7f272c0fb5d0 msgr2=0x7f272c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:45.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.809+0000 7f2730bde640 1 -- 172.21.15.16:0/3984813859 shutdown_connections 2024-09-13T16:30:45.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.809+0000 7f2730bde640 1 -- 172.21.15.16:0/3984813859 wait complete. 2024-09-13T16:30:45.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.809+0000 7f2730bde640 1 Processor -- start 2024-09-13T16:30:45.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.809+0000 7f2730bde640 1 -- start start 2024-09-13T16:30:45.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.810+0000 7f2730bde640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f272c0ffe00 0x7f272c195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:45.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.810+0000 7f2730bde640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f272c100f60 0x7f272c1961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:45.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.810+0000 7f2730bde640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f272c101920 0x7f272c19d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:45.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.810+0000 7f2730bde640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272c19d750 con 0x7f272c100f60 2024-09-13T16:30:45.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.810+0000 7f2730bde640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272c19d8c0 con 0x7f272c101920 2024-09-13T16:30:45.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.810+0000 7f2730bde640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272c19da80 con 0x7f272c0ffe00 2024-09-13T16:30:45.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.810+0000 7f2729d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f272c100f60 0x7f272c1961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:45.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.810+0000 7f2729d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f272c100f60 0x7f272c1961f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54728/0 (socket says 172.21.15.16:54728) 2024-09-13T16:30:45.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.810+0000 7f2729d9b640 1 -- 172.21.15.16:0/3328841760 learned_addr learned my addr 172.21.15.16:0/3328841760 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:45.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.810+0000 7f272a59c640 1 --2- 172.21.15.16:0/3328841760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f272c0ffe00 0x7f272c195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:45.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.810+0000 7f272ad9d640 1 --2- 172.21.15.16:0/3328841760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f272c101920 0x7f272c19d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:45.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.810+0000 7f2729d9b640 1 -- 172.21.15.16:0/3328841760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f272c0ffe00 msgr2=0x7f272c195cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:45.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.810+0000 7f2729d9b640 1 --2- 172.21.15.16:0/3328841760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f272c0ffe00 0x7f272c195cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:45.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.810+0000 7f2729d9b640 1 -- 172.21.15.16:0/3328841760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f272c101920 msgr2=0x7f272c19d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:45.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.810+0000 7f2729d9b640 1 --2- 172.21.15.16:0/3328841760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f272c101920 0x7f272c19d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:45.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.810+0000 7f2729d9b640 1 -- 172.21.15.16:0/3328841760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2714009560 con 0x7f272c100f60 2024-09-13T16:30:45.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.810+0000 7f2729d9b640 1 --2- 172.21.15.16:0/3328841760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f272c100f60 0x7f272c1961f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2714005ab0 tx=0x7f27140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:45.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.811+0000 7f27237fe640 1 -- 172.21.15.16:0/3328841760 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2714031d70 con 0x7f272c100f60 2024-09-13T16:30:45.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.811+0000 7f27237fe640 1 -- 172.21.15.16:0/3328841760 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f27140044e0 con 0x7f272c100f60 2024-09-13T16:30:45.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.811+0000 7f2730bde640 1 -- 172.21.15.16:0/3328841760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f272c19dd00 con 0x7f272c100f60 2024-09-13T16:30:45.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.811+0000 7f27237fe640 1 -- 172.21.15.16:0/3328841760 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2714003b00 con 0x7f272c100f60 2024-09-13T16:30:45.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.811+0000 7f2730bde640 1 -- 172.21.15.16:0/3328841760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f272c19e1a0 con 0x7f272c100f60 2024-09-13T16:30:45.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.812+0000 7f2730bde640 1 -- 172.21.15.16:0/3328841760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f272c107350 con 0x7f272c100f60 2024-09-13T16:30:45.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.813+0000 7f27237fe640 1 -- 172.21.15.16:0/3328841760 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f2714004c50 con 0x7f272c100f60 2024-09-13T16:30:45.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.814+0000 7f27237fe640 1 --2- 172.21.15.16:0/3328841760 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f27180a1100 0x7f27180a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:45.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.814+0000 7f27237fe640 1 -- 172.21.15.16:0/3328841760 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f27140edba0 con 0x7f272c100f60 2024-09-13T16:30:45.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.814+0000 7f272a59c640 1 --2- 172.21.15.16:0/3328841760 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f27180a1100 0x7f27180a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:45.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.814+0000 7f272a59c640 1 --2- 172.21.15.16:0/3328841760 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f27180a1100 0x7f27180a35c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f272c0fecd0 tx=0x7f271c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:45.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.818+0000 7f27237fe640 1 -- 172.21.15.16:0/3328841760 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f27140b7090 con 0x7f272c100f60 2024-09-13T16:30:46.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:45.999+0000 7f2730bde640 1 -- 172.21.15.16:0/3328841760 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f272c02c4f0 con 0x7f27180a1100 2024-09-13T16:30:46.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:46.000+0000 7f27237fe640 1 -- 172.21.15.16:0/3328841760 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f272c02c4f0 con 0x7f27180a1100 2024-09-13T16:30:46.001 INFO:teuthology.orchestra.run.smithi016.stdout:HOST ADDR LABELS STATUS 2024-09-13T16:30:46.001 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016 172.21.15.16 2024-09-13T16:30:46.001 INFO:teuthology.orchestra.run.smithi016.stdout:smithi107 172.21.15.107 2024-09-13T16:30:46.002 INFO:teuthology.orchestra.run.smithi016.stdout:smithi111 172.21.15.111 2024-09-13T16:30:46.002 INFO:teuthology.orchestra.run.smithi016.stdout:3 hosts in cluster 2024-09-13T16:30:46.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:46.002+0000 7f2730bde640 1 -- 172.21.15.16:0/3328841760 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f27180a1100 msgr2=0x7f27180a35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:46.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:46.002+0000 7f2730bde640 1 --2- 172.21.15.16:0/3328841760 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f27180a1100 0x7f27180a35c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f272c0fecd0 tx=0x7f271c00b390 comp rx=0 tx=0).stop 2024-09-13T16:30:46.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:46.002+0000 7f2730bde640 1 -- 172.21.15.16:0/3328841760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f272c100f60 msgr2=0x7f272c1961f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:46.002+0000 7f2730bde640 1 --2- 172.21.15.16:0/3328841760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f272c100f60 0x7f272c1961f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2714005ab0 tx=0x7f27140042f0 comp rx=0 tx=0).stop 2024-09-13T16:30:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:46.002+0000 7f2730bde640 1 -- 172.21.15.16:0/3328841760 shutdown_connections 2024-09-13T16:30:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:46.002+0000 7f2730bde640 1 --2- 172.21.15.16:0/3328841760 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f27180a1100 0x7f27180a35c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:46.002+0000 7f2730bde640 1 --2- 172.21.15.16:0/3328841760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f272c101920 0x7f272c19d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:46.002+0000 7f2730bde640 1 --2- 172.21.15.16:0/3328841760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f272c100f60 0x7f272c1961f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:46.002+0000 7f2730bde640 1 --2- 172.21.15.16:0/3328841760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f272c0ffe00 0x7f272c195cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:46.002+0000 7f2730bde640 1 -- 172.21.15.16:0/3328841760 >> 172.21.15.16:0/3328841760 conn(0x7f272c0fb5d0 msgr2=0x7f272c104520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:46.002+0000 7f2730bde640 1 -- 172.21.15.16:0/3328841760 shutdown_connections 2024-09-13T16:30:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:46.002+0000 7f2730bde640 1 -- 172.21.15.16:0/3328841760 wait complete. 2024-09-13T16:30:46.897 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-13T16:30:46.922 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:46 smithi016 ceph-mon[31187]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:30:46.922 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:46 smithi016 ceph-mon[31187]: pgmap v344: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-13T16:30:46.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:46 smithi111 ceph-mon[34600]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:30:46.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:46 smithi111 ceph-mon[34600]: pgmap v344: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-13T16:30:47.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:46 smithi107 ceph-mon[34139]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:30:47.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:46 smithi107 ceph-mon[34139]: pgmap v344: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-13T16:30:47.106 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:30:48.825 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:48 smithi016 ceph-mon[31187]: pgmap v345: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 13 op/s 2024-09-13T16:30:48.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:48 smithi111 ceph-mon[34600]: pgmap v345: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 13 op/s 2024-09-13T16:30:49.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.046+0000 7fadf6c23640 1 -- 172.21.15.16:0/4025428566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadf0100890 msgr2=0x7fadf0102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:49.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.046+0000 7fadf6c23640 1 --2- 172.21.15.16:0/4025428566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadf0100890 0x7fadf0102c80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fadd8009900 tx=0x7fadd802a410 comp rx=0 tx=0).stop 2024-09-13T16:30:49.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.046+0000 7fadf6c23640 1 -- 172.21.15.16:0/4025428566 shutdown_connections 2024-09-13T16:30:49.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.046+0000 7fadf6c23640 1 --2- 172.21.15.16:0/4025428566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadf0105af0 0x7fadf0107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:49.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.046+0000 7fadf6c23640 1 --2- 172.21.15.16:0/4025428566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadf01031c0 0x7fadf01055b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:49.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.046+0000 7fadf6c23640 1 --2- 172.21.15.16:0/4025428566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadf0100890 0x7fadf0102c80 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:49.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.046+0000 7fadf6c23640 1 -- 172.21.15.16:0/4025428566 >> 172.21.15.16:0/4025428566 conn(0x7fadf00fa490 msgr2=0x7fadf00fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:49.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.046+0000 7fadf6c23640 1 -- 172.21.15.16:0/4025428566 shutdown_connections 2024-09-13T16:30:49.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.046+0000 7fadf6c23640 1 -- 172.21.15.16:0/4025428566 wait complete. 2024-09-13T16:30:49.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.047+0000 7fadf6c23640 1 Processor -- start 2024-09-13T16:30:49.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.047+0000 7fadf6c23640 1 -- start start 2024-09-13T16:30:49.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.047+0000 7fadf6c23640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadf0100890 0x7fadf01001f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:49.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.047+0000 7fadf6c23640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadf01031c0 0x7fadf00fe840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:49.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.047+0000 7fadf6c23640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadf0105af0 0x7fadf00fed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:49.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.047+0000 7fadf6c23640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf00ff510 con 0x7fadf0105af0 2024-09-13T16:30:49.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.047+0000 7fadf6c23640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf00ff680 con 0x7fadf01031c0 2024-09-13T16:30:49.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.047+0000 7fadf6c23640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf019fd20 con 0x7fadf0100890 2024-09-13T16:30:49.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.047+0000 7fadf51c0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadf0105af0 0x7fadf00fed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:49.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.047+0000 7fadf51c0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadf0105af0 0x7fadf00fed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54748/0 (socket says 172.21.15.16:54748) 2024-09-13T16:30:49.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.047+0000 7fadf51c0640 1 -- 172.21.15.16:0/3260431467 learned_addr learned my addr 172.21.15.16:0/3260431467 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:49.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.048+0000 7fadf51c0640 1 -- 172.21.15.16:0/3260431467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadf0100890 msgr2=0x7fadf01001f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:49.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.048+0000 7fadf51c0640 1 --2- 172.21.15.16:0/3260431467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadf0100890 0x7fadf01001f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:49.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.048+0000 7fadf51c0640 1 -- 172.21.15.16:0/3260431467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadf01031c0 msgr2=0x7fadf00fe840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:49.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.048+0000 7fadf51c0640 1 --2- 172.21.15.16:0/3260431467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadf01031c0 0x7fadf00fe840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:49.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.048+0000 7fadf51c0640 1 -- 172.21.15.16:0/3260431467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd8009560 con 0x7fadf0105af0 2024-09-13T16:30:49.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.048+0000 7fadf51c0640 1 --2- 172.21.15.16:0/3260431467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadf0105af0 0x7fadf00fed80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fade800d7b0 tx=0x7fade800dc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:49.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.048+0000 7fadedffb640 1 -- 172.21.15.16:0/3260431467 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade8009bd0 con 0x7fadf0105af0 2024-09-13T16:30:49.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.048+0000 7fadedffb640 1 -- 172.21.15.16:0/3260431467 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fade800dec0 con 0x7fadf0105af0 2024-09-13T16:30:49.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.048+0000 7fadf6c23640 1 -- 172.21.15.16:0/3260431467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf019ffd0 con 0x7fadf0105af0 2024-09-13T16:30:49.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.048+0000 7fadedffb640 1 -- 172.21.15.16:0/3260431467 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade8004d60 con 0x7fadf0105af0 2024-09-13T16:30:49.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.048+0000 7fadf6c23640 1 -- 172.21.15.16:0/3260431467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf01a04a0 con 0x7fadf0105af0 2024-09-13T16:30:49.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.049+0000 7fadf6c23640 1 -- 172.21.15.16:0/3260431467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf00fbf80 con 0x7fadf0105af0 2024-09-13T16:30:49.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.050+0000 7fadedffb640 1 -- 172.21.15.16:0/3260431467 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fade8004ec0 con 0x7fadf0105af0 2024-09-13T16:30:49.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.051+0000 7fadedffb640 1 --2- 172.21.15.16:0/3260431467 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7faddc0a1080 0x7faddc0a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:49.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.051+0000 7fadedffb640 1 -- 172.21.15.16:0/3260431467 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fade80cc6a0 con 0x7fadf0105af0 2024-09-13T16:30:49.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.051+0000 7fadf49bf640 1 --2- 172.21.15.16:0/3260431467 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7faddc0a1080 0x7faddc0a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:49.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.051+0000 7fadf49bf640 1 --2- 172.21.15.16:0/3260431467 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7faddc0a1080 0x7faddc0a3540 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fadd802a930 tx=0x7fadd8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:49.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.055+0000 7fadedffb640 1 -- 172.21.15.16:0/3260431467 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fade8095a80 con 0x7fadf0105af0 2024-09-13T16:30:49.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:48 smithi107 ceph-mon[34139]: pgmap v345: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 13 op/s 2024-09-13T16:30:49.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.234+0000 7fadf6c23640 1 -- 172.21.15.16:0/3260431467 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fadf002c4f0 con 0x7faddc0a1080 2024-09-13T16:30:49.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.235+0000 7fadedffb640 1 -- 172.21.15.16:0/3260431467 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7fadf002c4f0 con 0x7faddc0a1080 2024-09-13T16:30:49.237 INFO:teuthology.orchestra.run.smithi016.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-13T16:30:49.238 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513002C480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-09-13T16:30:49.238 INFO:teuthology.orchestra.run.smithi016.stdout:smithi107 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NK400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-13T16:30:49.238 INFO:teuthology.orchestra.run.smithi016.stdout:smithi111 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400VZ400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-13T16:30:49.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.238+0000 7fadf6c23640 1 -- 172.21.15.16:0/3260431467 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7faddc0a1080 msgr2=0x7faddc0a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:49.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.238+0000 7fadf6c23640 1 --2- 172.21.15.16:0/3260431467 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7faddc0a1080 0x7faddc0a3540 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fadd802a930 tx=0x7fadd8000bc0 comp rx=0 tx=0).stop 2024-09-13T16:30:49.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.238+0000 7fadf6c23640 1 -- 172.21.15.16:0/3260431467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadf0105af0 msgr2=0x7fadf00fed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:49.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.238+0000 7fadf6c23640 1 --2- 172.21.15.16:0/3260431467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadf0105af0 0x7fadf00fed80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fade800d7b0 tx=0x7fade800dc70 comp rx=0 tx=0).stop 2024-09-13T16:30:49.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.238+0000 7fadf6c23640 1 -- 172.21.15.16:0/3260431467 shutdown_connections 2024-09-13T16:30:49.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.238+0000 7fadf6c23640 1 --2- 172.21.15.16:0/3260431467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadf0105af0 0x7fadf00fed80 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:49.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.238+0000 7fadf6c23640 1 --2- 172.21.15.16:0/3260431467 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7faddc0a1080 0x7faddc0a3540 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:49.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.238+0000 7fadf6c23640 1 --2- 172.21.15.16:0/3260431467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadf01031c0 0x7fadf00fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:49.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.238+0000 7fadf6c23640 1 --2- 172.21.15.16:0/3260431467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadf0100890 0x7fadf01001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:49.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.238+0000 7fadf6c23640 1 -- 172.21.15.16:0/3260431467 >> 172.21.15.16:0/3260431467 conn(0x7fadf00fa490 msgr2=0x7fadf01010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:49.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.238+0000 7fadf6c23640 1 -- 172.21.15.16:0/3260431467 shutdown_connections 2024-09-13T16:30:49.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:49.238+0000 7fadf6c23640 1 -- 172.21.15.16:0/3260431467 wait complete. 2024-09-13T16:30:49.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:49 smithi016 ceph-mon[31187]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:30:49.917 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-13T16:30:49.928 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-09-13T16:30:49.928 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-13T16:30:49.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:49 smithi111 ceph-mon[34600]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:30:50.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:49 smithi107 ceph-mon[34139]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:30:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:30:50.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:50 smithi111 ceph-mon[34600]: pgmap v346: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T16:30:51.086 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:50 smithi016 ceph-mon[31187]: pgmap v346: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T16:30:51.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:50 smithi107 ceph-mon[34139]: pgmap v346: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T16:30:52.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.017+0000 7fbfabdc8640 1 -- 172.21.15.16:0/1534107814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfa4105b70 msgr2=0x7fbfa4107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:52.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.017+0000 7fbfabdc8640 1 --2- 172.21.15.16:0/1534107814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfa4105b70 0x7fbfa4107f60 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c009900 tx=0x7fbf9c02a410 comp rx=0 tx=0).stop 2024-09-13T16:30:52.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.017+0000 7fbfabdc8640 1 -- 172.21.15.16:0/1534107814 shutdown_connections 2024-09-13T16:30:52.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.017+0000 7fbfabdc8640 1 --2- 172.21.15.16:0/1534107814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfa4105b70 0x7fbfa4107f60 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:52.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.017+0000 7fbfabdc8640 1 --2- 172.21.15.16:0/1534107814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfa4103240 0x7fbfa4105630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:52.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.017+0000 7fbfabdc8640 1 --2- 172.21.15.16:0/1534107814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfa4100910 0x7fbfa4102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.017+0000 7fbfabdc8640 1 -- 172.21.15.16:0/1534107814 >> 172.21.15.16:0/1534107814 conn(0x7fbfa40fa490 msgr2=0x7fbfa40fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.017+0000 7fbfabdc8640 1 -- 172.21.15.16:0/1534107814 shutdown_connections 2024-09-13T16:30:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.017+0000 7fbfabdc8640 1 -- 172.21.15.16:0/1534107814 wait complete. 2024-09-13T16:30:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.018+0000 7fbfabdc8640 1 Processor -- start 2024-09-13T16:30:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.018+0000 7fbfabdc8640 1 -- start start 2024-09-13T16:30:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.018+0000 7fbfabdc8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfa4100910 0x7fbfa4195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.018+0000 7fbfabdc8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfa4103240 0x7fbfa4196230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.018+0000 7fbfabdc8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfa4105b70 0x7fbfa419d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.018+0000 7fbfabdc8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa419d7e0 con 0x7fbfa4105b70 2024-09-13T16:30:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.018+0000 7fbfabdc8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa419d950 con 0x7fbfa4100910 2024-09-13T16:30:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.018+0000 7fbfabdc8640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa419db10 con 0x7fbfa4103240 2024-09-13T16:30:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.018+0000 7fbfaa365640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfa4105b70 0x7fbfa419d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.018+0000 7fbfaa365640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfa4105b70 0x7fbfa419d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54772/0 (socket says 172.21.15.16:54772) 2024-09-13T16:30:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.018+0000 7fbfaa365640 1 -- 172.21.15.16:0/1123853112 learned_addr learned my addr 172.21.15.16:0/1123853112 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.018+0000 7fbfa9b64640 1 --2- 172.21.15.16:0/1123853112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfa4100910 0x7fbfa4195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.018+0000 7fbfa9363640 1 --2- 172.21.15.16:0/1123853112 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfa4103240 0x7fbfa4196230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.018+0000 7fbfaa365640 1 -- 172.21.15.16:0/1123853112 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfa4103240 msgr2=0x7fbfa4196230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.018+0000 7fbfaa365640 1 --2- 172.21.15.16:0/1123853112 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfa4103240 0x7fbfa4196230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.018+0000 7fbfaa365640 1 -- 172.21.15.16:0/1123853112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfa4100910 msgr2=0x7fbfa4195cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.018+0000 7fbfaa365640 1 --2- 172.21.15.16:0/1123853112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfa4100910 0x7fbfa4195cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.018+0000 7fbfaa365640 1 -- 172.21.15.16:0/1123853112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9c009560 con 0x7fbfa4105b70 2024-09-13T16:30:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.019+0000 7fbfaa365640 1 --2- 172.21.15.16:0/1123853112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfa4105b70 0x7fbfa419d270 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c02aa00 tx=0x7fbf9c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.019+0000 7fbf96ffd640 1 -- 172.21.15.16:0/1123853112 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c031d70 con 0x7fbfa4105b70 2024-09-13T16:30:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.019+0000 7fbfabdc8640 1 -- 172.21.15.16:0/1123853112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa419dd90 con 0x7fbfa4105b70 2024-09-13T16:30:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.019+0000 7fbf96ffd640 1 -- 172.21.15.16:0/1123853112 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf9c0044e0 con 0x7fbfa4105b70 2024-09-13T16:30:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.019+0000 7fbf96ffd640 1 -- 172.21.15.16:0/1123853112 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c003b60 con 0x7fbfa4105b70 2024-09-13T16:30:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.019+0000 7fbfabdc8640 1 -- 172.21.15.16:0/1123853112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa419e170 con 0x7fbfa4105b70 2024-09-13T16:30:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.020+0000 7fbfabdc8640 1 -- 172.21.15.16:0/1123853112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfa40fbf80 con 0x7fbfa4105b70 2024-09-13T16:30:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.021+0000 7fbf96ffd640 1 -- 172.21.15.16:0/1123853112 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fbf9c004c50 con 0x7fbfa4105b70 2024-09-13T16:30:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.022+0000 7fbf96ffd640 1 --2- 172.21.15.16:0/1123853112 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fbf900a1080 0x7fbf900a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.022+0000 7fbf96ffd640 1 -- 172.21.15.16:0/1123853112 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fbf9c0edf10 con 0x7fbfa4105b70 2024-09-13T16:30:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.023+0000 7fbfa9b64640 1 --2- 172.21.15.16:0/1123853112 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fbf900a1080 0x7fbf900a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.023+0000 7fbfa9b64640 1 --2- 172.21.15.16:0/1123853112 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fbf900a1080 0x7fbf900a3540 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0005c40 tx=0x7fbfa000a590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.027+0000 7fbf96ffd640 1 -- 172.21.15.16:0/1123853112 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fbf9c0b72f0 con 0x7fbfa4105b70 2024-09-13T16:30:52.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.237+0000 7fbfabdc8640 1 -- 172.21.15.16:0/1123853112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbfa402c480 con 0x7fbfa4105b70 2024-09-13T16:30:52.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.237+0000 7fbf96ffd640 1 -- 172.21.15.16:0/1123853112 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+2933 (secure 0 0 0) 0x7fbf9c04ab90 con 0x7fbfa4105b70 2024-09-13T16:30:52.240 INFO:teuthology.orchestra.run.smithi016.stdout:e28 2024-09-13T16:30:52.241 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-13T16:30:52.241 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-13T16:30:52.241 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-09-13T16:30:52.241 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:30:52.242 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-09-13T16:30:52.242 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-09-13T16:30:52.242 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 28 2024-09-13T16:30:52.242 INFO:teuthology.orchestra.run.smithi016.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-13T16:30:52.242 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-09-13T16:28:39.184734+0000 2024-09-13T16:30:52.242 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-09-13T16:29:06.636937+0000 2024-09-13T16:30:52.243 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-09-13T16:30:52.243 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-09-13T16:30:52.243 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 300 2024-09-13T16:30:52.243 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-09-13T16:30:52.243 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-09-13T16:30:52.244 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-09-13T16:30:52.244 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-09-13T16:30:52.244 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-09-13T16:30:52.244 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 0 2024-09-13T16:30:52.244 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-13T16:30:52.245 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 5 2024-09-13T16:30:52.245 INFO:teuthology.orchestra.run.smithi016.stdout:in 0,1,2,3,4 2024-09-13T16:30:52.245 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=24518,1=24490,2=24485,3=24422,4=24418} 2024-09-13T16:30:52.245 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-09-13T16:30:52.246 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-09-13T16:30:52.246 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-09-13T16:30:52.246 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3,4] 2024-09-13T16:30:52.246 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-09-13T16:30:52.246 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data disabled 2024-09-13T16:30:52.246 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-09-13T16:30:52.247 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-09-13T16:30:52.247 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 0 2024-09-13T16:30:52.247 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b{0:24518} state up:active seq 3 addr [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T16:30:52.247 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c{0:24472} state up:standby-replay seq 1 addr [v2:172.21.15.111:6836/1877128453,v1:172.21.15.111:6837/1877128453] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T16:30:52.247 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.i{1:24490} state up:active seq 10 addr [v2:172.21.15.111:6838/1423842285,v1:172.21.15.111:6839/1423842285] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T16:30:52.248 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.f{1:24460} state up:standby-replay seq 1 addr [v2:172.21.15.111:6834/662156111,v1:172.21.15.111:6835/662156111] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T16:30:52.248 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.h{2:24485} state up:active seq 11 addr [v2:172.21.15.107:6836/2838457611,v1:172.21.15.107:6837/2838457611] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T16:30:52.248 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.e{2:24446} state up:standby-replay seq 1 addr [v2:172.21.15.107:6834/338548468,v1:172.21.15.107:6835/338548468] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T16:30:52.248 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.k{3:24422} state up:active seq 15 addr [v2:172.21.15.107:6832/3797712690,v1:172.21.15.107:6833/3797712690] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T16:30:52.248 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a{3:14598} state up:standby-replay seq 1 addr [v2:172.21.15.16:6840/521042681,v1:172.21.15.16:6841/521042681] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T16:30:52.248 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.l{4:24418} state up:active seq 16 addr [v2:172.21.15.111:6832/1984275936,v1:172.21.15.111:6833/1984275936] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T16:30:52.249 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.g{4:14586} state up:standby-replay seq 1 addr [v2:172.21.15.16:6838/2391618693,v1:172.21.15.16:6839/2391618693] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T16:30:52.249 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:30:52.249 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:30:52.249 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-09-13T16:30:52.249 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:30:52.250 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.j{-1:14505} state up:standby seq 1 addr [v2:172.21.15.16:6834/3292781664,v1:172.21.15.16:6835/3292781664] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T16:30:52.250 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d{-1:14517} state up:standby seq 1 addr [v2:172.21.15.16:6836/4109039027,v1:172.21.15.16:6837/4109039027] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T16:30:52.251 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-13T16:30:52.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.240+0000 7fbfabdc8640 1 -- 172.21.15.16:0/1123853112 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fbf900a1080 msgr2=0x7fbf900a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:52.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.240+0000 7fbfabdc8640 1 --2- 172.21.15.16:0/1123853112 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fbf900a1080 0x7fbf900a3540 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0005c40 tx=0x7fbfa000a590 comp rx=0 tx=0).stop 2024-09-13T16:30:52.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.240+0000 7fbfabdc8640 1 -- 172.21.15.16:0/1123853112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfa4105b70 msgr2=0x7fbfa419d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:52.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.240+0000 7fbfabdc8640 1 --2- 172.21.15.16:0/1123853112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfa4105b70 0x7fbfa419d270 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c02aa00 tx=0x7fbf9c0042f0 comp rx=0 tx=0).stop 2024-09-13T16:30:52.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.240+0000 7fbfabdc8640 1 -- 172.21.15.16:0/1123853112 shutdown_connections 2024-09-13T16:30:52.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.240+0000 7fbfabdc8640 1 --2- 172.21.15.16:0/1123853112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfa4105b70 0x7fbfa419d270 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:52.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.240+0000 7fbfabdc8640 1 --2- 172.21.15.16:0/1123853112 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfa4103240 0x7fbfa4196230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:52.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.240+0000 7fbfabdc8640 1 --2- 172.21.15.16:0/1123853112 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fbf900a1080 0x7fbf900a3540 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:52.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.240+0000 7fbfabdc8640 1 --2- 172.21.15.16:0/1123853112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfa4100910 0x7fbfa4195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:52.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.240+0000 7fbfabdc8640 1 -- 172.21.15.16:0/1123853112 >> 172.21.15.16:0/1123853112 conn(0x7fbfa40fa490 msgr2=0x7fbfa4101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:52.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.240+0000 7fbfabdc8640 1 -- 172.21.15.16:0/1123853112 shutdown_connections 2024-09-13T16:30:52.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:52.240+0000 7fbfabdc8640 1 -- 172.21.15.16:0/1123853112 wait complete. 2024-09-13T16:30:52.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:52 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1123853112' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-13T16:30:52.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:52 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1123853112' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-13T16:30:52.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:52 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1123853112' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-13T16:30:53.267 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-13T16:30:53.484 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:30:53.599 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:53 smithi107 ceph-mon[34139]: pgmap v347: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-13T16:30:53.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:53 smithi016 ceph-mon[31187]: pgmap v347: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-13T16:30:53.687 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:53 smithi111 ceph-mon[34600]: pgmap v347: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-13T16:30:54.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:54 smithi016 ceph-mon[31187]: pgmap v348: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T16:30:54.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:54 smithi111 ceph-mon[34600]: pgmap v348: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T16:30:55.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:54 smithi107 ceph-mon[34139]: pgmap v348: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T16:30:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.530+0000 7fd00931f640 1 -- 172.21.15.16:0/361781014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0040fed00 msgr2=0x7fd0040ff100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.530+0000 7fd00931f640 1 --2- 172.21.15.16:0/361781014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0040fed00 0x7fd0040ff100 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009900 tx=0x7fcff402a410 comp rx=0 tx=0).stop 2024-09-13T16:30:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.530+0000 7fd00931f640 1 -- 172.21.15.16:0/361781014 shutdown_connections 2024-09-13T16:30:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.530+0000 7fd00931f640 1 --2- 172.21.15.16:0/361781014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0041008c0 0x7fd004107150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.530+0000 7fd00931f640 1 --2- 172.21.15.16:0/361781014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0040fff00 0x7fd004100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.530+0000 7fd00931f640 1 --2- 172.21.15.16:0/361781014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0040fed00 0x7fd0040ff100 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.530+0000 7fd00931f640 1 -- 172.21.15.16:0/361781014 >> 172.21.15.16:0/361781014 conn(0x7fd0040fa490 msgr2=0x7fd0040fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.530+0000 7fd00931f640 1 -- 172.21.15.16:0/361781014 shutdown_connections 2024-09-13T16:30:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.530+0000 7fd00931f640 1 -- 172.21.15.16:0/361781014 wait complete. 2024-09-13T16:30:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.531+0000 7fd00931f640 1 Processor -- start 2024-09-13T16:30:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.531+0000 7fd00931f640 1 -- start start 2024-09-13T16:30:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.531+0000 7fd00931f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0040fed00 0x7fd004069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.531+0000 7fd00931f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0040fff00 0x7fd00419a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.531+0000 7fd00931f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0041008c0 0x7fd00419cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.531+0000 7fd00931f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00419d780 con 0x7fd0040fff00 2024-09-13T16:30:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.531+0000 7fd00931f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00419d8f0 con 0x7fd0040fed00 2024-09-13T16:30:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.531+0000 7fd00931f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00419dab0 con 0x7fd0041008c0 2024-09-13T16:30:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.531+0000 7fcffbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0040fff00 0x7fd00419a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.531+0000 7fcffbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0040fff00 0x7fd00419a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40468/0 (socket says 172.21.15.16:40468) 2024-09-13T16:30:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.531+0000 7fcffbfff640 1 -- 172.21.15.16:0/2295748755 learned_addr learned my addr 172.21.15.16:0/2295748755 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.531+0000 7fd002ffd640 1 --2- 172.21.15.16:0/2295748755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0040fed00 0x7fd004069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.531+0000 7fd0037fe640 1 --2- 172.21.15.16:0/2295748755 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0041008c0 0x7fd00419cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.532+0000 7fcffbfff640 1 -- 172.21.15.16:0/2295748755 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0041008c0 msgr2=0x7fd00419cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.532+0000 7fcffbfff640 1 --2- 172.21.15.16:0/2295748755 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0041008c0 0x7fd00419cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.532+0000 7fcffbfff640 1 -- 172.21.15.16:0/2295748755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0040fed00 msgr2=0x7fd004069280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.532+0000 7fcffbfff640 1 --2- 172.21.15.16:0/2295748755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0040fed00 0x7fd004069280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.532+0000 7fcffbfff640 1 -- 172.21.15.16:0/2295748755 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff4009560 con 0x7fd0040fff00 2024-09-13T16:30:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.532+0000 7fcffbfff640 1 --2- 172.21.15.16:0/2295748755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0040fff00 0x7fd00419a980 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcfec00c920 tx=0x7fcfec00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.532+0000 7fd000ff9640 1 -- 172.21.15.16:0/2295748755 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec007c70 con 0x7fd0040fff00 2024-09-13T16:30:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.532+0000 7fd000ff9640 1 -- 172.21.15.16:0/2295748755 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcfec007dd0 con 0x7fd0040fff00 2024-09-13T16:30:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.532+0000 7fd00931f640 1 -- 172.21.15.16:0/2295748755 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd00419dd30 con 0x7fd0040fff00 2024-09-13T16:30:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.532+0000 7fd000ff9640 1 -- 172.21.15.16:0/2295748755 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec018730 con 0x7fd0040fff00 2024-09-13T16:30:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.532+0000 7fd00931f640 1 -- 172.21.15.16:0/2295748755 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd00419e220 con 0x7fd0040fff00 2024-09-13T16:30:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.533+0000 7fd00931f640 1 -- 172.21.15.16:0/2295748755 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd004106190 con 0x7fd0040fff00 2024-09-13T16:30:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.534+0000 7fd000ff9640 1 -- 172.21.15.16:0/2295748755 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fcfec00f450 con 0x7fd0040fff00 2024-09-13T16:30:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.535+0000 7fd000ff9640 1 --2- 172.21.15.16:0/2295748755 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fcff00a1100 0x7fcff00a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.536+0000 7fd002ffd640 1 --2- 172.21.15.16:0/2295748755 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fcff00a1100 0x7fcff00a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.536+0000 7fd000ff9640 1 -- 172.21.15.16:0/2295748755 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fcfec0ccdc0 con 0x7fd0040fff00 2024-09-13T16:30:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.536+0000 7fd002ffd640 1 --2- 172.21.15.16:0/2295748755 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fcff00a1100 0x7fcff00a35c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcff4006010 tx=0x7fcff4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.540+0000 7fd000ff9640 1 -- 172.21.15.16:0/2295748755 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fcfec096200 con 0x7fd0040fff00 2024-09-13T16:30:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.713+0000 7fd00931f640 1 -- 172.21.15.16:0/2295748755 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fd00402c480 con 0x7fd0040fff00 2024-09-13T16:30:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.714+0000 7fd000ff9640 1 -- 172.21.15.16:0/2295748755 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v93) v1 ==== 56+0+5657 (secure 0 0 0) 0x7fcfec099820 con 0x7fd0040fff00 2024-09-13T16:30:55.717 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 93 2024-09-13T16:30:55.718 INFO:teuthology.orchestra.run.smithi016.stdout:fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 2024-09-13T16:30:55.718 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-09-13T16:21:17.971738+0000 2024-09-13T16:30:55.718 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-09-13T16:30:05.681521+0000 2024-09-13T16:30:55.718 INFO:teuthology.orchestra.run.smithi016.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-13T16:30:55.718 INFO:teuthology.orchestra.run.smithi016.stdout:crush_version 27 2024-09-13T16:30:55.718 INFO:teuthology.orchestra.run.smithi016.stdout:full_ratio 0.95 2024-09-13T16:30:55.718 INFO:teuthology.orchestra.run.smithi016.stdout:backfillfull_ratio 0.9 2024-09-13T16:30:55.718 INFO:teuthology.orchestra.run.smithi016.stdout:nearfull_ratio 0.85 2024-09-13T16:30:55.718 INFO:teuthology.orchestra.run.smithi016.stdout:require_min_compat_client luminous 2024-09-13T16:30:55.718 INFO:teuthology.orchestra.run.smithi016.stdout:min_compat_client luminous 2024-09-13T16:30:55.718 INFO:teuthology.orchestra.run.smithi016.stdout:require_osd_release reef 2024-09-13T16:30:55.718 INFO:teuthology.orchestra.run.smithi016.stdout:stretch_mode_enabled false 2024-09-13T16:30:55.718 INFO:teuthology.orchestra.run.smithi016.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-13T16:30:55.719 INFO:teuthology.orchestra.run.smithi016.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-13T16:30:55.719 INFO:teuthology.orchestra.run.smithi016.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-13T16:30:55.719 INFO:teuthology.orchestra.run.smithi016.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-13T16:30:55.719 INFO:teuthology.orchestra.run.smithi016.stdout:max_osd 12 2024-09-13T16:30:55.719 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6802/2079394186,v1:172.21.15.16:6803/2079394186] [v2:172.21.15.16:6804/2079394186,v1:172.21.15.16:6805/2079394186] exists,up 0e11ef8f-08d6-4e74-afd6-f49f4bd40032 2024-09-13T16:30:55.719 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 up in weight 1 up_from 13 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6800/4021360958,v1:172.21.15.107:6801/4021360958] [v2:172.21.15.107:6802/4021360958,v1:172.21.15.107:6803/4021360958] exists,up f2cd3694-9c9e-4da7-976e-f80a8df120f5 2024-09-13T16:30:55.719 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 up in weight 1 up_from 18 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6800/3227179825,v1:172.21.15.111:6801/3227179825] [v2:172.21.15.111:6802/3227179825,v1:172.21.15.111:6803/3227179825] exists,up a5bbf2eb-6599-44d5-ad96-68bb89ee19f4 2024-09-13T16:30:55.719 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 up in weight 1 up_from 25 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6810/2390375937,v1:172.21.15.16:6811/2390375937] [v2:172.21.15.16:6812/2390375937,v1:172.21.15.16:6813/2390375937] exists,up d6295ffe-11bc-4964-a250-9fecca870b97 2024-09-13T16:30:55.719 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 up in weight 1 up_from 33 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6808/1936582048,v1:172.21.15.107:6809/1936582048] [v2:172.21.15.107:6810/1936582048,v1:172.21.15.107:6811/1936582048] exists,up 0a8446e0-fc86-4646-beb6-a7b9d433093d 2024-09-13T16:30:55.719 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 up in weight 1 up_from 38 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6808/483858448,v1:172.21.15.111:6809/483858448] [v2:172.21.15.111:6810/483858448,v1:172.21.15.111:6811/483858448] exists,up 2c97a446-357e-4975-9e51-9c95fe84a663 2024-09-13T16:30:55.719 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 up in weight 1 up_from 46 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6818/1330214141,v1:172.21.15.16:6819/1330214141] [v2:172.21.15.16:6820/1330214141,v1:172.21.15.16:6821/1330214141] exists,up 890239e6-e6fb-4f9c-b973-05cc66d199bf 2024-09-13T16:30:55.719 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 up in weight 1 up_from 51 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6816/1133876754,v1:172.21.15.107:6817/1133876754] [v2:172.21.15.107:6818/1133876754,v1:172.21.15.107:6819/1133876754] exists,up ee826a9c-f441-4903-9b1b-66b627f7b712 2024-09-13T16:30:55.719 INFO:teuthology.orchestra.run.smithi016.stdout:osd.8 up in weight 1 up_from 59 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6816/837548190,v1:172.21.15.111:6817/837548190] [v2:172.21.15.111:6818/837548190,v1:172.21.15.111:6819/837548190] exists,up e19773f2-dec8-44e0-9d33-0252e07d6415 2024-09-13T16:30:55.720 INFO:teuthology.orchestra.run.smithi016.stdout:osd.9 up in weight 1 up_from 67 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6826/4175383686,v1:172.21.15.16:6827/4175383686] [v2:172.21.15.16:6828/4175383686,v1:172.21.15.16:6829/4175383686] exists,up bae4d6db-8b90-4d02-8e83-7d8f2a6f8fdb 2024-09-13T16:30:55.720 INFO:teuthology.orchestra.run.smithi016.stdout:osd.10 up in weight 1 up_from 72 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6824/2659550426,v1:172.21.15.107:6825/2659550426] [v2:172.21.15.107:6826/2659550426,v1:172.21.15.107:6827/2659550426] exists,up fd0e7740-76c8-4c99-a349-7077b4905143 2024-09-13T16:30:55.720 INFO:teuthology.orchestra.run.smithi016.stdout:osd.11 up in weight 1 up_from 80 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6824/1633548683,v1:172.21.15.111:6825/1633548683] [v2:172.21.15.111:6826/1633548683,v1:172.21.15.111:6827/1633548683] exists,up 94a1751d-b00c-4e23-815d-3a8bc3dea146 2024-09-13T16:30:55.720 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.10 [2,4] 2024-09-13T16:30:55.720 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.13 [2,11] 2024-09-13T16:30:55.720 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.1c [8,11] 2024-09-13T16:30:55.720 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.1f [8,1] 2024-09-13T16:30:55.720 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.0 [5,9] 2024-09-13T16:30:55.720 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.14 [4,6] 2024-09-13T16:30:55.720 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.36 [4,9] 2024-09-13T16:30:55.720 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.3c [4,9] 2024-09-13T16:30:55.720 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 4.e [5,8] 2024-09-13T16:30:55.720 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 4.f [9,8] 2024-09-13T16:30:55.721 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 4.13 [9,8] 2024-09-13T16:30:55.721 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 4.17 [9,10] 2024-09-13T16:30:55.721 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/2098378667 expires 2024-09-14T16:22:04.485277+0000 2024-09-13T16:30:55.721 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/3536229016 expires 2024-09-14T16:22:04.485277+0000 2024-09-13T16:30:55.721 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6801/1756940569 expires 2024-09-14T16:22:04.485277+0000 2024-09-13T16:30:55.721 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/2684687547 expires 2024-09-14T16:22:04.485277+0000 2024-09-13T16:30:55.721 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6800/1756940569 expires 2024-09-14T16:22:04.485277+0000 2024-09-13T16:30:55.721 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/368410612 expires 2024-09-14T16:21:40.126459+0000 2024-09-13T16:30:55.721 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/12158160 expires 2024-09-14T16:21:40.126459+0000 2024-09-13T16:30:55.721 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6800/1147765163 expires 2024-09-14T16:21:40.126459+0000 2024-09-13T16:30:55.721 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6801/1147765163 expires 2024-09-14T16:21:40.126459+0000 2024-09-13T16:30:55.721 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/3754423379 expires 2024-09-14T16:21:40.126459+0000 2024-09-13T16:30:55.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.718+0000 7fd00931f640 1 -- 172.21.15.16:0/2295748755 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fcff00a1100 msgr2=0x7fcff00a35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:55.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.718+0000 7fd00931f640 1 --2- 172.21.15.16:0/2295748755 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fcff00a1100 0x7fcff00a35c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcff4006010 tx=0x7fcff4005a20 comp rx=0 tx=0).stop 2024-09-13T16:30:55.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.718+0000 7fd00931f640 1 -- 172.21.15.16:0/2295748755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0040fff00 msgr2=0x7fd00419a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:55.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.718+0000 7fd00931f640 1 --2- 172.21.15.16:0/2295748755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0040fff00 0x7fd00419a980 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcfec00c920 tx=0x7fcfec00cde0 comp rx=0 tx=0).stop 2024-09-13T16:30:55.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.718+0000 7fd00931f640 1 -- 172.21.15.16:0/2295748755 shutdown_connections 2024-09-13T16:30:55.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.718+0000 7fd00931f640 1 --2- 172.21.15.16:0/2295748755 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0041008c0 0x7fd00419cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:55.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.718+0000 7fd00931f640 1 --2- 172.21.15.16:0/2295748755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0040fff00 0x7fd00419a980 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:55.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.718+0000 7fd00931f640 1 --2- 172.21.15.16:0/2295748755 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fcff00a1100 0x7fcff00a35c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:55.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.718+0000 7fd00931f640 1 --2- 172.21.15.16:0/2295748755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0040fed00 0x7fd004069280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:55.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.718+0000 7fd00931f640 1 -- 172.21.15.16:0/2295748755 >> 172.21.15.16:0/2295748755 conn(0x7fd0040fa490 msgr2=0x7fd004103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:55.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.718+0000 7fd00931f640 1 -- 172.21.15.16:0/2295748755 shutdown_connections 2024-09-13T16:30:55.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:55.718+0000 7fd00931f640 1 -- 172.21.15.16:0/2295748755 wait complete. 2024-09-13T16:30:56.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:55 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/2295748755' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-13T16:30:56.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:55 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2295748755' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-13T16:30:56.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:55 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/2295748755' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-13T16:30:56.416 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-13T16:30:56.430 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:30:56.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.541+0000 7fb7a275e640 1 -- 172.21.15.16:0/2541608105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb79c103180 msgr2=0x7fb79c105570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:56.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.541+0000 7fb7a275e640 1 --2- 172.21.15.16:0/2541608105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb79c103180 0x7fb79c105570 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb790009900 tx=0x7fb79002a410 comp rx=0 tx=0).stop 2024-09-13T16:30:56.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.541+0000 7fb7a275e640 1 -- 172.21.15.16:0/2541608105 shutdown_connections 2024-09-13T16:30:56.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.541+0000 7fb7a275e640 1 --2- 172.21.15.16:0/2541608105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb79c105ab0 0x7fb79c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:56.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.541+0000 7fb7a275e640 1 --2- 172.21.15.16:0/2541608105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb79c103180 0x7fb79c105570 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:56.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.541+0000 7fb7a275e640 1 --2- 172.21.15.16:0/2541608105 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb79c100850 0x7fb79c102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:56.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.541+0000 7fb7a275e640 1 -- 172.21.15.16:0/2541608105 >> 172.21.15.16:0/2541608105 conn(0x7fb79c0fa430 msgr2=0x7fb79c0fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:56.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.541+0000 7fb7a275e640 1 -- 172.21.15.16:0/2541608105 shutdown_connections 2024-09-13T16:30:56.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.541+0000 7fb7a275e640 1 -- 172.21.15.16:0/2541608105 wait complete. 2024-09-13T16:30:56.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.541+0000 7fb7a275e640 1 Processor -- start 2024-09-13T16:30:56.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.542+0000 7fb7a275e640 1 -- start start 2024-09-13T16:30:56.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.542+0000 7fb7a275e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb79c100850 0x7fb79c195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:56.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.542+0000 7fb7a275e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb79c103180 0x7fb79c196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:56.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.542+0000 7fb7a275e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb79c105ab0 0x7fb79c19d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:56.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.542+0000 7fb7a275e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c19d810 con 0x7fb79c100850 2024-09-13T16:30:56.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.542+0000 7fb7a275e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c19d980 con 0x7fb79c105ab0 2024-09-13T16:30:56.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.542+0000 7fb7a275e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c19db40 con 0x7fb79c103180 2024-09-13T16:30:56.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.542+0000 7fb79bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb79c100850 0x7fb79c195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:56.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.542+0000 7fb79bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb79c100850 0x7fb79c195d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40488/0 (socket says 172.21.15.16:40488) 2024-09-13T16:30:56.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.542+0000 7fb79bfff640 1 -- 172.21.15.16:0/1108057803 learned_addr learned my addr 172.21.15.16:0/1108057803 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:56.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.542+0000 7fb7a0cfb640 1 --2- 172.21.15.16:0/1108057803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb79c105ab0 0x7fb79c19d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:56.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.542+0000 7fb79b7fe640 1 --2- 172.21.15.16:0/1108057803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb79c103180 0x7fb79c196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:56.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.542+0000 7fb79bfff640 1 -- 172.21.15.16:0/1108057803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb79c103180 msgr2=0x7fb79c196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:56.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.542+0000 7fb79bfff640 1 --2- 172.21.15.16:0/1108057803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb79c103180 0x7fb79c196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:56.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.542+0000 7fb79bfff640 1 -- 172.21.15.16:0/1108057803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb79c105ab0 msgr2=0x7fb79c19d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:56.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.542+0000 7fb79bfff640 1 --2- 172.21.15.16:0/1108057803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb79c105ab0 0x7fb79c19d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:56.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.542+0000 7fb79bfff640 1 -- 172.21.15.16:0/1108057803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb790009560 con 0x7fb79c100850 2024-09-13T16:30:56.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.542+0000 7fb79bfff640 1 --2- 172.21.15.16:0/1108057803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb79c100850 0x7fb79c195d30 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb78c00e9f0 tx=0x7fb78c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:56.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.543+0000 7fb7997fa640 1 -- 172.21.15.16:0/1108057803 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c009720 con 0x7fb79c100850 2024-09-13T16:30:56.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.543+0000 7fb7997fa640 1 -- 172.21.15.16:0/1108057803 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb78c00cd80 con 0x7fb79c100850 2024-09-13T16:30:56.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.543+0000 7fb7a275e640 1 -- 172.21.15.16:0/1108057803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb79c19de20 con 0x7fb79c100850 2024-09-13T16:30:56.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.543+0000 7fb7997fa640 1 -- 172.21.15.16:0/1108057803 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c010430 con 0x7fb79c100850 2024-09-13T16:30:56.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.543+0000 7fb7a275e640 1 -- 172.21.15.16:0/1108057803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb79c19e240 con 0x7fb79c100850 2024-09-13T16:30:56.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.544+0000 7fb7a275e640 1 -- 172.21.15.16:0/1108057803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb79c0fbf20 con 0x7fb79c100850 2024-09-13T16:30:56.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.545+0000 7fb7997fa640 1 -- 172.21.15.16:0/1108057803 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb78c004fb0 con 0x7fb79c100850 2024-09-13T16:30:56.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.546+0000 7fb7997fa640 1 --2- 172.21.15.16:0/1108057803 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb7840a1110 0x7fb7840a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:56.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.546+0000 7fb79b7fe640 1 --2- 172.21.15.16:0/1108057803 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb7840a1110 0x7fb7840a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:56.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.546+0000 7fb7997fa640 1 -- 172.21.15.16:0/1108057803 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb78c014070 con 0x7fb79c100850 2024-09-13T16:30:56.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.546+0000 7fb79b7fe640 1 --2- 172.21.15.16:0/1108057803 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb7840a1110 0x7fb7840a35d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb790006010 tx=0x7fb790005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:56.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.550+0000 7fb7997fa640 1 -- 172.21.15.16:0/1108057803 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb78c0964a0 con 0x7fb79c100850 2024-09-13T16:30:56.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.726+0000 7fb7a275e640 1 -- 172.21.15.16:0/1108057803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb79c02c480 con 0x7fb79c100850 2024-09-13T16:30:56.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.727+0000 7fb7997fa640 1 -- 172.21.15.16:0/1108057803 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24593 (secure 0 0 0) 0x7fb78c0983d0 con 0x7fb79c100850 2024-09-13T16:30:56.965 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:30:56.965 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":93,"fsid":"0d241f6a-71ec-11ef-bceb-c7b262605968","created":"2024-09-13T16:21:17.971738+0000","modified":"2024-09-13T16:30:05.681521+0000","last_up_change":"2024-09-13T16:27:54.673413+0000","last_in_change":"2024-09-13T16:27:35.718390+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:24:22.981019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:28:37.746027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:28:38.602731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:28:41.633322+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e11ef8f-08d6-4e74-afd6-f49f4bd40032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6803","nonce":2079394186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6805","nonce":2079394186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6809","nonce":2079394186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6807","nonce":2079394186}]},"public_addr":"172.21.15.16:6803/2079394186","cluster_addr":"172.21.15.16:6805/2079394186","heartbeat_back_addr":"172.21.15.16:6809/2079394186","heartbeat_front_addr":"172.21.15.16:6807/2079394186","state":["exists","up"]},{"osd":1,"uuid":"f2cd3694-9c9e-4da7-976e-f80a8df120f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6801","nonce":4021360958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6803","nonce":4021360958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6807","nonce":4021360958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6805","nonce":4021360958}]},"public_addr":"172.21.15.107:6801/4021360958","cluster_addr":"172.21.15.107:6803/4021360958","heartbeat_back_addr":"172.21.15.107:6807/4021360958","heartbeat_front_addr":"172.21.15.107:6805/4021360958","state":["exists","up"]},{"osd":2,"uuid":"a5bbf2eb-6599-44d5-ad96-68bb89ee19f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6801","nonce":3227179825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6803","nonce":3227179825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6807","nonce":3227179825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6805","nonce":3227179825}]},"public_addr":"172.21.15.111:6801/3227179825","cluster_addr":"172.21.15.111:6803/3227179825","heartbeat_back_addr":"172.21.15.111:6807/3227179825","heartbeat_front_addr":"172.21.15.111:6805/3227179825","state":["exists","up"]},{"osd":3,"uuid":"d6295ffe-11bc-4964-a250-9fecca870b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6811","nonce":2390375937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6813","nonce":2390375937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6817","nonce":2390375937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6815","nonce":2390375937}]},"public_addr":"172.21.15.16:6811/2390375937","cluster_addr":"172.21.15.16:6813/2390375937","heartbeat_back_addr":"172.21.15.16:6817/2390375937","heartbeat_front_addr":"172.21.15.16:6815/2390375937","state":["exists","up"]},{"osd":4,"uuid":"0a8446e0-fc86-4646-beb6-a7b9d433093d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6809","nonce":1936582048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6811","nonce":1936582048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6815","nonce":1936582048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6813","nonce":1936582048}]},"public_addr":"172.21.15.107:6809/1936582048","cluster_addr":"172.21.15.107:6811/1936582048","heartbeat_back_addr":"172.21.15.107:6815/1936582048","heartbeat_front_addr":"172.21.15.107:6813/1936582048","state":["exists","up"]},{"osd":5,"uuid":"2c97a446-357e-4975-9e51-9c95fe84a663","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6809","nonce":483858448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6811","nonce":483858448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6815","nonce":483858448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6813","nonce":483858448}]},"public_addr":"172.21.15.111:6809/483858448","cluster_addr":"172.21.15.111:6811/483858448","heartbeat_back_addr":"172.21.15.111:6815/483858448","heartbeat_front_addr":"172.21.15.111:6813/483858448","state":["exists","up"]},{"osd":6,"uuid":"890239e6-e6fb-4f9c-b973-05cc66d199bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6819","nonce":1330214141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6821","nonce":1330214141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6825","nonce":1330214141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6823","nonce":1330214141}]},"public_addr":"172.21.15.16:6819/1330214141","cluster_addr":"172.21.15.16:6821/1330214141","heartbeat_back_addr":"172.21.15.16:6825/1330214141","heartbeat_front_addr":"172.21.15.16:6823/1330214141","state":["exists","up"]},{"osd":7,"uuid":"ee826a9c-f441-4903-9b1b-66b627f7b712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6817","nonce":1133876754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6819","nonce":1133876754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6823","nonce":1133876754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6821","nonce":1133876754}]},"public_addr":"172.21.15.107:6817/1133876754","cluster_addr":"172.21.15.107:6819/1133876754","heartbeat_back_addr":"172.21.15.107:6823/1133876754","heartbeat_front_addr":"172.21.15.107:6821/1133876754","state":["exists","up"]},{"osd":8,"uuid":"e19773f2-dec8-44e0-9d33-0252e07d6415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6817","nonce":837548190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6819","nonce":837548190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6823","nonce":837548190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6821","nonce":837548190}]},"public_addr":"172.21.15.111:6817/837548190","cluster_addr":"172.21.15.111:6819/837548190","heartbeat_back_addr":"172.21.15.111:6823/837548190","heartbeat_front_addr":"172.21.15.111:6821/837548190","state":["exists","up"]},{"osd":9,"uuid":"bae4d6db-8b90-4d02-8e83-7d8f2a6f8fdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6827","nonce":4175383686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6829","nonce":4175383686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6833","nonce":4175383686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6831","nonce":4175383686}]},"public_addr":"172.21.15.16:6827/4175383686","cluster_addr":"172.21.15.16:6829/4175383686","heartbeat_back_addr":"172.21.15.16:6833/4175383686","heartbeat_front_addr":"172.21.15.16:6831/4175383686","state":["exists","up"]},{"osd":10,"uuid":"fd0e7740-76c8-4c99-a349-7077b4905143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6825","nonce":2659550426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6827","nonce":2659550426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6831","nonce":2659550426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6829","nonce":2659550426}]},"public_addr":"172.21.15.107:6825/2659550426","cluster_addr":"172.21.15.107:6827/2659550426","heartbeat_back_addr":"172.21.15.107:6831/2659550426","heartbeat_front_addr":"172.21.15.107:6829/2659550426","state":["exists","up"]},{"osd":11,"uuid":"94a1751d-b00c-4e23-815d-3a8bc3dea146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6825","nonce":1633548683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6827","nonce":1633548683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6831","nonce":1633548683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6829","nonce":1633548683}]},"public_addr":"172.21.15.111:6825/1633548683","cluster_addr":"172.21.15.111:6827/1633548683","heartbeat_back_addr":"172.21.15.111:6831/1633548683","heartbeat_front_addr":"172.21.15.111:6829/1633548683","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:36.359852+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:58.014202+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:19.753672+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:43.404817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:06.001677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:29.361587+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:52.554847+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:15.957553+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:39.581093+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:04.911389+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:28.971280+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:53.473561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/2098378667":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/3536229016":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6801/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/2684687547":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6800/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/368410612":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/12158160":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6800/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6801/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/3754423379":"2024-09-14T16:21:40.126459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:30:56.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.729+0000 7fb7a275e640 1 -- 172.21.15.16:0/1108057803 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb7840a1110 msgr2=0x7fb7840a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:56.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.730+0000 7fb7a275e640 1 --2- 172.21.15.16:0/1108057803 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb7840a1110 0x7fb7840a35d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb790006010 tx=0x7fb790005a20 comp rx=0 tx=0).stop 2024-09-13T16:30:56.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.730+0000 7fb7a275e640 1 -- 172.21.15.16:0/1108057803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb79c100850 msgr2=0x7fb79c195d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:56.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.730+0000 7fb7a275e640 1 --2- 172.21.15.16:0/1108057803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb79c100850 0x7fb79c195d30 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb78c00e9f0 tx=0x7fb78c00edf0 comp rx=0 tx=0).stop 2024-09-13T16:30:56.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.730+0000 7fb7a275e640 1 -- 172.21.15.16:0/1108057803 shutdown_connections 2024-09-13T16:30:56.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.730+0000 7fb7a275e640 1 --2- 172.21.15.16:0/1108057803 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb7840a1110 0x7fb7840a35d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:56.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.730+0000 7fb7a275e640 1 --2- 172.21.15.16:0/1108057803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb79c105ab0 0x7fb79c19d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:56.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.730+0000 7fb7a275e640 1 --2- 172.21.15.16:0/1108057803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb79c103180 0x7fb79c196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:56.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.730+0000 7fb7a275e640 1 --2- 172.21.15.16:0/1108057803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb79c100850 0x7fb79c195d30 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:56.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.730+0000 7fb7a275e640 1 -- 172.21.15.16:0/1108057803 >> 172.21.15.16:0/1108057803 conn(0x7fb79c0fa430 msgr2=0x7fb79c101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:56.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.730+0000 7fb7a275e640 1 -- 172.21.15.16:0/1108057803 shutdown_connections 2024-09-13T16:30:56.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:56.730+0000 7fb7a275e640 1 -- 172.21.15.16:0/1108057803 wait complete. 2024-09-13T16:30:56.968 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:24:22.981019+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T16:28:37.746027+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T16:28:38.602731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T16:28:41.633322+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T16:30:56.969 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:30:56.993 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:56 smithi016 ceph-mon[31187]: pgmap v349: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T16:30:56.993 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:56 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1108057803' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:30:57.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.074+0000 7fea86b9c640 1 -- 172.21.15.16:0/2435783813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea80105a70 msgr2=0x7fea80107ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:57.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.074+0000 7fea86b9c640 1 --2- 172.21.15.16:0/2435783813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea80105a70 0x7fea80107ee0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fea78009ca0 tx=0x7fea7802a8c0 comp rx=0 tx=0).stop 2024-09-13T16:30:57.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.074+0000 7fea86b9c640 1 -- 172.21.15.16:0/2435783813 shutdown_connections 2024-09-13T16:30:57.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.074+0000 7fea86b9c640 1 --2- 172.21.15.16:0/2435783813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea80105a70 0x7fea80107ee0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:57.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.074+0000 7fea86b9c640 1 --2- 172.21.15.16:0/2435783813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea80103140 0x7fea80105530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:57.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.074+0000 7fea86b9c640 1 --2- 172.21.15.16:0/2435783813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea80100810 0x7fea80102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:57.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.074+0000 7fea86b9c640 1 -- 172.21.15.16:0/2435783813 >> 172.21.15.16:0/2435783813 conn(0x7fea800fa3f0 msgr2=0x7fea800fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:57.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.074+0000 7fea86b9c640 1 -- 172.21.15.16:0/2435783813 shutdown_connections 2024-09-13T16:30:57.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.074+0000 7fea86b9c640 1 -- 172.21.15.16:0/2435783813 wait complete. 2024-09-13T16:30:57.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.075+0000 7fea86b9c640 1 Processor -- start 2024-09-13T16:30:57.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.075+0000 7fea86b9c640 1 -- start start 2024-09-13T16:30:57.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.075+0000 7fea86b9c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea80100810 0x7fea80100180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:57.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.075+0000 7fea86b9c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea80103140 0x7fea800fe7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:57.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.075+0000 7fea86b9c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea80105a70 0x7fea800fed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:57.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.075+0000 7fea86b9c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea800ff460 con 0x7fea80100810 2024-09-13T16:30:57.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.075+0000 7fea86b9c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea800ff5d0 con 0x7fea80103140 2024-09-13T16:30:57.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.075+0000 7fea86b9c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8019fdb0 con 0x7fea80105a70 2024-09-13T16:30:57.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.075+0000 7fea84938640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea80100810 0x7fea80100180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:57.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.075+0000 7fea84938640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea80100810 0x7fea80100180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40506/0 (socket says 172.21.15.16:40506) 2024-09-13T16:30:57.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.075+0000 7fea84938640 1 -- 172.21.15.16:0/565342985 learned_addr learned my addr 172.21.15.16:0/565342985 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:57.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.075+0000 7fea7ffff640 1 --2- 172.21.15.16:0/565342985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea80103140 0x7fea800fe7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:57.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.075+0000 7fea85139640 1 --2- 172.21.15.16:0/565342985 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea80105a70 0x7fea800fed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:57.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.075+0000 7fea84938640 1 -- 172.21.15.16:0/565342985 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea80105a70 msgr2=0x7fea800fed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:57.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.075+0000 7fea84938640 1 --2- 172.21.15.16:0/565342985 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea80105a70 0x7fea800fed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:57.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.075+0000 7fea84938640 1 -- 172.21.15.16:0/565342985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea80103140 msgr2=0x7fea800fe7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:57.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.075+0000 7fea84938640 1 --2- 172.21.15.16:0/565342985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea80103140 0x7fea800fe7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:57.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.075+0000 7fea84938640 1 -- 172.21.15.16:0/565342985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea78009900 con 0x7fea80100810 2024-09-13T16:30:57.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.076+0000 7fea84938640 1 --2- 172.21.15.16:0/565342985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea80100810 0x7fea80100180 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fea7400b4d0 tx=0x7fea7400b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:57.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.076+0000 7fea7dffb640 1 -- 172.21.15.16:0/565342985 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea74010870 con 0x7fea80100810 2024-09-13T16:30:57.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.076+0000 7fea7dffb640 1 -- 172.21.15.16:0/565342985 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fea74010e60 con 0x7fea80100810 2024-09-13T16:30:57.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.076+0000 7fea86b9c640 1 -- 172.21.15.16:0/565342985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea801a0060 con 0x7fea80100810 2024-09-13T16:30:57.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.076+0000 7fea7dffb640 1 -- 172.21.15.16:0/565342985 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7400fe70 con 0x7fea80100810 2024-09-13T16:30:57.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.076+0000 7fea86b9c640 1 -- 172.21.15.16:0/565342985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea801a0640 con 0x7fea80100810 2024-09-13T16:30:57.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.077+0000 7fea86b9c640 1 -- 172.21.15.16:0/565342985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea800fbee0 con 0x7fea80100810 2024-09-13T16:30:57.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.078+0000 7fea7dffb640 1 -- 172.21.15.16:0/565342985 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fea740109d0 con 0x7fea80100810 2024-09-13T16:30:57.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.079+0000 7fea7dffb640 1 --2- 172.21.15.16:0/565342985 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fea700a1110 0x7fea700a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:57.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.079+0000 7fea7ffff640 1 --2- 172.21.15.16:0/565342985 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fea700a1110 0x7fea700a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:57.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.079+0000 7fea7dffb640 1 -- 172.21.15.16:0/565342985 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fea740cc630 con 0x7fea80100810 2024-09-13T16:30:57.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.079+0000 7fea7ffff640 1 --2- 172.21.15.16:0/565342985 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fea700a1110 0x7fea700a35d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fea800ffef0 tx=0x7fea6c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:57.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.083+0000 7fea7dffb640 1 -- 172.21.15.16:0/565342985 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fea74095a70 con 0x7fea80100810 2024-09-13T16:30:57.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:56 smithi107 ceph-mon[34139]: pgmap v349: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T16:30:57.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:56 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1108057803' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:30:57.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:56 smithi111 ceph-mon[34600]: pgmap v349: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T16:30:57.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:56 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1108057803' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:30:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.256+0000 7fea86b9c640 1 -- 172.21.15.16:0/565342985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fea800fb950 con 0x7fea80100810 2024-09-13T16:30:57.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.257+0000 7fea7dffb640 1 -- 172.21.15.16:0/565342985 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7fea74099090 con 0x7fea80100810 2024-09-13T16:30:57.259 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-13T16:30:57.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.259+0000 7fea86b9c640 1 -- 172.21.15.16:0/565342985 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fea700a1110 msgr2=0x7fea700a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:57.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.259+0000 7fea86b9c640 1 --2- 172.21.15.16:0/565342985 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fea700a1110 0x7fea700a35d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fea800ffef0 tx=0x7fea6c008040 comp rx=0 tx=0).stop 2024-09-13T16:30:57.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.259+0000 7fea86b9c640 1 -- 172.21.15.16:0/565342985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea80100810 msgr2=0x7fea80100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:57.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.259+0000 7fea86b9c640 1 --2- 172.21.15.16:0/565342985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea80100810 0x7fea80100180 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fea7400b4d0 tx=0x7fea7400b990 comp rx=0 tx=0).stop 2024-09-13T16:30:57.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.259+0000 7fea86b9c640 1 -- 172.21.15.16:0/565342985 shutdown_connections 2024-09-13T16:30:57.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.259+0000 7fea86b9c640 1 --2- 172.21.15.16:0/565342985 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea80105a70 0x7fea800fed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:57.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.259+0000 7fea86b9c640 1 --2- 172.21.15.16:0/565342985 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fea700a1110 0x7fea700a35d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:57.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.259+0000 7fea86b9c640 1 --2- 172.21.15.16:0/565342985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea80103140 0x7fea800fe7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:57.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.259+0000 7fea86b9c640 1 --2- 172.21.15.16:0/565342985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea80100810 0x7fea80100180 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:57.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.259+0000 7fea86b9c640 1 -- 172.21.15.16:0/565342985 >> 172.21.15.16:0/565342985 conn(0x7fea800fa3f0 msgr2=0x7fea80101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:57.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.259+0000 7fea86b9c640 1 -- 172.21.15.16:0/565342985 shutdown_connections 2024-09-13T16:30:57.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.259+0000 7fea86b9c640 1 -- 172.21.15.16:0/565342985 wait complete. 2024-09-13T16:30:57.272 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T16:30:57.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.420+0000 7f727c1c7640 1 -- 172.21.15.16:0/1453751285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7274101950 msgr2=0x7f72741081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:57.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.420+0000 7f727c1c7640 1 --2- 172.21.15.16:0/1453751285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7274101950 0x7f72741081e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f726c009ca0 tx=0x7f726c02a8c0 comp rx=0 tx=0).stop 2024-09-13T16:30:57.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.420+0000 7f727c1c7640 1 -- 172.21.15.16:0/1453751285 shutdown_connections 2024-09-13T16:30:57.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.420+0000 7f727c1c7640 1 --2- 172.21.15.16:0/1453751285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7274101950 0x7f72741081e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:57.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.420+0000 7f727c1c7640 1 --2- 172.21.15.16:0/1453751285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7274100f90 0x7f7274101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.420+0000 7f727c1c7640 1 --2- 172.21.15.16:0/1453751285 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72740ffda0 0x7f72741001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.420+0000 7f727c1c7640 1 -- 172.21.15.16:0/1453751285 >> 172.21.15.16:0/1453751285 conn(0x7f72740fb520 msgr2=0x7f72740fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.420+0000 7f727c1c7640 1 -- 172.21.15.16:0/1453751285 shutdown_connections 2024-09-13T16:30:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.420+0000 7f727c1c7640 1 -- 172.21.15.16:0/1453751285 wait complete. 2024-09-13T16:30:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.421+0000 7f727c1c7640 1 Processor -- start 2024-09-13T16:30:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.421+0000 7f727c1c7640 1 -- start start 2024-09-13T16:30:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.421+0000 7f727c1c7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72740ffda0 0x7f7274195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.421+0000 7f727c1c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7274100f90 0x7f7274196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.421+0000 7f727c1c7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7274101950 0x7f727419d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.421+0000 7f727c1c7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727419d730 con 0x7f7274100f90 2024-09-13T16:30:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.421+0000 7f727c1c7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727419d8a0 con 0x7f7274101950 2024-09-13T16:30:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.421+0000 7f727c1c7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727419da60 con 0x7f72740ffda0 2024-09-13T16:30:57.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.421+0000 7f7279762640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7274100f90 0x7f7274196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:57.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.421+0000 7f7279762640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7274100f90 0x7f7274196190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40526/0 (socket says 172.21.15.16:40526) 2024-09-13T16:30:57.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.421+0000 7f7279762640 1 -- 172.21.15.16:0/1679783263 learned_addr learned my addr 172.21.15.16:0/1679783263 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:57.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.421+0000 7f727a764640 1 --2- 172.21.15.16:0/1679783263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7274101950 0x7f727419d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:57.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.421+0000 7f7279f63640 1 --2- 172.21.15.16:0/1679783263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72740ffda0 0x7f7274195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:57.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.421+0000 7f7279762640 1 -- 172.21.15.16:0/1679783263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72740ffda0 msgr2=0x7f7274195c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:57.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.421+0000 7f7279762640 1 --2- 172.21.15.16:0/1679783263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72740ffda0 0x7f7274195c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:57.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.421+0000 7f7279762640 1 -- 172.21.15.16:0/1679783263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7274101950 msgr2=0x7f727419d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:57.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.421+0000 7f7279762640 1 --2- 172.21.15.16:0/1679783263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7274101950 0x7f727419d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:57.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.421+0000 7f7279762640 1 -- 172.21.15.16:0/1679783263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726c009900 con 0x7f7274100f90 2024-09-13T16:30:57.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.421+0000 7f727a764640 1 --2- 172.21.15.16:0/1679783263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7274101950 0x7f727419d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:30:57.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.421+0000 7f7279762640 1 --2- 172.21.15.16:0/1679783263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7274100f90 0x7f7274196190 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f726400e9f0 tx=0x7f726400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:57.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.422+0000 7f726affd640 1 -- 172.21.15.16:0/1679783263 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7264009700 con 0x7f7274100f90 2024-09-13T16:30:57.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.422+0000 7f726affd640 1 -- 172.21.15.16:0/1679783263 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f726400cd80 con 0x7f7274100f90 2024-09-13T16:30:57.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.422+0000 7f727c1c7640 1 -- 172.21.15.16:0/1679783263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f727419dd40 con 0x7f7274100f90 2024-09-13T16:30:57.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.422+0000 7f726affd640 1 -- 172.21.15.16:0/1679783263 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72640187e0 con 0x7f7274100f90 2024-09-13T16:30:57.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.422+0000 7f727c1c7640 1 -- 172.21.15.16:0/1679783263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f727419e1a0 con 0x7f7274100f90 2024-09-13T16:30:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.423+0000 7f727c1c7640 1 -- 172.21.15.16:0/1679783263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7274107230 con 0x7f7274100f90 2024-09-13T16:30:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.424+0000 7f726affd640 1 -- 172.21.15.16:0/1679783263 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7264010430 con 0x7f7274100f90 2024-09-13T16:30:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.424+0000 7f726affd640 1 --2- 172.21.15.16:0/1679783263 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f72600a1090 0x7f72600a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.425+0000 7f7279f63640 1 --2- 172.21.15.16:0/1679783263 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f72600a1090 0x7f72600a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.425+0000 7f726affd640 1 -- 172.21.15.16:0/1679783263 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f7264014070 con 0x7f7274100f90 2024-09-13T16:30:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.425+0000 7f7279f63640 1 --2- 172.21.15.16:0/1679783263 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f72600a1090 0x7f72600a3550 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7274197120 tx=0x7f7270005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:57.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.428+0000 7f726affd640 1 -- 172.21.15.16:0/1679783263 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7264052e20 con 0x7f7274100f90 2024-09-13T16:30:57.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.602+0000 7f727c1c7640 1 -- 172.21.15.16:0/1679783263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f727402c480 con 0x7f7274100f90 2024-09-13T16:30:57.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.602+0000 7f726affd640 1 -- 172.21.15.16:0/1679783263 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f72640964e0 con 0x7f7274100f90 2024-09-13T16:30:57.604 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-13T16:30:57.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.604+0000 7f727c1c7640 1 -- 172.21.15.16:0/1679783263 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f72600a1090 msgr2=0x7f72600a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:57.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.604+0000 7f727c1c7640 1 --2- 172.21.15.16:0/1679783263 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f72600a1090 0x7f72600a3550 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7274197120 tx=0x7f7270005f90 comp rx=0 tx=0).stop 2024-09-13T16:30:57.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.604+0000 7f727c1c7640 1 -- 172.21.15.16:0/1679783263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7274100f90 msgr2=0x7f7274196190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:57.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.604+0000 7f727c1c7640 1 --2- 172.21.15.16:0/1679783263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7274100f90 0x7f7274196190 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f726400e9f0 tx=0x7f726400eeb0 comp rx=0 tx=0).stop 2024-09-13T16:30:57.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.605+0000 7f727c1c7640 1 -- 172.21.15.16:0/1679783263 shutdown_connections 2024-09-13T16:30:57.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.605+0000 7f727c1c7640 1 --2- 172.21.15.16:0/1679783263 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f72600a1090 0x7f72600a3550 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:57.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.605+0000 7f727c1c7640 1 --2- 172.21.15.16:0/1679783263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7274101950 0x7f727419d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:57.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.605+0000 7f727c1c7640 1 --2- 172.21.15.16:0/1679783263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7274100f90 0x7f7274196190 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:57.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.605+0000 7f727c1c7640 1 --2- 172.21.15.16:0/1679783263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f72740ffda0 0x7f7274195c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:57.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.605+0000 7f727c1c7640 1 -- 172.21.15.16:0/1679783263 >> 172.21.15.16:0/1679783263 conn(0x7f72740fb520 msgr2=0x7f7274104400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:57.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.605+0000 7f727c1c7640 1 -- 172.21.15.16:0/1679783263 shutdown_connections 2024-09-13T16:30:57.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.605+0000 7f727c1c7640 1 -- 172.21.15.16:0/1679783263 wait complete. 2024-09-13T16:30:57.618 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T16:30:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.721+0000 7efc7bf26640 1 -- 172.21.15.16:0/3736794818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc740fe600 msgr2=0x7efc74100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.721+0000 7efc7bf26640 1 --2- 172.21.15.16:0/3736794818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc740fe600 0x7efc74100a10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7efc70009900 tx=0x7efc7002a410 comp rx=0 tx=0).stop 2024-09-13T16:30:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.721+0000 7efc7bf26640 1 -- 172.21.15.16:0/3736794818 shutdown_connections 2024-09-13T16:30:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.722+0000 7efc7bf26640 1 --2- 172.21.15.16:0/3736794818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc74103900 0x7efc74103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.722+0000 7efc7bf26640 1 --2- 172.21.15.16:0/3736794818 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc74100f50 0x7efc741033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.722+0000 7efc7bf26640 1 --2- 172.21.15.16:0/3736794818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc740fe600 0x7efc74100a10 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.722+0000 7efc7bf26640 1 -- 172.21.15.16:0/3736794818 >> 172.21.15.16:0/3736794818 conn(0x7efc740fa3f0 msgr2=0x7efc740fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.722+0000 7efc7bf26640 1 -- 172.21.15.16:0/3736794818 shutdown_connections 2024-09-13T16:30:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.722+0000 7efc7bf26640 1 -- 172.21.15.16:0/3736794818 wait complete. 2024-09-13T16:30:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.722+0000 7efc7bf26640 1 Processor -- start 2024-09-13T16:30:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.722+0000 7efc7bf26640 1 -- start start 2024-09-13T16:30:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.722+0000 7efc7bf26640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc740fe600 0x7efc7419c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.722+0000 7efc7bf26640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc74100f50 0x7efc7419c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.722+0000 7efc7bf26640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc74103900 0x7efc741a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.722+0000 7efc7bf26640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc741a3d00 con 0x7efc740fe600 2024-09-13T16:30:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.722+0000 7efc7bf26640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc741a3e70 con 0x7efc74100f50 2024-09-13T16:30:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.722+0000 7efc7bf26640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc741a4030 con 0x7efc74103900 2024-09-13T16:30:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.723+0000 7efc79cc2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc740fe600 0x7efc7419c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.723+0000 7efc79cc2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc740fe600 0x7efc7419c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40542/0 (socket says 172.21.15.16:40542) 2024-09-13T16:30:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.723+0000 7efc79cc2640 1 -- 172.21.15.16:0/734300677 learned_addr learned my addr 172.21.15.16:0/734300677 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.723+0000 7efc794c1640 1 --2- 172.21.15.16:0/734300677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc74100f50 0x7efc7419c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.723+0000 7efc7a4c3640 1 --2- 172.21.15.16:0/734300677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc74103900 0x7efc741a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.723+0000 7efc79cc2640 1 -- 172.21.15.16:0/734300677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc74103900 msgr2=0x7efc741a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.723+0000 7efc79cc2640 1 --2- 172.21.15.16:0/734300677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc74103900 0x7efc741a37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:57.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.723+0000 7efc79cc2640 1 -- 172.21.15.16:0/734300677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc74100f50 msgr2=0x7efc7419c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:57.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.723+0000 7efc79cc2640 1 --2- 172.21.15.16:0/734300677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc74100f50 0x7efc7419c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:57.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.723+0000 7efc79cc2640 1 -- 172.21.15.16:0/734300677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc70009560 con 0x7efc740fe600 2024-09-13T16:30:57.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.723+0000 7efc79cc2640 1 --2- 172.21.15.16:0/734300677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc740fe600 0x7efc7419c210 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7efc700057c0 tx=0x7efc700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:57.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.723+0000 7efc6affd640 1 -- 172.21.15.16:0/734300677 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc70031d70 con 0x7efc740fe600 2024-09-13T16:30:57.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.723+0000 7efc6affd640 1 -- 172.21.15.16:0/734300677 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efc700044e0 con 0x7efc740fe600 2024-09-13T16:30:57.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.723+0000 7efc7bf26640 1 -- 172.21.15.16:0/734300677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc741a42b0 con 0x7efc740fe600 2024-09-13T16:30:57.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.723+0000 7efc6affd640 1 -- 172.21.15.16:0/734300677 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc70003a80 con 0x7efc740fe600 2024-09-13T16:30:57.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.723+0000 7efc7bf26640 1 -- 172.21.15.16:0/734300677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc741a47b0 con 0x7efc740fe600 2024-09-13T16:30:57.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.724+0000 7efc7bf26640 1 -- 172.21.15.16:0/734300677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc740fbee0 con 0x7efc740fe600 2024-09-13T16:30:57.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.726+0000 7efc6affd640 1 -- 172.21.15.16:0/734300677 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7efc70003be0 con 0x7efc740fe600 2024-09-13T16:30:57.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.726+0000 7efc6affd640 1 --2- 172.21.15.16:0/734300677 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7efc600a1110 0x7efc600a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:57.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.726+0000 7efc6affd640 1 -- 172.21.15.16:0/734300677 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7efc700ee010 con 0x7efc740fe600 2024-09-13T16:30:57.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.726+0000 7efc794c1640 1 --2- 172.21.15.16:0/734300677 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7efc600a1110 0x7efc600a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:57.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.727+0000 7efc794c1640 1 --2- 172.21.15.16:0/734300677 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7efc600a1110 0x7efc600a35d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efc7419d720 tx=0x7efc6400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:57.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.730+0000 7efc6affd640 1 -- 172.21.15.16:0/734300677 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7efc700b7450 con 0x7efc740fe600 2024-09-13T16:30:57.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.902+0000 7efc7bf26640 1 -- 172.21.15.16:0/734300677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efc7402c3e0 con 0x7efc740fe600 2024-09-13T16:30:57.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.903+0000 7efc6affd640 1 -- 172.21.15.16:0/734300677 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7efc7004a020 con 0x7efc740fe600 2024-09-13T16:30:57.905 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-09-13T16:30:57.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.905+0000 7efc7bf26640 1 -- 172.21.15.16:0/734300677 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7efc600a1110 msgr2=0x7efc600a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:57.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.905+0000 7efc7bf26640 1 --2- 172.21.15.16:0/734300677 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7efc600a1110 0x7efc600a35d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efc7419d720 tx=0x7efc6400a2d0 comp rx=0 tx=0).stop 2024-09-13T16:30:57.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.905+0000 7efc7bf26640 1 -- 172.21.15.16:0/734300677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc740fe600 msgr2=0x7efc7419c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:57.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.905+0000 7efc7bf26640 1 --2- 172.21.15.16:0/734300677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc740fe600 0x7efc7419c210 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7efc700057c0 tx=0x7efc700042f0 comp rx=0 tx=0).stop 2024-09-13T16:30:57.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.905+0000 7efc7bf26640 1 -- 172.21.15.16:0/734300677 shutdown_connections 2024-09-13T16:30:57.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.905+0000 7efc7bf26640 1 --2- 172.21.15.16:0/734300677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc74103900 0x7efc741a37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:57.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.905+0000 7efc7bf26640 1 --2- 172.21.15.16:0/734300677 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7efc600a1110 0x7efc600a35d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:57.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.905+0000 7efc7bf26640 1 --2- 172.21.15.16:0/734300677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc74100f50 0x7efc7419c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:57.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.905+0000 7efc7bf26640 1 --2- 172.21.15.16:0/734300677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc740fe600 0x7efc7419c210 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:57.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.905+0000 7efc7bf26640 1 -- 172.21.15.16:0/734300677 >> 172.21.15.16:0/734300677 conn(0x7efc740fa3f0 msgr2=0x7efc741019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:57.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.905+0000 7efc7bf26640 1 -- 172.21.15.16:0/734300677 shutdown_connections 2024-09-13T16:30:57.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:57.905+0000 7efc7bf26640 1 -- 172.21.15.16:0/734300677 wait complete. 2024-09-13T16:30:57.916 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:57 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/565342985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:30:57.917 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:57 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1679783263' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T16:30:57.918 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T16:30:58.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.021+0000 7fd572261640 1 -- 172.21.15.16:0/3645215788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd56c0ffe60 msgr2=0x7fd56c1002e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:58.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.021+0000 7fd572261640 1 --2- 172.21.15.16:0/3645215788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd56c0ffe60 0x7fd56c1002e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd560009900 tx=0x7fd56002a410 comp rx=0 tx=0).stop 2024-09-13T16:30:58.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.021+0000 7fd572261640 1 -- 172.21.15.16:0/3645215788 shutdown_connections 2024-09-13T16:30:58.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.021+0000 7fd572261640 1 --2- 172.21.15.16:0/3645215788 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd56c100820 0x7fd56c1070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.021+0000 7fd572261640 1 --2- 172.21.15.16:0/3645215788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd56c0ffe60 0x7fd56c1002e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.021+0000 7fd572261640 1 --2- 172.21.15.16:0/3645215788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd56c0fec70 0x7fd56c0ff070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.021+0000 7fd572261640 1 -- 172.21.15.16:0/3645215788 >> 172.21.15.16:0/3645215788 conn(0x7fd56c0fa3d0 msgr2=0x7fd56c0fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:58.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.021+0000 7fd572261640 1 -- 172.21.15.16:0/3645215788 shutdown_connections 2024-09-13T16:30:58.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.021+0000 7fd572261640 1 -- 172.21.15.16:0/3645215788 wait complete. 2024-09-13T16:30:58.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.022+0000 7fd572261640 1 Processor -- start 2024-09-13T16:30:58.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.022+0000 7fd572261640 1 -- start start 2024-09-13T16:30:58.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.022+0000 7fd572261640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd56c0fec70 0x7fd56c195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:58.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.022+0000 7fd572261640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd56c0ffe60 0x7fd56c196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:58.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.022+0000 7fd572261640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd56c100820 0x7fd56c19d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:58.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.022+0000 7fd572261640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c19d7d0 con 0x7fd56c0ffe60 2024-09-13T16:30:58.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.022+0000 7fd572261640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c19d940 con 0x7fd56c0fec70 2024-09-13T16:30:58.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.022+0000 7fd572261640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c19db00 con 0x7fd56c100820 2024-09-13T16:30:58.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.022+0000 7fd56affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd56c0ffe60 0x7fd56c196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:58.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.023+0000 7fd56affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd56c0ffe60 0x7fd56c196200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40558/0 (socket says 172.21.15.16:40558) 2024-09-13T16:30:58.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.023+0000 7fd56affd640 1 -- 172.21.15.16:0/3640727750 learned_addr learned my addr 172.21.15.16:0/3640727750 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:58.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.023+0000 7fd56b7fe640 1 --2- 172.21.15.16:0/3640727750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd56c0fec70 0x7fd56c195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:58.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.023+0000 7fd56bfff640 1 --2- 172.21.15.16:0/3640727750 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd56c100820 0x7fd56c19d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:58.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.023+0000 7fd56affd640 1 -- 172.21.15.16:0/3640727750 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd56c100820 msgr2=0x7fd56c19d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:58.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.023+0000 7fd56affd640 1 --2- 172.21.15.16:0/3640727750 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd56c100820 0x7fd56c19d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.023+0000 7fd56affd640 1 -- 172.21.15.16:0/3640727750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd56c0fec70 msgr2=0x7fd56c195cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:58.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.023+0000 7fd56affd640 1 --2- 172.21.15.16:0/3640727750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd56c0fec70 0x7fd56c195cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.023+0000 7fd56affd640 1 -- 172.21.15.16:0/3640727750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd560009560 con 0x7fd56c0ffe60 2024-09-13T16:30:58.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.023+0000 7fd56bfff640 1 --2- 172.21.15.16:0/3640727750 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd56c100820 0x7fd56c19d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:30:58.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.023+0000 7fd56affd640 1 --2- 172.21.15.16:0/3640727750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd56c0ffe60 0x7fd56c196200 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd5600057c0 tx=0x7fd5600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:58.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.023+0000 7fd568ff9640 1 -- 172.21.15.16:0/3640727750 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd560031d70 con 0x7fd56c0ffe60 2024-09-13T16:30:58.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.023+0000 7fd568ff9640 1 -- 172.21.15.16:0/3640727750 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd5600044e0 con 0x7fd56c0ffe60 2024-09-13T16:30:58.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.023+0000 7fd572261640 1 -- 172.21.15.16:0/3640727750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd56c19dd80 con 0x7fd56c0ffe60 2024-09-13T16:30:58.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.023+0000 7fd568ff9640 1 -- 172.21.15.16:0/3640727750 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd560003b80 con 0x7fd56c0ffe60 2024-09-13T16:30:58.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.023+0000 7fd572261640 1 -- 172.21.15.16:0/3640727750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56c19e1d0 con 0x7fd56c0ffe60 2024-09-13T16:30:58.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.024+0000 7fd572261640 1 -- 172.21.15.16:0/3640727750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd56c10a3c0 con 0x7fd56c0ffe60 2024-09-13T16:30:58.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.026+0000 7fd568ff9640 1 -- 172.21.15.16:0/3640727750 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd560003ce0 con 0x7fd56c0ffe60 2024-09-13T16:30:58.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.026+0000 7fd568ff9640 1 --2- 172.21.15.16:0/3640727750 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd5540a1090 0x7fd5540a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:58.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.026+0000 7fd56b7fe640 1 --2- 172.21.15.16:0/3640727750 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd5540a1090 0x7fd5540a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:58.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.026+0000 7fd568ff9640 1 -- 172.21.15.16:0/3640727750 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd5600ee280 con 0x7fd56c0ffe60 2024-09-13T16:30:58.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.027+0000 7fd56b7fe640 1 --2- 172.21.15.16:0/3640727750 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd5540a1090 0x7fd5540a3550 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd56c196b90 tx=0x7fd55c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:58.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.029+0000 7fd568ff9640 1 -- 172.21.15.16:0/3640727750 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd5600b76c0 con 0x7fd56c0ffe60 2024-09-13T16:30:58.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:57 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/565342985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:30:58.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:57 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1679783263' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T16:30:58.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:57 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/565342985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:30:58.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:57 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1679783263' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T16:30:58.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.204+0000 7fd572261640 1 -- 172.21.15.16:0/3640727750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd56c19e3c0 con 0x7fd56c0ffe60 2024-09-13T16:30:58.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.204+0000 7fd568ff9640 1 -- 172.21.15.16:0/3640727750 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd5600bace0 con 0x7fd56c0ffe60 2024-09-13T16:30:58.206 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-13T16:30:58.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.206+0000 7fd572261640 1 -- 172.21.15.16:0/3640727750 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd5540a1090 msgr2=0x7fd5540a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:58.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.206+0000 7fd572261640 1 --2- 172.21.15.16:0/3640727750 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd5540a1090 0x7fd5540a3550 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd56c196b90 tx=0x7fd55c00b390 comp rx=0 tx=0).stop 2024-09-13T16:30:58.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.206+0000 7fd572261640 1 -- 172.21.15.16:0/3640727750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd56c0ffe60 msgr2=0x7fd56c196200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:58.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.206+0000 7fd572261640 1 --2- 172.21.15.16:0/3640727750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd56c0ffe60 0x7fd56c196200 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd5600057c0 tx=0x7fd5600042f0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.206+0000 7fd572261640 1 -- 172.21.15.16:0/3640727750 shutdown_connections 2024-09-13T16:30:58.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.206+0000 7fd572261640 1 --2- 172.21.15.16:0/3640727750 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd56c100820 0x7fd56c19d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.206+0000 7fd572261640 1 --2- 172.21.15.16:0/3640727750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd56c0ffe60 0x7fd56c196200 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.206+0000 7fd572261640 1 --2- 172.21.15.16:0/3640727750 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd5540a1090 0x7fd5540a3550 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.206+0000 7fd572261640 1 --2- 172.21.15.16:0/3640727750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd56c0fec70 0x7fd56c195cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.206+0000 7fd572261640 1 -- 172.21.15.16:0/3640727750 >> 172.21.15.16:0/3640727750 conn(0x7fd56c0fa3d0 msgr2=0x7fd56c068620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:58.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.206+0000 7fd572261640 1 -- 172.21.15.16:0/3640727750 shutdown_connections 2024-09-13T16:30:58.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.206+0000 7fd572261640 1 -- 172.21.15.16:0/3640727750 wait complete. 2024-09-13T16:30:58.220 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:30:58.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.322+0000 7ff123f18640 1 -- 172.21.15.16:0/1955356155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff11c1031f0 msgr2=0x7ff11c1055e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:58.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.322+0000 7ff123f18640 1 --2- 172.21.15.16:0/1955356155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff11c1031f0 0x7ff11c1055e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff10c009900 tx=0x7ff10c02a410 comp rx=0 tx=0).stop 2024-09-13T16:30:58.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.323+0000 7ff123f18640 1 -- 172.21.15.16:0/1955356155 shutdown_connections 2024-09-13T16:30:58.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.323+0000 7ff123f18640 1 --2- 172.21.15.16:0/1955356155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff11c105b20 0x7ff11c107f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.323+0000 7ff123f18640 1 --2- 172.21.15.16:0/1955356155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff11c1031f0 0x7ff11c1055e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.323+0000 7ff123f18640 1 --2- 172.21.15.16:0/1955356155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff11c1008c0 0x7ff11c102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.323+0000 7ff123f18640 1 -- 172.21.15.16:0/1955356155 >> 172.21.15.16:0/1955356155 conn(0x7ff11c0fa440 msgr2=0x7ff11c0fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:58.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.323+0000 7ff123f18640 1 -- 172.21.15.16:0/1955356155 shutdown_connections 2024-09-13T16:30:58.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.323+0000 7ff123f18640 1 -- 172.21.15.16:0/1955356155 wait complete. 2024-09-13T16:30:58.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.323+0000 7ff123f18640 1 Processor -- start 2024-09-13T16:30:58.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.324+0000 7ff123f18640 1 -- start start 2024-09-13T16:30:58.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.324+0000 7ff123f18640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff11c1008c0 0x7ff11c195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:58.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.324+0000 7ff123f18640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff11c1031f0 0x7ff11c196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:58.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.324+0000 7ff123f18640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff11c105b20 0x7ff11c19d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:58.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.324+0000 7ff123f18640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11c19d7f0 con 0x7ff11c1008c0 2024-09-13T16:30:58.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.324+0000 7ff123f18640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11c19d960 con 0x7ff11c105b20 2024-09-13T16:30:58.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.324+0000 7ff123f18640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11c19db20 con 0x7ff11c1031f0 2024-09-13T16:30:58.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.324+0000 7ff121cb4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff11c1008c0 0x7ff11c195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:58.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.324+0000 7ff121cb4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff11c1008c0 0x7ff11c195d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40576/0 (socket says 172.21.15.16:40576) 2024-09-13T16:30:58.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.324+0000 7ff121cb4640 1 -- 172.21.15.16:0/3708252689 learned_addr learned my addr 172.21.15.16:0/3708252689 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:58.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.324+0000 7ff1224b5640 1 --2- 172.21.15.16:0/3708252689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff11c105b20 0x7ff11c19d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:58.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.324+0000 7ff1214b3640 1 --2- 172.21.15.16:0/3708252689 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff11c1031f0 0x7ff11c196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:58.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.324+0000 7ff121cb4640 1 -- 172.21.15.16:0/3708252689 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff11c1031f0 msgr2=0x7ff11c196250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:58.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.324+0000 7ff121cb4640 1 --2- 172.21.15.16:0/3708252689 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff11c1031f0 0x7ff11c196250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.324+0000 7ff121cb4640 1 -- 172.21.15.16:0/3708252689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff11c105b20 msgr2=0x7ff11c19d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:58.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.324+0000 7ff121cb4640 1 --2- 172.21.15.16:0/3708252689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff11c105b20 0x7ff11c19d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.324+0000 7ff121cb4640 1 -- 172.21.15.16:0/3708252689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff10c009560 con 0x7ff11c1008c0 2024-09-13T16:30:58.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.324+0000 7ff1224b5640 1 --2- 172.21.15.16:0/3708252689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff11c105b20 0x7ff11c19d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:30:58.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.324+0000 7ff121cb4640 1 --2- 172.21.15.16:0/3708252689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff11c1008c0 0x7ff11c195d10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff11800d950 tx=0x7ff11800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:58.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.325+0000 7ff112ffd640 1 -- 172.21.15.16:0/3708252689 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff118009a30 con 0x7ff11c1008c0 2024-09-13T16:30:58.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.325+0000 7ff112ffd640 1 -- 172.21.15.16:0/3708252689 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff1180044b0 con 0x7ff11c1008c0 2024-09-13T16:30:58.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.325+0000 7ff123f18640 1 -- 172.21.15.16:0/3708252689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff11c19de00 con 0x7ff11c1008c0 2024-09-13T16:30:58.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.325+0000 7ff112ffd640 1 -- 172.21.15.16:0/3708252689 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1180047a0 con 0x7ff11c1008c0 2024-09-13T16:30:58.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.325+0000 7ff123f18640 1 -- 172.21.15.16:0/3708252689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff11c19e280 con 0x7ff11c1008c0 2024-09-13T16:30:58.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.326+0000 7ff123f18640 1 -- 172.21.15.16:0/3708252689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff11c0fbf30 con 0x7ff11c1008c0 2024-09-13T16:30:58.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.327+0000 7ff112ffd640 1 -- 172.21.15.16:0/3708252689 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff118004020 con 0x7ff11c1008c0 2024-09-13T16:30:58.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.328+0000 7ff112ffd640 1 --2- 172.21.15.16:0/3708252689 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff1080a1110 0x7ff1080a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:58.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.328+0000 7ff1214b3640 1 --2- 172.21.15.16:0/3708252689 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff1080a1110 0x7ff1080a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:58.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.328+0000 7ff112ffd640 1 -- 172.21.15.16:0/3708252689 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7ff1180ccdc0 con 0x7ff11c1008c0 2024-09-13T16:30:58.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.329+0000 7ff1214b3640 1 --2- 172.21.15.16:0/3708252689 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff1080a1110 0x7ff1080a35d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff10c006010 tx=0x7ff10c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:58.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.331+0000 7ff112ffd640 1 -- 172.21.15.16:0/3708252689 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff1180961f0 con 0x7ff11c1008c0 2024-09-13T16:30:58.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.540+0000 7ff123f18640 1 -- 172.21.15.16:0/3708252689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff11c10b500 con 0x7ff11c1008c0 2024-09-13T16:30:58.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.540+0000 7ff112ffd640 1 -- 172.21.15.16:0/3708252689 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10924 (secure 0 0 0) 0x7ff118017070 con 0x7ff11c1008c0 2024-09-13T16:30:58.542 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:30:58.543 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3292781664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3292781664},{"type":"v1","addr":"172.21.15.16:6835","nonce":3292781664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4109039027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4109039027},{"type":"v1","addr":"172.21.15.16:6837","nonce":4109039027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T16:28:39.184734+0000","modified":"2024-09-13T16:29:06.636937+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24518,"mds_1":24490,"mds_2":24485,"mds_3":24422,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/2391618693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2391618693},{"type":"v1","addr":"172.21.15.16:6839","nonce":2391618693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/521042681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":521042681},{"type":"v1","addr":"172.21.15.16:6841","nonce":521042681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24418":{"gid":24418,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.111:6833/1984275936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1984275936},{"type":"v1","addr":"172.21.15.111:6833","nonce":1984275936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.107:6833/3797712690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3797712690},{"type":"v1","addr":"172.21.15.107:6833","nonce":3797712690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/338548468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":338548468},{"type":"v1","addr":"172.21.15.107:6835","nonce":338548468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/662156111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":662156111},{"type":"v1","addr":"172.21.15.111:6835","nonce":662156111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/1877128453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1877128453},{"type":"v1","addr":"172.21.15.111:6837","nonce":1877128453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.107:6837/2838457611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2838457611},{"type":"v1","addr":"172.21.15.107:6837","nonce":2838457611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.111:6839/1423842285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":1423842285},{"type":"v1","addr":"172.21.15.111:6839","nonce":1423842285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6839/1914689806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1914689806},{"type":"v1","addr":"172.21.15.107:6839","nonce":1914689806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T16:30:58.543 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-13T16:30:58.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.542+0000 7ff123f18640 1 -- 172.21.15.16:0/3708252689 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff1080a1110 msgr2=0x7ff1080a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:58.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.542+0000 7ff123f18640 1 --2- 172.21.15.16:0/3708252689 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff1080a1110 0x7ff1080a35d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff10c006010 tx=0x7ff10c005a20 comp rx=0 tx=0).stop 2024-09-13T16:30:58.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.542+0000 7ff123f18640 1 -- 172.21.15.16:0/3708252689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff11c1008c0 msgr2=0x7ff11c195d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:58.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.542+0000 7ff123f18640 1 --2- 172.21.15.16:0/3708252689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff11c1008c0 0x7ff11c195d10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff11800d950 tx=0x7ff11800de10 comp rx=0 tx=0).stop 2024-09-13T16:30:58.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.542+0000 7ff123f18640 1 -- 172.21.15.16:0/3708252689 shutdown_connections 2024-09-13T16:30:58.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.542+0000 7ff123f18640 1 --2- 172.21.15.16:0/3708252689 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff1080a1110 0x7ff1080a35d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.542+0000 7ff123f18640 1 --2- 172.21.15.16:0/3708252689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff11c105b20 0x7ff11c19d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.542+0000 7ff123f18640 1 --2- 172.21.15.16:0/3708252689 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff11c1031f0 0x7ff11c196250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.542+0000 7ff123f18640 1 --2- 172.21.15.16:0/3708252689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff11c1008c0 0x7ff11c195d10 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.542+0000 7ff123f18640 1 -- 172.21.15.16:0/3708252689 >> 172.21.15.16:0/3708252689 conn(0x7ff11c0fa440 msgr2=0x7ff11c1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:58.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.542+0000 7ff123f18640 1 -- 172.21.15.16:0/3708252689 shutdown_connections 2024-09-13T16:30:58.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.542+0000 7ff123f18640 1 -- 172.21.15.16:0/3708252689 wait complete. 2024-09-13T16:30:58.556 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:30:58.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.659+0000 7f780faf7640 1 -- 172.21.15.16:0/3854958893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7808100850 msgr2=0x7f78081070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:58.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.659+0000 7f780faf7640 1 --2- 172.21.15.16:0/3854958893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7808100850 0x7f78081070e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7800009ca0 tx=0x7f780002a8c0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.659+0000 7f780faf7640 1 -- 172.21.15.16:0/3854958893 shutdown_connections 2024-09-13T16:30:58.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.659+0000 7f780faf7640 1 --2- 172.21.15.16:0/3854958893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7808100850 0x7f78081070e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.659+0000 7f780faf7640 1 --2- 172.21.15.16:0/3854958893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78080ffe90 0x7f7808100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.659+0000 7f780faf7640 1 --2- 172.21.15.16:0/3854958893 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78080feca0 0x7f78080ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.659+0000 7f780faf7640 1 -- 172.21.15.16:0/3854958893 >> 172.21.15.16:0/3854958893 conn(0x7f78080fa440 msgr2=0x7f78080fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:58.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.660+0000 7f780faf7640 1 -- 172.21.15.16:0/3854958893 shutdown_connections 2024-09-13T16:30:58.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.660+0000 7f780faf7640 1 -- 172.21.15.16:0/3854958893 wait complete. 2024-09-13T16:30:58.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.660+0000 7f780faf7640 1 Processor -- start 2024-09-13T16:30:58.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.660+0000 7f780faf7640 1 -- start start 2024-09-13T16:30:58.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.660+0000 7f780faf7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78080feca0 0x7f7808102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.660+0000 7f780faf7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78080ffe90 0x7f7808101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.660+0000 7f780faf7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7808100850 0x7f78081017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.660+0000 7f780faf7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7808101f10 con 0x7f7808100850 2024-09-13T16:30:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.660+0000 7f780faf7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7808102080 con 0x7f78080feca0 2024-09-13T16:30:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.660+0000 7f780faf7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780819fcc0 con 0x7f78080ffe90 2024-09-13T16:30:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.661+0000 7f780e094640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7808100850 0x7f78081017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.661+0000 7f780e094640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7808100850 0x7f78081017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40604/0 (socket says 172.21.15.16:40604) 2024-09-13T16:30:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.661+0000 7f780e094640 1 -- 172.21.15.16:0/2275889841 learned_addr learned my addr 172.21.15.16:0/2275889841 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.661+0000 7f780d893640 1 --2- 172.21.15.16:0/2275889841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78080feca0 0x7f7808102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.661+0000 7f780d092640 1 --2- 172.21.15.16:0/2275889841 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78080ffe90 0x7f7808101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.661+0000 7f780e094640 1 -- 172.21.15.16:0/2275889841 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78080ffe90 msgr2=0x7f7808101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.661+0000 7f780e094640 1 --2- 172.21.15.16:0/2275889841 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78080ffe90 0x7f7808101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.661+0000 7f780e094640 1 -- 172.21.15.16:0/2275889841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78080feca0 msgr2=0x7f7808102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.661+0000 7f780e094640 1 --2- 172.21.15.16:0/2275889841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78080feca0 0x7f7808102c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.661+0000 7f780e094640 1 -- 172.21.15.16:0/2275889841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7800009900 con 0x7f7808100850 2024-09-13T16:30:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.661+0000 7f780d893640 1 --2- 172.21.15.16:0/2275889841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78080feca0 0x7f7808102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:30:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.661+0000 7f780d092640 1 --2- 172.21.15.16:0/2275889841 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78080ffe90 0x7f7808101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:30:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.661+0000 7f780e094640 1 --2- 172.21.15.16:0/2275889841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7808100850 0x7f78081017b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7800009430 tx=0x7f7800003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.661+0000 7f77faffd640 1 -- 172.21.15.16:0/2275889841 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78000307c0 con 0x7f7808100850 2024-09-13T16:30:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.661+0000 7f780faf7640 1 -- 172.21.15.16:0/2275889841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f780819ff10 con 0x7f7808100850 2024-09-13T16:30:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.661+0000 7f77faffd640 1 -- 172.21.15.16:0/2275889841 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7800040070 con 0x7f7808100850 2024-09-13T16:30:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.661+0000 7f77faffd640 1 -- 172.21.15.16:0/2275889841 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78000043f0 con 0x7f7808100850 2024-09-13T16:30:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.661+0000 7f780faf7640 1 -- 172.21.15.16:0/2275889841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78081a0390 con 0x7f7808100850 2024-09-13T16:30:58.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.663+0000 7f780faf7640 1 -- 172.21.15.16:0/2275889841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f780810a410 con 0x7f7808100850 2024-09-13T16:30:58.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.663+0000 7f77faffd640 1 -- 172.21.15.16:0/2275889841 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7800042030 con 0x7f7808100850 2024-09-13T16:30:58.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.664+0000 7f77faffd640 1 --2- 172.21.15.16:0/2275889841 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f77f40a1090 0x7f77f40a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:58.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.664+0000 7f77faffd640 1 -- 172.21.15.16:0/2275889841 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f78000ee2f0 con 0x7f7808100850 2024-09-13T16:30:58.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.664+0000 7f780d893640 1 --2- 172.21.15.16:0/2275889841 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f77f40a1090 0x7f77f40a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:58.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.665+0000 7f780d893640 1 --2- 172.21.15.16:0/2275889841 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f77f40a1090 0x7f77f40a3550 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f78080ffcf0 tx=0x7f7804005e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:58.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.673+0000 7f77faffd640 1 -- 172.21.15.16:0/2275889841 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f78000b77d0 con 0x7f7808100850 2024-09-13T16:30:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.850+0000 7f780faf7640 1 -- 172.21.15.16:0/2275889841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78081a0690 con 0x7f7808100850 2024-09-13T16:30:58.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.851+0000 7f77faffd640 1 -- 172.21.15.16:0/2275889841 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24593 (secure 0 0 0) 0x7f78000badf0 con 0x7f7808100850 2024-09-13T16:30:58.853 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:30:58.854 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":93,"fsid":"0d241f6a-71ec-11ef-bceb-c7b262605968","created":"2024-09-13T16:21:17.971738+0000","modified":"2024-09-13T16:30:05.681521+0000","last_up_change":"2024-09-13T16:27:54.673413+0000","last_in_change":"2024-09-13T16:27:35.718390+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:24:22.981019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:28:37.746027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:28:38.602731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:28:41.633322+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e11ef8f-08d6-4e74-afd6-f49f4bd40032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6803","nonce":2079394186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6805","nonce":2079394186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6809","nonce":2079394186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6807","nonce":2079394186}]},"public_addr":"172.21.15.16:6803/2079394186","cluster_addr":"172.21.15.16:6805/2079394186","heartbeat_back_addr":"172.21.15.16:6809/2079394186","heartbeat_front_addr":"172.21.15.16:6807/2079394186","state":["exists","up"]},{"osd":1,"uuid":"f2cd3694-9c9e-4da7-976e-f80a8df120f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6801","nonce":4021360958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6803","nonce":4021360958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6807","nonce":4021360958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6805","nonce":4021360958}]},"public_addr":"172.21.15.107:6801/4021360958","cluster_addr":"172.21.15.107:6803/4021360958","heartbeat_back_addr":"172.21.15.107:6807/4021360958","heartbeat_front_addr":"172.21.15.107:6805/4021360958","state":["exists","up"]},{"osd":2,"uuid":"a5bbf2eb-6599-44d5-ad96-68bb89ee19f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6801","nonce":3227179825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6803","nonce":3227179825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6807","nonce":3227179825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6805","nonce":3227179825}]},"public_addr":"172.21.15.111:6801/3227179825","cluster_addr":"172.21.15.111:6803/3227179825","heartbeat_back_addr":"172.21.15.111:6807/3227179825","heartbeat_front_addr":"172.21.15.111:6805/3227179825","state":["exists","up"]},{"osd":3,"uuid":"d6295ffe-11bc-4964-a250-9fecca870b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6811","nonce":2390375937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6813","nonce":2390375937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6817","nonce":2390375937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6815","nonce":2390375937}]},"public_addr":"172.21.15.16:6811/2390375937","cluster_addr":"172.21.15.16:6813/2390375937","heartbeat_back_addr":"172.21.15.16:6817/2390375937","heartbeat_front_addr":"172.21.15.16:6815/2390375937","state":["exists","up"]},{"osd":4,"uuid":"0a8446e0-fc86-4646-beb6-a7b9d433093d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6809","nonce":1936582048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6811","nonce":1936582048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6815","nonce":1936582048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6813","nonce":1936582048}]},"public_addr":"172.21.15.107:6809/1936582048","cluster_addr":"172.21.15.107:6811/1936582048","heartbeat_back_addr":"172.21.15.107:6815/1936582048","heartbeat_front_addr":"172.21.15.107:6813/1936582048","state":["exists","up"]},{"osd":5,"uuid":"2c97a446-357e-4975-9e51-9c95fe84a663","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6809","nonce":483858448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6811","nonce":483858448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6815","nonce":483858448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6813","nonce":483858448}]},"public_addr":"172.21.15.111:6809/483858448","cluster_addr":"172.21.15.111:6811/483858448","heartbeat_back_addr":"172.21.15.111:6815/483858448","heartbeat_front_addr":"172.21.15.111:6813/483858448","state":["exists","up"]},{"osd":6,"uuid":"890239e6-e6fb-4f9c-b973-05cc66d199bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6819","nonce":1330214141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6821","nonce":1330214141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6825","nonce":1330214141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6823","nonce":1330214141}]},"public_addr":"172.21.15.16:6819/1330214141","cluster_addr":"172.21.15.16:6821/1330214141","heartbeat_back_addr":"172.21.15.16:6825/1330214141","heartbeat_front_addr":"172.21.15.16:6823/1330214141","state":["exists","up"]},{"osd":7,"uuid":"ee826a9c-f441-4903-9b1b-66b627f7b712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6817","nonce":1133876754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6819","nonce":1133876754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6823","nonce":1133876754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6821","nonce":1133876754}]},"public_addr":"172.21.15.107:6817/1133876754","cluster_addr":"172.21.15.107:6819/1133876754","heartbeat_back_addr":"172.21.15.107:6823/1133876754","heartbeat_front_addr":"172.21.15.107:6821/1133876754","state":["exists","up"]},{"osd":8,"uuid":"e19773f2-dec8-44e0-9d33-0252e07d6415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6817","nonce":837548190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6819","nonce":837548190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6823","nonce":837548190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6821","nonce":837548190}]},"public_addr":"172.21.15.111:6817/837548190","cluster_addr":"172.21.15.111:6819/837548190","heartbeat_back_addr":"172.21.15.111:6823/837548190","heartbeat_front_addr":"172.21.15.111:6821/837548190","state":["exists","up"]},{"osd":9,"uuid":"bae4d6db-8b90-4d02-8e83-7d8f2a6f8fdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6827","nonce":4175383686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6829","nonce":4175383686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6833","nonce":4175383686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6831","nonce":4175383686}]},"public_addr":"172.21.15.16:6827/4175383686","cluster_addr":"172.21.15.16:6829/4175383686","heartbeat_back_addr":"172.21.15.16:6833/4175383686","heartbeat_front_addr":"172.21.15.16:6831/4175383686","state":["exists","up"]},{"osd":10,"uuid":"fd0e7740-76c8-4c99-a349-7077b4905143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6825","nonce":2659550426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6827","nonce":2659550426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6831","nonce":2659550426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6829","nonce":2659550426}]},"public_addr":"172.21.15.107:6825/2659550426","cluster_addr":"172.21.15.107:6827/2659550426","heartbeat_back_addr":"172.21.15.107:6831/2659550426","heartbeat_front_addr":"172.21.15.107:6829/2659550426","state":["exists","up"]},{"osd":11,"uuid":"94a1751d-b00c-4e23-815d-3a8bc3dea146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6825","nonce":1633548683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6827","nonce":1633548683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6831","nonce":1633548683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6829","nonce":1633548683}]},"public_addr":"172.21.15.111:6825/1633548683","cluster_addr":"172.21.15.111:6827/1633548683","heartbeat_back_addr":"172.21.15.111:6831/1633548683","heartbeat_front_addr":"172.21.15.111:6829/1633548683","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:36.359852+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:58.014202+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:19.753672+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:43.404817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:06.001677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:29.361587+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:52.554847+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:15.957553+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:39.581093+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:04.911389+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:28.971280+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:53.473561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/2098378667":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/3536229016":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6801/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/2684687547":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6800/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/368410612":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/12158160":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6800/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6801/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/3754423379":"2024-09-14T16:21:40.126459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:30:58.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.853+0000 7f780faf7640 1 -- 172.21.15.16:0/2275889841 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f77f40a1090 msgr2=0x7f77f40a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:58.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.853+0000 7f780faf7640 1 --2- 172.21.15.16:0/2275889841 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f77f40a1090 0x7f77f40a3550 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f78080ffcf0 tx=0x7f7804005e70 comp rx=0 tx=0).stop 2024-09-13T16:30:58.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.853+0000 7f780faf7640 1 -- 172.21.15.16:0/2275889841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7808100850 msgr2=0x7f78081017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:58.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.853+0000 7f780faf7640 1 --2- 172.21.15.16:0/2275889841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7808100850 0x7f78081017b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7800009430 tx=0x7f7800003c20 comp rx=0 tx=0).stop 2024-09-13T16:30:58.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.853+0000 7f780faf7640 1 -- 172.21.15.16:0/2275889841 shutdown_connections 2024-09-13T16:30:58.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.853+0000 7f780faf7640 1 --2- 172.21.15.16:0/2275889841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7808100850 0x7f78081017b0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.853+0000 7f780faf7640 1 --2- 172.21.15.16:0/2275889841 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78080ffe90 0x7f7808101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.853+0000 7f780faf7640 1 --2- 172.21.15.16:0/2275889841 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f77f40a1090 0x7f77f40a3550 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.853+0000 7f780faf7640 1 --2- 172.21.15.16:0/2275889841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78080feca0 0x7f7808102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.853+0000 7f780faf7640 1 -- 172.21.15.16:0/2275889841 >> 172.21.15.16:0/2275889841 conn(0x7f78080fa440 msgr2=0x7f7808067690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:58.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.853+0000 7f780faf7640 1 -- 172.21.15.16:0/2275889841 shutdown_connections 2024-09-13T16:30:58.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.853+0000 7f780faf7640 1 -- 172.21.15.16:0/2275889841 wait complete. 2024-09-13T16:30:58.867 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:24:22.981019+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T16:28:37.746027+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T16:28:38.602731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T16:28:41.633322+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T16:30:58.867 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:30:58.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.972+0000 7f6d8ffdb640 1 -- 172.21.15.16:0/3352539838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d88101930 msgr2=0x7f6d881081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:58.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.972+0000 7f6d8ffdb640 1 --2- 172.21.15.16:0/3352539838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d88101930 0x7f6d881081c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009ca0 tx=0x7f6d8002a8c0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.972+0000 7f6d8ffdb640 1 -- 172.21.15.16:0/3352539838 shutdown_connections 2024-09-13T16:30:58.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.972+0000 7f6d8ffdb640 1 --2- 172.21.15.16:0/3352539838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d88101930 0x7f6d881081c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.972+0000 7f6d8ffdb640 1 --2- 172.21.15.16:0/3352539838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d88100f70 0x7f6d881013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.972+0000 7f6d8ffdb640 1 --2- 172.21.15.16:0/3352539838 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d880ffd80 0x7f6d88100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.972+0000 7f6d8ffdb640 1 -- 172.21.15.16:0/3352539838 >> 172.21.15.16:0/3352539838 conn(0x7f6d880fb4e0 msgr2=0x7f6d880fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:58.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.972+0000 7f6d8ffdb640 1 -- 172.21.15.16:0/3352539838 shutdown_connections 2024-09-13T16:30:58.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.972+0000 7f6d8ffdb640 1 -- 172.21.15.16:0/3352539838 wait complete. 2024-09-13T16:30:58.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.973+0000 7f6d8ffdb640 1 Processor -- start 2024-09-13T16:30:58.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.973+0000 7f6d8ffdb640 1 -- start start 2024-09-13T16:30:58.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.973+0000 7f6d8ffdb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d880ffd80 0x7f6d88195ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:58.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.973+0000 7f6d8ffdb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d88100f70 0x7f6d881961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:58.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.973+0000 7f6d8ffdb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d88101930 0x7f6d8819d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:58.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.973+0000 7f6d8ffdb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8819d7b0 con 0x7f6d880ffd80 2024-09-13T16:30:58.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.973+0000 7f6d8ffdb640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8819d920 con 0x7f6d88101930 2024-09-13T16:30:58.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.973+0000 7f6d8ffdb640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8819dae0 con 0x7f6d88100f70 2024-09-13T16:30:58.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.973+0000 7f6d8dd77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d880ffd80 0x7f6d88195ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:58.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.973+0000 7f6d8dd77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d880ffd80 0x7f6d88195ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40620/0 (socket says 172.21.15.16:40620) 2024-09-13T16:30:58.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.973+0000 7f6d8dd77640 1 -- 172.21.15.16:0/4096585515 learned_addr learned my addr 172.21.15.16:0/4096585515 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:58.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.973+0000 7f6d8d576640 1 --2- 172.21.15.16:0/4096585515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d88100f70 0x7f6d881961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:58.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.973+0000 7f6d8e578640 1 --2- 172.21.15.16:0/4096585515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d88101930 0x7f6d8819d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:58.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.973+0000 7f6d8dd77640 1 -- 172.21.15.16:0/4096585515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d88100f70 msgr2=0x7f6d881961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:58.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.973+0000 7f6d8dd77640 1 --2- 172.21.15.16:0/4096585515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d88100f70 0x7f6d881961e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.973+0000 7f6d8dd77640 1 -- 172.21.15.16:0/4096585515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d88101930 msgr2=0x7f6d8819d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:58.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.973+0000 7f6d8dd77640 1 --2- 172.21.15.16:0/4096585515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d88101930 0x7f6d8819d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:58.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.973+0000 7f6d8dd77640 1 -- 172.21.15.16:0/4096585515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d80009900 con 0x7f6d880ffd80 2024-09-13T16:30:58.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.973+0000 7f6d8dd77640 1 --2- 172.21.15.16:0/4096585515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d880ffd80 0x7f6d88195ca0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400e9c0 tx=0x7f6d8400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:58.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.974+0000 7f6d7effd640 1 -- 172.21.15.16:0/4096585515 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d84009700 con 0x7f6d880ffd80 2024-09-13T16:30:58.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.974+0000 7f6d7effd640 1 -- 172.21.15.16:0/4096585515 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d840044b0 con 0x7f6d880ffd80 2024-09-13T16:30:58.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.974+0000 7f6d8ffdb640 1 -- 172.21.15.16:0/4096585515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d8819ddc0 con 0x7f6d880ffd80 2024-09-13T16:30:58.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.974+0000 7f6d7effd640 1 -- 172.21.15.16:0/4096585515 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d84003e40 con 0x7f6d880ffd80 2024-09-13T16:30:58.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.974+0000 7f6d8ffdb640 1 -- 172.21.15.16:0/4096585515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d8819e210 con 0x7f6d880ffd80 2024-09-13T16:30:58.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.975+0000 7f6d8ffdb640 1 -- 172.21.15.16:0/4096585515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d88107230 con 0x7f6d880ffd80 2024-09-13T16:30:58.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.976+0000 7f6d7effd640 1 -- 172.21.15.16:0/4096585515 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f6d84003fe0 con 0x7f6d880ffd80 2024-09-13T16:30:58.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.977+0000 7f6d7effd640 1 --2- 172.21.15.16:0/4096585515 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f6d740a1090 0x7f6d740a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:58.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.977+0000 7f6d8d576640 1 --2- 172.21.15.16:0/4096585515 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f6d740a1090 0x7f6d740a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:58.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.977+0000 7f6d7effd640 1 -- 172.21.15.16:0/4096585515 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f6d84014070 con 0x7f6d880ffd80 2024-09-13T16:30:58.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.977+0000 7f6d8d576640 1 --2- 172.21.15.16:0/4096585515 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f6d740a1090 0x7f6d740a3550 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6d881971a0 tx=0x7f6d78005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:58.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:58.981+0000 7f6d7effd640 1 -- 172.21.15.16:0/4096585515 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f6d84096600 con 0x7f6d880ffd80 2024-09-13T16:30:59.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:58 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/734300677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T16:30:59.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:58 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3640727750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T16:30:59.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:58 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3708252689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:30:59.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:58 smithi107 ceph-mon[34139]: pgmap v350: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-13T16:30:59.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:58 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/734300677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T16:30:59.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:58 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3640727750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T16:30:59.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:58 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3708252689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:30:59.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:58 smithi016 ceph-mon[31187]: pgmap v350: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-13T16:30:59.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.153+0000 7f6d8ffdb640 1 -- 172.21.15.16:0/4096585515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6d8802c480 con 0x7f6d880ffd80 2024-09-13T16:30:59.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.154+0000 7f6d7effd640 1 -- 172.21.15.16:0/4096585515 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6d84099c20 con 0x7f6d880ffd80 2024-09-13T16:30:59.156 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-13T16:30:59.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.156+0000 7f6d8ffdb640 1 -- 172.21.15.16:0/4096585515 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f6d740a1090 msgr2=0x7f6d740a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:59.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.156+0000 7f6d8ffdb640 1 --2- 172.21.15.16:0/4096585515 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f6d740a1090 0x7f6d740a3550 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6d881971a0 tx=0x7f6d78005f90 comp rx=0 tx=0).stop 2024-09-13T16:30:59.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.156+0000 7f6d8ffdb640 1 -- 172.21.15.16:0/4096585515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d880ffd80 msgr2=0x7f6d88195ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:59.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.156+0000 7f6d8ffdb640 1 --2- 172.21.15.16:0/4096585515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d880ffd80 0x7f6d88195ca0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400e9c0 tx=0x7f6d8400ee80 comp rx=0 tx=0).stop 2024-09-13T16:30:59.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.156+0000 7f6d8ffdb640 1 -- 172.21.15.16:0/4096585515 shutdown_connections 2024-09-13T16:30:59.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.156+0000 7f6d8ffdb640 1 --2- 172.21.15.16:0/4096585515 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f6d740a1090 0x7f6d740a3550 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:59.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.156+0000 7f6d8ffdb640 1 --2- 172.21.15.16:0/4096585515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d88101930 0x7f6d8819d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:59.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.156+0000 7f6d8ffdb640 1 --2- 172.21.15.16:0/4096585515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d88100f70 0x7f6d881961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:59.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.156+0000 7f6d8ffdb640 1 --2- 172.21.15.16:0/4096585515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d880ffd80 0x7f6d88195ca0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:59.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.156+0000 7f6d8ffdb640 1 -- 172.21.15.16:0/4096585515 >> 172.21.15.16:0/4096585515 conn(0x7f6d880fb4e0 msgr2=0x7f6d88104400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:59.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.156+0000 7f6d8ffdb640 1 -- 172.21.15.16:0/4096585515 shutdown_connections 2024-09-13T16:30:59.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.156+0000 7f6d8ffdb640 1 -- 172.21.15.16:0/4096585515 wait complete. 2024-09-13T16:30:59.169 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T16:30:59.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:58 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/734300677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T16:30:59.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:58 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3640727750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T16:30:59.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:58 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3708252689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:30:59.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:58 smithi111 ceph-mon[34600]: pgmap v350: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-13T16:30:59.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.275+0000 7f439167d640 1 -- 172.21.15.16:0/3669151953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f438c0ffe50 msgr2=0x7f438c1002d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:59.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.275+0000 7f439167d640 1 --2- 172.21.15.16:0/3669151953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f438c0ffe50 0x7f438c1002d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4374009900 tx=0x7f437402a410 comp rx=0 tx=0).stop 2024-09-13T16:30:59.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.275+0000 7f439167d640 1 -- 172.21.15.16:0/3669151953 shutdown_connections 2024-09-13T16:30:59.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.275+0000 7f439167d640 1 --2- 172.21.15.16:0/3669151953 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f438c100810 0x7f438c1070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:59.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.275+0000 7f439167d640 1 --2- 172.21.15.16:0/3669151953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f438c0ffe50 0x7f438c1002d0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:59.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.275+0000 7f439167d640 1 --2- 172.21.15.16:0/3669151953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f438c0fec60 0x7f438c0ff060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:59.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.275+0000 7f439167d640 1 -- 172.21.15.16:0/3669151953 >> 172.21.15.16:0/3669151953 conn(0x7f438c0fa3e0 msgr2=0x7f438c0fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:59.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.275+0000 7f439167d640 1 -- 172.21.15.16:0/3669151953 shutdown_connections 2024-09-13T16:30:59.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.275+0000 7f439167d640 1 -- 172.21.15.16:0/3669151953 wait complete. 2024-09-13T16:30:59.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.275+0000 7f439167d640 1 Processor -- start 2024-09-13T16:30:59.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.275+0000 7f439167d640 1 -- start start 2024-09-13T16:30:59.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.276+0000 7f439167d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f438c0fec60 0x7f438c068100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:59.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.276+0000 7f439167d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f438c0ffe50 0x7f438c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:59.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.276+0000 7f439167d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f438c100810 0x7f438c19d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:59.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.276+0000 7f439167d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438c19d7e0 con 0x7f438c0fec60 2024-09-13T16:30:59.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.276+0000 7f439167d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438c19d950 con 0x7f438c0ffe50 2024-09-13T16:30:59.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.276+0000 7f439167d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438c19db10 con 0x7f438c100810 2024-09-13T16:30:59.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.276+0000 7f438affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f438c0fec60 0x7f438c068100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.276+0000 7f438affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f438c0fec60 0x7f438c068100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40626/0 (socket says 172.21.15.16:40626) 2024-09-13T16:30:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.276+0000 7f438affd640 1 -- 172.21.15.16:0/2407328232 learned_addr learned my addr 172.21.15.16:0/2407328232 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.276+0000 7f438b7fe640 1 --2- 172.21.15.16:0/2407328232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f438c100810 0x7f438c19d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.276+0000 7f438a7fc640 1 --2- 172.21.15.16:0/2407328232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f438c0ffe50 0x7f438c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.276+0000 7f438affd640 1 -- 172.21.15.16:0/2407328232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f438c100810 msgr2=0x7f438c19d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.276+0000 7f438affd640 1 --2- 172.21.15.16:0/2407328232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f438c100810 0x7f438c19d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.276+0000 7f438affd640 1 -- 172.21.15.16:0/2407328232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f438c0ffe50 msgr2=0x7f438c19a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.276+0000 7f438affd640 1 --2- 172.21.15.16:0/2407328232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f438c0ffe50 0x7f438c19a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.276+0000 7f438affd640 1 -- 172.21.15.16:0/2407328232 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4374009560 con 0x7f438c0fec60 2024-09-13T16:30:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.276+0000 7f438affd640 1 --2- 172.21.15.16:0/2407328232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f438c0fec60 0x7f438c068100 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f437c00d950 tx=0x7f437c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.276+0000 7f4383fff640 1 -- 172.21.15.16:0/2407328232 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437c009c30 con 0x7f438c0fec60 2024-09-13T16:30:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.277+0000 7f439167d640 1 -- 172.21.15.16:0/2407328232 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f438c19dd90 con 0x7f438c0fec60 2024-09-13T16:30:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.277+0000 7f4383fff640 1 -- 172.21.15.16:0/2407328232 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f437c009d90 con 0x7f438c0fec60 2024-09-13T16:30:59.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.277+0000 7f4383fff640 1 -- 172.21.15.16:0/2407328232 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437c00f6b0 con 0x7f438c0fec60 2024-09-13T16:30:59.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.277+0000 7f439167d640 1 -- 172.21.15.16:0/2407328232 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f438c19e210 con 0x7f438c0fec60 2024-09-13T16:30:59.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.277+0000 7f439167d640 1 -- 172.21.15.16:0/2407328232 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f438c106190 con 0x7f438c0fec60 2024-09-13T16:30:59.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.280+0000 7f4383fff640 1 -- 172.21.15.16:0/2407328232 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f437c010460 con 0x7f438c0fec60 2024-09-13T16:30:59.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.280+0000 7f4383fff640 1 --2- 172.21.15.16:0/2407328232 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f43780a1090 0x7f43780a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:59.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.280+0000 7f4383fff640 1 -- 172.21.15.16:0/2407328232 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f437c0ccc90 con 0x7f438c0fec60 2024-09-13T16:30:59.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.280+0000 7f438a7fc640 1 --2- 172.21.15.16:0/2407328232 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f43780a1090 0x7f43780a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:59.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.281+0000 7f438a7fc640 1 --2- 172.21.15.16:0/2407328232 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f43780a1090 0x7f43780a3550 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4374000c00 tx=0x7f4374005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:59.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.283+0000 7f4383fff640 1 -- 172.21.15.16:0/2407328232 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f437c0960d0 con 0x7f438c0fec60 2024-09-13T16:30:59.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.456+0000 7f439167d640 1 -- 172.21.15.16:0/2407328232 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f438c10a3d0 con 0x7f438c0fec60 2024-09-13T16:30:59.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.456+0000 7f4383fff640 1 -- 172.21.15.16:0/2407328232 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f437c010710 con 0x7f438c0fec60 2024-09-13T16:30:59.458 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-13T16:30:59.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.458+0000 7f439167d640 1 -- 172.21.15.16:0/2407328232 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f43780a1090 msgr2=0x7f43780a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:59.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.458+0000 7f439167d640 1 --2- 172.21.15.16:0/2407328232 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f43780a1090 0x7f43780a3550 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4374000c00 tx=0x7f4374005a40 comp rx=0 tx=0).stop 2024-09-13T16:30:59.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.458+0000 7f439167d640 1 -- 172.21.15.16:0/2407328232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f438c0fec60 msgr2=0x7f438c068100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:59.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.458+0000 7f439167d640 1 --2- 172.21.15.16:0/2407328232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f438c0fec60 0x7f438c068100 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f437c00d950 tx=0x7f437c00de10 comp rx=0 tx=0).stop 2024-09-13T16:30:59.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.458+0000 7f439167d640 1 -- 172.21.15.16:0/2407328232 shutdown_connections 2024-09-13T16:30:59.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.458+0000 7f439167d640 1 --2- 172.21.15.16:0/2407328232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f438c100810 0x7f438c19d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:59.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.458+0000 7f439167d640 1 --2- 172.21.15.16:0/2407328232 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f43780a1090 0x7f43780a3550 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:59.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.458+0000 7f439167d640 1 --2- 172.21.15.16:0/2407328232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f438c0ffe50 0x7f438c19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:59.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.458+0000 7f439167d640 1 --2- 172.21.15.16:0/2407328232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f438c0fec60 0x7f438c068100 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:59.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.458+0000 7f439167d640 1 -- 172.21.15.16:0/2407328232 >> 172.21.15.16:0/2407328232 conn(0x7f438c0fa3e0 msgr2=0x7f438c103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:59.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.458+0000 7f439167d640 1 -- 172.21.15.16:0/2407328232 shutdown_connections 2024-09-13T16:30:59.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.458+0000 7f439167d640 1 -- 172.21.15.16:0/2407328232 wait complete. 2024-09-13T16:30:59.471 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T16:30:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.575+0000 7ff68539c640 1 -- 172.21.15.16:0/901996993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6800feca0 msgr2=0x7ff6800ff0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.575+0000 7ff68539c640 1 --2- 172.21.15.16:0/901996993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6800feca0 0x7ff6800ff0a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff670009900 tx=0x7ff67002a410 comp rx=0 tx=0).stop 2024-09-13T16:30:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.575+0000 7ff68539c640 1 -- 172.21.15.16:0/901996993 shutdown_connections 2024-09-13T16:30:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.575+0000 7ff68539c640 1 --2- 172.21.15.16:0/901996993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff680100850 0x7ff6801070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.575+0000 7ff68539c640 1 --2- 172.21.15.16:0/901996993 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6800ffe90 0x7ff680100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.575+0000 7ff68539c640 1 --2- 172.21.15.16:0/901996993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6800feca0 0x7ff6800ff0a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.575+0000 7ff68539c640 1 -- 172.21.15.16:0/901996993 >> 172.21.15.16:0/901996993 conn(0x7ff6800fa420 msgr2=0x7ff6800fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.575+0000 7ff68539c640 1 -- 172.21.15.16:0/901996993 shutdown_connections 2024-09-13T16:30:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.575+0000 7ff68539c640 1 -- 172.21.15.16:0/901996993 wait complete. 2024-09-13T16:30:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.576+0000 7ff68539c640 1 Processor -- start 2024-09-13T16:30:59.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.576+0000 7ff68539c640 1 -- start start 2024-09-13T16:30:59.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.576+0000 7ff68539c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6800feca0 0x7ff680195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:59.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.576+0000 7ff68539c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6800ffe90 0x7ff680196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:59.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.576+0000 7ff68539c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff680100850 0x7ff68019d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:59.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.576+0000 7ff68539c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68019d8a0 con 0x7ff680100850 2024-09-13T16:30:59.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.576+0000 7ff68539c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68019da10 con 0x7ff6800feca0 2024-09-13T16:30:59.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.576+0000 7ff68539c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68019dbd0 con 0x7ff6800ffe90 2024-09-13T16:30:59.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.576+0000 7ff67f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff680100850 0x7ff68019d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:59.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.576+0000 7ff67f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff680100850 0x7ff68019d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40656/0 (socket says 172.21.15.16:40656) 2024-09-13T16:30:59.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.576+0000 7ff67f7fe640 1 -- 172.21.15.16:0/1814225868 learned_addr learned my addr 172.21.15.16:0/1814225868 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.576+0000 7ff67effd640 1 --2- 172.21.15.16:0/1814225868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6800feca0 0x7ff680195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.576+0000 7ff67e7fc640 1 --2- 172.21.15.16:0/1814225868 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6800ffe90 0x7ff680196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.577+0000 7ff67f7fe640 1 -- 172.21.15.16:0/1814225868 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6800ffe90 msgr2=0x7ff680196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.577+0000 7ff67f7fe640 1 --2- 172.21.15.16:0/1814225868 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6800ffe90 0x7ff680196300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.577+0000 7ff67f7fe640 1 -- 172.21.15.16:0/1814225868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6800feca0 msgr2=0x7ff680195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.577+0000 7ff67f7fe640 1 --2- 172.21.15.16:0/1814225868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6800feca0 0x7ff680195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.577+0000 7ff67f7fe640 1 -- 172.21.15.16:0/1814225868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff670009560 con 0x7ff680100850 2024-09-13T16:30:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.577+0000 7ff67effd640 1 --2- 172.21.15.16:0/1814225868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6800feca0 0x7ff680195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:30:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.577+0000 7ff67f7fe640 1 --2- 172.21.15.16:0/1814225868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff680100850 0x7ff68019d330 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff67800cad0 tx=0x7ff67800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.577+0000 7ff667fff640 1 -- 172.21.15.16:0/1814225868 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67800ae20 con 0x7ff680100850 2024-09-13T16:30:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.577+0000 7ff667fff640 1 -- 172.21.15.16:0/1814225868 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff678005020 con 0x7ff680100850 2024-09-13T16:30:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.577+0000 7ff68539c640 1 -- 172.21.15.16:0/1814225868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff68019deb0 con 0x7ff680100850 2024-09-13T16:30:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.577+0000 7ff667fff640 1 -- 172.21.15.16:0/1814225868 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67800f6b0 con 0x7ff680100850 2024-09-13T16:30:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.577+0000 7ff68539c640 1 -- 172.21.15.16:0/1814225868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff68019e280 con 0x7ff680100850 2024-09-13T16:30:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.578+0000 7ff68539c640 1 -- 172.21.15.16:0/1814225868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6800ff0a0 con 0x7ff680100850 2024-09-13T16:30:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.579+0000 7ff667fff640 1 -- 172.21.15.16:0/1814225868 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff678005190 con 0x7ff680100850 2024-09-13T16:30:59.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.580+0000 7ff667fff640 1 --2- 172.21.15.16:0/1814225868 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff6680a1090 0x7ff6680a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:59.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.580+0000 7ff67effd640 1 --2- 172.21.15.16:0/1814225868 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff6680a1090 0x7ff6680a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:59.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.580+0000 7ff667fff640 1 -- 172.21.15.16:0/1814225868 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7ff6780cce30 con 0x7ff680100850 2024-09-13T16:30:59.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.580+0000 7ff67effd640 1 --2- 172.21.15.16:0/1814225868 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff6680a1090 0x7ff6680a3550 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff67002aa00 tx=0x7ff670031470 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:59.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.584+0000 7ff667fff640 1 -- 172.21.15.16:0/1814225868 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff678096180 con 0x7ff680100850 2024-09-13T16:30:59.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.755+0000 7ff68539c640 1 -- 172.21.15.16:0/1814225868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff68019e750 con 0x7ff680100850 2024-09-13T16:30:59.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.756+0000 7ff667fff640 1 -- 172.21.15.16:0/1814225868 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff67801d020 con 0x7ff680100850 2024-09-13T16:30:59.758 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-09-13T16:30:59.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.757+0000 7ff68539c640 1 -- 172.21.15.16:0/1814225868 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff6680a1090 msgr2=0x7ff6680a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:59.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.757+0000 7ff68539c640 1 --2- 172.21.15.16:0/1814225868 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff6680a1090 0x7ff6680a3550 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff67002aa00 tx=0x7ff670031470 comp rx=0 tx=0).stop 2024-09-13T16:30:59.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.758+0000 7ff68539c640 1 -- 172.21.15.16:0/1814225868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff680100850 msgr2=0x7ff68019d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:59.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.758+0000 7ff68539c640 1 --2- 172.21.15.16:0/1814225868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff680100850 0x7ff68019d330 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff67800cad0 tx=0x7ff67800cf90 comp rx=0 tx=0).stop 2024-09-13T16:30:59.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.758+0000 7ff68539c640 1 -- 172.21.15.16:0/1814225868 shutdown_connections 2024-09-13T16:30:59.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.758+0000 7ff68539c640 1 --2- 172.21.15.16:0/1814225868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff680100850 0x7ff68019d330 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:59.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.758+0000 7ff68539c640 1 --2- 172.21.15.16:0/1814225868 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6800ffe90 0x7ff680196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:59.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.758+0000 7ff68539c640 1 --2- 172.21.15.16:0/1814225868 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff6680a1090 0x7ff6680a3550 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:59.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.758+0000 7ff68539c640 1 --2- 172.21.15.16:0/1814225868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6800feca0 0x7ff680195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:59.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.758+0000 7ff68539c640 1 -- 172.21.15.16:0/1814225868 >> 172.21.15.16:0/1814225868 conn(0x7ff6800fa420 msgr2=0x7ff6800fbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:59.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.758+0000 7ff68539c640 1 -- 172.21.15.16:0/1814225868 shutdown_connections 2024-09-13T16:30:59.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.758+0000 7ff68539c640 1 -- 172.21.15.16:0/1814225868 wait complete. 2024-09-13T16:30:59.771 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T16:30:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.876+0000 7f33ebe44640 1 -- 172.21.15.16:0/3983522737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33e41008a0 msgr2=0x7f33e4102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.876+0000 7f33ebe44640 1 --2- 172.21.15.16:0/3983522737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33e41008a0 0x7f33e4102c90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f33e0009900 tx=0x7f33e002a410 comp rx=0 tx=0).stop 2024-09-13T16:30:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.876+0000 7f33ebe44640 1 -- 172.21.15.16:0/3983522737 shutdown_connections 2024-09-13T16:30:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.876+0000 7f33ebe44640 1 --2- 172.21.15.16:0/3983522737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33e4105b00 0x7f33e4107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.876+0000 7f33ebe44640 1 --2- 172.21.15.16:0/3983522737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33e41031d0 0x7f33e41055c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.876+0000 7f33ebe44640 1 --2- 172.21.15.16:0/3983522737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33e41008a0 0x7f33e4102c90 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.876+0000 7f33ebe44640 1 -- 172.21.15.16:0/3983522737 >> 172.21.15.16:0/3983522737 conn(0x7f33e40fa3e0 msgr2=0x7f33e40fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:30:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.876+0000 7f33ebe44640 1 -- 172.21.15.16:0/3983522737 shutdown_connections 2024-09-13T16:30:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.876+0000 7f33ebe44640 1 -- 172.21.15.16:0/3983522737 wait complete. 2024-09-13T16:30:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.876+0000 7f33ebe44640 1 Processor -- start 2024-09-13T16:30:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.876+0000 7f33ebe44640 1 -- start start 2024-09-13T16:30:59.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.877+0000 7f33ebe44640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33e41008a0 0x7f33e4195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:59.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.877+0000 7f33ebe44640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33e41031d0 0x7f33e4196230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:59.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.877+0000 7f33ebe44640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33e4105b00 0x7f33e419d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:59.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.877+0000 7f33ebe44640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e419d7d0 con 0x7f33e41008a0 2024-09-13T16:30:59.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.877+0000 7f33ebe44640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e419d940 con 0x7f33e4105b00 2024-09-13T16:30:59.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.877+0000 7f33ebe44640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e419db00 con 0x7f33e41031d0 2024-09-13T16:30:59.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.877+0000 7f33e9be0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33e41008a0 0x7f33e4195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:59.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.877+0000 7f33e9be0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33e41008a0 0x7f33e4195cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40682/0 (socket says 172.21.15.16:40682) 2024-09-13T16:30:59.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.877+0000 7f33e9be0640 1 -- 172.21.15.16:0/1253381200 learned_addr learned my addr 172.21.15.16:0/1253381200 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:30:59.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.877+0000 7f33e93df640 1 --2- 172.21.15.16:0/1253381200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33e41031d0 0x7f33e4196230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:59.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.877+0000 7f33e9be0640 1 -- 172.21.15.16:0/1253381200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33e41031d0 msgr2=0x7f33e4196230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:59.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.877+0000 7f33e9be0640 1 --2- 172.21.15.16:0/1253381200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33e41031d0 0x7f33e4196230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:59.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.877+0000 7f33e9be0640 1 -- 172.21.15.16:0/1253381200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33e4105b00 msgr2=0x7f33e419d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:30:59.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.877+0000 7f33ea3e1640 1 --2- 172.21.15.16:0/1253381200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33e4105b00 0x7f33e419d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:59.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.877+0000 7f33e9be0640 1 --2- 172.21.15.16:0/1253381200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33e4105b00 0x7f33e419d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:30:59.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.877+0000 7f33e9be0640 1 -- 172.21.15.16:0/1253381200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e0009560 con 0x7f33e41008a0 2024-09-13T16:30:59.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.877+0000 7f33e93df640 1 --2- 172.21.15.16:0/1253381200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33e41031d0 0x7f33e4196230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:30:59.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.877+0000 7f33e9be0640 1 --2- 172.21.15.16:0/1253381200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33e41008a0 0x7f33e4195cf0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f33e00057c0 tx=0x7f33e00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:59.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.877+0000 7f33daffd640 1 -- 172.21.15.16:0/1253381200 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e0031d70 con 0x7f33e41008a0 2024-09-13T16:30:59.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.877+0000 7f33daffd640 1 -- 172.21.15.16:0/1253381200 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f33e00044e0 con 0x7f33e41008a0 2024-09-13T16:30:59.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.877+0000 7f33ebe44640 1 -- 172.21.15.16:0/1253381200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e419dd80 con 0x7f33e41008a0 2024-09-13T16:30:59.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.877+0000 7f33daffd640 1 -- 172.21.15.16:0/1253381200 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e0003b80 con 0x7f33e41008a0 2024-09-13T16:30:59.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.878+0000 7f33ebe44640 1 -- 172.21.15.16:0/1253381200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e419e180 con 0x7f33e41008a0 2024-09-13T16:30:59.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.879+0000 7f33ebe44640 1 -- 172.21.15.16:0/1253381200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e40fbed0 con 0x7f33e41008a0 2024-09-13T16:30:59.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.880+0000 7f33daffd640 1 -- 172.21.15.16:0/1253381200 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f33e0003ce0 con 0x7f33e41008a0 2024-09-13T16:30:59.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.880+0000 7f33daffd640 1 --2- 172.21.15.16:0/1253381200 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f33d00a1090 0x7f33d00a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:30:59.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.880+0000 7f33e93df640 1 --2- 172.21.15.16:0/1253381200 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f33d00a1090 0x7f33d00a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:30:59.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.881+0000 7f33daffd640 1 -- 172.21.15.16:0/1253381200 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f33e00ee370 con 0x7f33e41008a0 2024-09-13T16:30:59.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.881+0000 7f33e93df640 1 --2- 172.21.15.16:0/1253381200 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f33d00a1090 0x7f33d00a3550 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f33d40058c0 tx=0x7f33d4005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:30:59.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:30:59.883+0000 7f33daffd640 1 -- 172.21.15.16:0/1253381200 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f33e00b7720 con 0x7f33e41008a0 2024-09-13T16:31:00.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.054+0000 7f33ebe44640 1 -- 172.21.15.16:0/1253381200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f33e402c3e0 con 0x7f33e41008a0 2024-09-13T16:31:00.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.054+0000 7f33daffd640 1 -- 172.21.15.16:0/1253381200 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f33e00b9650 con 0x7f33e41008a0 2024-09-13T16:31:00.057 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-13T16:31:00.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.056+0000 7f33ebe44640 1 -- 172.21.15.16:0/1253381200 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f33d00a1090 msgr2=0x7f33d00a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:00.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.056+0000 7f33ebe44640 1 --2- 172.21.15.16:0/1253381200 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f33d00a1090 0x7f33d00a3550 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f33d40058c0 tx=0x7f33d4005d30 comp rx=0 tx=0).stop 2024-09-13T16:31:00.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.056+0000 7f33ebe44640 1 -- 172.21.15.16:0/1253381200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33e41008a0 msgr2=0x7f33e4195cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:00.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.056+0000 7f33ebe44640 1 --2- 172.21.15.16:0/1253381200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33e41008a0 0x7f33e4195cf0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f33e00057c0 tx=0x7f33e00042f0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.056+0000 7f33ebe44640 1 -- 172.21.15.16:0/1253381200 shutdown_connections 2024-09-13T16:31:00.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.056+0000 7f33ebe44640 1 --2- 172.21.15.16:0/1253381200 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f33d00a1090 0x7f33d00a3550 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.056+0000 7f33ebe44640 1 --2- 172.21.15.16:0/1253381200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33e4105b00 0x7f33e419d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.056+0000 7f33ebe44640 1 --2- 172.21.15.16:0/1253381200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33e41031d0 0x7f33e4196230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.056+0000 7f33ebe44640 1 --2- 172.21.15.16:0/1253381200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33e41008a0 0x7f33e4195cf0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.056+0000 7f33ebe44640 1 -- 172.21.15.16:0/1253381200 >> 172.21.15.16:0/1253381200 conn(0x7f33e40fa3e0 msgr2=0x7f33e4101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:00.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.057+0000 7f33ebe44640 1 -- 172.21.15.16:0/1253381200 shutdown_connections 2024-09-13T16:31:00.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.057+0000 7f33ebe44640 1 -- 172.21.15.16:0/1253381200 wait complete. 2024-09-13T16:31:00.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:59 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2275889841' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:31:00.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:59 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/4096585515' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:31:00.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:59 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2407328232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T16:31:00.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:30:59 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1814225868' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T16:31:00.071 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:31:00.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:59 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/2275889841' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:31:00.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:59 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/4096585515' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:31:00.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:59 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/2407328232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T16:31:00.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:30:59 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1814225868' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T16:31:00.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.172+0000 7fe8c7925640 1 -- 172.21.15.16:0/291345160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8c0100880 msgr2=0x7fe8c0107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:00.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.172+0000 7fe8c7925640 1 --2- 172.21.15.16:0/291345160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8c0100880 0x7fe8c0107110 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009ca0 tx=0x7fe8b802a8c0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.172+0000 7fe8c7925640 1 -- 172.21.15.16:0/291345160 shutdown_connections 2024-09-13T16:31:00.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.172+0000 7fe8c7925640 1 --2- 172.21.15.16:0/291345160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8c0100880 0x7fe8c0107110 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.172+0000 7fe8c7925640 1 --2- 172.21.15.16:0/291345160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8c00ffec0 0x7fe8c0100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.172+0000 7fe8c7925640 1 --2- 172.21.15.16:0/291345160 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8c00fecd0 0x7fe8c00ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.172+0000 7fe8c7925640 1 -- 172.21.15.16:0/291345160 >> 172.21.15.16:0/291345160 conn(0x7fe8c00fa430 msgr2=0x7fe8c00fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:00.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.172+0000 7fe8c7925640 1 -- 172.21.15.16:0/291345160 shutdown_connections 2024-09-13T16:31:00.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.172+0000 7fe8c7925640 1 -- 172.21.15.16:0/291345160 wait complete. 2024-09-13T16:31:00.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.173+0000 7fe8c7925640 1 Processor -- start 2024-09-13T16:31:00.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.173+0000 7fe8c7925640 1 -- start start 2024-09-13T16:31:00.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.173+0000 7fe8c7925640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8c00fecd0 0x7fe8c0102c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.173+0000 7fe8c7925640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8c00ffec0 0x7fe8c01012a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.173+0000 7fe8c7925640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8c0100880 0x7fe8c01017e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.173+0000 7fe8c7925640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c0101f40 con 0x7fe8c0100880 2024-09-13T16:31:00.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.173+0000 7fe8c7925640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c01020b0 con 0x7fe8c00ffec0 2024-09-13T16:31:00.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.173+0000 7fe8c7925640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c019fd00 con 0x7fe8c00fecd0 2024-09-13T16:31:00.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.173+0000 7fe8c5ec2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8c0100880 0x7fe8c01017e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.173+0000 7fe8c5ec2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8c0100880 0x7fe8c01017e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40692/0 (socket says 172.21.15.16:40692) 2024-09-13T16:31:00.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.173+0000 7fe8c5ec2640 1 -- 172.21.15.16:0/1580887664 learned_addr learned my addr 172.21.15.16:0/1580887664 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:00.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.173+0000 7fe8c4ec0640 1 --2- 172.21.15.16:0/1580887664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8c00ffec0 0x7fe8c01012a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.173+0000 7fe8c56c1640 1 --2- 172.21.15.16:0/1580887664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8c00fecd0 0x7fe8c0102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.174+0000 7fe8c5ec2640 1 -- 172.21.15.16:0/1580887664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8c00fecd0 msgr2=0x7fe8c0102c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:00.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.174+0000 7fe8c5ec2640 1 --2- 172.21.15.16:0/1580887664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8c00fecd0 0x7fe8c0102c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.174+0000 7fe8c5ec2640 1 -- 172.21.15.16:0/1580887664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8c00ffec0 msgr2=0x7fe8c01012a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:00.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.174+0000 7fe8c5ec2640 1 --2- 172.21.15.16:0/1580887664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8c00ffec0 0x7fe8c01012a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.174+0000 7fe8c5ec2640 1 -- 172.21.15.16:0/1580887664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b8009900 con 0x7fe8c0100880 2024-09-13T16:31:00.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.174+0000 7fe8c5ec2640 1 --2- 172.21.15.16:0/1580887664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8c0100880 0x7fe8c01017e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009430 tx=0x7fe8b802f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:00.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.174+0000 7fe8b67fc640 1 -- 172.21.15.16:0/1580887664 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8003bf0 con 0x7fe8c0100880 2024-09-13T16:31:00.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.174+0000 7fe8b67fc640 1 -- 172.21.15.16:0/1580887664 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe8b8040070 con 0x7fe8c0100880 2024-09-13T16:31:00.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.174+0000 7fe8c7925640 1 -- 172.21.15.16:0/1580887664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c019ff50 con 0x7fe8c0100880 2024-09-13T16:31:00.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.174+0000 7fe8b67fc640 1 -- 172.21.15.16:0/1580887664 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b80044d0 con 0x7fe8c0100880 2024-09-13T16:31:00.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.174+0000 7fe8c7925640 1 -- 172.21.15.16:0/1580887664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c01a0420 con 0x7fe8c0100880 2024-09-13T16:31:00.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.175+0000 7fe8c7925640 1 -- 172.21.15.16:0/1580887664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c010a420 con 0x7fe8c0100880 2024-09-13T16:31:00.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.176+0000 7fe8b67fc640 1 -- 172.21.15.16:0/1580887664 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe8b8042030 con 0x7fe8c0100880 2024-09-13T16:31:00.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.177+0000 7fe8b67fc640 1 --2- 172.21.15.16:0/1580887664 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe8ac0a1110 0x7fe8ac0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.177+0000 7fe8c56c1640 1 --2- 172.21.15.16:0/1580887664 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe8ac0a1110 0x7fe8ac0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.177+0000 7fe8b67fc640 1 -- 172.21.15.16:0/1580887664 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fe8b80ee3a0 con 0x7fe8c0100880 2024-09-13T16:31:00.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.177+0000 7fe8c56c1640 1 --2- 172.21.15.16:0/1580887664 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe8ac0a1110 0x7fe8ac0a35d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc005fd0 tx=0x7fe8bc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:00.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.180+0000 7fe8b67fc640 1 -- 172.21.15.16:0/1580887664 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fe8b80b7780 con 0x7fe8c0100880 2024-09-13T16:31:00.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:59 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/2275889841' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:31:00.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:59 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/4096585515' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:31:00.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:59 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/2407328232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T16:31:00.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:30:59 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1814225868' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T16:31:00.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.388+0000 7fe8c7925640 1 -- 172.21.15.16:0/1580887664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8c010a600 con 0x7fe8c0100880 2024-09-13T16:31:00.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.389+0000 7fe8b67fc640 1 -- 172.21.15.16:0/1580887664 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10924 (secure 0 0 0) 0x7fe8b80bada0 con 0x7fe8c0100880 2024-09-13T16:31:00.391 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:31:00.391 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3292781664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3292781664},{"type":"v1","addr":"172.21.15.16:6835","nonce":3292781664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4109039027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4109039027},{"type":"v1","addr":"172.21.15.16:6837","nonce":4109039027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T16:28:39.184734+0000","modified":"2024-09-13T16:29:06.636937+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24518,"mds_1":24490,"mds_2":24485,"mds_3":24422,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/2391618693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2391618693},{"type":"v1","addr":"172.21.15.16:6839","nonce":2391618693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/521042681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":521042681},{"type":"v1","addr":"172.21.15.16:6841","nonce":521042681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24418":{"gid":24418,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.111:6833/1984275936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1984275936},{"type":"v1","addr":"172.21.15.111:6833","nonce":1984275936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.107:6833/3797712690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3797712690},{"type":"v1","addr":"172.21.15.107:6833","nonce":3797712690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/338548468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":338548468},{"type":"v1","addr":"172.21.15.107:6835","nonce":338548468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/662156111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":662156111},{"type":"v1","addr":"172.21.15.111:6835","nonce":662156111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/1877128453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1877128453},{"type":"v1","addr":"172.21.15.111:6837","nonce":1877128453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.107:6837/2838457611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2838457611},{"type":"v1","addr":"172.21.15.107:6837","nonce":2838457611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.111:6839/1423842285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":1423842285},{"type":"v1","addr":"172.21.15.111:6839","nonce":1423842285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6839/1914689806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1914689806},{"type":"v1","addr":"172.21.15.107:6839","nonce":1914689806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T16:31:00.391 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-13T16:31:00.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.391+0000 7fe8c7925640 1 -- 172.21.15.16:0/1580887664 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe8ac0a1110 msgr2=0x7fe8ac0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:00.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.391+0000 7fe8c7925640 1 --2- 172.21.15.16:0/1580887664 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe8ac0a1110 0x7fe8ac0a35d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc005fd0 tx=0x7fe8bc005850 comp rx=0 tx=0).stop 2024-09-13T16:31:00.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.391+0000 7fe8c7925640 1 -- 172.21.15.16:0/1580887664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8c0100880 msgr2=0x7fe8c01017e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:00.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.391+0000 7fe8c7925640 1 --2- 172.21.15.16:0/1580887664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8c0100880 0x7fe8c01017e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009430 tx=0x7fe8b802f310 comp rx=0 tx=0).stop 2024-09-13T16:31:00.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.391+0000 7fe8c7925640 1 -- 172.21.15.16:0/1580887664 shutdown_connections 2024-09-13T16:31:00.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.391+0000 7fe8c7925640 1 --2- 172.21.15.16:0/1580887664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8c0100880 0x7fe8c01017e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.391+0000 7fe8c7925640 1 --2- 172.21.15.16:0/1580887664 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe8ac0a1110 0x7fe8ac0a35d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.391+0000 7fe8c7925640 1 --2- 172.21.15.16:0/1580887664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8c00ffec0 0x7fe8c01012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.391+0000 7fe8c7925640 1 --2- 172.21.15.16:0/1580887664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8c00fecd0 0x7fe8c0102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.391+0000 7fe8c7925640 1 -- 172.21.15.16:0/1580887664 >> 172.21.15.16:0/1580887664 conn(0x7fe8c00fa430 msgr2=0x7fe8c0068660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:00.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.391+0000 7fe8c7925640 1 -- 172.21.15.16:0/1580887664 shutdown_connections 2024-09-13T16:31:00.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.391+0000 7fe8c7925640 1 -- 172.21.15.16:0/1580887664 wait complete. 2024-09-13T16:31:00.404 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:31:00.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.504+0000 7f0fd9b7d640 1 -- 172.21.15.16:0/3886392507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fd4105ab0 msgr2=0x7f0fd4107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:00.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.504+0000 7f0fd9b7d640 1 --2- 172.21.15.16:0/3886392507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fd4105ab0 0x7f0fd4107f20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc009ca0 tx=0x7f0fcc02a8c0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.504+0000 7f0fd9b7d640 1 -- 172.21.15.16:0/3886392507 shutdown_connections 2024-09-13T16:31:00.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.504+0000 7f0fd9b7d640 1 --2- 172.21.15.16:0/3886392507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fd4105ab0 0x7f0fd4107f20 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.504+0000 7f0fd9b7d640 1 --2- 172.21.15.16:0/3886392507 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0fd4103180 0x7f0fd4105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.504+0000 7f0fd9b7d640 1 --2- 172.21.15.16:0/3886392507 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fd4100850 0x7f0fd4102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.504+0000 7f0fd9b7d640 1 -- 172.21.15.16:0/3886392507 >> 172.21.15.16:0/3886392507 conn(0x7f0fd40fa430 msgr2=0x7f0fd40fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:00.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.504+0000 7f0fd9b7d640 1 -- 172.21.15.16:0/3886392507 shutdown_connections 2024-09-13T16:31:00.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.504+0000 7f0fd9b7d640 1 -- 172.21.15.16:0/3886392507 wait complete. 2024-09-13T16:31:00.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.505+0000 7f0fd9b7d640 1 Processor -- start 2024-09-13T16:31:00.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.505+0000 7f0fd9b7d640 1 -- start start 2024-09-13T16:31:00.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.505+0000 7f0fd9b7d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0fd4100850 0x7f0fd4197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.505+0000 7f0fd9b7d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fd4103180 0x7f0fd41984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.505+0000 7f0fd9b7d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fd4105ab0 0x7f0fd419f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.505+0000 7f0fd9b7d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd419fa60 con 0x7f0fd4105ab0 2024-09-13T16:31:00.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.505+0000 7f0fd9b7d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd419fbd0 con 0x7f0fd4103180 2024-09-13T16:31:00.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.505+0000 7f0fd9b7d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd419fd90 con 0x7f0fd4100850 2024-09-13T16:31:00.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.505+0000 7f0fd3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fd4105ab0 0x7f0fd419f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.505+0000 7f0fd3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fd4105ab0 0x7f0fd419f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40706/0 (socket says 172.21.15.16:40706) 2024-09-13T16:31:00.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.505+0000 7f0fd3fff640 1 -- 172.21.15.16:0/1872701749 learned_addr learned my addr 172.21.15.16:0/1872701749 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:00.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.505+0000 7f0fd37fe640 1 --2- 172.21.15.16:0/1872701749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0fd4100850 0x7f0fd4197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.505+0000 7f0fd2ffd640 1 --2- 172.21.15.16:0/1872701749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fd4103180 0x7f0fd41984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.505+0000 7f0fd3fff640 1 -- 172.21.15.16:0/1872701749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0fd4100850 msgr2=0x7f0fd4197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:00.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.505+0000 7f0fd3fff640 1 --2- 172.21.15.16:0/1872701749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0fd4100850 0x7f0fd4197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.505+0000 7f0fd3fff640 1 -- 172.21.15.16:0/1872701749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fd4103180 msgr2=0x7f0fd41984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:00.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.505+0000 7f0fd3fff640 1 --2- 172.21.15.16:0/1872701749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fd4103180 0x7f0fd41984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.506+0000 7f0fd3fff640 1 -- 172.21.15.16:0/1872701749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fcc009900 con 0x7f0fd4105ab0 2024-09-13T16:31:00.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.506+0000 7f0fd2ffd640 1 --2- 172.21.15.16:0/1872701749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fd4103180 0x7f0fd41984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:31:00.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.506+0000 7f0fd3fff640 1 --2- 172.21.15.16:0/1872701749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fd4105ab0 0x7f0fd419f520 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc02aeb0 tx=0x7f0fcc02f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:00.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.506+0000 7f0fd0ff9640 1 -- 172.21.15.16:0/1872701749 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc003bf0 con 0x7f0fd4105ab0 2024-09-13T16:31:00.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.506+0000 7f0fd0ff9640 1 -- 172.21.15.16:0/1872701749 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0fcc040070 con 0x7f0fd4105ab0 2024-09-13T16:31:00.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.506+0000 7f0fd9b7d640 1 -- 172.21.15.16:0/1872701749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd41a0010 con 0x7f0fd4105ab0 2024-09-13T16:31:00.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.506+0000 7f0fd0ff9640 1 -- 172.21.15.16:0/1872701749 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc0044a0 con 0x7f0fd4105ab0 2024-09-13T16:31:00.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.506+0000 7f0fd9b7d640 1 -- 172.21.15.16:0/1872701749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd41a0570 con 0x7f0fd4105ab0 2024-09-13T16:31:00.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.507+0000 7f0fd9b7d640 1 -- 172.21.15.16:0/1872701749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd41075a0 con 0x7f0fd4105ab0 2024-09-13T16:31:00.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.508+0000 7f0fd0ff9640 1 -- 172.21.15.16:0/1872701749 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f0fcc042030 con 0x7f0fd4105ab0 2024-09-13T16:31:00.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.509+0000 7f0fd0ff9640 1 --2- 172.21.15.16:0/1872701749 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f0fbc0a1110 0x7f0fbc0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.509+0000 7f0fd37fe640 1 --2- 172.21.15.16:0/1872701749 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f0fbc0a1110 0x7f0fbc0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.509+0000 7f0fd0ff9640 1 -- 172.21.15.16:0/1872701749 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f0fcc0ee350 con 0x7f0fd4105ab0 2024-09-13T16:31:00.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.510+0000 7f0fd37fe640 1 --2- 172.21.15.16:0/1872701749 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f0fbc0a1110 0x7f0fbc0a35d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4005d00 tx=0x7f0fc400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:00.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.513+0000 7f0fd0ff9640 1 -- 172.21.15.16:0/1872701749 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0fcc0b7790 con 0x7f0fd4105ab0 2024-09-13T16:31:00.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.687+0000 7f0fd9b7d640 1 -- 172.21.15.16:0/1872701749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0fd410b580 con 0x7f0fd4105ab0 2024-09-13T16:31:00.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.688+0000 7f0fd0ff9640 1 -- 172.21.15.16:0/1872701749 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24593 (secure 0 0 0) 0x7f0fcc0badb0 con 0x7f0fd4105ab0 2024-09-13T16:31:00.691 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:31:00.691 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":93,"fsid":"0d241f6a-71ec-11ef-bceb-c7b262605968","created":"2024-09-13T16:21:17.971738+0000","modified":"2024-09-13T16:30:05.681521+0000","last_up_change":"2024-09-13T16:27:54.673413+0000","last_in_change":"2024-09-13T16:27:35.718390+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:24:22.981019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:28:37.746027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:28:38.602731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:28:41.633322+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e11ef8f-08d6-4e74-afd6-f49f4bd40032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6803","nonce":2079394186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6805","nonce":2079394186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6809","nonce":2079394186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6807","nonce":2079394186}]},"public_addr":"172.21.15.16:6803/2079394186","cluster_addr":"172.21.15.16:6805/2079394186","heartbeat_back_addr":"172.21.15.16:6809/2079394186","heartbeat_front_addr":"172.21.15.16:6807/2079394186","state":["exists","up"]},{"osd":1,"uuid":"f2cd3694-9c9e-4da7-976e-f80a8df120f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6801","nonce":4021360958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6803","nonce":4021360958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6807","nonce":4021360958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6805","nonce":4021360958}]},"public_addr":"172.21.15.107:6801/4021360958","cluster_addr":"172.21.15.107:6803/4021360958","heartbeat_back_addr":"172.21.15.107:6807/4021360958","heartbeat_front_addr":"172.21.15.107:6805/4021360958","state":["exists","up"]},{"osd":2,"uuid":"a5bbf2eb-6599-44d5-ad96-68bb89ee19f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6801","nonce":3227179825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6803","nonce":3227179825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6807","nonce":3227179825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6805","nonce":3227179825}]},"public_addr":"172.21.15.111:6801/3227179825","cluster_addr":"172.21.15.111:6803/3227179825","heartbeat_back_addr":"172.21.15.111:6807/3227179825","heartbeat_front_addr":"172.21.15.111:6805/3227179825","state":["exists","up"]},{"osd":3,"uuid":"d6295ffe-11bc-4964-a250-9fecca870b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6811","nonce":2390375937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6813","nonce":2390375937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6817","nonce":2390375937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6815","nonce":2390375937}]},"public_addr":"172.21.15.16:6811/2390375937","cluster_addr":"172.21.15.16:6813/2390375937","heartbeat_back_addr":"172.21.15.16:6817/2390375937","heartbeat_front_addr":"172.21.15.16:6815/2390375937","state":["exists","up"]},{"osd":4,"uuid":"0a8446e0-fc86-4646-beb6-a7b9d433093d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6809","nonce":1936582048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6811","nonce":1936582048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6815","nonce":1936582048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6813","nonce":1936582048}]},"public_addr":"172.21.15.107:6809/1936582048","cluster_addr":"172.21.15.107:6811/1936582048","heartbeat_back_addr":"172.21.15.107:6815/1936582048","heartbeat_front_addr":"172.21.15.107:6813/1936582048","state":["exists","up"]},{"osd":5,"uuid":"2c97a446-357e-4975-9e51-9c95fe84a663","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6809","nonce":483858448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6811","nonce":483858448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6815","nonce":483858448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6813","nonce":483858448}]},"public_addr":"172.21.15.111:6809/483858448","cluster_addr":"172.21.15.111:6811/483858448","heartbeat_back_addr":"172.21.15.111:6815/483858448","heartbeat_front_addr":"172.21.15.111:6813/483858448","state":["exists","up"]},{"osd":6,"uuid":"890239e6-e6fb-4f9c-b973-05cc66d199bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6819","nonce":1330214141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6821","nonce":1330214141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6825","nonce":1330214141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6823","nonce":1330214141}]},"public_addr":"172.21.15.16:6819/1330214141","cluster_addr":"172.21.15.16:6821/1330214141","heartbeat_back_addr":"172.21.15.16:6825/1330214141","heartbeat_front_addr":"172.21.15.16:6823/1330214141","state":["exists","up"]},{"osd":7,"uuid":"ee826a9c-f441-4903-9b1b-66b627f7b712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6817","nonce":1133876754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6819","nonce":1133876754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6823","nonce":1133876754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6821","nonce":1133876754}]},"public_addr":"172.21.15.107:6817/1133876754","cluster_addr":"172.21.15.107:6819/1133876754","heartbeat_back_addr":"172.21.15.107:6823/1133876754","heartbeat_front_addr":"172.21.15.107:6821/1133876754","state":["exists","up"]},{"osd":8,"uuid":"e19773f2-dec8-44e0-9d33-0252e07d6415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6817","nonce":837548190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6819","nonce":837548190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6823","nonce":837548190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6821","nonce":837548190}]},"public_addr":"172.21.15.111:6817/837548190","cluster_addr":"172.21.15.111:6819/837548190","heartbeat_back_addr":"172.21.15.111:6823/837548190","heartbeat_front_addr":"172.21.15.111:6821/837548190","state":["exists","up"]},{"osd":9,"uuid":"bae4d6db-8b90-4d02-8e83-7d8f2a6f8fdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6827","nonce":4175383686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6829","nonce":4175383686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6833","nonce":4175383686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6831","nonce":4175383686}]},"public_addr":"172.21.15.16:6827/4175383686","cluster_addr":"172.21.15.16:6829/4175383686","heartbeat_back_addr":"172.21.15.16:6833/4175383686","heartbeat_front_addr":"172.21.15.16:6831/4175383686","state":["exists","up"]},{"osd":10,"uuid":"fd0e7740-76c8-4c99-a349-7077b4905143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6825","nonce":2659550426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6827","nonce":2659550426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6831","nonce":2659550426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6829","nonce":2659550426}]},"public_addr":"172.21.15.107:6825/2659550426","cluster_addr":"172.21.15.107:6827/2659550426","heartbeat_back_addr":"172.21.15.107:6831/2659550426","heartbeat_front_addr":"172.21.15.107:6829/2659550426","state":["exists","up"]},{"osd":11,"uuid":"94a1751d-b00c-4e23-815d-3a8bc3dea146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6825","nonce":1633548683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6827","nonce":1633548683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6831","nonce":1633548683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6829","nonce":1633548683}]},"public_addr":"172.21.15.111:6825/1633548683","cluster_addr":"172.21.15.111:6827/1633548683","heartbeat_back_addr":"172.21.15.111:6831/1633548683","heartbeat_front_addr":"172.21.15.111:6829/1633548683","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:36.359852+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:58.014202+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:19.753672+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:43.404817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:06.001677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:29.361587+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:52.554847+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:15.957553+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:39.581093+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:04.911389+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:28.971280+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:53.473561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/2098378667":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/3536229016":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6801/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/2684687547":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6800/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/368410612":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/12158160":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6800/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6801/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/3754423379":"2024-09-14T16:21:40.126459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:31:00.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.690+0000 7f0fd9b7d640 1 -- 172.21.15.16:0/1872701749 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f0fbc0a1110 msgr2=0x7f0fbc0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:00.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.690+0000 7f0fd9b7d640 1 --2- 172.21.15.16:0/1872701749 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f0fbc0a1110 0x7f0fbc0a35d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4005d00 tx=0x7f0fc400b410 comp rx=0 tx=0).stop 2024-09-13T16:31:00.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.690+0000 7f0fd9b7d640 1 -- 172.21.15.16:0/1872701749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fd4105ab0 msgr2=0x7f0fd419f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:00.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.690+0000 7f0fd9b7d640 1 --2- 172.21.15.16:0/1872701749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fd4105ab0 0x7f0fd419f520 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc02aeb0 tx=0x7f0fcc02f310 comp rx=0 tx=0).stop 2024-09-13T16:31:00.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.690+0000 7f0fd9b7d640 1 -- 172.21.15.16:0/1872701749 shutdown_connections 2024-09-13T16:31:00.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.690+0000 7f0fd9b7d640 1 --2- 172.21.15.16:0/1872701749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fd4105ab0 0x7f0fd419f520 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.690+0000 7f0fd9b7d640 1 --2- 172.21.15.16:0/1872701749 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f0fbc0a1110 0x7f0fbc0a35d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.690+0000 7f0fd9b7d640 1 --2- 172.21.15.16:0/1872701749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fd4103180 0x7f0fd41984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.690+0000 7f0fd9b7d640 1 --2- 172.21.15.16:0/1872701749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0fd4100850 0x7f0fd4197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.690+0000 7f0fd9b7d640 1 -- 172.21.15.16:0/1872701749 >> 172.21.15.16:0/1872701749 conn(0x7f0fd40fa430 msgr2=0x7f0fd4100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:00.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.690+0000 7f0fd9b7d640 1 -- 172.21.15.16:0/1872701749 shutdown_connections 2024-09-13T16:31:00.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.690+0000 7f0fd9b7d640 1 -- 172.21.15.16:0/1872701749 wait complete. 2024-09-13T16:31:00.704 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-13T16:28:39.184734+0000', 'modified': '2024-09-13T16:29:06.636937+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24518, 'mds_1': 24490, 'mds_2': 24485, 'mds_3': 24422, 'mds_4': 24418}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14586': {'gid': 14586, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6839/2391618693', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6838', 'nonce': 2391618693}, {'type': 'v1', 'addr': '172.21.15.16:6839', 'nonce': 2391618693}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14598': {'gid': 14598, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6841/521042681', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6840', 'nonce': 521042681}, {'type': 'v1', 'addr': '172.21.15.16:6841', 'nonce': 521042681}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24418': {'gid': 24418, 'name': 'l', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.111:6833/1984275936', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6832', 'nonce': 1984275936}, {'type': 'v1', 'addr': '172.21.15.111:6833', 'nonce': 1984275936}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24422': {'gid': 24422, 'name': 'k', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.107:6833/3797712690', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6832', 'nonce': 3797712690}, {'type': 'v1', 'addr': '172.21.15.107:6833', 'nonce': 3797712690}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24446': {'gid': 24446, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.107:6835/338548468', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6834', 'nonce': 338548468}, {'type': 'v1', 'addr': '172.21.15.107:6835', 'nonce': 338548468}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.111:6835/662156111', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6834', 'nonce': 662156111}, {'type': 'v1', 'addr': '172.21.15.111:6835', 'nonce': 662156111}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24472': {'gid': 24472, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.111:6837/1877128453', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6836', 'nonce': 1877128453}, {'type': 'v1', 'addr': '172.21.15.111:6837', 'nonce': 1877128453}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24485': {'gid': 24485, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.107:6837/2838457611', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6836', 'nonce': 2838457611}, {'type': 'v1', 'addr': '172.21.15.107:6837', 'nonce': 2838457611}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24490': {'gid': 24490, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.111:6839/1423842285', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6838', 'nonce': 1423842285}, {'type': 'v1', 'addr': '172.21.15.111:6839', 'nonce': 1423842285}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24518': {'gid': 24518, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.107:6839/1914689806', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6838', 'nonce': 1914689806}, {'type': 'v1', 'addr': '172.21.15.107:6839', 'nonce': 1914689806}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T16:31:00.704 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-13T16:31:00.704 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-09-13T16:31:00.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.807+0000 7f372cec9640 1 -- 172.21.15.16:0/3224423394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37280fec70 msgr2=0x7f37280ff070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:00.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.807+0000 7f372cec9640 1 --2- 172.21.15.16:0/3224423394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37280fec70 0x7f37280ff070 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3718009900 tx=0x7f371802a410 comp rx=0 tx=0).stop 2024-09-13T16:31:00.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.807+0000 7f372cec9640 1 -- 172.21.15.16:0/3224423394 shutdown_connections 2024-09-13T16:31:00.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.807+0000 7f372cec9640 1 --2- 172.21.15.16:0/3224423394 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3728100820 0x7f37281070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.807+0000 7f372cec9640 1 --2- 172.21.15.16:0/3224423394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37280ffe60 0x7f37281002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.807+0000 7f372cec9640 1 --2- 172.21.15.16:0/3224423394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37280fec70 0x7f37280ff070 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.807+0000 7f372cec9640 1 -- 172.21.15.16:0/3224423394 >> 172.21.15.16:0/3224423394 conn(0x7f37280fa3b0 msgr2=0x7f37280fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:00.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.807+0000 7f372cec9640 1 -- 172.21.15.16:0/3224423394 shutdown_connections 2024-09-13T16:31:00.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.807+0000 7f372cec9640 1 -- 172.21.15.16:0/3224423394 wait complete. 2024-09-13T16:31:00.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.808+0000 7f372cec9640 1 Processor -- start 2024-09-13T16:31:00.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.808+0000 7f372cec9640 1 -- start start 2024-09-13T16:31:00.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.808+0000 7f372cec9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37280fec70 0x7f3728102c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.808+0000 7f372cec9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37280ffe60 0x7f3728101240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.808+0000 7f372cec9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3728100820 0x7f3728101780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.808+0000 7f372cec9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3728101ee0 con 0x7f37280fec70 2024-09-13T16:31:00.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.808+0000 7f372cec9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3728102050 con 0x7f3728100820 2024-09-13T16:31:00.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.808+0000 7f372cec9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372819fca0 con 0x7f37280ffe60 2024-09-13T16:31:00.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.808+0000 7f372659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37280fec70 0x7f3728102c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.808+0000 7f372659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37280fec70 0x7f3728102c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40712/0 (socket says 172.21.15.16:40712) 2024-09-13T16:31:00.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.808+0000 7f372659c640 1 -- 172.21.15.16:0/536348231 learned_addr learned my addr 172.21.15.16:0/536348231 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:00.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.809+0000 7f3725d9b640 1 --2- 172.21.15.16:0/536348231 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37280ffe60 0x7f3728101240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.809+0000 7f3726d9d640 1 --2- 172.21.15.16:0/536348231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3728100820 0x7f3728101780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.809+0000 7f372659c640 1 -- 172.21.15.16:0/536348231 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37280ffe60 msgr2=0x7f3728101240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:00.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.809+0000 7f372659c640 1 --2- 172.21.15.16:0/536348231 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37280ffe60 0x7f3728101240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.809+0000 7f372659c640 1 -- 172.21.15.16:0/536348231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3728100820 msgr2=0x7f3728101780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:00.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.809+0000 7f372659c640 1 --2- 172.21.15.16:0/536348231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3728100820 0x7f3728101780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.809+0000 7f372659c640 1 -- 172.21.15.16:0/536348231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3718009560 con 0x7f37280fec70 2024-09-13T16:31:00.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.809+0000 7f372659c640 1 --2- 172.21.15.16:0/536348231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37280fec70 0x7f3728102c00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3718009230 tx=0x7f37180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:00.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.809+0000 7f37177fe640 1 -- 172.21.15.16:0/536348231 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3718031d70 con 0x7f37280fec70 2024-09-13T16:31:00.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.809+0000 7f37177fe640 1 -- 172.21.15.16:0/536348231 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f37180044e0 con 0x7f37280fec70 2024-09-13T16:31:00.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.809+0000 7f372cec9640 1 -- 172.21.15.16:0/536348231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f372819fef0 con 0x7f37280fec70 2024-09-13T16:31:00.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.809+0000 7f37177fe640 1 -- 172.21.15.16:0/536348231 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3718003a00 con 0x7f37280fec70 2024-09-13T16:31:00.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.809+0000 7f372cec9640 1 -- 172.21.15.16:0/536348231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37281a02c0 con 0x7f37280fec70 2024-09-13T16:31:00.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.811+0000 7f37177fe640 1 -- 172.21.15.16:0/536348231 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f3718003ba0 con 0x7f37280fec70 2024-09-13T16:31:00.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.812+0000 7f372fbe8740 1 Processor -- start 2024-09-13T16:31:00.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.812+0000 7f372fbe8740 1 -- start start 2024-09-13T16:31:00.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.812+0000 7f372fbe8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b37e832500 0x55b37e832900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.812+0000 7f372fbe8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b37e832e40 0x55b37e837900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.812+0000 7f372fbe8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b37e837ed0 0x55b37e83a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.812+0000 7f372fbe8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b37e78d7c0 con 0x55b37e832e40 2024-09-13T16:31:00.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.812+0000 7f372fbe8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b37e79cb70 con 0x55b37e837ed0 2024-09-13T16:31:00.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.812+0000 7f372fbe8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b37e79a300 con 0x55b37e832500 2024-09-13T16:31:00.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.812+0000 7f37177fe640 1 --2- 172.21.15.16:0/536348231 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f37100a1090 0x7f37100a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.812+0000 7f372659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b37e832e40 0x55b37e837900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.813+0000 7f3726d9d640 1 --2- 172.21.15.16:0/536348231 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f37100a1090 0x7f37100a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.813+0000 7f372659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b37e832e40 0x55b37e837900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40726/0 (socket says 172.21.15.16:40726) 2024-09-13T16:31:00.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.813+0000 7f372659c640 1 -- 172.21.15.16:0/2321564554 learned_addr learned my addr 172.21.15.16:0/2321564554 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:00.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.813+0000 7f3725d9b640 1 --2- 172.21.15.16:0/2321564554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b37e837ed0 0x55b37e83a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.813+0000 7f3726d9d640 1 --2- 172.21.15.16:0/2321564554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b37e832500 0x55b37e832900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.813+0000 7f37177fe640 1 -- 172.21.15.16:0/536348231 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f37180edd40 con 0x7f37280fec70 2024-09-13T16:31:00.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.813+0000 7f372659c640 1 -- 172.21.15.16:0/2321564554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b37e832500 msgr2=0x55b37e832900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:00.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.813+0000 7f372659c640 1 --2- 172.21.15.16:0/2321564554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b37e832500 0x55b37e832900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.813+0000 7f372659c640 1 -- 172.21.15.16:0/2321564554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b37e837ed0 msgr2=0x55b37e83a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:00.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.813+0000 7f372659c640 1 --2- 172.21.15.16:0/2321564554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b37e837ed0 0x55b37e83a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.813+0000 7f372659c640 1 -- 172.21.15.16:0/2321564554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b37e7224b0 con 0x55b37e832e40 2024-09-13T16:31:00.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.813+0000 7f3726d9d640 1 --2- 172.21.15.16:0/536348231 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f37100a1090 0x7f37100a3550 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3720009270 tx=0x7f3720009e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:00.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.813+0000 7f372659c640 1 --2- 172.21.15.16:0/2321564554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b37e832e40 0x55b37e837900 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f37180b72e0 tx=0x7f371808e080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7722e70a4c52eafc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:00.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.813+0000 7f3714ff9640 1 -- 172.21.15.16:0/2321564554 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37180ba740 con 0x55b37e832e40 2024-09-13T16:31:00.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.813+0000 7f3714ff9640 1 -- 172.21.15.16:0/2321564554 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f37180bb4e0 con 0x55b37e832e40 2024-09-13T16:31:00.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.813+0000 7f3714ff9640 1 -- 172.21.15.16:0/2321564554 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37180b74f0 con 0x55b37e832e40 2024-09-13T16:31:00.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.814+0000 7f372fbe8740 1 -- 172.21.15.16:0/2321564554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b37e832e40 msgr2=0x55b37e837900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:00.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.814+0000 7f372fbe8740 1 --2- 172.21.15.16:0/2321564554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b37e832e40 0x55b37e837900 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f37180b72e0 tx=0x7f371808e080 comp rx=0 tx=0).stop 2024-09-13T16:31:00.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.814+0000 7f372fbe8740 1 -- 172.21.15.16:0/2321564554 shutdown_connections 2024-09-13T16:31:00.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.814+0000 7f372fbe8740 1 --2- 172.21.15.16:0/2321564554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b37e837ed0 0x55b37e83a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.814+0000 7f372fbe8740 1 --2- 172.21.15.16:0/2321564554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b37e832e40 0x55b37e837900 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.814+0000 7f372fbe8740 1 --2- 172.21.15.16:0/2321564554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b37e832500 0x55b37e832900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.814+0000 7f372fbe8740 1 -- 172.21.15.16:0/2321564554 >> 172.21.15.16:0/2321564554 conn(0x55b37e82db70 msgr2=0x55b37e82df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:00.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.814+0000 7f372fbe8740 1 -- 172.21.15.16:0/2321564554 shutdown_connections 2024-09-13T16:31:00.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.814+0000 7f372fbe8740 1 -- 172.21.15.16:0/2321564554 wait complete. 2024-09-13T16:31:00.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.814+0000 7f372fbe8740 1 -- start start 2024-09-13T16:31:00.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.814+0000 7f372fbe8740 1 Processor -- start 2024-09-13T16:31:00.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.815+0000 7f372fbe8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b37e832500 0x55b37e83bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.815+0000 7f372fbe8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b37e832e40 0x55b37e83c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.815+0000 7f372fbe8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b37e837ed0 0x55b37e843300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.815+0000 7f372fbe8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b37e795bf0 con 0x55b37e832500 2024-09-13T16:31:00.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.815+0000 7f372fbe8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b37e754250 con 0x55b37e832e40 2024-09-13T16:31:00.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.815+0000 7f372fbe8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b37e753ef0 con 0x55b37e837ed0 2024-09-13T16:31:00.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.815+0000 7f372fbe8740 10 client.0 _ll_register_callbacks cb 0x55b37e786740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:31:00.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.815+0000 7f372659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b37e832500 0x55b37e83bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.815+0000 7f372659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b37e832500 0x55b37e83bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40734/0 (socket says 172.21.15.16:40734) 2024-09-13T16:31:00.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.815+0000 7f372659c640 1 -- 172.21.15.16:0/3395004962 learned_addr learned my addr 172.21.15.16:0/3395004962 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:00.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.815+0000 7f3725d9b640 1 --2- 172.21.15.16:0/3395004962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b37e832e40 0x55b37e83c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.815+0000 7f3726d9d640 1 --2- 172.21.15.16:0/3395004962 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b37e837ed0 0x55b37e843300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.815+0000 7f372659c640 1 -- 172.21.15.16:0/3395004962 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b37e837ed0 msgr2=0x55b37e843300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:00.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.815+0000 7f372659c640 1 --2- 172.21.15.16:0/3395004962 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b37e837ed0 0x55b37e843300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.815+0000 7f372659c640 1 -- 172.21.15.16:0/3395004962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b37e832e40 msgr2=0x55b37e83c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:00.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.815+0000 7f372659c640 1 --2- 172.21.15.16:0/3395004962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b37e832e40 0x55b37e83c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.815+0000 7f372659c640 1 -- 172.21.15.16:0/3395004962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37281a02c0 con 0x55b37e832500 2024-09-13T16:31:00.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.815+0000 7f3726d9d640 1 --2- 172.21.15.16:0/3395004962 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b37e837ed0 0x55b37e843300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:31:00.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.815+0000 7f372659c640 1 --2- 172.21.15.16:0/3395004962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b37e832500 0x55b37e83bdd0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f37180baae0 tx=0x7f37180bec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc6997f1ff499068 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:00.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.815+0000 7f370dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:31:00.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.815+0000 7f370dffb640 1 -- 172.21.15.16:0/3395004962 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37180bb040 con 0x55b37e832500 2024-09-13T16:31:00.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.815+0000 7f370dffb640 1 -- 172.21.15.16:0/3395004962 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f37180bb1a0 con 0x55b37e832500 2024-09-13T16:31:00.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.815+0000 7f372fbe8740 1 -- 172.21.15.16:0/3395004962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b37e4893c0 con 0x55b37e832500 2024-09-13T16:31:00.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.816+0000 7f370dffb640 1 -- 172.21.15.16:0/3395004962 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37180ba2d0 con 0x55b37e832500 2024-09-13T16:31:00.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.816+0000 7f372fbe8740 10 client.15420 fetch_fsmap learned FSMap version 28 2024-09-13T16:31:00.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.816+0000 7f372fbe8740 1 -- 172.21.15.16:0/3395004962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b37e794270 con 0x55b37e832500 2024-09-13T16:31:00.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.816+0000 7f370dffb640 1 -- 172.21.15.16:0/3395004962 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b37e4893c0 con 0x55b37e832500 2024-09-13T16:31:00.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.816+0000 7f370dffb640 1 -- 172.21.15.16:0/3395004962 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f37180d5d20 con 0x55b37e832500 2024-09-13T16:31:00.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.816+0000 7f372fbe8740 10 client.15420 fetch_fsmap finished waiting for FSMap version 28 2024-09-13T16:31:00.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.816+0000 7f372fbe8740 10 client.15420 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-13T16:31:00.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.816+0000 7f372fbe8740 20 client.15420 populate_metadata read hostname 'smithi016' 2024-09-13T16:31:00.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.816+0000 7f372fbe8740 1 --2- 172.21.15.16:0/3395004962 >> [v2:172.21.15.111:6832/1984275936,v1:172.21.15.111:6833/1984275936] conn(0x55b37e847df0 0x55b37e84bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.816+0000 7f372fbe8740 4 client.15420 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T16:31:00.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.816+0000 7f372fbe8740 1 -- 172.21.15.16:0/3395004962 --> [v2:172.21.15.111:6832/1984275936,v1:172.21.15.111:6833/1984275936] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b37e49a6c0 con 0x55b37e847df0 2024-09-13T16:31:00.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.816+0000 7f3725d9b640 1 --2- 172.21.15.16:0/3395004962 >> [v2:172.21.15.111:6832/1984275936,v1:172.21.15.111:6833/1984275936] conn(0x55b37e847df0 0x55b37e84bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.817+0000 7f370dffb640 10 client.15420 ms_handle_connect on v2:172.21.15.111:6832/1984275936 2024-09-13T16:31:00.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.817+0000 7f3725d9b640 1 --2- 172.21.15.16:0/3395004962 >> [v2:172.21.15.111:6832/1984275936,v1:172.21.15.111:6833/1984275936] conn(0x55b37e847df0 0x55b37e84bf70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8916abc1ba740298 server_cookie=854d757b2b843ad5 in_seq=0 out_seq=0 2024-09-13T16:31:00.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.817+0000 7f370dffb640 1 -- 172.21.15.16:0/3395004962 <== mds.4 v2:172.21.15.111:6832/1984275936 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b37e49a6c0 con 0x55b37e847df0 2024-09-13T16:31:00.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.817+0000 7f370dffb640 10 client.15420 handle_command_reply: tid=0 2024-09-13T16:31:00.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.817+0000 7f372fbe8740 1 client.15420 shutdown 2024-09-13T16:31:00.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.818+0000 7f372fbe8740 1 -- 172.21.15.16:0/3395004962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b37e832500 msgr2=0x55b37e83bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:00.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.818+0000 7f372fbe8740 1 --2- 172.21.15.16:0/3395004962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b37e832500 0x55b37e83bdd0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f37180baae0 tx=0x7f37180bec10 comp rx=0 tx=0).stop 2024-09-13T16:31:00.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.818+0000 7f372fbe8740 1 -- 172.21.15.16:0/3395004962 shutdown_connections 2024-09-13T16:31:00.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.818+0000 7f372fbe8740 1 --2- 172.21.15.16:0/3395004962 >> [v2:172.21.15.111:6832/1984275936,v1:172.21.15.111:6833/1984275936] conn(0x55b37e847df0 0x55b37e84bf70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.818+0000 7f372fbe8740 1 --2- 172.21.15.16:0/3395004962 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b37e837ed0 0x55b37e843300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.818+0000 7f372fbe8740 1 --2- 172.21.15.16:0/3395004962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b37e832e40 0x55b37e83c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.818+0000 7f372fbe8740 1 --2- 172.21.15.16:0/3395004962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b37e832500 0x55b37e83bdd0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.818+0000 7f370dffb640 0 client.15420 ms_handle_reset on v2:172.21.15.111:6832/1984275936 2024-09-13T16:31:00.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.818+0000 7f372fbe8740 1 -- 172.21.15.16:0/3395004962 >> 172.21.15.16:0/3395004962 conn(0x55b37e82db70 msgr2=0x55b37e82f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:00.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.818+0000 7f372fbe8740 1 -- 172.21.15.16:0/3395004962 shutdown_connections 2024-09-13T16:31:00.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.818+0000 7f372fbe8740 1 -- 172.21.15.16:0/3395004962 wait complete. 2024-09-13T16:31:00.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.818+0000 7f372fbe8740 20 client.15420 trim_cache size 0 max 16384 2024-09-13T16:31:00.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.832+0000 7f372fbe8740 1 Processor -- start 2024-09-13T16:31:00.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.832+0000 7f372fbe8740 1 -- start start 2024-09-13T16:31:00.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.833+0000 7f372fbe8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b37e832500 0x55b37e83bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.833+0000 7f372fbe8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b37e832e40 0x55b37e843220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.833+0000 7f372fbe8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b37e837ed0 0x55b37e83c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.833+0000 7f372fbe8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b37e719460 con 0x55b37e832500 2024-09-13T16:31:00.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.833+0000 7f372fbe8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b37e78d7c0 con 0x55b37e832e40 2024-09-13T16:31:00.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.833+0000 7f372fbe8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b37e83b4f0 con 0x55b37e837ed0 2024-09-13T16:31:00.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.833+0000 7f372659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b37e832500 0x55b37e83bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.833+0000 7f372659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b37e832500 0x55b37e83bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40744/0 (socket says 172.21.15.16:40744) 2024-09-13T16:31:00.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.833+0000 7f372659c640 1 -- 172.21.15.16:0/3747159112 learned_addr learned my addr 172.21.15.16:0/3747159112 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:00.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.833+0000 7f3726d9d640 1 --2- 172.21.15.16:0/3747159112 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b37e837ed0 0x55b37e83c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.833+0000 7f372659c640 1 -- 172.21.15.16:0/3747159112 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b37e837ed0 msgr2=0x55b37e83c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:00.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.833+0000 7f372659c640 1 --2- 172.21.15.16:0/3747159112 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b37e837ed0 0x55b37e83c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.833+0000 7f372659c640 1 -- 172.21.15.16:0/3747159112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b37e832e40 msgr2=0x55b37e843220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:00.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.833+0000 7f372659c640 1 --2- 172.21.15.16:0/3747159112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b37e832e40 0x55b37e843220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.833+0000 7f372659c640 1 -- 172.21.15.16:0/3747159112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b37e7365f0 con 0x55b37e832500 2024-09-13T16:31:00.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.833+0000 7f3726d9d640 1 --2- 172.21.15.16:0/3747159112 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b37e837ed0 0x55b37e83c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:31:00.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.833+0000 7f372659c640 1 --2- 172.21.15.16:0/3747159112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b37e832500 0x55b37e83bb80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f37180b72e0 tx=0x7f37180edf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1281c9c1c5a7da server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:00.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.833+0000 7f37157fa640 1 -- 172.21.15.16:0/3747159112 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371808ce20 con 0x55b37e832500 2024-09-13T16:31:00.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.833+0000 7f37157fa640 1 -- 172.21.15.16:0/3747159112 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f371808e030 con 0x55b37e832500 2024-09-13T16:31:00.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.833+0000 7f37157fa640 1 -- 172.21.15.16:0/3747159112 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37180ba4d0 con 0x55b37e832500 2024-09-13T16:31:00.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.834+0000 7f372fbe8740 1 -- 172.21.15.16:0/3747159112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b37e832500 msgr2=0x55b37e83bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:00.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.834+0000 7f372fbe8740 1 --2- 172.21.15.16:0/3747159112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b37e832500 0x55b37e83bb80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f37180b72e0 tx=0x7f37180edf80 comp rx=0 tx=0).stop 2024-09-13T16:31:00.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.834+0000 7f372fbe8740 1 -- 172.21.15.16:0/3747159112 shutdown_connections 2024-09-13T16:31:00.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.834+0000 7f372fbe8740 1 --2- 172.21.15.16:0/3747159112 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b37e837ed0 0x55b37e83c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.834+0000 7f372fbe8740 1 --2- 172.21.15.16:0/3747159112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b37e832e40 0x55b37e843220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.834+0000 7f372fbe8740 1 --2- 172.21.15.16:0/3747159112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b37e832500 0x55b37e83bb80 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.834+0000 7f372fbe8740 1 -- 172.21.15.16:0/3747159112 >> 172.21.15.16:0/3747159112 conn(0x55b37e82db70 msgr2=0x55b37e833d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:00.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.834+0000 7f372fbe8740 1 -- 172.21.15.16:0/3747159112 shutdown_connections 2024-09-13T16:31:00.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.834+0000 7f372fbe8740 1 -- 172.21.15.16:0/3747159112 wait complete. 2024-09-13T16:31:00.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.834+0000 7f372fbe8740 1 -- start start 2024-09-13T16:31:00.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.834+0000 7f372fbe8740 1 Processor -- start 2024-09-13T16:31:00.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.834+0000 7f372fbe8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b37e832500 0x55b37e83d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.834+0000 7f372fbe8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b37e832e40 0x55b37e83dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.834+0000 7f372fbe8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b37e837ed0 0x55b37e842d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.834+0000 7f372fbe8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b37e8432b0 con 0x55b37e837ed0 2024-09-13T16:31:00.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.834+0000 7f372fbe8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b37e843420 con 0x55b37e832e40 2024-09-13T16:31:00.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.834+0000 7f372fbe8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b37e8435e0 con 0x55b37e832500 2024-09-13T16:31:00.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.834+0000 7f372fbe8740 10 client.0 _ll_register_callbacks cb 0x7f3718005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:31:00.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.834+0000 7f3726d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b37e837ed0 0x55b37e842d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.835+0000 7f3726d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b37e837ed0 0x55b37e842d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40758/0 (socket says 172.21.15.16:40758) 2024-09-13T16:31:00.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.835+0000 7f3726d9d640 1 -- 172.21.15.16:0/1776304101 learned_addr learned my addr 172.21.15.16:0/1776304101 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:00.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.835+0000 7f372659c640 1 --2- 172.21.15.16:0/1776304101 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b37e832500 0x55b37e83d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.835+0000 7f3725d9b640 1 --2- 172.21.15.16:0/1776304101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b37e832e40 0x55b37e83dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.835+0000 7f3726d9d640 1 -- 172.21.15.16:0/1776304101 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b37e832500 msgr2=0x55b37e83d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:00.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.835+0000 7f3726d9d640 1 --2- 172.21.15.16:0/1776304101 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b37e832500 0x55b37e83d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.835+0000 7f3726d9d640 1 -- 172.21.15.16:0/1776304101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b37e832e40 msgr2=0x55b37e83dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:00.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.835+0000 7f3726d9d640 1 --2- 172.21.15.16:0/1776304101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b37e832e40 0x55b37e83dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.835+0000 7f3726d9d640 1 -- 172.21.15.16:0/1776304101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b37e794270 con 0x55b37e837ed0 2024-09-13T16:31:00.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.835+0000 7f3726d9d640 1 --2- 172.21.15.16:0/1776304101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b37e837ed0 0x55b37e842d70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3720010db0 tx=0x7f372001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79cfcbb88b521206 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:00.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.835+0000 7f370dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:31:00.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.835+0000 7f370dffb640 1 -- 172.21.15.16:0/1776304101 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37200126f0 con 0x55b37e837ed0 2024-09-13T16:31:00.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.835+0000 7f370dffb640 1 -- 172.21.15.16:0/1776304101 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f3720004040 con 0x55b37e837ed0 2024-09-13T16:31:00.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.835+0000 7f372fbe8740 1 -- 172.21.15.16:0/1776304101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f37180bb4e0 con 0x55b37e837ed0 2024-09-13T16:31:00.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.835+0000 7f370dffb640 1 -- 172.21.15.16:0/1776304101 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3720004200 con 0x55b37e837ed0 2024-09-13T16:31:00.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.835+0000 7f372fbe8740 10 client.15426 fetch_fsmap learned FSMap version 28 2024-09-13T16:31:00.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.835+0000 7f372fbe8740 1 -- 172.21.15.16:0/1776304101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b37e75bd00 con 0x55b37e837ed0 2024-09-13T16:31:00.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.836+0000 7f370dffb640 1 -- 172.21.15.16:0/1776304101 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f37180bb4e0 con 0x55b37e837ed0 2024-09-13T16:31:00.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.836+0000 7f370dffb640 1 -- 172.21.15.16:0/1776304101 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3720011c70 con 0x55b37e837ed0 2024-09-13T16:31:00.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.836+0000 7f372fbe8740 10 client.15426 fetch_fsmap finished waiting for FSMap version 28 2024-09-13T16:31:00.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.836+0000 7f372fbe8740 10 client.15426 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-13T16:31:00.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.836+0000 7f372fbe8740 20 client.15426 populate_metadata read hostname 'smithi016' 2024-09-13T16:31:00.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.836+0000 7f372fbe8740 1 --2- 172.21.15.16:0/1776304101 >> [v2:172.21.15.111:6832/1984275936,v1:172.21.15.111:6833/1984275936] conn(0x55b37e8a2700 0x55b37e89f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.836+0000 7f372fbe8740 4 client.15426 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T16:31:00.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.836+0000 7f372fbe8740 1 -- 172.21.15.16:0/1776304101 --> [v2:172.21.15.111:6832/1984275936,v1:172.21.15.111:6833/1984275936] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b37e4edd10 con 0x55b37e8a2700 2024-09-13T16:31:00.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.836+0000 7f372659c640 1 --2- 172.21.15.16:0/1776304101 >> [v2:172.21.15.111:6832/1984275936,v1:172.21.15.111:6833/1984275936] conn(0x55b37e8a2700 0x55b37e89f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.836+0000 7f372659c640 1 --2- 172.21.15.16:0/1776304101 >> [v2:172.21.15.111:6832/1984275936,v1:172.21.15.111:6833/1984275936] conn(0x55b37e8a2700 0x55b37e89f080 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6a33dcfcbd14cbab server_cookie=79453903fa314bcb in_seq=0 out_seq=0 2024-09-13T16:31:00.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.836+0000 7f370dffb640 10 client.15426 ms_handle_connect on v2:172.21.15.111:6832/1984275936 2024-09-13T16:31:00.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.837+0000 7f370dffb640 1 -- 172.21.15.16:0/1776304101 <== mds.4 v2:172.21.15.111:6832/1984275936 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x55b37e4edd10 con 0x55b37e8a2700 2024-09-13T16:31:00.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.837+0000 7f370dffb640 10 client.15426 handle_command_reply: tid=0 2024-09-13T16:31:00.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.837+0000 7f372fbe8740 1 client.15426 shutdown 2024-09-13T16:31:00.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.837+0000 7f372fbe8740 1 -- 172.21.15.16:0/1776304101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b37e837ed0 msgr2=0x55b37e842d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:00.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.837+0000 7f372fbe8740 1 --2- 172.21.15.16:0/1776304101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b37e837ed0 0x55b37e842d70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3720010db0 tx=0x7f372001c000 comp rx=0 tx=0).stop 2024-09-13T16:31:00.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.837+0000 7f372fbe8740 1 -- 172.21.15.16:0/1776304101 shutdown_connections 2024-09-13T16:31:00.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.837+0000 7f372fbe8740 1 --2- 172.21.15.16:0/1776304101 >> [v2:172.21.15.111:6832/1984275936,v1:172.21.15.111:6833/1984275936] conn(0x55b37e8a2700 0x55b37e89f080 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.837+0000 7f372fbe8740 1 --2- 172.21.15.16:0/1776304101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b37e837ed0 0x55b37e842d70 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.837+0000 7f372fbe8740 1 --2- 172.21.15.16:0/1776304101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b37e832e40 0x55b37e83dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.837+0000 7f372fbe8740 1 --2- 172.21.15.16:0/1776304101 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b37e832500 0x55b37e83d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.837+0000 7f370dffb640 0 client.15426 ms_handle_reset on v2:172.21.15.111:6832/1984275936 2024-09-13T16:31:00.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.837+0000 7f372fbe8740 1 -- 172.21.15.16:0/1776304101 >> 172.21.15.16:0/1776304101 conn(0x55b37e82db70 msgr2=0x55b37e83bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:00.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.837+0000 7f372fbe8740 1 -- 172.21.15.16:0/1776304101 shutdown_connections 2024-09-13T16:31:00.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.837+0000 7f372fbe8740 1 -- 172.21.15.16:0/1776304101 wait complete. 2024-09-13T16:31:00.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.837+0000 7f372fbe8740 20 client.15426 trim_cache size 0 max 16384 2024-09-13T16:31:00.848 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-13T16:31:00.848 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "0d241f6a-71ec-11ef-bceb-c7b262605968", 2024-09-13T16:31:00.848 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 4, 2024-09-13T16:31:00.848 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24418, 2024-09-13T16:31:00.848 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-13T16:31:00.848 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-13T16:31:00.848 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-13T16:31:00.848 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 116.210168333, 2024-09-13T16:31:00.848 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 28, 2024-09-13T16:31:00.848 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 93, 2024-09-13T16:31:00.849 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 91, 2024-09-13T16:31:00.849 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 174.446862337 2024-09-13T16:31:00.849 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-13T16:31:00.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.837+0000 7f372cec9640 1 -- 172.21.15.16:0/536348231 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f37100a1090 msgr2=0x7f37100a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:00.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.837+0000 7f372cec9640 1 --2- 172.21.15.16:0/536348231 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f37100a1090 0x7f37100a3550 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3720009270 tx=0x7f3720009e30 comp rx=0 tx=0).stop 2024-09-13T16:31:00.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.837+0000 7f372cec9640 1 -- 172.21.15.16:0/536348231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37280fec70 msgr2=0x7f3728102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:00.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.837+0000 7f372cec9640 1 --2- 172.21.15.16:0/536348231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37280fec70 0x7f3728102c00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3718009230 tx=0x7f37180042f0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.837+0000 7f372cec9640 1 -- 172.21.15.16:0/536348231 shutdown_connections 2024-09-13T16:31:00.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.837+0000 7f372cec9640 1 --2- 172.21.15.16:0/536348231 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f37100a1090 0x7f37100a3550 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.837+0000 7f372cec9640 1 --2- 172.21.15.16:0/536348231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3728100820 0x7f3728101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.837+0000 7f372cec9640 1 --2- 172.21.15.16:0/536348231 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37280ffe60 0x7f3728101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.837+0000 7f372cec9640 1 --2- 172.21.15.16:0/536348231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37280fec70 0x7f3728102c00 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.837+0000 7f372cec9640 1 -- 172.21.15.16:0/536348231 >> 172.21.15.16:0/536348231 conn(0x7f37280fa3b0 msgr2=0x7f37280685e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.837+0000 7f372cec9640 1 -- 172.21.15.16:0/536348231 shutdown_connections 2024-09-13T16:31:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.837+0000 7f372cec9640 1 -- 172.21.15.16:0/536348231 wait complete. 2024-09-13T16:31:00.852 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-09-13T16:31:00.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.953+0000 7fe3d54d0640 1 -- 172.21.15.16:0/433882675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3d0100820 msgr2=0x7fe3d01070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:00.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.953+0000 7fe3d54d0640 1 --2- 172.21.15.16:0/433882675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3d0100820 0x7fe3d01070b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8009ca0 tx=0x7fe3c802a8c0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.953+0000 7fe3d54d0640 1 -- 172.21.15.16:0/433882675 shutdown_connections 2024-09-13T16:31:00.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.953+0000 7fe3d54d0640 1 --2- 172.21.15.16:0/433882675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3d0100820 0x7fe3d01070b0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.953+0000 7fe3d54d0640 1 --2- 172.21.15.16:0/433882675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3d00ffe60 0x7fe3d01002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.953+0000 7fe3d54d0640 1 --2- 172.21.15.16:0/433882675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3d00fec70 0x7fe3d00ff070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.953+0000 7fe3d54d0640 1 -- 172.21.15.16:0/433882675 >> 172.21.15.16:0/433882675 conn(0x7fe3d00fa410 msgr2=0x7fe3d00fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:00.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.953+0000 7fe3d54d0640 1 -- 172.21.15.16:0/433882675 shutdown_connections 2024-09-13T16:31:00.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.953+0000 7fe3d54d0640 1 -- 172.21.15.16:0/433882675 wait complete. 2024-09-13T16:31:00.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.954+0000 7fe3d54d0640 1 Processor -- start 2024-09-13T16:31:00.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.954+0000 7fe3d54d0640 1 -- start start 2024-09-13T16:31:00.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.954+0000 7fe3d54d0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3d00fec70 0x7fe3d0102c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.954+0000 7fe3d54d0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3d00ffe60 0x7fe3d0101240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.954+0000 7fe3d54d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3d0100820 0x7fe3d0101780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.954+0000 7fe3d54d0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d0101ee0 con 0x7fe3d0100820 2024-09-13T16:31:00.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.954+0000 7fe3d54d0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d0102050 con 0x7fe3d00ffe60 2024-09-13T16:31:00.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.954+0000 7fe3d54d0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d019fc80 con 0x7fe3d00fec70 2024-09-13T16:31:00.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.954+0000 7fe3cf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3d0100820 0x7fe3d0101780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.954+0000 7fe3cf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3d0100820 0x7fe3d0101780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40776/0 (socket says 172.21.15.16:40776) 2024-09-13T16:31:00.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.954+0000 7fe3cf7fe640 1 -- 172.21.15.16:0/1321451280 learned_addr learned my addr 172.21.15.16:0/1321451280 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:00.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.955+0000 7fe3ceffd640 1 --2- 172.21.15.16:0/1321451280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3d00fec70 0x7fe3d0102c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.955+0000 7fe3ce7fc640 1 --2- 172.21.15.16:0/1321451280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3d00ffe60 0x7fe3d0101240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.955+0000 7fe3cf7fe640 1 -- 172.21.15.16:0/1321451280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3d00fec70 msgr2=0x7fe3d0102c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:00.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.955+0000 7fe3cf7fe640 1 --2- 172.21.15.16:0/1321451280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3d00fec70 0x7fe3d0102c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.955+0000 7fe3cf7fe640 1 -- 172.21.15.16:0/1321451280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3d00ffe60 msgr2=0x7fe3d0101240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:00.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.955+0000 7fe3cf7fe640 1 --2- 172.21.15.16:0/1321451280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3d00ffe60 0x7fe3d0101240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.955+0000 7fe3cf7fe640 1 -- 172.21.15.16:0/1321451280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c8009900 con 0x7fe3d0100820 2024-09-13T16:31:00.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.955+0000 7fe3ceffd640 1 --2- 172.21.15.16:0/1321451280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3d00fec70 0x7fe3d0102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:31:00.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.955+0000 7fe3cf7fe640 1 --2- 172.21.15.16:0/1321451280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3d0100820 0x7fe3d0101780 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8009430 tx=0x7fe3c8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:00.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.955+0000 7fe3b7fff640 1 -- 172.21.15.16:0/1321451280 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c80307c0 con 0x7fe3d0100820 2024-09-13T16:31:00.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.955+0000 7fe3b7fff640 1 -- 172.21.15.16:0/1321451280 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe3c8040070 con 0x7fe3d0100820 2024-09-13T16:31:00.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.955+0000 7fe3d54d0640 1 -- 172.21.15.16:0/1321451280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3d019fed0 con 0x7fe3d0100820 2024-09-13T16:31:00.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.955+0000 7fe3b7fff640 1 -- 172.21.15.16:0/1321451280 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8004570 con 0x7fe3d0100820 2024-09-13T16:31:00.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.955+0000 7fe3d54d0640 1 -- 172.21.15.16:0/1321451280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d01a03a0 con 0x7fe3d0100820 2024-09-13T16:31:00.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.957+0000 7fe3b7fff640 1 -- 172.21.15.16:0/1321451280 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe3c8042030 con 0x7fe3d0100820 2024-09-13T16:31:00.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.958+0000 7fe3d81ee740 1 Processor -- start 2024-09-13T16:31:00.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.958+0000 7fe3d81ee740 1 -- start start 2024-09-13T16:31:00.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.958+0000 7fe3d81ee740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf5f79e500 0x55cf5f79e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.958+0000 7fe3d81ee740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf5f79ee40 0x55cf5f7a3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.958+0000 7fe3d81ee740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5f7a3ed0 0x55cf5f7a6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.958+0000 7fe3d81ee740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf5f6f97c0 con 0x55cf5f79ee40 2024-09-13T16:31:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.958+0000 7fe3d81ee740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf5f708b70 con 0x55cf5f79e500 2024-09-13T16:31:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.958+0000 7fe3d81ee740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf5f706300 con 0x55cf5f7a3ed0 2024-09-13T16:31:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.958+0000 7fe3b7fff640 1 --2- 172.21.15.16:0/1321451280 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe3b80a1110 0x7fe3b80a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.958+0000 7fe3ceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf5f79ee40 0x55cf5f7a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.958+0000 7fe3ceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf5f79ee40 0x55cf5f7a3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40788/0 (socket says 172.21.15.16:40788) 2024-09-13T16:31:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.958+0000 7fe3ceffd640 1 -- 172.21.15.16:0/3598463905 learned_addr learned my addr 172.21.15.16:0/3598463905 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.958+0000 7fe3cf7fe640 1 --2- 172.21.15.16:0/1321451280 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe3b80a1110 0x7fe3b80a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.958+0000 7fe3ce7fc640 1 --2- 172.21.15.16:0/3598463905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf5f79e500 0x55cf5f79e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.958+0000 7fe3ce7fc640 1 --2- 172.21.15.16:0/3598463905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5f7a3ed0 0x55cf5f7a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.958+0000 7fe3b7fff640 1 -- 172.21.15.16:0/1321451280 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fe3c80ee410 con 0x7fe3d0100820 2024-09-13T16:31:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.958+0000 7fe3ceffd640 1 -- 172.21.15.16:0/3598463905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5f7a3ed0 msgr2=0x55cf5f7a6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.958+0000 7fe3ceffd640 1 --2- 172.21.15.16:0/3598463905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5f7a3ed0 0x55cf5f7a6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.958+0000 7fe3ceffd640 1 -- 172.21.15.16:0/3598463905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf5f79e500 msgr2=0x55cf5f79e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.958+0000 7fe3ceffd640 1 --2- 172.21.15.16:0/3598463905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf5f79e500 0x55cf5f79e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.958+0000 7fe3ceffd640 1 -- 172.21.15.16:0/3598463905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf5f68e4b0 con 0x55cf5f79ee40 2024-09-13T16:31:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.958+0000 7fe3ce7fc640 1 --2- 172.21.15.16:0/3598463905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf5f79e500 0x55cf5f79e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:31:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.959+0000 7fe3ceffd640 1 --2- 172.21.15.16:0/3598463905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf5f79ee40 0x55cf5f7a3900 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000bd40 tx=0x7fe3c000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd914ed6675c0144 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.959+0000 7fe3cf7fe640 1 --2- 172.21.15.16:0/1321451280 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe3b80a1110 0x7fe3b80a35d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80c3400 tx=0x7fe3c8086990 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:00.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.959+0000 7fe3b57fa640 1 -- 172.21.15.16:0/3598463905 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0011640 con 0x55cf5f79ee40 2024-09-13T16:31:00.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.959+0000 7fe3b57fa640 1 -- 172.21.15.16:0/3598463905 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe3c000be30 con 0x55cf5f79ee40 2024-09-13T16:31:00.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.959+0000 7fe3b57fa640 1 -- 172.21.15.16:0/3598463905 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c00109a0 con 0x55cf5f79ee40 2024-09-13T16:31:00.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.959+0000 7fe3d81ee740 1 -- 172.21.15.16:0/3598463905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf5f79ee40 msgr2=0x55cf5f7a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:00.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.959+0000 7fe3d81ee740 1 --2- 172.21.15.16:0/3598463905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf5f79ee40 0x55cf5f7a3900 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000bd40 tx=0x7fe3c000f730 comp rx=0 tx=0).stop 2024-09-13T16:31:00.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.959+0000 7fe3d81ee740 1 -- 172.21.15.16:0/3598463905 shutdown_connections 2024-09-13T16:31:00.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.959+0000 7fe3d81ee740 1 --2- 172.21.15.16:0/3598463905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5f7a3ed0 0x55cf5f7a6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.959+0000 7fe3d81ee740 1 --2- 172.21.15.16:0/3598463905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf5f79ee40 0x55cf5f7a3900 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.959+0000 7fe3d81ee740 1 --2- 172.21.15.16:0/3598463905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf5f79e500 0x55cf5f79e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.959+0000 7fe3d81ee740 1 -- 172.21.15.16:0/3598463905 >> 172.21.15.16:0/3598463905 conn(0x55cf5f799b70 msgr2=0x55cf5f799f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:00.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.960+0000 7fe3d81ee740 1 -- 172.21.15.16:0/3598463905 shutdown_connections 2024-09-13T16:31:00.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.960+0000 7fe3d81ee740 1 -- 172.21.15.16:0/3598463905 wait complete. 2024-09-13T16:31:00.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.960+0000 7fe3d81ee740 1 -- start start 2024-09-13T16:31:00.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.960+0000 7fe3d81ee740 1 Processor -- start 2024-09-13T16:31:00.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.960+0000 7fe3d81ee740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf5f79e500 0x55cf5f7a8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.960+0000 7fe3d81ee740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf5f79ee40 0x55cf5f7a86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.960+0000 7fe3d81ee740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5f7a3ed0 0x55cf5f7af6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.960+0000 7fe3d81ee740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf5f701bf0 con 0x55cf5f79ee40 2024-09-13T16:31:00.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.960+0000 7fe3d81ee740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf5f6c0250 con 0x55cf5f79e500 2024-09-13T16:31:00.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.960+0000 7fe3d81ee740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf5f6bfef0 con 0x55cf5f7a3ed0 2024-09-13T16:31:00.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.960+0000 7fe3d81ee740 10 client.0 _ll_register_callbacks cb 0x55cf5f6f2740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:31:00.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.960+0000 7fe3ceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf5f79ee40 0x55cf5f7a86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.960+0000 7fe3ceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf5f79ee40 0x55cf5f7a86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40800/0 (socket says 172.21.15.16:40800) 2024-09-13T16:31:00.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.960+0000 7fe3ceffd640 1 -- 172.21.15.16:0/2218487829 learned_addr learned my addr 172.21.15.16:0/2218487829 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:00.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.961+0000 7fe3ce7fc640 1 --2- 172.21.15.16:0/2218487829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf5f79e500 0x55cf5f7a8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.961+0000 7fe3ceffd640 1 -- 172.21.15.16:0/2218487829 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5f7a3ed0 msgr2=0x55cf5f7af6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:00.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.961+0000 7fe3ceffd640 1 --2- 172.21.15.16:0/2218487829 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5f7a3ed0 0x55cf5f7af6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.961+0000 7fe3ceffd640 1 -- 172.21.15.16:0/2218487829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf5f79e500 msgr2=0x55cf5f7a8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:00.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.961+0000 7fe3ceffd640 1 --2- 172.21.15.16:0/2218487829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf5f79e500 0x55cf5f7a8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.961+0000 7fe3ceffd640 1 -- 172.21.15.16:0/2218487829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c000b9a0 con 0x55cf5f79ee40 2024-09-13T16:31:00.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.961+0000 7fe3ceffd640 1 --2- 172.21.15.16:0/2218487829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf5f79ee40 0x55cf5f7a86b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0000c00 tx=0x7fe3c0005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb58d87f1e7d314 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:00.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.961+0000 7fe3b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:31:00.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.961+0000 7fe3b2ff5640 1 -- 172.21.15.16:0/2218487829 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c001f040 con 0x55cf5f79ee40 2024-09-13T16:31:00.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.961+0000 7fe3b2ff5640 1 -- 172.21.15.16:0/2218487829 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fe3c00108e0 con 0x55cf5f79ee40 2024-09-13T16:31:00.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.961+0000 7fe3d81ee740 1 -- 172.21.15.16:0/2218487829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cf5f3f53c0 con 0x55cf5f79ee40 2024-09-13T16:31:00.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.961+0000 7fe3b2ff5640 1 -- 172.21.15.16:0/2218487829 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3c0005670 con 0x55cf5f79ee40 2024-09-13T16:31:00.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.961+0000 7fe3d81ee740 10 client.15438 fetch_fsmap learned FSMap version 28 2024-09-13T16:31:00.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.961+0000 7fe3d81ee740 1 -- 172.21.15.16:0/2218487829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55cf5f700270 con 0x55cf5f79ee40 2024-09-13T16:31:00.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.961+0000 7fe3b2ff5640 1 -- 172.21.15.16:0/2218487829 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cf5f3f53c0 con 0x55cf5f79ee40 2024-09-13T16:31:00.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.962+0000 7fe3b2ff5640 1 -- 172.21.15.16:0/2218487829 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe3c0011440 con 0x55cf5f79ee40 2024-09-13T16:31:00.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.962+0000 7fe3d81ee740 10 client.15438 fetch_fsmap finished waiting for FSMap version 28 2024-09-13T16:31:00.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.962+0000 7fe3d81ee740 10 client.15438 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-13T16:31:00.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.962+0000 7fe3d81ee740 20 client.15438 populate_metadata read hostname 'smithi016' 2024-09-13T16:31:00.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.962+0000 7fe3d81ee740 1 --2- 172.21.15.16:0/2218487829 >> [v2:172.21.15.107:6832/3797712690,v1:172.21.15.107:6833/3797712690] conn(0x55cf5f7b4520 0x55cf5f7b86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.962+0000 7fe3d81ee740 4 client.15438 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T16:31:00.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.962+0000 7fe3d81ee740 1 -- 172.21.15.16:0/2218487829 --> [v2:172.21.15.107:6832/3797712690,v1:172.21.15.107:6833/3797712690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cf5f4066c0 con 0x55cf5f7b4520 2024-09-13T16:31:00.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.962+0000 7fe3ce7fc640 1 --2- 172.21.15.16:0/2218487829 >> [v2:172.21.15.107:6832/3797712690,v1:172.21.15.107:6833/3797712690] conn(0x55cf5f7b4520 0x55cf5f7b86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.962+0000 7fe3b2ff5640 10 client.15438 ms_handle_connect on v2:172.21.15.107:6832/3797712690 2024-09-13T16:31:00.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.962+0000 7fe3ce7fc640 1 --2- 172.21.15.16:0/2218487829 >> [v2:172.21.15.107:6832/3797712690,v1:172.21.15.107:6833/3797712690] conn(0x55cf5f7b4520 0x55cf5f7b86c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f8278925f69fe7aa server_cookie=4251832fe1b284da in_seq=0 out_seq=0 2024-09-13T16:31:00.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.963+0000 7fe3b2ff5640 1 -- 172.21.15.16:0/2218487829 <== mds.3 v2:172.21.15.107:6832/3797712690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cf5f4066c0 con 0x55cf5f7b4520 2024-09-13T16:31:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.963+0000 7fe3b2ff5640 10 client.15438 handle_command_reply: tid=0 2024-09-13T16:31:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.963+0000 7fe3d81ee740 1 client.15438 shutdown 2024-09-13T16:31:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.963+0000 7fe3d81ee740 1 -- 172.21.15.16:0/2218487829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf5f79ee40 msgr2=0x55cf5f7a86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.963+0000 7fe3d81ee740 1 --2- 172.21.15.16:0/2218487829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf5f79ee40 0x55cf5f7a86b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0000c00 tx=0x7fe3c0005270 comp rx=0 tx=0).stop 2024-09-13T16:31:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.963+0000 7fe3d81ee740 1 -- 172.21.15.16:0/2218487829 shutdown_connections 2024-09-13T16:31:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.963+0000 7fe3d81ee740 1 --2- 172.21.15.16:0/2218487829 >> [v2:172.21.15.107:6832/3797712690,v1:172.21.15.107:6833/3797712690] conn(0x55cf5f7b4520 0x55cf5f7b86c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.963+0000 7fe3d81ee740 1 --2- 172.21.15.16:0/2218487829 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5f7a3ed0 0x55cf5f7af6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.963+0000 7fe3d81ee740 1 --2- 172.21.15.16:0/2218487829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf5f79ee40 0x55cf5f7a86b0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.963+0000 7fe3b2ff5640 0 client.15438 ms_handle_reset on v2:172.21.15.107:6832/3797712690 2024-09-13T16:31:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.963+0000 7fe3d81ee740 1 --2- 172.21.15.16:0/2218487829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf5f79e500 0x55cf5f7a8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.963+0000 7fe3d81ee740 1 -- 172.21.15.16:0/2218487829 >> 172.21.15.16:0/2218487829 conn(0x55cf5f799b70 msgr2=0x55cf5f79b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.963+0000 7fe3d81ee740 1 -- 172.21.15.16:0/2218487829 shutdown_connections 2024-09-13T16:31:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.963+0000 7fe3d81ee740 1 -- 172.21.15.16:0/2218487829 wait complete. 2024-09-13T16:31:00.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.963+0000 7fe3d81ee740 20 client.15438 trim_cache size 0 max 16384 2024-09-13T16:31:00.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.978+0000 7fe3d81ee740 1 Processor -- start 2024-09-13T16:31:00.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.978+0000 7fe3d81ee740 1 -- start start 2024-09-13T16:31:00.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.978+0000 7fe3d81ee740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf5f799b70 0x55cf5f7a7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.978+0000 7fe3d81ee740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf5f79e500 0x55cf5f7af6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.978+0000 7fe3d81ee740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5f79ee40 0x55cf5f7a8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.978+0000 7fe3d81ee740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf5f7999d0 con 0x55cf5f799b70 2024-09-13T16:31:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.978+0000 7fe3d81ee740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf5f685460 con 0x55cf5f79e500 2024-09-13T16:31:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.978+0000 7fe3d81ee740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf5f7a8ed0 con 0x55cf5f79ee40 2024-09-13T16:31:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.978+0000 7fe3ce7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf5f799b70 0x55cf5f7a7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.978+0000 7fe3ce7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf5f799b70 0x55cf5f7a7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40808/0 (socket says 172.21.15.16:40808) 2024-09-13T16:31:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.978+0000 7fe3ce7fc640 1 -- 172.21.15.16:0/1801748701 learned_addr learned my addr 172.21.15.16:0/1801748701 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.978+0000 7fe3ce7fc640 1 -- 172.21.15.16:0/1801748701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5f79ee40 msgr2=0x55cf5f7a8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.978+0000 7fe3ce7fc640 1 --2- 172.21.15.16:0/1801748701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5f79ee40 0x55cf5f7a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.978+0000 7fe3ce7fc640 1 -- 172.21.15.16:0/1801748701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf5f79e500 msgr2=0x55cf5f7af6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:00.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.978+0000 7fe3ce7fc640 1 --2- 172.21.15.16:0/1801748701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf5f79e500 0x55cf5f7af6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.979+0000 7fe3ce7fc640 1 -- 172.21.15.16:0/1801748701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf5f6a25f0 con 0x55cf5f799b70 2024-09-13T16:31:00.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.979+0000 7fe3ce7fc640 1 --2- 172.21.15.16:0/1801748701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf5f799b70 0x55cf5f7a7f20 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4000c00 tx=0x7fe3c4005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90d24f9b7bb6946 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:00.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.979+0000 7fe3b5ffb640 1 -- 172.21.15.16:0/1801748701 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c401cde0 con 0x55cf5f799b70 2024-09-13T16:31:00.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.979+0000 7fe3b5ffb640 1 -- 172.21.15.16:0/1801748701 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe3c401c6c0 con 0x55cf5f799b70 2024-09-13T16:31:00.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.979+0000 7fe3b5ffb640 1 -- 172.21.15.16:0/1801748701 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4011af0 con 0x55cf5f799b70 2024-09-13T16:31:00.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.979+0000 7fe3d81ee740 1 -- 172.21.15.16:0/1801748701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf5f799b70 msgr2=0x55cf5f7a7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:00.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.979+0000 7fe3d81ee740 1 --2- 172.21.15.16:0/1801748701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf5f799b70 0x55cf5f7a7f20 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4000c00 tx=0x7fe3c4005d40 comp rx=0 tx=0).stop 2024-09-13T16:31:00.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.979+0000 7fe3d81ee740 1 -- 172.21.15.16:0/1801748701 shutdown_connections 2024-09-13T16:31:00.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.979+0000 7fe3d81ee740 1 --2- 172.21.15.16:0/1801748701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5f79ee40 0x55cf5f7a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.979+0000 7fe3d81ee740 1 --2- 172.21.15.16:0/1801748701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf5f79e500 0x55cf5f7af6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.979+0000 7fe3d81ee740 1 --2- 172.21.15.16:0/1801748701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf5f799b70 0x55cf5f7a7f20 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.979+0000 7fe3d81ee740 1 -- 172.21.15.16:0/1801748701 >> 172.21.15.16:0/1801748701 conn(0x55cf5f7a6630 msgr2=0x55cf5f79fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:00.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.979+0000 7fe3d81ee740 1 -- 172.21.15.16:0/1801748701 shutdown_connections 2024-09-13T16:31:00.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.980+0000 7fe3d81ee740 1 -- 172.21.15.16:0/1801748701 wait complete. 2024-09-13T16:31:00.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.980+0000 7fe3d81ee740 1 -- start start 2024-09-13T16:31:00.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.980+0000 7fe3d81ee740 1 Processor -- start 2024-09-13T16:31:00.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.980+0000 7fe3d81ee740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf5f799b70 0x55cf5f7ac410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.980+0000 7fe3d81ee740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf5f79e500 0x55cf5f7ac950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.980+0000 7fe3d81ee740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5f79ee40 0x55cf5f80a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.980+0000 7fe3d81ee740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf5f7ace90 con 0x55cf5f799b70 2024-09-13T16:31:00.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.980+0000 7fe3d81ee740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf5f7ad000 con 0x55cf5f79e500 2024-09-13T16:31:00.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.980+0000 7fe3d81ee740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf5f80a8c0 con 0x55cf5f79ee40 2024-09-13T16:31:00.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.980+0000 7fe3d81ee740 10 client.0 _ll_register_callbacks cb 0x7fe3c000b500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:31:00.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.980+0000 7fe3ce7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf5f799b70 0x55cf5f7ac410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.980+0000 7fe3ce7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf5f799b70 0x55cf5f7ac410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40818/0 (socket says 172.21.15.16:40818) 2024-09-13T16:31:00.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.980+0000 7fe3ce7fc640 1 -- 172.21.15.16:0/2324210574 learned_addr learned my addr 172.21.15.16:0/2324210574 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:00.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.980+0000 7fe3ce7fc640 1 --2- 172.21.15.16:0/2324210574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5f79ee40 0x55cf5f80a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.980+0000 7fe3ceffd640 1 --2- 172.21.15.16:0/2324210574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf5f79e500 0x55cf5f7ac950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.980+0000 7fe3ce7fc640 1 -- 172.21.15.16:0/2324210574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5f79ee40 msgr2=0x55cf5f80a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:00.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.980+0000 7fe3ce7fc640 1 --2- 172.21.15.16:0/2324210574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5f79ee40 0x55cf5f80a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.980+0000 7fe3ce7fc640 1 -- 172.21.15.16:0/2324210574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf5f79e500 msgr2=0x55cf5f7ac950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:00.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.980+0000 7fe3ce7fc640 1 --2- 172.21.15.16:0/2324210574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf5f79e500 0x55cf5f7ac950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.980+0000 7fe3ce7fc640 1 -- 172.21.15.16:0/2324210574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c400ce80 con 0x55cf5f799b70 2024-09-13T16:31:00.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.981+0000 7fe3ce7fc640 1 --2- 172.21.15.16:0/2324210574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf5f799b70 0x55cf5f7ac410 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4006010 tx=0x7fe3c4005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=523dfb6e7d959466 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:00.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.981+0000 7fe3b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:31:00.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.981+0000 7fe3b2ff5640 1 -- 172.21.15.16:0/2324210574 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4011870 con 0x55cf5f799b70 2024-09-13T16:31:00.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.981+0000 7fe3b2ff5640 1 -- 172.21.15.16:0/2324210574 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fe3c40119d0 con 0x55cf5f799b70 2024-09-13T16:31:00.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.981+0000 7fe3d81ee740 1 -- 172.21.15.16:0/2324210574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3c000be30 con 0x55cf5f799b70 2024-09-13T16:31:00.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.981+0000 7fe3b2ff5640 1 -- 172.21.15.16:0/2324210574 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3c4011b90 con 0x55cf5f799b70 2024-09-13T16:31:00.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.981+0000 7fe3d81ee740 10 client.15444 fetch_fsmap learned FSMap version 28 2024-09-13T16:31:00.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.981+0000 7fe3d81ee740 1 -- 172.21.15.16:0/2324210574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55cf5f6c7d00 con 0x55cf5f799b70 2024-09-13T16:31:00.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.981+0000 7fe3b2ff5640 1 -- 172.21.15.16:0/2324210574 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe3c000be30 con 0x55cf5f799b70 2024-09-13T16:31:00.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.981+0000 7fe3b2ff5640 1 -- 172.21.15.16:0/2324210574 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe3c4027c00 con 0x55cf5f799b70 2024-09-13T16:31:00.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.981+0000 7fe3d81ee740 10 client.15444 fetch_fsmap finished waiting for FSMap version 28 2024-09-13T16:31:00.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.981+0000 7fe3d81ee740 10 client.15444 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-13T16:31:00.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.981+0000 7fe3d81ee740 20 client.15444 populate_metadata read hostname 'smithi016' 2024-09-13T16:31:00.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.981+0000 7fe3d81ee740 1 --2- 172.21.15.16:0/2324210574 >> [v2:172.21.15.107:6832/3797712690,v1:172.21.15.107:6833/3797712690] conn(0x55cf5f7a3ed0 0x55cf5f80bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:00.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.981+0000 7fe3d81ee740 4 client.15444 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T16:31:00.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.981+0000 7fe3d81ee740 1 -- 172.21.15.16:0/2324210574 --> [v2:172.21.15.107:6832/3797712690,v1:172.21.15.107:6833/3797712690] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cf5f459d10 con 0x55cf5f7a3ed0 2024-09-13T16:31:00.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.982+0000 7fe3ceffd640 1 --2- 172.21.15.16:0/2324210574 >> [v2:172.21.15.107:6832/3797712690,v1:172.21.15.107:6833/3797712690] conn(0x55cf5f7a3ed0 0x55cf5f80bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:00.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.982+0000 7fe3ceffd640 1 --2- 172.21.15.16:0/2324210574 >> [v2:172.21.15.107:6832/3797712690,v1:172.21.15.107:6833/3797712690] conn(0x55cf5f7a3ed0 0x55cf5f80bea0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=346ee3a06689a6fa server_cookie=c5bc436bfeef1c89 in_seq=0 out_seq=0 2024-09-13T16:31:00.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.982+0000 7fe3b2ff5640 10 client.15444 ms_handle_connect on v2:172.21.15.107:6832/3797712690 2024-09-13T16:31:00.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.982+0000 7fe3b2ff5640 1 -- 172.21.15.16:0/2324210574 <== mds.3 v2:172.21.15.107:6832/3797712690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x55cf5f459d10 con 0x55cf5f7a3ed0 2024-09-13T16:31:00.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.982+0000 7fe3b2ff5640 10 client.15444 handle_command_reply: tid=0 2024-09-13T16:31:00.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.982+0000 7fe3d81ee740 1 client.15444 shutdown 2024-09-13T16:31:00.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.982+0000 7fe3d81ee740 1 -- 172.21.15.16:0/2324210574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf5f799b70 msgr2=0x55cf5f7ac410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:00.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.982+0000 7fe3d81ee740 1 --2- 172.21.15.16:0/2324210574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf5f799b70 0x55cf5f7ac410 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4006010 tx=0x7fe3c4005f80 comp rx=0 tx=0).stop 2024-09-13T16:31:00.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.982+0000 7fe3d81ee740 1 -- 172.21.15.16:0/2324210574 shutdown_connections 2024-09-13T16:31:00.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.982+0000 7fe3d81ee740 1 --2- 172.21.15.16:0/2324210574 >> [v2:172.21.15.107:6832/3797712690,v1:172.21.15.107:6833/3797712690] conn(0x55cf5f7a3ed0 0x55cf5f80bea0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.982+0000 7fe3d81ee740 1 --2- 172.21.15.16:0/2324210574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5f79ee40 0x55cf5f80a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.982+0000 7fe3d81ee740 1 --2- 172.21.15.16:0/2324210574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf5f79e500 0x55cf5f7ac950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.982+0000 7fe3b2ff5640 0 client.15444 ms_handle_reset on v2:172.21.15.107:6832/3797712690 2024-09-13T16:31:00.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.982+0000 7fe3d81ee740 1 --2- 172.21.15.16:0/2324210574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf5f799b70 0x55cf5f7ac410 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.982+0000 7fe3d81ee740 1 -- 172.21.15.16:0/2324210574 >> 172.21.15.16:0/2324210574 conn(0x55cf5f7a6630 msgr2=0x55cf5f7a7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:00.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.982+0000 7fe3d81ee740 1 -- 172.21.15.16:0/2324210574 shutdown_connections 2024-09-13T16:31:00.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.983+0000 7fe3d81ee740 1 -- 172.21.15.16:0/2324210574 wait complete. 2024-09-13T16:31:00.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.983+0000 7fe3d81ee740 20 client.15444 trim_cache size 0 max 16384 2024-09-13T16:31:00.994 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-13T16:31:00.995 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "0d241f6a-71ec-11ef-bceb-c7b262605968", 2024-09-13T16:31:00.995 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 3, 2024-09-13T16:31:00.995 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24422, 2024-09-13T16:31:00.995 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-13T16:31:00.995 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-13T16:31:00.995 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-13T16:31:00.995 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 117.367162736, 2024-09-13T16:31:00.995 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 28, 2024-09-13T16:31:00.995 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 93, 2024-09-13T16:31:00.995 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 91, 2024-09-13T16:31:00.995 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 172.009917616 2024-09-13T16:31:00.995 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-13T16:31:00.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.983+0000 7fe3d54d0640 1 -- 172.21.15.16:0/1321451280 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe3b80a1110 msgr2=0x7fe3b80a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:00.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.983+0000 7fe3d54d0640 1 --2- 172.21.15.16:0/1321451280 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe3b80a1110 0x7fe3b80a35d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80c3400 tx=0x7fe3c8086990 comp rx=0 tx=0).stop 2024-09-13T16:31:00.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.983+0000 7fe3d54d0640 1 -- 172.21.15.16:0/1321451280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3d0100820 msgr2=0x7fe3d0101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:00.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.983+0000 7fe3d54d0640 1 --2- 172.21.15.16:0/1321451280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3d0100820 0x7fe3d0101780 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8009430 tx=0x7fe3c8003c20 comp rx=0 tx=0).stop 2024-09-13T16:31:00.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.983+0000 7fe3d54d0640 1 -- 172.21.15.16:0/1321451280 shutdown_connections 2024-09-13T16:31:00.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.983+0000 7fe3d54d0640 1 --2- 172.21.15.16:0/1321451280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3d0100820 0x7fe3d0101780 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.983+0000 7fe3d54d0640 1 --2- 172.21.15.16:0/1321451280 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe3b80a1110 0x7fe3b80a35d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.983+0000 7fe3d54d0640 1 --2- 172.21.15.16:0/1321451280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3d00ffe60 0x7fe3d0101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.983+0000 7fe3d54d0640 1 --2- 172.21.15.16:0/1321451280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3d00fec70 0x7fe3d0102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:00.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.983+0000 7fe3d54d0640 1 -- 172.21.15.16:0/1321451280 >> 172.21.15.16:0/1321451280 conn(0x7fe3d00fa410 msgr2=0x7fe3d0067650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:00.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.983+0000 7fe3d54d0640 1 -- 172.21.15.16:0/1321451280 shutdown_connections 2024-09-13T16:31:00.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:00.983+0000 7fe3d54d0640 1 -- 172.21.15.16:0/1321451280 wait complete. 2024-09-13T16:31:00.997 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:00 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1253381200' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T16:31:00.998 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:00 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1580887664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:31:00.998 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:00 smithi016 ceph-mon[31187]: pgmap v351: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T16:31:00.998 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:00 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1872701749' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:31:00.999 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-09-13T16:31:01.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:00 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1253381200' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T16:31:01.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:00 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1580887664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:31:01.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:00 smithi107 ceph-mon[34139]: pgmap v351: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T16:31:01.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:00 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1872701749' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:31:01.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.100+0000 7f326e74c640 1 -- 172.21.15.16:0/3460663322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3268101940 msgr2=0x7f32681081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.100+0000 7f326e74c640 1 --2- 172.21.15.16:0/3460663322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3268101940 0x7f32681081d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3260009ca0 tx=0x7f326002a8c0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.100+0000 7f326e74c640 1 -- 172.21.15.16:0/3460663322 shutdown_connections 2024-09-13T16:31:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.100+0000 7f326e74c640 1 --2- 172.21.15.16:0/3460663322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3268101940 0x7f32681081d0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.100+0000 7f326e74c640 1 --2- 172.21.15.16:0/3460663322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3268100f80 0x7f3268101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.100+0000 7f326e74c640 1 --2- 172.21.15.16:0/3460663322 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32680ffd90 0x7f3268100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.100+0000 7f326e74c640 1 -- 172.21.15.16:0/3460663322 >> 172.21.15.16:0/3460663322 conn(0x7f32680fb4d0 msgr2=0x7f32680fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.100+0000 7f326e74c640 1 -- 172.21.15.16:0/3460663322 shutdown_connections 2024-09-13T16:31:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.100+0000 7f326e74c640 1 -- 172.21.15.16:0/3460663322 wait complete. 2024-09-13T16:31:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.101+0000 7f326e74c640 1 Processor -- start 2024-09-13T16:31:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.101+0000 7f326e74c640 1 -- start start 2024-09-13T16:31:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.101+0000 7f326e74c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32680ffd90 0x7f3268195ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.101+0000 7f326e74c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3268100f80 0x7f32681960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.101+0000 7f326e74c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3268101940 0x7f326819d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.101+0000 7f326e74c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326819d690 con 0x7f3268101940 2024-09-13T16:31:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.101+0000 7f326e74c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326819d800 con 0x7f32680ffd90 2024-09-13T16:31:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.101+0000 7f326e74c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326819d9c0 con 0x7f3268100f80 2024-09-13T16:31:01.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.101+0000 7f326cce9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3268101940 0x7f326819d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.101+0000 7f326cce9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3268101940 0x7f326819d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40834/0 (socket says 172.21.15.16:40834) 2024-09-13T16:31:01.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.101+0000 7f326cce9640 1 -- 172.21.15.16:0/202430809 learned_addr learned my addr 172.21.15.16:0/202430809 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:01.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.101+0000 7f32677fe640 1 --2- 172.21.15.16:0/202430809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3268100f80 0x7f32681960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.101+0000 7f3267fff640 1 --2- 172.21.15.16:0/202430809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32680ffd90 0x7f3268195ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.102+0000 7f326cce9640 1 -- 172.21.15.16:0/202430809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3268100f80 msgr2=0x7f32681960e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:01.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.102+0000 7f326cce9640 1 --2- 172.21.15.16:0/202430809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3268100f80 0x7f32681960e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.102+0000 7f326cce9640 1 -- 172.21.15.16:0/202430809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32680ffd90 msgr2=0x7f3268195ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:01.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.102+0000 7f326cce9640 1 --2- 172.21.15.16:0/202430809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32680ffd90 0x7f3268195ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.102+0000 7f326cce9640 1 -- 172.21.15.16:0/202430809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3260009900 con 0x7f3268101940 2024-09-13T16:31:01.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.102+0000 7f32677fe640 1 --2- 172.21.15.16:0/202430809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3268100f80 0x7f32681960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:31:01.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.102+0000 7f326cce9640 1 --2- 172.21.15.16:0/202430809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3268101940 0x7f326819d150 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f326002aeb0 tx=0x7f3260003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:01.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.102+0000 7f32657fa640 1 -- 172.21.15.16:0/202430809 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32600307c0 con 0x7f3268101940 2024-09-13T16:31:01.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.102+0000 7f32657fa640 1 -- 172.21.15.16:0/202430809 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3260040070 con 0x7f3268101940 2024-09-13T16:31:01.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.102+0000 7f32657fa640 1 -- 172.21.15.16:0/202430809 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32600044e0 con 0x7f3268101940 2024-09-13T16:31:01.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.102+0000 7f326e74c640 1 -- 172.21.15.16:0/202430809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f326819dc40 con 0x7f3268101940 2024-09-13T16:31:01.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.102+0000 7f326e74c640 1 -- 172.21.15.16:0/202430809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f326819e100 con 0x7f3268101940 2024-09-13T16:31:01.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.104+0000 7f32657fa640 1 -- 172.21.15.16:0/202430809 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f3260042030 con 0x7f3268101940 2024-09-13T16:31:01.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.104+0000 7f327146f740 1 Processor -- start 2024-09-13T16:31:01.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.104+0000 7f327146f740 1 -- start start 2024-09-13T16:31:01.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.105+0000 7f327146f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dec38d1500 0x55dec38d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.105+0000 7f327146f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec38d1e40 0x55dec38d6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.105+0000 7f327146f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dec38d6ed0 0x55dec38d9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.105+0000 7f327146f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dec382c7c0 con 0x55dec38d1500 2024-09-13T16:31:01.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.105+0000 7f327146f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dec383bb70 con 0x55dec38d1e40 2024-09-13T16:31:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.105+0000 7f327146f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dec3839300 con 0x55dec38d6ed0 2024-09-13T16:31:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.105+0000 7f32677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dec38d1500 0x55dec38d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.105+0000 7f32677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dec38d1500 0x55dec38d1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40840/0 (socket says 172.21.15.16:40840) 2024-09-13T16:31:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.105+0000 7f32677fe640 1 -- 172.21.15.16:0/4113461482 learned_addr learned my addr 172.21.15.16:0/4113461482 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.105+0000 7f3267fff640 1 --2- 172.21.15.16:0/4113461482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec38d1e40 0x55dec38d6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.105+0000 7f32677fe640 1 --2- 172.21.15.16:0/4113461482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dec38d6ed0 0x55dec38d9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.105+0000 7f32657fa640 1 --2- 172.21.15.16:0/202430809 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f32500a1090 0x7f32500a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.105+0000 7f32677fe640 1 -- 172.21.15.16:0/4113461482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dec38d6ed0 msgr2=0x55dec38d9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.105+0000 7f32677fe640 1 --2- 172.21.15.16:0/4113461482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dec38d6ed0 0x55dec38d9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.105+0000 7f32677fe640 1 -- 172.21.15.16:0/4113461482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec38d1e40 msgr2=0x55dec38d6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.105+0000 7f32677fe640 1 --2- 172.21.15.16:0/4113461482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec38d1e40 0x55dec38d6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.105+0000 7f32677fe640 1 -- 172.21.15.16:0/4113461482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dec37c14b0 con 0x55dec38d1500 2024-09-13T16:31:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.105+0000 7f326cce9640 1 --2- 172.21.15.16:0/202430809 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f32500a1090 0x7f32500a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.106+0000 7f32677fe640 1 --2- 172.21.15.16:0/4113461482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dec38d1500 0x55dec38d1900 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f325c00be80 tx=0x7f325c00b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d62dc7555649cbb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:01.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.106+0000 7f32657fa640 1 -- 172.21.15.16:0/202430809 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f32600ee470 con 0x7f3268101940 2024-09-13T16:31:01.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.106+0000 7f324e7fc640 1 -- 172.21.15.16:0/4113461482 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c011540 con 0x55dec38d1500 2024-09-13T16:31:01.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.106+0000 7f324e7fc640 1 -- 172.21.15.16:0/4113461482 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f325c011b30 con 0x55dec38d1500 2024-09-13T16:31:01.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.106+0000 7f324e7fc640 1 -- 172.21.15.16:0/4113461482 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c01d930 con 0x55dec38d1500 2024-09-13T16:31:01.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.106+0000 7f326cce9640 1 --2- 172.21.15.16:0/202430809 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f32500a1090 0x7f32500a3550 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f32600b7fe0 tx=0x7f32600e5ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:01.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.106+0000 7f327146f740 1 -- 172.21.15.16:0/4113461482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dec38d1500 msgr2=0x55dec38d1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:01.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.106+0000 7f327146f740 1 --2- 172.21.15.16:0/4113461482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dec38d1500 0x55dec38d1900 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f325c00be80 tx=0x7f325c00b730 comp rx=0 tx=0).stop 2024-09-13T16:31:01.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.106+0000 7f327146f740 1 -- 172.21.15.16:0/4113461482 shutdown_connections 2024-09-13T16:31:01.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.106+0000 7f327146f740 1 --2- 172.21.15.16:0/4113461482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dec38d6ed0 0x55dec38d9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.106+0000 7f327146f740 1 --2- 172.21.15.16:0/4113461482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec38d1e40 0x55dec38d6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.106+0000 7f327146f740 1 --2- 172.21.15.16:0/4113461482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dec38d1500 0x55dec38d1900 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.106+0000 7f327146f740 1 -- 172.21.15.16:0/4113461482 >> 172.21.15.16:0/4113461482 conn(0x55dec38ccb70 msgr2=0x55dec38ccf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:01.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.106+0000 7f327146f740 1 -- 172.21.15.16:0/4113461482 shutdown_connections 2024-09-13T16:31:01.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.106+0000 7f327146f740 1 -- 172.21.15.16:0/4113461482 wait complete. 2024-09-13T16:31:01.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.107+0000 7f327146f740 1 -- start start 2024-09-13T16:31:01.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.107+0000 7f327146f740 1 Processor -- start 2024-09-13T16:31:01.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.107+0000 7f327146f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dec38d1500 0x55dec38db220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.107+0000 7f327146f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dec38d1e40 0x55dec38db760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.107+0000 7f327146f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec38d6ed0 0x55dec38e27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.107+0000 7f327146f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dec3834bf0 con 0x55dec38d1e40 2024-09-13T16:31:01.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.107+0000 7f327146f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dec37f3250 con 0x55dec38d6ed0 2024-09-13T16:31:01.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.107+0000 7f327146f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dec37f2ef0 con 0x55dec38d1500 2024-09-13T16:31:01.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.107+0000 7f327146f740 10 client.0 _ll_register_callbacks cb 0x55dec3825740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:31:01.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.107+0000 7f3267fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dec38d1e40 0x55dec38db760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.107+0000 7f3267fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dec38d1e40 0x55dec38db760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40852/0 (socket says 172.21.15.16:40852) 2024-09-13T16:31:01.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.107+0000 7f3267fff640 1 -- 172.21.15.16:0/2106783405 learned_addr learned my addr 172.21.15.16:0/2106783405 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:01.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.108+0000 7f32677fe640 1 --2- 172.21.15.16:0/2106783405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec38d6ed0 0x55dec38e27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.108+0000 7f32677fe640 1 --2- 172.21.15.16:0/2106783405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dec38d1500 0x55dec38db220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.108+0000 7f3267fff640 1 -- 172.21.15.16:0/2106783405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dec38d1500 msgr2=0x55dec38db220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:01.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.108+0000 7f3267fff640 1 --2- 172.21.15.16:0/2106783405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dec38d1500 0x55dec38db220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.108+0000 7f3267fff640 1 -- 172.21.15.16:0/2106783405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec38d6ed0 msgr2=0x55dec38e27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:01.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.108+0000 7f3267fff640 1 --2- 172.21.15.16:0/2106783405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec38d6ed0 0x55dec38e27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.108+0000 7f3267fff640 1 -- 172.21.15.16:0/2106783405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f325c013400 con 0x55dec38d1e40 2024-09-13T16:31:01.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.108+0000 7f32677fe640 1 --2- 172.21.15.16:0/2106783405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec38d6ed0 0x55dec38e27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:31:01.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.108+0000 7f3267fff640 1 --2- 172.21.15.16:0/2106783405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dec38d1e40 0x55dec38db760 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3258009e00 tx=0x7f32580117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e88ab9fc355e3d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:01.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.108+0000 7f32477fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:31:01.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.108+0000 7f32477fe640 1 -- 172.21.15.16:0/2106783405 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32580096e0 con 0x55dec38d1e40 2024-09-13T16:31:01.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.108+0000 7f32477fe640 1 -- 172.21.15.16:0/2106783405 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f3258012860 con 0x55dec38d1e40 2024-09-13T16:31:01.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.108+0000 7f327146f740 1 -- 172.21.15.16:0/2106783405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dec35283c0 con 0x55dec38d1e40 2024-09-13T16:31:01.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.108+0000 7f32477fe640 1 -- 172.21.15.16:0/2106783405 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3258012a20 con 0x55dec38d1e40 2024-09-13T16:31:01.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.108+0000 7f327146f740 10 client.15456 fetch_fsmap learned FSMap version 28 2024-09-13T16:31:01.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.108+0000 7f327146f740 1 -- 172.21.15.16:0/2106783405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55dec3833270 con 0x55dec38d1e40 2024-09-13T16:31:01.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.108+0000 7f32477fe640 1 -- 172.21.15.16:0/2106783405 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55dec35283c0 con 0x55dec38d1e40 2024-09-13T16:31:01.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.109+0000 7f32477fe640 1 -- 172.21.15.16:0/2106783405 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f325800c070 con 0x55dec38d1e40 2024-09-13T16:31:01.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.109+0000 7f327146f740 10 client.15456 fetch_fsmap finished waiting for FSMap version 28 2024-09-13T16:31:01.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.109+0000 7f327146f740 10 client.15456 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-13T16:31:01.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.109+0000 7f327146f740 20 client.15456 populate_metadata read hostname 'smithi016' 2024-09-13T16:31:01.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.109+0000 7f327146f740 1 --2- 172.21.15.16:0/2106783405 >> [v2:172.21.15.107:6836/2838457611,v1:172.21.15.107:6837/2838457611] conn(0x55dec38e7850 0x55dec38eb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.109+0000 7f327146f740 4 client.15456 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T16:31:01.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.109+0000 7f327146f740 1 -- 172.21.15.16:0/2106783405 --> [v2:172.21.15.107:6836/2838457611,v1:172.21.15.107:6837/2838457611] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dec35396c0 con 0x55dec38e7850 2024-09-13T16:31:01.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.109+0000 7f32677fe640 1 --2- 172.21.15.16:0/2106783405 >> [v2:172.21.15.107:6836/2838457611,v1:172.21.15.107:6837/2838457611] conn(0x55dec38e7850 0x55dec38eb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.109+0000 7f32477fe640 10 client.15456 ms_handle_connect on v2:172.21.15.107:6836/2838457611 2024-09-13T16:31:01.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.109+0000 7f32677fe640 1 --2- 172.21.15.16:0/2106783405 >> [v2:172.21.15.107:6836/2838457611,v1:172.21.15.107:6837/2838457611] conn(0x55dec38e7850 0x55dec38eb9f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ec043a0062eac9e0 server_cookie=19bb6343d3f6d9b4 in_seq=0 out_seq=0 2024-09-13T16:31:01.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.110+0000 7f32477fe640 1 -- 172.21.15.16:0/2106783405 <== mds.2 v2:172.21.15.107:6836/2838457611 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dec35396c0 con 0x55dec38e7850 2024-09-13T16:31:01.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.110+0000 7f32477fe640 10 client.15456 handle_command_reply: tid=0 2024-09-13T16:31:01.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.110+0000 7f327146f740 1 client.15456 shutdown 2024-09-13T16:31:01.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.110+0000 7f327146f740 1 -- 172.21.15.16:0/2106783405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dec38d1e40 msgr2=0x55dec38db760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:01.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.110+0000 7f327146f740 1 --2- 172.21.15.16:0/2106783405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dec38d1e40 0x55dec38db760 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3258009e00 tx=0x7f32580117b0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.110+0000 7f327146f740 1 -- 172.21.15.16:0/2106783405 shutdown_connections 2024-09-13T16:31:01.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.110+0000 7f327146f740 1 --2- 172.21.15.16:0/2106783405 >> [v2:172.21.15.107:6836/2838457611,v1:172.21.15.107:6837/2838457611] conn(0x55dec38e7850 0x55dec38eb9f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.110+0000 7f327146f740 1 --2- 172.21.15.16:0/2106783405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec38d6ed0 0x55dec38e27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.110+0000 7f32477fe640 0 client.15456 ms_handle_reset on v2:172.21.15.107:6836/2838457611 2024-09-13T16:31:01.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.110+0000 7f327146f740 1 --2- 172.21.15.16:0/2106783405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dec38d1e40 0x55dec38db760 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.110+0000 7f327146f740 1 --2- 172.21.15.16:0/2106783405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dec38d1500 0x55dec38db220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.111+0000 7f327146f740 1 -- 172.21.15.16:0/2106783405 >> 172.21.15.16:0/2106783405 conn(0x55dec38ccb70 msgr2=0x55dec38ce380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:01.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.111+0000 7f327146f740 1 -- 172.21.15.16:0/2106783405 shutdown_connections 2024-09-13T16:31:01.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.111+0000 7f327146f740 1 -- 172.21.15.16:0/2106783405 wait complete. 2024-09-13T16:31:01.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.111+0000 7f327146f740 20 client.15456 trim_cache size 0 max 16384 2024-09-13T16:31:01.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.125+0000 7f327146f740 1 Processor -- start 2024-09-13T16:31:01.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.125+0000 7f327146f740 1 -- start start 2024-09-13T16:31:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.125+0000 7f327146f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dec38d1500 0x55dec38d2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.125+0000 7f327146f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dec38d1e40 0x55dec38d32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.125+0000 7f327146f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec38d6ed0 0x55dec38db1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.125+0000 7f327146f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dec37b8460 con 0x55dec38d1500 2024-09-13T16:31:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.125+0000 7f327146f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dec35396c0 con 0x55dec38d6ed0 2024-09-13T16:31:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.126+0000 7f327146f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dec35283c0 con 0x55dec38d1e40 2024-09-13T16:31:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.126+0000 7f32677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dec38d1500 0x55dec38d2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.126+0000 7f32677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dec38d1500 0x55dec38d2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40862/0 (socket says 172.21.15.16:40862) 2024-09-13T16:31:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.126+0000 7f32677fe640 1 -- 172.21.15.16:0/667974442 learned_addr learned my addr 172.21.15.16:0/667974442 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.126+0000 7f32677fe640 1 -- 172.21.15.16:0/667974442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dec38d1e40 msgr2=0x55dec38d32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.126+0000 7f32677fe640 1 --2- 172.21.15.16:0/667974442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dec38d1e40 0x55dec38d32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.126+0000 7f32677fe640 1 -- 172.21.15.16:0/667974442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec38d6ed0 msgr2=0x55dec38db1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.126+0000 7f32677fe640 1 --2- 172.21.15.16:0/667974442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec38d6ed0 0x55dec38db1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.126+0000 7f32677fe640 1 -- 172.21.15.16:0/667974442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dec37d55f0 con 0x55dec38d1500 2024-09-13T16:31:01.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.126+0000 7f32677fe640 1 --2- 172.21.15.16:0/667974442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dec38d1500 0x55dec38d2da0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f325c013fa0 tx=0x7f325c005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748800f8c7d172c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:01.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.126+0000 7f324effd640 1 -- 172.21.15.16:0/667974442 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c024030 con 0x55dec38d1500 2024-09-13T16:31:01.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.126+0000 7f324effd640 1 -- 172.21.15.16:0/667974442 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f325c025040 con 0x55dec38d1500 2024-09-13T16:31:01.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.126+0000 7f324effd640 1 -- 172.21.15.16:0/667974442 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c028a90 con 0x55dec38d1500 2024-09-13T16:31:01.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.127+0000 7f327146f740 1 -- 172.21.15.16:0/667974442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dec38d1500 msgr2=0x55dec38d2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:01.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.127+0000 7f327146f740 1 --2- 172.21.15.16:0/667974442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dec38d1500 0x55dec38d2da0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f325c013fa0 tx=0x7f325c005fb0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.127+0000 7f327146f740 1 -- 172.21.15.16:0/667974442 shutdown_connections 2024-09-13T16:31:01.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.127+0000 7f327146f740 1 --2- 172.21.15.16:0/667974442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec38d6ed0 0x55dec38db1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.127+0000 7f327146f740 1 --2- 172.21.15.16:0/667974442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dec38d1e40 0x55dec38d32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.127+0000 7f327146f740 1 --2- 172.21.15.16:0/667974442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dec38d1500 0x55dec38d2da0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.127+0000 7f327146f740 1 -- 172.21.15.16:0/667974442 >> 172.21.15.16:0/667974442 conn(0x55dec38ccb70 msgr2=0x55dec38d63b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:01.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.127+0000 7f327146f740 1 -- 172.21.15.16:0/667974442 shutdown_connections 2024-09-13T16:31:01.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.127+0000 7f327146f740 1 -- 172.21.15.16:0/667974442 wait complete. 2024-09-13T16:31:01.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.127+0000 7f327146f740 1 -- start start 2024-09-13T16:31:01.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.127+0000 7f327146f740 1 Processor -- start 2024-09-13T16:31:01.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.127+0000 7f327146f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dec38d1500 0x55dec3939e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.127+0000 7f327146f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dec38d1e40 0x55dec393a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.128+0000 7f327146f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec38d6ed0 0x55dec39413a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.128+0000 7f327146f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dec39418e0 con 0x55dec38d1500 2024-09-13T16:31:01.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.128+0000 7f327146f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dec3941a20 con 0x55dec38d6ed0 2024-09-13T16:31:01.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.128+0000 7f327146f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dec3941bb0 con 0x55dec38d1e40 2024-09-13T16:31:01.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.128+0000 7f327146f740 10 client.0 _ll_register_callbacks cb 0x7f325c00b590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:31:01.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.128+0000 7f32677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dec38d1500 0x55dec3939e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.128+0000 7f32677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dec38d1500 0x55dec3939e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40878/0 (socket says 172.21.15.16:40878) 2024-09-13T16:31:01.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.128+0000 7f32677fe640 1 -- 172.21.15.16:0/3575580975 learned_addr learned my addr 172.21.15.16:0/3575580975 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:01.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.128+0000 7f3267fff640 1 --2- 172.21.15.16:0/3575580975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dec38d1e40 0x55dec393a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.128+0000 7f32677fe640 1 --2- 172.21.15.16:0/3575580975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec38d6ed0 0x55dec39413a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.128+0000 7f32677fe640 1 -- 172.21.15.16:0/3575580975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dec38d1e40 msgr2=0x55dec393a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:01.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.128+0000 7f32677fe640 1 --2- 172.21.15.16:0/3575580975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dec38d1e40 0x55dec393a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.128+0000 7f32677fe640 1 -- 172.21.15.16:0/3575580975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec38d6ed0 msgr2=0x55dec39413a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:01.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.128+0000 7f32677fe640 1 --2- 172.21.15.16:0/3575580975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec38d6ed0 0x55dec39413a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.128+0000 7f32677fe640 1 -- 172.21.15.16:0/3575580975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dec37c14b0 con 0x55dec38d1500 2024-09-13T16:31:01.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.128+0000 7f3267fff640 1 --2- 172.21.15.16:0/3575580975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dec38d1e40 0x55dec393a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:31:01.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.128+0000 7f32677fe640 1 --2- 172.21.15.16:0/3575580975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dec38d1500 0x55dec3939e70 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f325c011f60 tx=0x7f325c004020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc7fed3b7362d43e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:01.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.128+0000 7f32477fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:31:01.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.128+0000 7f32477fe640 1 -- 172.21.15.16:0/3575580975 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c024040 con 0x55dec38d1500 2024-09-13T16:31:01.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.128+0000 7f32477fe640 1 -- 172.21.15.16:0/3575580975 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f325c011750 con 0x55dec38d1500 2024-09-13T16:31:01.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.128+0000 7f327146f740 1 -- 172.21.15.16:0/3575580975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f325c011b30 con 0x55dec38d1500 2024-09-13T16:31:01.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.128+0000 7f32477fe640 1 -- 172.21.15.16:0/3575580975 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f325c011910 con 0x55dec38d1500 2024-09-13T16:31:01.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.128+0000 7f327146f740 10 client.15462 fetch_fsmap learned FSMap version 28 2024-09-13T16:31:01.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.128+0000 7f327146f740 1 -- 172.21.15.16:0/3575580975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55dec37fad00 con 0x55dec38d1500 2024-09-13T16:31:01.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.129+0000 7f32477fe640 1 -- 172.21.15.16:0/3575580975 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f325c03a080 con 0x55dec38d1500 2024-09-13T16:31:01.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.129+0000 7f32477fe640 1 -- 172.21.15.16:0/3575580975 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f325c028750 con 0x55dec38d1500 2024-09-13T16:31:01.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.129+0000 7f327146f740 10 client.15462 fetch_fsmap finished waiting for FSMap version 28 2024-09-13T16:31:01.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.129+0000 7f327146f740 10 client.15462 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-13T16:31:01.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.129+0000 7f327146f740 20 client.15462 populate_metadata read hostname 'smithi016' 2024-09-13T16:31:01.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.129+0000 7f327146f740 1 --2- 172.21.15.16:0/3575580975 >> [v2:172.21.15.107:6836/2838457611,v1:172.21.15.107:6837/2838457611] conn(0x55dec3946580 0x55dec3942ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.129+0000 7f327146f740 4 client.15462 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T16:31:01.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.129+0000 7f327146f740 1 -- 172.21.15.16:0/3575580975 --> [v2:172.21.15.107:6836/2838457611,v1:172.21.15.107:6837/2838457611] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55dec358cd10 con 0x55dec3946580 2024-09-13T16:31:01.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.129+0000 7f3267fff640 1 --2- 172.21.15.16:0/3575580975 >> [v2:172.21.15.107:6836/2838457611,v1:172.21.15.107:6837/2838457611] conn(0x55dec3946580 0x55dec3942ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.130+0000 7f3267fff640 1 --2- 172.21.15.16:0/3575580975 >> [v2:172.21.15.107:6836/2838457611,v1:172.21.15.107:6837/2838457611] conn(0x55dec3946580 0x55dec3942ef0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ad667c141120d8d6 server_cookie=4f06ba4f494d36a in_seq=0 out_seq=0 2024-09-13T16:31:01.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.130+0000 7f32477fe640 10 client.15462 ms_handle_connect on v2:172.21.15.107:6836/2838457611 2024-09-13T16:31:01.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.130+0000 7f32477fe640 1 -- 172.21.15.16:0/3575580975 <== mds.2 v2:172.21.15.107:6836/2838457611 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55dec358cd10 con 0x55dec3946580 2024-09-13T16:31:01.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.130+0000 7f32477fe640 10 client.15462 handle_command_reply: tid=0 2024-09-13T16:31:01.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.130+0000 7f327146f740 1 client.15462 shutdown 2024-09-13T16:31:01.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.130+0000 7f327146f740 1 -- 172.21.15.16:0/3575580975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dec38d1500 msgr2=0x55dec3939e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:01.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.130+0000 7f327146f740 1 --2- 172.21.15.16:0/3575580975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dec38d1500 0x55dec3939e70 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f325c011f60 tx=0x7f325c004020 comp rx=0 tx=0).stop 2024-09-13T16:31:01.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.130+0000 7f327146f740 1 -- 172.21.15.16:0/3575580975 shutdown_connections 2024-09-13T16:31:01.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.130+0000 7f327146f740 1 --2- 172.21.15.16:0/3575580975 >> [v2:172.21.15.107:6836/2838457611,v1:172.21.15.107:6837/2838457611] conn(0x55dec3946580 0x55dec3942ef0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.130+0000 7f327146f740 1 --2- 172.21.15.16:0/3575580975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dec38d6ed0 0x55dec39413a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.130+0000 7f327146f740 1 --2- 172.21.15.16:0/3575580975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dec38d1e40 0x55dec393a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.130+0000 7f32477fe640 0 client.15462 ms_handle_reset on v2:172.21.15.107:6836/2838457611 2024-09-13T16:31:01.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.130+0000 7f327146f740 1 --2- 172.21.15.16:0/3575580975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dec38d1500 0x55dec3939e70 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.130+0000 7f327146f740 1 -- 172.21.15.16:0/3575580975 >> 172.21.15.16:0/3575580975 conn(0x55dec38ccb70 msgr2=0x55dec38db1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:01.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.130+0000 7f327146f740 1 -- 172.21.15.16:0/3575580975 shutdown_connections 2024-09-13T16:31:01.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.130+0000 7f327146f740 1 -- 172.21.15.16:0/3575580975 wait complete. 2024-09-13T16:31:01.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.130+0000 7f327146f740 20 client.15462 trim_cache size 0 max 16384 2024-09-13T16:31:01.139 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-13T16:31:01.139 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "0d241f6a-71ec-11ef-bceb-c7b262605968", 2024-09-13T16:31:01.139 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 2, 2024-09-13T16:31:01.139 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24485, 2024-09-13T16:31:01.139 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-13T16:31:01.139 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-13T16:31:01.139 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-13T16:31:01.139 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 117.54898208, 2024-09-13T16:31:01.140 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 28, 2024-09-13T16:31:01.140 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 93, 2024-09-13T16:31:01.140 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 91, 2024-09-13T16:31:01.140 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 156.61407688700001 2024-09-13T16:31:01.140 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-13T16:31:01.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.131+0000 7f326e74c640 1 -- 172.21.15.16:0/202430809 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f32500a1090 msgr2=0x7f32500a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.131+0000 7f326e74c640 1 --2- 172.21.15.16:0/202430809 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f32500a1090 0x7f32500a3550 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f32600b7fe0 tx=0x7f32600e5ab0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.131+0000 7f326e74c640 1 -- 172.21.15.16:0/202430809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3268101940 msgr2=0x7f326819d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.131+0000 7f326e74c640 1 --2- 172.21.15.16:0/202430809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3268101940 0x7f326819d150 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f326002aeb0 tx=0x7f3260003c20 comp rx=0 tx=0).stop 2024-09-13T16:31:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.131+0000 7f326e74c640 1 -- 172.21.15.16:0/202430809 shutdown_connections 2024-09-13T16:31:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.131+0000 7f326e74c640 1 --2- 172.21.15.16:0/202430809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3268101940 0x7f326819d150 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.131+0000 7f326e74c640 1 --2- 172.21.15.16:0/202430809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3268100f80 0x7f32681960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.131+0000 7f326e74c640 1 --2- 172.21.15.16:0/202430809 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f32500a1090 0x7f32500a3550 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.131+0000 7f326e74c640 1 --2- 172.21.15.16:0/202430809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32680ffd90 0x7f3268195ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.131+0000 7f326e74c640 1 -- 172.21.15.16:0/202430809 >> 172.21.15.16:0/202430809 conn(0x7f32680fb4d0 msgr2=0x7f3268104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.131+0000 7f326e74c640 1 -- 172.21.15.16:0/202430809 shutdown_connections 2024-09-13T16:31:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.131+0000 7f326e74c640 1 -- 172.21.15.16:0/202430809 wait complete. 2024-09-13T16:31:01.143 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-09-13T16:31:01.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:00 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1253381200' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T16:31:01.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:00 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1580887664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:31:01.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:00 smithi111 ceph-mon[34600]: pgmap v351: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T16:31:01.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:00 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1872701749' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:31:01.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.244+0000 7fabff6b7640 1 -- 172.21.15.16:0/2640304627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabf80fe5e0 msgr2=0x7fabf80fe9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:01.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.244+0000 7fabff6b7640 1 --2- 172.21.15.16:0/2640304627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabf80fe5e0 0x7fabf80fe9e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fabf4009900 tx=0x7fabf402a410 comp rx=0 tx=0).stop 2024-09-13T16:31:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.245+0000 7fabff6b7640 1 -- 172.21.15.16:0/2640304627 shutdown_connections 2024-09-13T16:31:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.245+0000 7fabff6b7640 1 --2- 172.21.15.16:0/2640304627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabf80ff8c0 0x7fabf80ffd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.245+0000 7fabff6b7640 1 --2- 172.21.15.16:0/2640304627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabf80fef20 0x7fabf80ff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.245+0000 7fabff6b7640 1 --2- 172.21.15.16:0/2640304627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabf80fe5e0 0x7fabf80fe9e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.245+0000 7fabff6b7640 1 -- 172.21.15.16:0/2640304627 >> 172.21.15.16:0/2640304627 conn(0x7fabf80fa410 msgr2=0x7fabf80fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.245+0000 7fabff6b7640 1 -- 172.21.15.16:0/2640304627 shutdown_connections 2024-09-13T16:31:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.245+0000 7fabff6b7640 1 -- 172.21.15.16:0/2640304627 wait complete. 2024-09-13T16:31:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.245+0000 7fabff6b7640 1 Processor -- start 2024-09-13T16:31:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.245+0000 7fabff6b7640 1 -- start start 2024-09-13T16:31:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.245+0000 7fabff6b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabf80fe5e0 0x7fabf8197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.245+0000 7fabff6b7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabf80fef20 0x7fabf81983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.245+0000 7fabff6b7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabf80ff8c0 0x7fabf819f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.245+0000 7fabff6b7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf819f970 con 0x7fabf80fe5e0 2024-09-13T16:31:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.245+0000 7fabff6b7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf819fae0 con 0x7fabf80ff8c0 2024-09-13T16:31:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.245+0000 7fabff6b7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf819fca0 con 0x7fabf80fef20 2024-09-13T16:31:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.245+0000 7fabfd453640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabf80fe5e0 0x7fabf8197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.245+0000 7fabfd453640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabf80fe5e0 0x7fabf8197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40896/0 (socket says 172.21.15.16:40896) 2024-09-13T16:31:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.245+0000 7fabfd453640 1 -- 172.21.15.16:0/1811918063 learned_addr learned my addr 172.21.15.16:0/1811918063 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.246+0000 7fabfd453640 1 -- 172.21.15.16:0/1811918063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabf80fef20 msgr2=0x7fabf81983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.246+0000 7fabfd453640 1 --2- 172.21.15.16:0/1811918063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabf80fef20 0x7fabf81983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.246+0000 7fabfd453640 1 -- 172.21.15.16:0/1811918063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabf80ff8c0 msgr2=0x7fabf819f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.246+0000 7fabfd453640 1 --2- 172.21.15.16:0/1811918063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabf80ff8c0 0x7fabf819f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.246+0000 7fabfd453640 1 -- 172.21.15.16:0/1811918063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf4009560 con 0x7fabf80fe5e0 2024-09-13T16:31:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.246+0000 7fabfd453640 1 --2- 172.21.15.16:0/1811918063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabf80fe5e0 0x7fabf8197e80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fabf40057c0 tx=0x7fabf40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.246+0000 7fabee7fc640 1 -- 172.21.15.16:0/1811918063 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf4031d70 con 0x7fabf80fe5e0 2024-09-13T16:31:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.246+0000 7fabee7fc640 1 -- 172.21.15.16:0/1811918063 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fabf40044e0 con 0x7fabf80fe5e0 2024-09-13T16:31:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.246+0000 7fabee7fc640 1 -- 172.21.15.16:0/1811918063 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf4003b80 con 0x7fabf80fe5e0 2024-09-13T16:31:01.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.246+0000 7fabff6b7640 1 -- 172.21.15.16:0/1811918063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf819ff20 con 0x7fabf80fe5e0 2024-09-13T16:31:01.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.246+0000 7fabff6b7640 1 -- 172.21.15.16:0/1811918063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf81a02f0 con 0x7fabf80fe5e0 2024-09-13T16:31:01.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.248+0000 7fabee7fc640 1 -- 172.21.15.16:0/1811918063 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fabf4003ce0 con 0x7fabf80fe5e0 2024-09-13T16:31:01.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.249+0000 7fac023dd740 1 Processor -- start 2024-09-13T16:31:01.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.249+0000 7fac023dd740 1 -- start start 2024-09-13T16:31:01.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.249+0000 7fac023dd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e296fa8500 0x55e296fa8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.249+0000 7fac023dd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e296fa8e40 0x55e296fad900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.249+0000 7fac023dd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e296faded0 0x55e296fb0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.249+0000 7fac023dd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e296f037c0 con 0x55e296fa8e40 2024-09-13T16:31:01.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.249+0000 7fac023dd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e296f12b70 con 0x55e296fa8500 2024-09-13T16:31:01.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.249+0000 7fac023dd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e296f10300 con 0x55e296faded0 2024-09-13T16:31:01.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.249+0000 7fabfd453640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e296fa8e40 0x55e296fad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.250+0000 7fabfd453640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e296fa8e40 0x55e296fad900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40898/0 (socket says 172.21.15.16:40898) 2024-09-13T16:31:01.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.250+0000 7fabfd453640 1 -- 172.21.15.16:0/3481284845 learned_addr learned my addr 172.21.15.16:0/3481284845 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:01.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.250+0000 7fabfdc54640 1 --2- 172.21.15.16:0/3481284845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e296fa8500 0x55e296fa8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.250+0000 7fabee7fc640 1 --2- 172.21.15.16:0/1811918063 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fabe40a1090 0x7fabe40a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.250+0000 7fabfcc52640 1 --2- 172.21.15.16:0/3481284845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e296faded0 0x55e296fb0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.250+0000 7fabfdc54640 1 --2- 172.21.15.16:0/1811918063 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fabe40a1090 0x7fabe40a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.250+0000 7fabee7fc640 1 -- 172.21.15.16:0/1811918063 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fabf40ee190 con 0x7fabf80fe5e0 2024-09-13T16:31:01.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.250+0000 7fabfd453640 1 -- 172.21.15.16:0/3481284845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e296faded0 msgr2=0x55e296fb0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:01.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.250+0000 7fabfd453640 1 --2- 172.21.15.16:0/3481284845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e296faded0 0x55e296fb0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.250+0000 7fabfd453640 1 -- 172.21.15.16:0/3481284845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e296fa8500 msgr2=0x55e296fa8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:01.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.250+0000 7fabfd453640 1 --2- 172.21.15.16:0/3481284845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e296fa8500 0x55e296fa8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.250+0000 7fabfd453640 1 -- 172.21.15.16:0/3481284845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e296e984b0 con 0x55e296fa8e40 2024-09-13T16:31:01.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.250+0000 7fabfdc54640 1 --2- 172.21.15.16:0/3481284845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e296fa8500 0x55e296fa8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:31:01.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.250+0000 7fabfd453640 1 --2- 172.21.15.16:0/3481284845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e296fa8e40 0x55e296fad900 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fabf40eb8e0 tx=0x7fabf402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d659e843f7e6214 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:01.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.250+0000 7fabfdc54640 1 --2- 172.21.15.16:0/1811918063 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fabe40a1090 0x7fabe40a3550 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fabf00137d0 tx=0x7fabf0005bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:01.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.250+0000 7fabe37fe640 1 -- 172.21.15.16:0/3481284845 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf40bab90 con 0x55e296fa8e40 2024-09-13T16:31:01.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.250+0000 7fabe37fe640 1 -- 172.21.15.16:0/3481284845 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fabf40bb930 con 0x55e296fa8e40 2024-09-13T16:31:01.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.250+0000 7fabe37fe640 1 -- 172.21.15.16:0/3481284845 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf40b7940 con 0x55e296fa8e40 2024-09-13T16:31:01.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.251+0000 7fac023dd740 1 -- 172.21.15.16:0/3481284845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e296fa8e40 msgr2=0x55e296fad900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:01.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.251+0000 7fac023dd740 1 --2- 172.21.15.16:0/3481284845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e296fa8e40 0x55e296fad900 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fabf40eb8e0 tx=0x7fabf402a410 comp rx=0 tx=0).stop 2024-09-13T16:31:01.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.251+0000 7fac023dd740 1 -- 172.21.15.16:0/3481284845 shutdown_connections 2024-09-13T16:31:01.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.251+0000 7fac023dd740 1 --2- 172.21.15.16:0/3481284845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e296faded0 0x55e296fb0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.251+0000 7fac023dd740 1 --2- 172.21.15.16:0/3481284845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e296fa8e40 0x55e296fad900 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.251+0000 7fac023dd740 1 --2- 172.21.15.16:0/3481284845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e296fa8500 0x55e296fa8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.251+0000 7fac023dd740 1 -- 172.21.15.16:0/3481284845 >> 172.21.15.16:0/3481284845 conn(0x55e296fa3b70 msgr2=0x55e296fa3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:01.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.251+0000 7fac023dd740 1 -- 172.21.15.16:0/3481284845 shutdown_connections 2024-09-13T16:31:01.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.251+0000 7fac023dd740 1 -- 172.21.15.16:0/3481284845 wait complete. 2024-09-13T16:31:01.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.251+0000 7fac023dd740 1 -- start start 2024-09-13T16:31:01.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.252+0000 7fac023dd740 1 Processor -- start 2024-09-13T16:31:01.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.252+0000 7fac023dd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e296fa8500 0x55e296fb1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.252+0000 7fac023dd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e296fa8e40 0x55e296fb2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.252+0000 7fac023dd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e296faded0 0x55e296fb93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.252+0000 7fac023dd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e296f0bbf0 con 0x55e296fa8e40 2024-09-13T16:31:01.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.252+0000 7fac023dd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e296eca250 con 0x55e296fa8500 2024-09-13T16:31:01.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.252+0000 7fac023dd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e296ec9ef0 con 0x55e296faded0 2024-09-13T16:31:01.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.252+0000 7fac023dd740 10 client.0 _ll_register_callbacks cb 0x55e296efc740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:31:01.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.252+0000 7fabfcc52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e296fa8e40 0x55e296fb2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.252+0000 7fabfcc52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e296fa8e40 0x55e296fb2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40912/0 (socket says 172.21.15.16:40912) 2024-09-13T16:31:01.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.252+0000 7fabfcc52640 1 -- 172.21.15.16:0/4062281962 learned_addr learned my addr 172.21.15.16:0/4062281962 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:01.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.252+0000 7fabfd453640 1 --2- 172.21.15.16:0/4062281962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e296fa8500 0x55e296fb1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.252+0000 7fabfdc54640 1 --2- 172.21.15.16:0/4062281962 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e296faded0 0x55e296fb93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.252+0000 7fabfcc52640 1 -- 172.21.15.16:0/4062281962 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e296faded0 msgr2=0x55e296fb93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:01.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.252+0000 7fabfcc52640 1 --2- 172.21.15.16:0/4062281962 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e296faded0 0x55e296fb93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.252+0000 7fabfcc52640 1 -- 172.21.15.16:0/4062281962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e296fa8500 msgr2=0x55e296fb1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:01.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.252+0000 7fabfcc52640 1 --2- 172.21.15.16:0/4062281962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e296fa8500 0x55e296fb1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.252+0000 7fabfcc52640 1 -- 172.21.15.16:0/4062281962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf81a02f0 con 0x55e296fa8e40 2024-09-13T16:31:01.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.253+0000 7fabfcc52640 1 --2- 172.21.15.16:0/4062281962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e296fa8e40 0x55e296fb2440 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fabe800be80 tx=0x7fabe800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07498aaebd708be server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:01.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.253+0000 7fabe0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:31:01.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.253+0000 7fabe0ff9640 1 -- 172.21.15.16:0/4062281962 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe8011850 con 0x55e296fa8e40 2024-09-13T16:31:01.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.253+0000 7fabe0ff9640 1 -- 172.21.15.16:0/4062281962 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fabe80119b0 con 0x55e296fa8e40 2024-09-13T16:31:01.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.253+0000 7fac023dd740 1 -- 172.21.15.16:0/4062281962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e296bff3c0 con 0x55e296fa8e40 2024-09-13T16:31:01.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.253+0000 7fabe0ff9640 1 -- 172.21.15.16:0/4062281962 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabe8011b90 con 0x55e296fa8e40 2024-09-13T16:31:01.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.253+0000 7fac023dd740 10 client.15474 fetch_fsmap learned FSMap version 28 2024-09-13T16:31:01.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.253+0000 7fac023dd740 1 -- 172.21.15.16:0/4062281962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e296f0a270 con 0x55e296fa8e40 2024-09-13T16:31:01.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.253+0000 7fabe0ff9640 1 -- 172.21.15.16:0/4062281962 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e296bff3c0 con 0x55e296fa8e40 2024-09-13T16:31:01.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.253+0000 7fabe0ff9640 1 -- 172.21.15.16:0/4062281962 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fabe80103d0 con 0x55e296fa8e40 2024-09-13T16:31:01.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.253+0000 7fac023dd740 10 client.15474 fetch_fsmap finished waiting for FSMap version 28 2024-09-13T16:31:01.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.253+0000 7fac023dd740 10 client.15474 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-13T16:31:01.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.253+0000 7fac023dd740 20 client.15474 populate_metadata read hostname 'smithi016' 2024-09-13T16:31:01.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.253+0000 7fac023dd740 1 --2- 172.21.15.16:0/4062281962 >> [v2:172.21.15.111:6838/1423842285,v1:172.21.15.111:6839/1423842285] conn(0x55e296fbe3b0 0x55e296fc2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.253+0000 7fac023dd740 4 client.15474 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T16:31:01.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.253+0000 7fac023dd740 1 -- 172.21.15.16:0/4062281962 --> [v2:172.21.15.111:6838/1423842285,v1:172.21.15.111:6839/1423842285] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e296c106c0 con 0x55e296fbe3b0 2024-09-13T16:31:01.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.254+0000 7fabfd453640 1 --2- 172.21.15.16:0/4062281962 >> [v2:172.21.15.111:6838/1423842285,v1:172.21.15.111:6839/1423842285] conn(0x55e296fbe3b0 0x55e296fc2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.254+0000 7fabe0ff9640 10 client.15474 ms_handle_connect on v2:172.21.15.111:6838/1423842285 2024-09-13T16:31:01.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.254+0000 7fabfd453640 1 --2- 172.21.15.16:0/4062281962 >> [v2:172.21.15.111:6838/1423842285,v1:172.21.15.111:6839/1423842285] conn(0x55e296fbe3b0 0x55e296fc2510 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cd7f9d09344262e1 server_cookie=8bc07f0166a4d5d4 in_seq=0 out_seq=0 2024-09-13T16:31:01.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.255+0000 7fabe0ff9640 1 -- 172.21.15.16:0/4062281962 <== mds.1 v2:172.21.15.111:6838/1423842285 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e296c106c0 con 0x55e296fbe3b0 2024-09-13T16:31:01.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.255+0000 7fabe0ff9640 10 client.15474 handle_command_reply: tid=0 2024-09-13T16:31:01.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.255+0000 7fac023dd740 1 client.15474 shutdown 2024-09-13T16:31:01.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.255+0000 7fac023dd740 1 -- 172.21.15.16:0/4062281962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e296fa8e40 msgr2=0x55e296fb2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:01.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.255+0000 7fac023dd740 1 --2- 172.21.15.16:0/4062281962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e296fa8e40 0x55e296fb2440 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fabe800be80 tx=0x7fabe800f730 comp rx=0 tx=0).stop 2024-09-13T16:31:01.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.255+0000 7fac023dd740 1 -- 172.21.15.16:0/4062281962 shutdown_connections 2024-09-13T16:31:01.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.255+0000 7fac023dd740 1 --2- 172.21.15.16:0/4062281962 >> [v2:172.21.15.111:6838/1423842285,v1:172.21.15.111:6839/1423842285] conn(0x55e296fbe3b0 0x55e296fc2510 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.255+0000 7fac023dd740 1 --2- 172.21.15.16:0/4062281962 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e296faded0 0x55e296fb93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.255+0000 7fac023dd740 1 --2- 172.21.15.16:0/4062281962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e296fa8e40 0x55e296fb2440 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.255+0000 7fac023dd740 1 --2- 172.21.15.16:0/4062281962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e296fa8500 0x55e296fb1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.255+0000 7fabe0ff9640 0 client.15474 ms_handle_reset on v2:172.21.15.111:6838/1423842285 2024-09-13T16:31:01.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.255+0000 7fac023dd740 1 -- 172.21.15.16:0/4062281962 >> 172.21.15.16:0/4062281962 conn(0x55e296fa3b70 msgr2=0x55e296fa5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:01.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.255+0000 7fac023dd740 1 -- 172.21.15.16:0/4062281962 shutdown_connections 2024-09-13T16:31:01.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.255+0000 7fac023dd740 1 -- 172.21.15.16:0/4062281962 wait complete. 2024-09-13T16:31:01.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.255+0000 7fac023dd740 20 client.15474 trim_cache size 0 max 16384 2024-09-13T16:31:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.270+0000 7fac023dd740 1 Processor -- start 2024-09-13T16:31:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.270+0000 7fac023dd740 1 -- start start 2024-09-13T16:31:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.270+0000 7fac023dd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e296fa8500 0x55e296fb23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.270+0000 7fac023dd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e296fa8e40 0x55e296fb93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.270+0000 7fac023dd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e296faded0 0x55e296fc6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.270+0000 7fac023dd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e296e8f460 con 0x55e296fa8500 2024-09-13T16:31:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.270+0000 7fac023dd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e296f04990 con 0x55e296faded0 2024-09-13T16:31:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.270+0000 7fac023dd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e296f08850 con 0x55e296fa8e40 2024-09-13T16:31:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.270+0000 7fabfd453640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e296fa8500 0x55e296fb23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.270+0000 7fabfd453640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e296fa8500 0x55e296fb23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40922/0 (socket says 172.21.15.16:40922) 2024-09-13T16:31:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.270+0000 7fabfd453640 1 -- 172.21.15.16:0/2473150556 learned_addr learned my addr 172.21.15.16:0/2473150556 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.270+0000 7fabfcc52640 1 --2- 172.21.15.16:0/2473150556 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e296fa8e40 0x55e296fb93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.270+0000 7fabfd453640 1 -- 172.21.15.16:0/2473150556 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e296fa8e40 msgr2=0x55e296fb93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.270+0000 7fabfd453640 1 --2- 172.21.15.16:0/2473150556 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e296fa8e40 0x55e296fb93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.270+0000 7fabfd453640 1 -- 172.21.15.16:0/2473150556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e296faded0 msgr2=0x55e296fc6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.270+0000 7fabfdc54640 1 --2- 172.21.15.16:0/2473150556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e296faded0 0x55e296fc6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.270+0000 7fabfd453640 1 --2- 172.21.15.16:0/2473150556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e296faded0 0x55e296fc6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.270+0000 7fabfd453640 1 -- 172.21.15.16:0/2473150556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e296eac5f0 con 0x55e296fa8500 2024-09-13T16:31:01.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.271+0000 7fabfcc52640 1 --2- 172.21.15.16:0/2473150556 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e296fa8e40 0x55e296fb93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:31:01.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.271+0000 7fabfdc54640 1 --2- 172.21.15.16:0/2473150556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e296faded0 0x55e296fc6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:31:01.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.271+0000 7fabfd453640 1 --2- 172.21.15.16:0/2473150556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e296fa8500 0x55e296fb23c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fabf4005940 tx=0x7fabf40c36b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac38a844c6834e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:01.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.271+0000 7fabe3fff640 1 -- 172.21.15.16:0/2473150556 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf4086a90 con 0x55e296fa8500 2024-09-13T16:31:01.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.271+0000 7fabe3fff640 1 -- 172.21.15.16:0/2473150556 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fabf40b9650 con 0x55e296fa8500 2024-09-13T16:31:01.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.271+0000 7fabe3fff640 1 -- 172.21.15.16:0/2473150556 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf40bb490 con 0x55e296fa8500 2024-09-13T16:31:01.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.271+0000 7fac023dd740 1 -- 172.21.15.16:0/2473150556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e296fa8500 msgr2=0x55e296fb23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:01.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.271+0000 7fac023dd740 1 --2- 172.21.15.16:0/2473150556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e296fa8500 0x55e296fb23c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fabf4005940 tx=0x7fabf40c36b0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.271+0000 7fac023dd740 1 -- 172.21.15.16:0/2473150556 shutdown_connections 2024-09-13T16:31:01.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.271+0000 7fac023dd740 1 --2- 172.21.15.16:0/2473150556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e296faded0 0x55e296fc6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.271+0000 7fac023dd740 1 --2- 172.21.15.16:0/2473150556 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e296fa8e40 0x55e296fb93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.271+0000 7fac023dd740 1 --2- 172.21.15.16:0/2473150556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e296fa8500 0x55e296fb23c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.271+0000 7fac023dd740 1 -- 172.21.15.16:0/2473150556 >> 172.21.15.16:0/2473150556 conn(0x55e296fa3b70 msgr2=0x55e296fb1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:01.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.271+0000 7fac023dd740 1 -- 172.21.15.16:0/2473150556 shutdown_connections 2024-09-13T16:31:01.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.271+0000 7fac023dd740 1 -- 172.21.15.16:0/2473150556 wait complete. 2024-09-13T16:31:01.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.272+0000 7fac023dd740 1 -- start start 2024-09-13T16:31:01.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.272+0000 7fac023dd740 1 Processor -- start 2024-09-13T16:31:01.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.272+0000 7fac023dd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e296fa8500 0x55e296fb3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.272+0000 7fac023dd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e296fa8e40 0x55e296fb93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.272+0000 7fac023dd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e296faded0 0x55e296fb4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.272+0000 7fac023dd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e296fb4b30 con 0x55e296fa8e40 2024-09-13T16:31:01.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.272+0000 7fac023dd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e297014200 con 0x55e296fa8500 2024-09-13T16:31:01.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.272+0000 7fac023dd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2970143c0 con 0x55e296faded0 2024-09-13T16:31:01.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.272+0000 7fac023dd740 10 client.0 _ll_register_callbacks cb 0x7fabf40b75f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:31:01.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.272+0000 7fabfcc52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e296fa8e40 0x55e296fb93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.272+0000 7fabfcc52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e296fa8e40 0x55e296fb93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40930/0 (socket says 172.21.15.16:40930) 2024-09-13T16:31:01.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.272+0000 7fabfcc52640 1 -- 172.21.15.16:0/2636867020 learned_addr learned my addr 172.21.15.16:0/2636867020 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:01.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.272+0000 7fabfd453640 1 --2- 172.21.15.16:0/2636867020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e296fa8500 0x55e296fb3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.273+0000 7fabfdc54640 1 --2- 172.21.15.16:0/2636867020 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e296faded0 0x55e296fb4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.273+0000 7fabfcc52640 1 -- 172.21.15.16:0/2636867020 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e296faded0 msgr2=0x55e296fb4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:01.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.273+0000 7fabfcc52640 1 --2- 172.21.15.16:0/2636867020 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e296faded0 0x55e296fb4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.273+0000 7fabfcc52640 1 -- 172.21.15.16:0/2636867020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e296fa8500 msgr2=0x55e296fb3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:01.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.273+0000 7fabfcc52640 1 --2- 172.21.15.16:0/2636867020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e296fa8500 0x55e296fb3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.273+0000 7fabfcc52640 1 -- 172.21.15.16:0/2636867020 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e296e984b0 con 0x55e296fa8e40 2024-09-13T16:31:01.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.273+0000 7fabfd453640 1 --2- 172.21.15.16:0/2636867020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e296fa8500 0x55e296fb3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:31:01.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.273+0000 7fabfdc54640 1 --2- 172.21.15.16:0/2636867020 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e296faded0 0x55e296fb4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:31:01.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.273+0000 7fabfcc52640 1 --2- 172.21.15.16:0/2636867020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e296fa8e40 0x55e296fb93d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fabe8001eb0 tx=0x7fabe800ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e1d4a54d1a9f10 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.273+0000 7fabe0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:31:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.273+0000 7fabe0ff9640 1 -- 172.21.15.16:0/2636867020 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe8010a50 con 0x55e296fa8e40 2024-09-13T16:31:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.273+0000 7fabe0ff9640 1 -- 172.21.15.16:0/2636867020 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fabe80103d0 con 0x55e296fa8e40 2024-09-13T16:31:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.273+0000 7fac023dd740 1 -- 172.21.15.16:0/2636867020 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e296c63d10 con 0x55e296fa8e40 2024-09-13T16:31:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.273+0000 7fabe0ff9640 1 -- 172.21.15.16:0/2636867020 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabe8010590 con 0x55e296fa8e40 2024-09-13T16:31:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.273+0000 7fac023dd740 10 client.15480 fetch_fsmap learned FSMap version 28 2024-09-13T16:31:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.273+0000 7fac023dd740 1 -- 172.21.15.16:0/2636867020 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e296ed1d00 con 0x55e296fa8e40 2024-09-13T16:31:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.273+0000 7fabe0ff9640 1 -- 172.21.15.16:0/2636867020 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e296c63d10 con 0x55e296fa8e40 2024-09-13T16:31:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.274+0000 7fabe0ff9640 1 -- 172.21.15.16:0/2636867020 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fabe801a460 con 0x55e296fa8e40 2024-09-13T16:31:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.274+0000 7fac023dd740 10 client.15480 fetch_fsmap finished waiting for FSMap version 28 2024-09-13T16:31:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.274+0000 7fac023dd740 10 client.15480 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-13T16:31:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.274+0000 7fac023dd740 20 client.15480 populate_metadata read hostname 'smithi016' 2024-09-13T16:31:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.274+0000 7fac023dd740 1 --2- 172.21.15.16:0/2636867020 >> [v2:172.21.15.111:6838/1423842285,v1:172.21.15.111:6839/1423842285] conn(0x55e297019160 0x55e297015af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.274+0000 7fac023dd740 4 client.15480 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T16:31:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.274+0000 7fac023dd740 1 -- 172.21.15.16:0/2636867020 --> [v2:172.21.15.111:6838/1423842285,v1:172.21.15.111:6839/1423842285] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e296fbe7e0 con 0x55e297019160 2024-09-13T16:31:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.274+0000 7fabfd453640 1 --2- 172.21.15.16:0/2636867020 >> [v2:172.21.15.111:6838/1423842285,v1:172.21.15.111:6839/1423842285] conn(0x55e297019160 0x55e297015af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.274+0000 7fabe0ff9640 10 client.15480 ms_handle_connect on v2:172.21.15.111:6838/1423842285 2024-09-13T16:31:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.274+0000 7fabfd453640 1 --2- 172.21.15.16:0/2636867020 >> [v2:172.21.15.111:6838/1423842285,v1:172.21.15.111:6839/1423842285] conn(0x55e297019160 0x55e297015af0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8a1dd650b14d7dfc server_cookie=80f69f0cfaafaef0 in_seq=0 out_seq=0 2024-09-13T16:31:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.274+0000 7fabe0ff9640 1 -- 172.21.15.16:0/2636867020 <== mds.1 v2:172.21.15.111:6838/1423842285 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x7fabf40c36b0 con 0x55e297019160 2024-09-13T16:31:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.274+0000 7fabe0ff9640 10 client.15480 handle_command_reply: tid=0 2024-09-13T16:31:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.274+0000 7fac023dd740 1 client.15480 shutdown 2024-09-13T16:31:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.275+0000 7fac023dd740 1 -- 172.21.15.16:0/2636867020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e296fa8e40 msgr2=0x55e296fb93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.275+0000 7fac023dd740 1 --2- 172.21.15.16:0/2636867020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e296fa8e40 0x55e296fb93d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fabe8001eb0 tx=0x7fabe800ff80 comp rx=0 tx=0).stop 2024-09-13T16:31:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.275+0000 7fac023dd740 1 -- 172.21.15.16:0/2636867020 shutdown_connections 2024-09-13T16:31:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.275+0000 7fac023dd740 1 --2- 172.21.15.16:0/2636867020 >> [v2:172.21.15.111:6838/1423842285,v1:172.21.15.111:6839/1423842285] conn(0x55e297019160 0x55e297015af0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.275+0000 7fabe0ff9640 0 client.15480 ms_handle_reset on v2:172.21.15.111:6838/1423842285 2024-09-13T16:31:01.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.275+0000 7fac023dd740 1 --2- 172.21.15.16:0/2636867020 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e296faded0 0x55e296fb4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.275+0000 7fac023dd740 1 --2- 172.21.15.16:0/2636867020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e296fa8e40 0x55e296fb93d0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.275+0000 7fac023dd740 1 --2- 172.21.15.16:0/2636867020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e296fa8500 0x55e296fb3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.275+0000 7fac023dd740 1 -- 172.21.15.16:0/2636867020 >> 172.21.15.16:0/2636867020 conn(0x55e296fa3b70 msgr2=0x55e296fc6530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:01.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.275+0000 7fac023dd740 1 -- 172.21.15.16:0/2636867020 shutdown_connections 2024-09-13T16:31:01.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.275+0000 7fac023dd740 1 -- 172.21.15.16:0/2636867020 wait complete. 2024-09-13T16:31:01.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.275+0000 7fac023dd740 20 client.15480 trim_cache size 0 max 16384 2024-09-13T16:31:01.284 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-13T16:31:01.284 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "0d241f6a-71ec-11ef-bceb-c7b262605968", 2024-09-13T16:31:01.284 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 1, 2024-09-13T16:31:01.285 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24490, 2024-09-13T16:31:01.285 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-13T16:31:01.285 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-13T16:31:01.285 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-13T16:31:01.285 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 118.705291449, 2024-09-13T16:31:01.285 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 28, 2024-09-13T16:31:01.285 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 93, 2024-09-13T16:31:01.285 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 91, 2024-09-13T16:31:01.285 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 154.229374777 2024-09-13T16:31:01.285 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-13T16:31:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.276+0000 7fabff6b7640 1 -- 172.21.15.16:0/1811918063 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fabe40a1090 msgr2=0x7fabe40a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.276+0000 7fabff6b7640 1 --2- 172.21.15.16:0/1811918063 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fabe40a1090 0x7fabe40a3550 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fabf00137d0 tx=0x7fabf0005bf0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.276+0000 7fabff6b7640 1 -- 172.21.15.16:0/1811918063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabf80fe5e0 msgr2=0x7fabf8197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.276+0000 7fabff6b7640 1 --2- 172.21.15.16:0/1811918063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabf80fe5e0 0x7fabf8197e80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fabf40057c0 tx=0x7fabf40042f0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.276+0000 7fabff6b7640 1 -- 172.21.15.16:0/1811918063 shutdown_connections 2024-09-13T16:31:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.276+0000 7fabff6b7640 1 --2- 172.21.15.16:0/1811918063 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fabe40a1090 0x7fabe40a3550 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.276+0000 7fabff6b7640 1 --2- 172.21.15.16:0/1811918063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabf80ff8c0 0x7fabf819f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.276+0000 7fabff6b7640 1 --2- 172.21.15.16:0/1811918063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabf80fef20 0x7fabf81983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.276+0000 7fabff6b7640 1 --2- 172.21.15.16:0/1811918063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabf80fe5e0 0x7fabf8197e80 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.276+0000 7fabff6b7640 1 -- 172.21.15.16:0/1811918063 >> 172.21.15.16:0/1811918063 conn(0x7fabf80fa410 msgr2=0x7fabf80fbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.276+0000 7fabff6b7640 1 -- 172.21.15.16:0/1811918063 shutdown_connections 2024-09-13T16:31:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.276+0000 7fabff6b7640 1 -- 172.21.15.16:0/1811918063 wait complete. 2024-09-13T16:31:01.288 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-13T16:31:01.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.389+0000 7f1711dc7640 1 -- 172.21.15.16:0/1334663168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f170c0ffd90 msgr2=0x7f170c100190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:01.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.389+0000 7f1711dc7640 1 --2- 172.21.15.16:0/1334663168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f170c0ffd90 0x7f170c100190 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009900 tx=0x7f16fc02a410 comp rx=0 tx=0).stop 2024-09-13T16:31:01.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.389+0000 7f1711dc7640 1 -- 172.21.15.16:0/1334663168 shutdown_connections 2024-09-13T16:31:01.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.389+0000 7f1711dc7640 1 --2- 172.21.15.16:0/1334663168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f170c101940 0x7f170c1081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.389+0000 7f1711dc7640 1 --2- 172.21.15.16:0/1334663168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f170c100f80 0x7f170c101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.389+0000 7f1711dc7640 1 --2- 172.21.15.16:0/1334663168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f170c0ffd90 0x7f170c100190 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.389+0000 7f1711dc7640 1 -- 172.21.15.16:0/1334663168 >> 172.21.15.16:0/1334663168 conn(0x7f170c0fb4d0 msgr2=0x7f170c0fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:01.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.389+0000 7f1711dc7640 1 -- 172.21.15.16:0/1334663168 shutdown_connections 2024-09-13T16:31:01.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.389+0000 7f1711dc7640 1 -- 172.21.15.16:0/1334663168 wait complete. 2024-09-13T16:31:01.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.390+0000 7f1711dc7640 1 Processor -- start 2024-09-13T16:31:01.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.390+0000 7f1711dc7640 1 -- start start 2024-09-13T16:31:01.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.390+0000 7f1711dc7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f170c0ffd90 0x7f170c195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.390+0000 7f1711dc7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f170c100f80 0x7f170c196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.390+0000 7f1711dc7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f170c101940 0x7f170c19d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.390+0000 7f1711dc7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c19d7d0 con 0x7f170c0ffd90 2024-09-13T16:31:01.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.390+0000 7f1711dc7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c19d940 con 0x7f170c101940 2024-09-13T16:31:01.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.390+0000 7f1711dc7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c19db00 con 0x7f170c100f80 2024-09-13T16:31:01.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.390+0000 7f170b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f170c0ffd90 0x7f170c195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.390+0000 7f170b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f170c0ffd90 0x7f170c195cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40956/0 (socket says 172.21.15.16:40956) 2024-09-13T16:31:01.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.391+0000 7f170b7fe640 1 -- 172.21.15.16:0/3842377618 learned_addr learned my addr 172.21.15.16:0/3842377618 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:01.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.391+0000 7f170bfff640 1 --2- 172.21.15.16:0/3842377618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f170c101940 0x7f170c19d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.391+0000 7f170affd640 1 --2- 172.21.15.16:0/3842377618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f170c100f80 0x7f170c196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.391+0000 7f170b7fe640 1 -- 172.21.15.16:0/3842377618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f170c100f80 msgr2=0x7f170c196200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:01.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.391+0000 7f170b7fe640 1 --2- 172.21.15.16:0/3842377618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f170c100f80 0x7f170c196200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.391+0000 7f170b7fe640 1 -- 172.21.15.16:0/3842377618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f170c101940 msgr2=0x7f170c19d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:01.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.391+0000 7f170b7fe640 1 --2- 172.21.15.16:0/3842377618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f170c101940 0x7f170c19d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.391+0000 7f170b7fe640 1 -- 172.21.15.16:0/3842377618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16fc009560 con 0x7f170c0ffd90 2024-09-13T16:31:01.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.391+0000 7f170affd640 1 --2- 172.21.15.16:0/3842377618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f170c100f80 0x7f170c196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:31:01.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.391+0000 7f170b7fe640 1 --2- 172.21.15.16:0/3842377618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f170c0ffd90 0x7f170c195cc0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f16fc005ab0 tx=0x7f16fc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:01.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.391+0000 7f1708ff9640 1 -- 172.21.15.16:0/3842377618 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc031d70 con 0x7f170c0ffd90 2024-09-13T16:31:01.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.391+0000 7f1708ff9640 1 -- 172.21.15.16:0/3842377618 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f16fc0044e0 con 0x7f170c0ffd90 2024-09-13T16:31:01.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.391+0000 7f1711dc7640 1 -- 172.21.15.16:0/3842377618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f170c19dd80 con 0x7f170c0ffd90 2024-09-13T16:31:01.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.391+0000 7f1708ff9640 1 -- 172.21.15.16:0/3842377618 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc003a00 con 0x7f170c0ffd90 2024-09-13T16:31:01.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.391+0000 7f1711dc7640 1 -- 172.21.15.16:0/3842377618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f170c19e180 con 0x7f170c0ffd90 2024-09-13T16:31:01.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.394+0000 7f1708ff9640 1 -- 172.21.15.16:0/3842377618 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f16fc003ba0 con 0x7f170c0ffd90 2024-09-13T16:31:01.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.394+0000 7f1714bc4740 1 Processor -- start 2024-09-13T16:31:01.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.394+0000 7f1714bc4740 1 -- start start 2024-09-13T16:31:01.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.394+0000 7f1714bc4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557753edb500 0x557753edb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.394+0000 7f1714bc4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557753edbe40 0x557753ee0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.394+0000 7f1714bc4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557753ee0ed0 0x557753ee3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.394+0000 7f1714bc4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557753e367c0 con 0x557753edbe40 2024-09-13T16:31:01.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.394+0000 7f1714bc4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557753e45b70 con 0x557753edb500 2024-09-13T16:31:01.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.394+0000 7f1714bc4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x557753e43300 con 0x557753ee0ed0 2024-09-13T16:31:01.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.395+0000 7f170b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557753edbe40 0x557753ee0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.395+0000 7f170b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557753edbe40 0x557753ee0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40960/0 (socket says 172.21.15.16:40960) 2024-09-13T16:31:01.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.395+0000 7f170b7fe640 1 -- 172.21.15.16:0/3031751968 learned_addr learned my addr 172.21.15.16:0/3031751968 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:01.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.395+0000 7f170bfff640 1 --2- 172.21.15.16:0/3031751968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557753edb500 0x557753edb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.395+0000 7f170affd640 1 --2- 172.21.15.16:0/3031751968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557753ee0ed0 0x557753ee3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.395+0000 7f1708ff9640 1 --2- 172.21.15.16:0/3842377618 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f16f40a1090 0x7f16f40a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.395+0000 7f170b7fe640 1 -- 172.21.15.16:0/3031751968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557753ee0ed0 msgr2=0x557753ee3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:01.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.395+0000 7f170b7fe640 1 --2- 172.21.15.16:0/3031751968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557753ee0ed0 0x557753ee3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.395+0000 7f170b7fe640 1 -- 172.21.15.16:0/3031751968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557753edb500 msgr2=0x557753edb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:01.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.395+0000 7f170b7fe640 1 --2- 172.21.15.16:0/3031751968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557753edb500 0x557753edb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.395+0000 7f170b7fe640 1 -- 172.21.15.16:0/3031751968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557753dcb4b0 con 0x557753edbe40 2024-09-13T16:31:01.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.395+0000 7f170affd640 1 --2- 172.21.15.16:0/3031751968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557753ee0ed0 0x557753ee3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:31:01.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.395+0000 7f170bfff640 1 --2- 172.21.15.16:0/3842377618 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f16f40a1090 0x7f16f40a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.395+0000 7f1708ff9640 1 -- 172.21.15.16:0/3842377618 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f16fc0edd70 con 0x7f170c0ffd90 2024-09-13T16:31:01.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.395+0000 7f170b7fe640 1 --2- 172.21.15.16:0/3031751968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557753edbe40 0x557753ee0900 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f16fc08a9b0 tx=0x7f16fc0863a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abef8c1a47f28b11 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:01.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.395+0000 7f16f1ffb640 1 -- 172.21.15.16:0/3031751968 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc0ba770 con 0x557753edbe40 2024-09-13T16:31:01.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.396+0000 7f16f1ffb640 1 -- 172.21.15.16:0/3031751968 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f16fc0c8e90 con 0x557753edbe40 2024-09-13T16:31:01.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.396+0000 7f16f1ffb640 1 -- 172.21.15.16:0/3031751968 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc08a5d0 con 0x557753edbe40 2024-09-13T16:31:01.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.396+0000 7f170bfff640 1 --2- 172.21.15.16:0/3842377618 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f16f40a1090 0x7f16f40a3550 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1704005fd0 tx=0x7f1704004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:01.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.396+0000 7f1714bc4740 1 -- 172.21.15.16:0/3031751968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557753edbe40 msgr2=0x557753ee0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:01.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.396+0000 7f1714bc4740 1 --2- 172.21.15.16:0/3031751968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557753edbe40 0x557753ee0900 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f16fc08a9b0 tx=0x7f16fc0863a0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.396+0000 7f1714bc4740 1 -- 172.21.15.16:0/3031751968 shutdown_connections 2024-09-13T16:31:01.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.396+0000 7f1714bc4740 1 --2- 172.21.15.16:0/3031751968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557753ee0ed0 0x557753ee3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.396+0000 7f1714bc4740 1 --2- 172.21.15.16:0/3031751968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557753edbe40 0x557753ee0900 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.396+0000 7f1714bc4740 1 --2- 172.21.15.16:0/3031751968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557753edb500 0x557753edb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.396+0000 7f1714bc4740 1 -- 172.21.15.16:0/3031751968 >> 172.21.15.16:0/3031751968 conn(0x557753ed6b70 msgr2=0x557753ed6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:01.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.396+0000 7f1714bc4740 1 -- 172.21.15.16:0/3031751968 shutdown_connections 2024-09-13T16:31:01.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.396+0000 7f1714bc4740 1 -- 172.21.15.16:0/3031751968 wait complete. 2024-09-13T16:31:01.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.397+0000 7f1714bc4740 1 -- start start 2024-09-13T16:31:01.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.397+0000 7f1714bc4740 1 Processor -- start 2024-09-13T16:31:01.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.397+0000 7f1714bc4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557753edb500 0x557753ee4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.397+0000 7f1714bc4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557753edbe40 0x557753ee5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.397+0000 7f1714bc4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557753ee0ed0 0x557753eec260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.397+0000 7f1714bc4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557753e3ebf0 con 0x557753edb500 2024-09-13T16:31:01.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.397+0000 7f1714bc4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557753dfd250 con 0x557753ee0ed0 2024-09-13T16:31:01.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.397+0000 7f1714bc4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x557753dfcef0 con 0x557753edbe40 2024-09-13T16:31:01.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.397+0000 7f1714bc4740 10 client.0 _ll_register_callbacks cb 0x557753e2f740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:31:01.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.397+0000 7f170b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557753edb500 0x557753ee4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.397+0000 7f170b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557753edb500 0x557753ee4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40962/0 (socket says 172.21.15.16:40962) 2024-09-13T16:31:01.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.397+0000 7f170b7fe640 1 -- 172.21.15.16:0/658909860 learned_addr learned my addr 172.21.15.16:0/658909860 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:01.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.397+0000 7f170affd640 1 --2- 172.21.15.16:0/658909860 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557753edbe40 0x557753ee5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.397+0000 7f170bfff640 1 --2- 172.21.15.16:0/658909860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557753ee0ed0 0x557753eec260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.397+0000 7f170b7fe640 1 -- 172.21.15.16:0/658909860 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557753edbe40 msgr2=0x557753ee5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:01.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.397+0000 7f170b7fe640 1 --2- 172.21.15.16:0/658909860 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557753edbe40 0x557753ee5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.397+0000 7f170b7fe640 1 -- 172.21.15.16:0/658909860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557753ee0ed0 msgr2=0x557753eec260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:01.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.397+0000 7f170b7fe640 1 --2- 172.21.15.16:0/658909860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557753ee0ed0 0x557753eec260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.398+0000 7f170b7fe640 1 -- 172.21.15.16:0/658909860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f170c19e180 con 0x557753edb500 2024-09-13T16:31:01.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.398+0000 7f170bfff640 1 --2- 172.21.15.16:0/658909860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557753ee0ed0 0x557753eec260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:31:01.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.398+0000 7f170b7fe640 1 --2- 172.21.15.16:0/658909860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557753edb500 0x557753ee4ce0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0c2d60 tx=0x7f16fc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e328adde7abed8f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:01.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.398+0000 7f16eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:31:01.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.398+0000 7f16eaffd640 1 -- 172.21.15.16:0/658909860 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc0bb510 con 0x557753edb500 2024-09-13T16:31:01.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.398+0000 7f16eaffd640 1 -- 172.21.15.16:0/658909860 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f16fc0bae10 con 0x557753edb500 2024-09-13T16:31:01.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.398+0000 7f1714bc4740 1 -- 172.21.15.16:0/658909860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557753b323c0 con 0x557753edb500 2024-09-13T16:31:01.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.398+0000 7f16eaffd640 1 -- 172.21.15.16:0/658909860 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16fc0bec40 con 0x557753edb500 2024-09-13T16:31:01.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.398+0000 7f1714bc4740 10 client.15492 fetch_fsmap learned FSMap version 28 2024-09-13T16:31:01.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.398+0000 7f1714bc4740 1 -- 172.21.15.16:0/658909860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557753e3d270 con 0x557753edb500 2024-09-13T16:31:01.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.398+0000 7f16eaffd640 1 -- 172.21.15.16:0/658909860 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557753b323c0 con 0x557753edb500 2024-09-13T16:31:01.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.398+0000 7f16eaffd640 1 -- 172.21.15.16:0/658909860 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f16fc0f12e0 con 0x557753edb500 2024-09-13T16:31:01.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.398+0000 7f1714bc4740 10 client.15492 fetch_fsmap finished waiting for FSMap version 28 2024-09-13T16:31:01.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.398+0000 7f1714bc4740 10 client.15492 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-13T16:31:01.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.398+0000 7f1714bc4740 20 client.15492 populate_metadata read hostname 'smithi016' 2024-09-13T16:31:01.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.398+0000 7f1714bc4740 1 --2- 172.21.15.16:0/658909860 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x557753ef0e40 0x557753ef4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.398+0000 7f1714bc4740 4 client.15492 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T16:31:01.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.398+0000 7f1714bc4740 1 -- 172.21.15.16:0/658909860 --> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557753b436c0 con 0x557753ef0e40 2024-09-13T16:31:01.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.399+0000 7f170affd640 1 --2- 172.21.15.16:0/658909860 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x557753ef0e40 0x557753ef4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.399+0000 7f170affd640 1 --2- 172.21.15.16:0/658909860 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x557753ef0e40 0x557753ef4f80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd85c5543ca81785 server_cookie=1811c0e90ad958b1 in_seq=0 out_seq=0 2024-09-13T16:31:01.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.399+0000 7f16eaffd640 10 client.15492 ms_handle_connect on v2:172.21.15.107:6838/1914689806 2024-09-13T16:31:01.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.400+0000 7f16eaffd640 1 -- 172.21.15.16:0/658909860 <== mds.0 v2:172.21.15.107:6838/1914689806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557753b436c0 con 0x557753ef0e40 2024-09-13T16:31:01.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.400+0000 7f16eaffd640 10 client.15492 handle_command_reply: tid=0 2024-09-13T16:31:01.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.400+0000 7f1714bc4740 1 client.15492 shutdown 2024-09-13T16:31:01.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.400+0000 7f1714bc4740 1 -- 172.21.15.16:0/658909860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557753edb500 msgr2=0x557753ee4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:01.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.400+0000 7f1714bc4740 1 --2- 172.21.15.16:0/658909860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557753edb500 0x557753ee4ce0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0c2d60 tx=0x7f16fc009260 comp rx=0 tx=0).stop 2024-09-13T16:31:01.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.400+0000 7f1714bc4740 1 -- 172.21.15.16:0/658909860 shutdown_connections 2024-09-13T16:31:01.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.400+0000 7f1714bc4740 1 --2- 172.21.15.16:0/658909860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557753ee0ed0 0x557753eec260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.400+0000 7f1714bc4740 1 --2- 172.21.15.16:0/658909860 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x557753ef0e40 0x557753ef4f80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.400+0000 7f1714bc4740 1 --2- 172.21.15.16:0/658909860 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557753edbe40 0x557753ee5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.400+0000 7f1714bc4740 1 --2- 172.21.15.16:0/658909860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557753edb500 0x557753ee4ce0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.400+0000 7f16eaffd640 0 client.15492 ms_handle_reset on v2:172.21.15.107:6838/1914689806 2024-09-13T16:31:01.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.400+0000 7f1714bc4740 1 -- 172.21.15.16:0/658909860 >> 172.21.15.16:0/658909860 conn(0x557753ed6b70 msgr2=0x557753ed8380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:01.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.400+0000 7f1714bc4740 1 -- 172.21.15.16:0/658909860 shutdown_connections 2024-09-13T16:31:01.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.400+0000 7f1714bc4740 1 -- 172.21.15.16:0/658909860 wait complete. 2024-09-13T16:31:01.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.400+0000 7f1714bc4740 20 client.15492 trim_cache size 0 max 16384 2024-09-13T16:31:01.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.415+0000 7f1714bc4740 1 Processor -- start 2024-09-13T16:31:01.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.415+0000 7f1714bc4740 1 -- start start 2024-09-13T16:31:01.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.415+0000 7f1714bc4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557753edb500 0x557753ee51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.415+0000 7f1714bc4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557753edbe40 0x557753eec180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.415+0000 7f1714bc4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557753ee0ed0 0x557753ef8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.415+0000 7f1714bc4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557753dc2460 con 0x557753edb500 2024-09-13T16:31:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.415+0000 7f1714bc4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557753b436f0 con 0x557753edbe40 2024-09-13T16:31:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.415+0000 7f1714bc4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x557753ee5a40 con 0x557753ee0ed0 2024-09-13T16:31:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.415+0000 7f170b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557753edb500 0x557753ee51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.415+0000 7f170b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557753edb500 0x557753ee51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40972/0 (socket says 172.21.15.16:40972) 2024-09-13T16:31:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.415+0000 7f170b7fe640 1 -- 172.21.15.16:0/3283829579 learned_addr learned my addr 172.21.15.16:0/3283829579 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.416+0000 7f170bfff640 1 --2- 172.21.15.16:0/3283829579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557753ee0ed0 0x557753ef8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.416+0000 7f170b7fe640 1 -- 172.21.15.16:0/3283829579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557753ee0ed0 msgr2=0x557753ef8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.416+0000 7f170b7fe640 1 --2- 172.21.15.16:0/3283829579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557753ee0ed0 0x557753ef8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.416+0000 7f170affd640 1 --2- 172.21.15.16:0/3283829579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557753edbe40 0x557753eec180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.416+0000 7f170b7fe640 1 -- 172.21.15.16:0/3283829579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557753edbe40 msgr2=0x557753eec180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.416+0000 7f170b7fe640 1 --2- 172.21.15.16:0/3283829579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557753edbe40 0x557753eec180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.416+0000 7f170b7fe640 1 -- 172.21.15.16:0/3283829579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557753ddf5f0 con 0x557753edb500 2024-09-13T16:31:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.416+0000 7f170bfff640 1 --2- 172.21.15.16:0/3283829579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557753ee0ed0 0x557753ef8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:31:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.416+0000 7f170b7fe640 1 --2- 172.21.15.16:0/3283829579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557753edb500 0x557753ee51a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0bab10 tx=0x7f16fc0ba830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f11255c211a1e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.416+0000 7f16f27fc640 1 -- 172.21.15.16:0/3283829579 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc08c070 con 0x557753edb500 2024-09-13T16:31:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.416+0000 7f16f27fc640 1 -- 172.21.15.16:0/3283829579 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f16fc0cb070 con 0x557753edb500 2024-09-13T16:31:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.416+0000 7f16f27fc640 1 -- 172.21.15.16:0/3283829579 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc0bae10 con 0x557753edb500 2024-09-13T16:31:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.417+0000 7f1714bc4740 1 -- 172.21.15.16:0/3283829579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557753edb500 msgr2=0x557753ee51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.417+0000 7f1714bc4740 1 --2- 172.21.15.16:0/3283829579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557753edb500 0x557753ee51a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0bab10 tx=0x7f16fc0ba830 comp rx=0 tx=0).stop 2024-09-13T16:31:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.417+0000 7f1714bc4740 1 -- 172.21.15.16:0/3283829579 shutdown_connections 2024-09-13T16:31:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.417+0000 7f1714bc4740 1 --2- 172.21.15.16:0/3283829579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557753ee0ed0 0x557753ef8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.417+0000 7f1714bc4740 1 --2- 172.21.15.16:0/3283829579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557753edbe40 0x557753eec180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.417+0000 7f1714bc4740 1 --2- 172.21.15.16:0/3283829579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557753edb500 0x557753ee51a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.417+0000 7f1714bc4740 1 -- 172.21.15.16:0/3283829579 >> 172.21.15.16:0/3283829579 conn(0x557753ed6b70 msgr2=0x557753ee4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.417+0000 7f1714bc4740 1 -- 172.21.15.16:0/3283829579 shutdown_connections 2024-09-13T16:31:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.417+0000 7f1714bc4740 1 -- 172.21.15.16:0/3283829579 wait complete. 2024-09-13T16:31:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.417+0000 7f1714bc4740 1 -- start start 2024-09-13T16:31:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.417+0000 7f1714bc4740 1 Processor -- start 2024-09-13T16:31:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.417+0000 7f1714bc4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557753edb500 0x557753eec2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.417+0000 7f1714bc4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557753edbe40 0x557753ee8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.417+0000 7f1714bc4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557753ee0ed0 0x557753ee93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.417+0000 7f1714bc4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557753f46c40 con 0x557753ee0ed0 2024-09-13T16:31:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.417+0000 7f1714bc4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557753f46d80 con 0x557753edb500 2024-09-13T16:31:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.417+0000 7f1714bc4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x557753f46ec0 con 0x557753edbe40 2024-09-13T16:31:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.417+0000 7f1714bc4740 10 client.0 _ll_register_callbacks cb 0x7f16fc0f1640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:31:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.418+0000 7f170bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557753ee0ed0 0x557753ee93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.418+0000 7f170bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557753ee0ed0 0x557753ee93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40976/0 (socket says 172.21.15.16:40976) 2024-09-13T16:31:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.418+0000 7f170bfff640 1 -- 172.21.15.16:0/1063865138 learned_addr learned my addr 172.21.15.16:0/1063865138 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.418+0000 7f170affd640 1 --2- 172.21.15.16:0/1063865138 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557753edbe40 0x557753ee8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.418+0000 7f170b7fe640 1 --2- 172.21.15.16:0/1063865138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557753edb500 0x557753eec2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.418+0000 7f170bfff640 1 -- 172.21.15.16:0/1063865138 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557753edbe40 msgr2=0x557753ee8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:01.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.418+0000 7f170bfff640 1 --2- 172.21.15.16:0/1063865138 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557753edbe40 0x557753ee8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.418+0000 7f170bfff640 1 -- 172.21.15.16:0/1063865138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557753edb500 msgr2=0x557753eec2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:01.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.418+0000 7f170bfff640 1 --2- 172.21.15.16:0/1063865138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557753edb500 0x557753eec2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.418+0000 7f170bfff640 1 -- 172.21.15.16:0/1063865138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557753e3d270 con 0x557753ee0ed0 2024-09-13T16:31:01.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.418+0000 7f170b7fe640 1 --2- 172.21.15.16:0/1063865138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557753edb500 0x557753eec2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:31:01.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.418+0000 7f170bfff640 1 --2- 172.21.15.16:0/1063865138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557753ee0ed0 0x557753ee93f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1704021770 tx=0x7f1704004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd514b5613107e03 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:01.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.418+0000 7f16eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:31:01.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.418+0000 7f16eaffd640 1 -- 172.21.15.16:0/1063865138 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1704011990 con 0x557753ee0ed0 2024-09-13T16:31:01.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.418+0000 7f16eaffd640 1 -- 172.21.15.16:0/1063865138 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f1704011af0 con 0x557753ee0ed0 2024-09-13T16:31:01.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.418+0000 7f1714bc4740 1 -- 172.21.15.16:0/1063865138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f16fc0c8e90 con 0x557753ee0ed0 2024-09-13T16:31:01.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.418+0000 7f16eaffd640 1 -- 172.21.15.16:0/1063865138 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1704011cb0 con 0x557753ee0ed0 2024-09-13T16:31:01.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.418+0000 7f1714bc4740 10 client.15498 fetch_fsmap learned FSMap version 28 2024-09-13T16:31:01.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.418+0000 7f1714bc4740 1 -- 172.21.15.16:0/1063865138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557753e04d00 con 0x557753ee0ed0 2024-09-13T16:31:01.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.419+0000 7f16eaffd640 1 -- 172.21.15.16:0/1063865138 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1704033020 con 0x557753ee0ed0 2024-09-13T16:31:01.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.419+0000 7f16eaffd640 1 -- 172.21.15.16:0/1063865138 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1704010430 con 0x557753ee0ed0 2024-09-13T16:31:01.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.419+0000 7f1714bc4740 10 client.15498 fetch_fsmap finished waiting for FSMap version 28 2024-09-13T16:31:01.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.419+0000 7f1714bc4740 10 client.15498 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-13T16:31:01.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.419+0000 7f1714bc4740 20 client.15498 populate_metadata read hostname 'smithi016' 2024-09-13T16:31:01.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.419+0000 7f1714bc4740 1 --2- 172.21.15.16:0/1063865138 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x557753f4bd60 0x557753f486b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:01.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.419+0000 7f1714bc4740 4 client.15498 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T16:31:01.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.419+0000 7f1714bc4740 1 -- 172.21.15.16:0/1063865138 --> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557753b96d10 con 0x557753f4bd60 2024-09-13T16:31:01.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.419+0000 7f170b7fe640 1 --2- 172.21.15.16:0/1063865138 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x557753f4bd60 0x557753f486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:01.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.419+0000 7f16eaffd640 10 client.15498 ms_handle_connect on v2:172.21.15.107:6838/1914689806 2024-09-13T16:31:01.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.419+0000 7f170b7fe640 1 --2- 172.21.15.16:0/1063865138 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x557753f4bd60 0x557753f486b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d4d469923786b96 server_cookie=1dc3c27e471d5440 in_seq=0 out_seq=0 2024-09-13T16:31:01.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.420+0000 7f16eaffd640 1 -- 172.21.15.16:0/1063865138 <== mds.0 v2:172.21.15.107:6838/1914689806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x557753b96d10 con 0x557753f4bd60 2024-09-13T16:31:01.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.420+0000 7f16eaffd640 10 client.15498 handle_command_reply: tid=0 2024-09-13T16:31:01.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.420+0000 7f1714bc4740 1 client.15498 shutdown 2024-09-13T16:31:01.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.420+0000 7f1714bc4740 1 -- 172.21.15.16:0/1063865138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557753ee0ed0 msgr2=0x557753ee93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:01.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.420+0000 7f1714bc4740 1 --2- 172.21.15.16:0/1063865138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557753ee0ed0 0x557753ee93f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1704021770 tx=0x7f1704004ab0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.420+0000 7f1714bc4740 1 -- 172.21.15.16:0/1063865138 shutdown_connections 2024-09-13T16:31:01.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.420+0000 7f1714bc4740 1 --2- 172.21.15.16:0/1063865138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557753ee0ed0 0x557753ee93f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.420+0000 7f1714bc4740 1 --2- 172.21.15.16:0/1063865138 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x557753f4bd60 0x557753f486b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.420+0000 7f1714bc4740 1 --2- 172.21.15.16:0/1063865138 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557753edbe40 0x557753ee8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.420+0000 7f16eaffd640 0 client.15498 ms_handle_reset on v2:172.21.15.107:6838/1914689806 2024-09-13T16:31:01.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.420+0000 7f1714bc4740 1 --2- 172.21.15.16:0/1063865138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557753edb500 0x557753eec2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.420+0000 7f1714bc4740 1 -- 172.21.15.16:0/1063865138 >> 172.21.15.16:0/1063865138 conn(0x557753ed6b70 msgr2=0x557753ef8fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:01.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.420+0000 7f1714bc4740 1 -- 172.21.15.16:0/1063865138 shutdown_connections 2024-09-13T16:31:01.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.420+0000 7f1714bc4740 1 -- 172.21.15.16:0/1063865138 wait complete. 2024-09-13T16:31:01.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.420+0000 7f1714bc4740 20 client.15498 trim_cache size 0 max 16384 2024-09-13T16:31:01.429 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-13T16:31:01.429 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "0d241f6a-71ec-11ef-bceb-c7b262605968", 2024-09-13T16:31:01.429 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-13T16:31:01.429 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24518, 2024-09-13T16:31:01.429 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-13T16:31:01.429 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-13T16:31:01.429 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-13T16:31:01.429 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 142.234101192, 2024-09-13T16:31:01.429 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 28, 2024-09-13T16:31:01.429 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 93, 2024-09-13T16:31:01.429 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 91, 2024-09-13T16:31:01.429 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 146.634884306 2024-09-13T16:31:01.430 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-13T16:31:01.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.421+0000 7f1711dc7640 1 -- 172.21.15.16:0/3842377618 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f16f40a1090 msgr2=0x7f16f40a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:01.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.421+0000 7f1711dc7640 1 --2- 172.21.15.16:0/3842377618 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f16f40a1090 0x7f16f40a3550 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1704005fd0 tx=0x7f1704004a40 comp rx=0 tx=0).stop 2024-09-13T16:31:01.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.421+0000 7f1711dc7640 1 -- 172.21.15.16:0/3842377618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f170c0ffd90 msgr2=0x7f170c195cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:01.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.421+0000 7f1711dc7640 1 --2- 172.21.15.16:0/3842377618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f170c0ffd90 0x7f170c195cc0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f16fc005ab0 tx=0x7f16fc0042f0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.421+0000 7f1711dc7640 1 -- 172.21.15.16:0/3842377618 shutdown_connections 2024-09-13T16:31:01.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.421+0000 7f1711dc7640 1 --2- 172.21.15.16:0/3842377618 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f16f40a1090 0x7f16f40a3550 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.421+0000 7f1711dc7640 1 --2- 172.21.15.16:0/3842377618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f170c101940 0x7f170c19d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.421+0000 7f1711dc7640 1 --2- 172.21.15.16:0/3842377618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f170c100f80 0x7f170c196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.421+0000 7f1711dc7640 1 --2- 172.21.15.16:0/3842377618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f170c0ffd90 0x7f170c195cc0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:01.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.421+0000 7f1711dc7640 1 -- 172.21.15.16:0/3842377618 >> 172.21.15.16:0/3842377618 conn(0x7f170c0fb4d0 msgr2=0x7f170c104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:01.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.421+0000 7f1711dc7640 1 -- 172.21.15.16:0/3842377618 shutdown_connections 2024-09-13T16:31:01.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:01.421+0000 7f1711dc7640 1 -- 172.21.15.16:0/3842377618 wait complete. 2024-09-13T16:31:01.433 INFO:teuthology.run_tasks:Running task sequential... 2024-09-13T16:31:01.447 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-13T16:31:01.447 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-13T16:31:01.448 INFO:teuthology.task.print:Enabling mgr modules 2024-09-13T16:31:01.448 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-13T16:31:01.458 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-13T16:31:01.458 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-09-13T16:31:01.459 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-09-13T16:31:01.459 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T16:31:01.459 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T16:31:01.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:31:01.459 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-13T16:31:01.476 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-13T16:31:01.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:31:01.477 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-13T16:31:01.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:31:01.543 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-09-13T16:31:01.637 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-13T16:31:01.637 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-13T16:31:01.637 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-13T16:31:01.846 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:31:02.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:02 smithi111 ceph-mon[34600]: pgmap v352: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-13T16:31:03.026 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:02 smithi016 ceph-mon[31187]: pgmap v352: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-13T16:31:03.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:02 smithi107 ceph-mon[34139]: pgmap v352: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-13T16:31:03.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.604+0000 7fe9e099b640 1 -- 172.21.15.16:0/4234239970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9dc10a3a0 msgr2=0x7fe9dc10c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:03.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.604+0000 7fe9e099b640 1 --2- 172.21.15.16:0/4234239970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9dc10a3a0 0x7fe9dc10c840 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4009ca0 tx=0x7fe9d402a8c0 comp rx=0 tx=0).stop 2024-09-13T16:31:03.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.604+0000 7fe9e099b640 1 -- 172.21.15.16:0/4234239970 shutdown_connections 2024-09-13T16:31:03.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.604+0000 7fe9e099b640 1 --2- 172.21.15.16:0/4234239970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9dc10a3a0 0x7fe9dc10c840 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:03.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.604+0000 7fe9e099b640 1 --2- 172.21.15.16:0/4234239970 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9dc0ff540 0x7fe9dc109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:03.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.604+0000 7fe9e099b640 1 --2- 172.21.15.16:0/4234239970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9dc0feb90 0x7fe9dc0fef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:03.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.604+0000 7fe9e099b640 1 -- 172.21.15.16:0/4234239970 >> 172.21.15.16:0/4234239970 conn(0x7fe9dc074130 msgr2=0x7fe9dc074540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:03.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.604+0000 7fe9e099b640 1 -- 172.21.15.16:0/4234239970 shutdown_connections 2024-09-13T16:31:03.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.604+0000 7fe9e099b640 1 -- 172.21.15.16:0/4234239970 wait complete. 2024-09-13T16:31:03.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.605+0000 7fe9e099b640 1 Processor -- start 2024-09-13T16:31:03.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.605+0000 7fe9e099b640 1 -- start start 2024-09-13T16:31:03.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.606+0000 7fe9e099b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9dc0feb90 0x7fe9dc1960d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:03.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.606+0000 7fe9e099b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9dc0ff540 0x7fe9dc196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:03.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.606+0000 7fe9e099b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9dc10a3a0 0x7fe9dc19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:03.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.606+0000 7fe9e099b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc19b010 con 0x7fe9dc0feb90 2024-09-13T16:31:03.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.606+0000 7fe9e099b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc19b180 con 0x7fe9dc0ff540 2024-09-13T16:31:03.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.606+0000 7fe9e099b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc19b2f0 con 0x7fe9dc10a3a0 2024-09-13T16:31:03.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.606+0000 7fe9da59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9dc0feb90 0x7fe9dc1960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:03.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.606+0000 7fe9da59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9dc0feb90 0x7fe9dc1960d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44328/0 (socket says 172.21.15.16:44328) 2024-09-13T16:31:03.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.606+0000 7fe9da59c640 1 -- 172.21.15.16:0/3636232436 learned_addr learned my addr 172.21.15.16:0/3636232436 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:03.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.606+0000 7fe9dad9d640 1 --2- 172.21.15.16:0/3636232436 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9dc10a3a0 0x7fe9dc19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:03.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.606+0000 7fe9d9d9b640 1 --2- 172.21.15.16:0/3636232436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9dc0ff540 0x7fe9dc196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:03.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.606+0000 7fe9da59c640 1 -- 172.21.15.16:0/3636232436 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9dc10a3a0 msgr2=0x7fe9dc19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:03.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.606+0000 7fe9da59c640 1 --2- 172.21.15.16:0/3636232436 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9dc10a3a0 0x7fe9dc19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:03.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.606+0000 7fe9da59c640 1 -- 172.21.15.16:0/3636232436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9dc0ff540 msgr2=0x7fe9dc196610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:03.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.606+0000 7fe9da59c640 1 --2- 172.21.15.16:0/3636232436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9dc0ff540 0x7fe9dc196610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:03.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.606+0000 7fe9da59c640 1 -- 172.21.15.16:0/3636232436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d4009900 con 0x7fe9dc0feb90 2024-09-13T16:31:03.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.606+0000 7fe9da59c640 1 --2- 172.21.15.16:0/3636232436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9dc0feb90 0x7fe9dc1960d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00e9f0 tx=0x7fe9cc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:03.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.607+0000 7fe9cb7fe640 1 -- 172.21.15.16:0/3636232436 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc009700 con 0x7fe9dc0feb90 2024-09-13T16:31:03.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.607+0000 7fe9e099b640 1 -- 172.21.15.16:0/3636232436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9dc19b5d0 con 0x7fe9dc0feb90 2024-09-13T16:31:03.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.607+0000 7fe9cb7fe640 1 -- 172.21.15.16:0/3636232436 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9cc00cd80 con 0x7fe9dc0feb90 2024-09-13T16:31:03.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.607+0000 7fe9cb7fe640 1 -- 172.21.15.16:0/3636232436 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc0187e0 con 0x7fe9dc0feb90 2024-09-13T16:31:03.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.607+0000 7fe9e099b640 1 -- 172.21.15.16:0/3636232436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9dc1a2e90 con 0x7fe9dc0feb90 2024-09-13T16:31:03.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.608+0000 7fe9e099b640 1 -- 172.21.15.16:0/3636232436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9dc0680a0 con 0x7fe9dc0feb90 2024-09-13T16:31:03.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.609+0000 7fe9cb7fe640 1 -- 172.21.15.16:0/3636232436 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe9cc010430 con 0x7fe9dc0feb90 2024-09-13T16:31:03.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.609+0000 7fe9cb7fe640 1 --2- 172.21.15.16:0/3636232436 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe9c40a1080 0x7fe9c40a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:03.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.610+0000 7fe9cb7fe640 1 -- 172.21.15.16:0/3636232436 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fe9cc014070 con 0x7fe9dc0feb90 2024-09-13T16:31:03.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.610+0000 7fe9d9d9b640 1 --2- 172.21.15.16:0/3636232436 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe9c40a1080 0x7fe9c40a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:03.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.610+0000 7fe9d9d9b640 1 --2- 172.21.15.16:0/3636232436 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe9c40a1080 0x7fe9c40a3540 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc197680 tx=0x7fe9d0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:03.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.613+0000 7fe9cb7fe640 1 -- 172.21.15.16:0/3636232436 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fe9cc0968d0 con 0x7fe9dc0feb90 2024-09-13T16:31:03.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.820+0000 7fe9e099b640 1 -- 172.21.15.16:0/3636232436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe9dc0fef70 con 0x7fe9dc0feb90 2024-09-13T16:31:03.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.820+0000 7fe9cb7fe640 1 -- 172.21.15.16:0/3636232436 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) v1 ==== 53+0+88 (secure 0 0 0) 0x7fe9cc099ef0 con 0x7fe9dc0feb90 2024-09-13T16:31:03.823 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-13T16:31:03.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.823+0000 7fe9e099b640 1 -- 172.21.15.16:0/3636232436 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe9c40a1080 msgr2=0x7fe9c40a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:03.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.823+0000 7fe9e099b640 1 --2- 172.21.15.16:0/3636232436 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe9c40a1080 0x7fe9c40a3540 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc197680 tx=0x7fe9d0005f90 comp rx=0 tx=0).stop 2024-09-13T16:31:03.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.823+0000 7fe9e099b640 1 -- 172.21.15.16:0/3636232436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9dc0feb90 msgr2=0x7fe9dc1960d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:03.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.823+0000 7fe9e099b640 1 --2- 172.21.15.16:0/3636232436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9dc0feb90 0x7fe9dc1960d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00e9f0 tx=0x7fe9cc00eeb0 comp rx=0 tx=0).stop 2024-09-13T16:31:03.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.823+0000 7fe9e099b640 1 -- 172.21.15.16:0/3636232436 shutdown_connections 2024-09-13T16:31:03.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.823+0000 7fe9e099b640 1 --2- 172.21.15.16:0/3636232436 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9dc10a3a0 0x7fe9dc19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:03.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.823+0000 7fe9e099b640 1 --2- 172.21.15.16:0/3636232436 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe9c40a1080 0x7fe9c40a3540 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:03.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.823+0000 7fe9e099b640 1 --2- 172.21.15.16:0/3636232436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9dc0ff540 0x7fe9dc196610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:03.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.823+0000 7fe9e099b640 1 --2- 172.21.15.16:0/3636232436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9dc0feb90 0x7fe9dc1960d0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:03.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.823+0000 7fe9e099b640 1 -- 172.21.15.16:0/3636232436 >> 172.21.15.16:0/3636232436 conn(0x7fe9dc074130 msgr2=0x7fe9dc0fbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:03.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.823+0000 7fe9e099b640 1 -- 172.21.15.16:0/3636232436 shutdown_connections 2024-09-13T16:31:03.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:03.823+0000 7fe9e099b640 1 -- 172.21.15.16:0/3636232436 wait complete. 2024-09-13T16:31:04.435 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T16:31:04.435 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T16:31:04.436 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-13T16:31:04.436 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-13T16:31:04.436 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T16:31:04.436 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T16:31:04.436 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T16:31:04.436 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T16:31:04.436 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T16:31:04.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:31:04.436 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-13T16:31:04.455 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T16:31:04.455 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T16:31:04.456 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T16:31:04.456 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:31:04.456 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-13T16:31:04.456 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:31:04.456 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T16:31:04.456 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-13T16:31:04.456 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-13T16:31:04.456 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T16:31:04.456 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-13T16:31:04.456 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-13T16:31:04.456 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T16:31:04.456 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-13T16:31:04.456 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-13T16:31:04.456 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-13T16:31:04.457 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42186sec preferred_lft 42186sec 2024-09-13T16:31:04.457 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-13T16:31:04.457 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:31:04.457 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T16:31:04.457 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-13T16:31:04.457 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-13T16:31:04.457 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-13T16:31:04.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:31:04.457 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-09-13T16:31:04.457 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-13T16:31:04.458 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2024-09-13T16:31:04.458 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2024-09-13T16:31:04.458 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2024-09-13T16:31:04.458 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-13T16:31:04.458 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-13T16:31:04.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:31:04.566 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T16:31:04.592 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-13T16:31:04.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:31:04.594 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-13T16:31:04.653 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-13T16:31:04.653 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T16:31:04.653 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-13T16:31:04.653 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-13T16:31:04.653 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T16:31:04.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:31:04.654 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-09-13T16:31:04.654 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-13T16:31:04.654 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-13T16:31:04.654 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-13T16:31:04.654 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-13T16:31:04.654 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-13T16:31:04.718 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:04 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3636232436' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-13T16:31:04.718 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:04 smithi016 ceph-mon[31187]: pgmap v353: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T16:31:04.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:31:04.744 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-13T16:31:04.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:31:04.802 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2024-09-13T16:31:04.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:31:04.858 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-09-13T16:31:04.858 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-13T16:31:04.858 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T16:31:04.858 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-13T16:31:04.858 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-13T16:31:04.937 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-13T16:31:04.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:31:04.938 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-09-13T16:31:04.938 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-13T16:31:04.938 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-13T16:31:04.938 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-13T16:31:04.938 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-13T16:31:04.938 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-13T16:31:04.938 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-13T16:31:04.938 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-13T16:31:04.939 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:04 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3636232436' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-13T16:31:04.939 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:04 smithi111 ceph-mon[34600]: pgmap v353: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T16:31:05.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:31:05.056 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-09-13T16:31:05.056 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-13T16:31:05.056 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-13T16:31:05.056 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2024-09-13T16:31:05.056 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-13T16:31:05.094 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-09-13T16:31:05.094 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T16:31:05.095 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-13T16:31:05.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:04 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3636232436' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-13T16:31:05.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:04 smithi107 ceph-mon[34139]: pgmap v353: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T16:31:05.151 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-13T16:31:05.151 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T16:31:05.151 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-13T16:31:05.207 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-13T16:31:05.274 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T16:31:05.275 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T16:31:05.340 INFO:teuthology.orchestra.run.smithi016.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T16:31:05.342 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T16:31:05.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:31:05.342 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-13T16:31:05.399 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-13T16:31:05.399 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/d317f307-79ab-4bc1-8aca-0d73735f8789 --client_fs=cephfs) 2024-09-13T16:31:05.441 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-13T16:31:05.466 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T16:31:05.467 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T16:31:05.526 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2024-09-13T16:31:05.524+0000 7fa67cb0f480 -1 init, newargv = 0x55df315733c0 newargc=15 2024-09-13T16:31:05.526 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[85149]: starting ceph client 2024-09-13T16:31:05.533 INFO:teuthology.orchestra.run.smithi016.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T16:31:05.535 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T16:31:05.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:31:05.536 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-13T16:31:05.547 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[85149]: starting fuse 2024-09-13T16:31:05.562 INFO:teuthology.orchestra.run.smithi016.stdout:120 2024-09-13T16:31:05.563 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-13T16:31:05.563 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T16:31:05.563 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-13T16:31:05.563 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-13T16:31:05.563 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-13T16:31:05.563 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-13T16:31:05.564 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-13T16:31:05.564 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-13T16:31:05.564 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-13T16:31:05.564 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-13T16:31:05.564 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-13T16:31:05.564 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-13T16:31:05.564 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-13T16:31:05.564 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-13T16:31:05.564 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-13T16:31:05.564 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-13T16:31:05.564 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-13T16:31:05.564 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-13T16:31:05.565 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T16:31:05.565 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T16:31:05.565 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T16:31:05.565 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-13T16:31:05.565 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-13T16:31:05.565 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-13T16:31:05.565 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T16:31:05.565 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-13T16:31:05.565 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-09-13T16:31:05.565 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-13T16:31:05.679 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.85149.asok 2024-09-13T16:31:05.865 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.85149.asok 2024-09-13T16:31:05.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:31:05.866 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.85149.asok status 2024-09-13T16:31:06.003 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-13T16:31:06.003 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-13T16:31:06.003 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "50bbe7f395997dcd871813949c49addbbf630361", 2024-09-13T16:31:06.003 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 18.2.4-850-g50bbe7f3 (50bbe7f395997dcd871813949c49addbbf630361) reef (stable)", 2024-09-13T16:31:06.003 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2024-09-13T16:31:06.003 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-13T16:31:06.003 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-13T16:31:06.003 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "85149", 2024-09-13T16:31:06.003 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/volumes/qa/sv_0/d317f307-79ab-4bc1-8aca-0d73735f8789" 2024-09-13T16:31:06.003 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:31:06.003 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-09-13T16:31:06.004 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-09-13T16:31:06.004 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 15510, 2024-09-13T16:31:06.004 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-13T16:31:06.004 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-13T16:31:06.004 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-13T16:31:06.004 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 15510 2024-09-13T16:31:06.004 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:31:06.004 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-13T16:31:06.004 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-13T16:31:06.004 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-13T16:31:06.004 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 425144011 2024-09-13T16:31:06.004 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:31:06.004 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:31:06.005 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-13T16:31:06.005 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-13T16:31:06.005 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-13T16:31:06.005 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 425144011 2024-09-13T16:31:06.005 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:31:06.005 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.15510 192.168.0.1:0/425144011", 2024-09-13T16:31:06.005 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/425144011", 2024-09-13T16:31:06.005 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 5, 2024-09-13T16:31:06.005 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 28, 2024-09-13T16:31:06.005 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 93, 2024-09-13T16:31:06.005 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-09-13T16:31:06.005 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-13T16:31:06.005 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-13T16:31:06.006 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-13T16:31:06.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:31:06.011 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-09-13T16:31:06.012 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-13T16:31:06.028 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-09-13T16:31:06.028 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-13T16:31:06.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:31:06.029 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-13T16:31:06.097 INFO:teuthology.run_tasks:Running task exec... 2024-09-13T16:31:06.108 INFO:teuthology.task.exec:Executing custom commands... 2024-09-13T16:31:06.108 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-09-13T16:31:06.109 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-13T16:31:06.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.239+0000 7feaec75a640 1 -- 172.21.15.16:0/4191844767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae41018f0 msgr2=0x7feae4108180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:06.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.239+0000 7feaec75a640 1 --2- 172.21.15.16:0/4191844767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae41018f0 0x7feae4108180 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7feadc009ca0 tx=0x7feadc02a8c0 comp rx=0 tx=0).stop 2024-09-13T16:31:06.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.239+0000 7feaec75a640 1 -- 172.21.15.16:0/4191844767 shutdown_connections 2024-09-13T16:31:06.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.239+0000 7feaec75a640 1 --2- 172.21.15.16:0/4191844767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae41018f0 0x7feae4108180 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:06.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.239+0000 7feaec75a640 1 --2- 172.21.15.16:0/4191844767 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feae4100f30 0x7feae41013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:06.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.239+0000 7feaec75a640 1 --2- 172.21.15.16:0/4191844767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feae40ffd40 0x7feae4100140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:06.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.239+0000 7feaec75a640 1 -- 172.21.15.16:0/4191844767 >> 172.21.15.16:0/4191844767 conn(0x7feae40fb4c0 msgr2=0x7feae40fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:06.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.239+0000 7feaec75a640 1 -- 172.21.15.16:0/4191844767 shutdown_connections 2024-09-13T16:31:06.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.239+0000 7feaec75a640 1 -- 172.21.15.16:0/4191844767 wait complete. 2024-09-13T16:31:06.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.240+0000 7feaec75a640 1 Processor -- start 2024-09-13T16:31:06.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.240+0000 7feaec75a640 1 -- start start 2024-09-13T16:31:06.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.240+0000 7feaec75a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feae40ffd40 0x7feae4195c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:06.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.240+0000 7feaec75a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae4100f30 0x7feae41961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:06.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.240+0000 7feaec75a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feae41018f0 0x7feae419d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:06.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.240+0000 7feaec75a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae419d700 con 0x7feae4100f30 2024-09-13T16:31:06.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.240+0000 7feaec75a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae419d870 con 0x7feae41018f0 2024-09-13T16:31:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.240+0000 7feaec75a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae419da30 con 0x7feae40ffd40 2024-09-13T16:31:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.240+0000 7feae9cf5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae4100f30 0x7feae41961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.240+0000 7feae9cf5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae4100f30 0x7feae41961b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44356/0 (socket says 172.21.15.16:44356) 2024-09-13T16:31:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.240+0000 7feae9cf5640 1 -- 172.21.15.16:0/1060390976 learned_addr learned my addr 172.21.15.16:0/1060390976 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.240+0000 7feaea4f6640 1 --2- 172.21.15.16:0/1060390976 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feae40ffd40 0x7feae4195c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.240+0000 7feae9cf5640 1 -- 172.21.15.16:0/1060390976 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feae40ffd40 msgr2=0x7feae4195c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.240+0000 7feae9cf5640 1 --2- 172.21.15.16:0/1060390976 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feae40ffd40 0x7feae4195c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.240+0000 7feaeacf7640 1 --2- 172.21.15.16:0/1060390976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feae41018f0 0x7feae419d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.240+0000 7feae9cf5640 1 -- 172.21.15.16:0/1060390976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feae41018f0 msgr2=0x7feae419d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.240+0000 7feae9cf5640 1 --2- 172.21.15.16:0/1060390976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feae41018f0 0x7feae419d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.240+0000 7feae9cf5640 1 -- 172.21.15.16:0/1060390976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feadc009900 con 0x7feae4100f30 2024-09-13T16:31:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.241+0000 7feae9cf5640 1 --2- 172.21.15.16:0/1060390976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae4100f30 0x7feae41961b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fead400d770 tx=0x7fead400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.241+0000 7feadb7fe640 1 -- 172.21.15.16:0/1060390976 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead4009c30 con 0x7feae4100f30 2024-09-13T16:31:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.241+0000 7feadb7fe640 1 -- 172.21.15.16:0/1060390976 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fead400de80 con 0x7feae4100f30 2024-09-13T16:31:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.241+0000 7feaec75a640 1 -- 172.21.15.16:0/1060390976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae419dd10 con 0x7feae4100f30 2024-09-13T16:31:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.241+0000 7feadb7fe640 1 -- 172.21.15.16:0/1060390976 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead4004d40 con 0x7feae4100f30 2024-09-13T16:31:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.241+0000 7feaec75a640 1 -- 172.21.15.16:0/1060390976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae419e1d0 con 0x7feae4100f30 2024-09-13T16:31:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.242+0000 7feaec75a640 1 -- 172.21.15.16:0/1060390976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae41072c0 con 0x7feae4100f30 2024-09-13T16:31:06.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.243+0000 7feadb7fe640 1 -- 172.21.15.16:0/1060390976 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fead4004ea0 con 0x7feae4100f30 2024-09-13T16:31:06.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.244+0000 7feadb7fe640 1 --2- 172.21.15.16:0/1060390976 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fead00a1090 0x7fead00a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:06.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.244+0000 7feadb7fe640 1 -- 172.21.15.16:0/1060390976 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fead40ccbd0 con 0x7feae4100f30 2024-09-13T16:31:06.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.244+0000 7feaea4f6640 1 --2- 172.21.15.16:0/1060390976 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fead00a1090 0x7fead00a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:06.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.244+0000 7feaea4f6640 1 --2- 172.21.15.16:0/1060390976 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fead00a1090 0x7fead00a3550 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7feae41970f0 tx=0x7feae000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:06.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.247+0000 7feadb7fe640 1 -- 172.21.15.16:0/1060390976 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fead4095f20 con 0x7feae4100f30 2024-09-13T16:31:06.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.454+0000 7feaec75a640 1 -- 172.21.15.16:0/1060390976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) v1 -- 0x7feae410b5a0 con 0x7feae4100f30 2024-09-13T16:31:06.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.509+0000 7feadb7fe640 1 -- 172.21.15.16:0/1060390976 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v29) v1 ==== 118+0+0 (secure 0 0 0) 0x7fead4017020 con 0x7feae4100f30 2024-09-13T16:31:06.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.511+0000 7feaec75a640 1 -- 172.21.15.16:0/1060390976 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fead00a1090 msgr2=0x7fead00a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:06.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.511+0000 7feaec75a640 1 --2- 172.21.15.16:0/1060390976 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fead00a1090 0x7fead00a3550 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7feae41970f0 tx=0x7feae000a2d0 comp rx=0 tx=0).stop 2024-09-13T16:31:06.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.511+0000 7feaec75a640 1 -- 172.21.15.16:0/1060390976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae4100f30 msgr2=0x7feae41961b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:06.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.511+0000 7feaec75a640 1 --2- 172.21.15.16:0/1060390976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae4100f30 0x7feae41961b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fead400d770 tx=0x7fead400dc30 comp rx=0 tx=0).stop 2024-09-13T16:31:06.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.511+0000 7feaec75a640 1 -- 172.21.15.16:0/1060390976 shutdown_connections 2024-09-13T16:31:06.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.511+0000 7feaec75a640 1 --2- 172.21.15.16:0/1060390976 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fead00a1090 0x7fead00a3550 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:06.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.511+0000 7feaec75a640 1 --2- 172.21.15.16:0/1060390976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feae41018f0 0x7feae419d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:06.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.511+0000 7feaec75a640 1 --2- 172.21.15.16:0/1060390976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae4100f30 0x7feae41961b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:06.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.511+0000 7feaec75a640 1 --2- 172.21.15.16:0/1060390976 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feae40ffd40 0x7feae4195c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:06.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.511+0000 7feaec75a640 1 -- 172.21.15.16:0/1060390976 >> 172.21.15.16:0/1060390976 conn(0x7feae40fb4c0 msgr2=0x7feae4104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:06.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.512+0000 7feaec75a640 1 -- 172.21.15.16:0/1060390976 shutdown_connections 2024-09-13T16:31:06.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.512+0000 7feaec75a640 1 -- 172.21.15.16:0/1060390976 wait complete. 2024-09-13T16:31:06.525 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-13T16:31:06.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.628+0000 7f9275f9d640 1 -- 172.21.15.16:0/8308013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9270100620 msgr2=0x7f9270106eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:06.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.628+0000 7f9275f9d640 1 --2- 172.21.15.16:0/8308013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9270100620 0x7f9270106eb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9268009ca0 tx=0x7f926802a8c0 comp rx=0 tx=0).stop 2024-09-13T16:31:06.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.628+0000 7f9275f9d640 1 -- 172.21.15.16:0/8308013 shutdown_connections 2024-09-13T16:31:06.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.628+0000 7f9275f9d640 1 --2- 172.21.15.16:0/8308013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9270100620 0x7f9270106eb0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:06.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.628+0000 7f9275f9d640 1 --2- 172.21.15.16:0/8308013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92700ffc60 0x7f92701000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:06.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.628+0000 7f9275f9d640 1 --2- 172.21.15.16:0/8308013 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92700fea70 0x7f92700fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:06.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.628+0000 7f9275f9d640 1 -- 172.21.15.16:0/8308013 >> 172.21.15.16:0/8308013 conn(0x7f92700fa1f0 msgr2=0x7f92700fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:06.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.628+0000 7f9275f9d640 1 -- 172.21.15.16:0/8308013 shutdown_connections 2024-09-13T16:31:06.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.628+0000 7f9275f9d640 1 -- 172.21.15.16:0/8308013 wait complete. 2024-09-13T16:31:06.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.629+0000 7f9275f9d640 1 Processor -- start 2024-09-13T16:31:06.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.629+0000 7f9275f9d640 1 -- start start 2024-09-13T16:31:06.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.629+0000 7f9275f9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92700fea70 0x7f92701947e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:06.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.629+0000 7f9275f9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92700ffc60 0x7f9270194d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:06.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.629+0000 7f9275f9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9270100620 0x7f927019bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:06.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.629+0000 7f9275f9d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927019c2d0 con 0x7f92700ffc60 2024-09-13T16:31:06.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.629+0000 7f9275f9d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927019c440 con 0x7f9270100620 2024-09-13T16:31:06.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.629+0000 7f9275f9d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927019c600 con 0x7f92700fea70 2024-09-13T16:31:06.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.629+0000 7f926ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92700ffc60 0x7f9270194d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:06.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.629+0000 7f926ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92700ffc60 0x7f9270194d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44376/0 (socket says 172.21.15.16:44376) 2024-09-13T16:31:06.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.629+0000 7f926ffff640 1 -- 172.21.15.16:0/2279852216 learned_addr learned my addr 172.21.15.16:0/2279852216 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:06.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.629+0000 7f9274f9b640 1 --2- 172.21.15.16:0/2279852216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92700fea70 0x7f92701947e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:06.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.629+0000 7f927579c640 1 --2- 172.21.15.16:0/2279852216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9270100620 0x7f927019bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:06.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.629+0000 7f926ffff640 1 -- 172.21.15.16:0/2279852216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92700fea70 msgr2=0x7f92701947e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:06.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.629+0000 7f926ffff640 1 --2- 172.21.15.16:0/2279852216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92700fea70 0x7f92701947e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:06.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.629+0000 7f926ffff640 1 -- 172.21.15.16:0/2279852216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9270100620 msgr2=0x7f927019bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:06.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.629+0000 7f926ffff640 1 --2- 172.21.15.16:0/2279852216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9270100620 0x7f927019bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:06.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.629+0000 7f926ffff640 1 -- 172.21.15.16:0/2279852216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9268009900 con 0x7f92700ffc60 2024-09-13T16:31:06.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.630+0000 7f926ffff640 1 --2- 172.21.15.16:0/2279852216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92700ffc60 0x7f9270194d20 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f925c00c920 tx=0x7f925c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:06.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.630+0000 7f926dffb640 1 -- 172.21.15.16:0/2279852216 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c007c70 con 0x7f92700ffc60 2024-09-13T16:31:06.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.630+0000 7f926dffb640 1 -- 172.21.15.16:0/2279852216 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f925c007dd0 con 0x7f92700ffc60 2024-09-13T16:31:06.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.630+0000 7f9275f9d640 1 -- 172.21.15.16:0/2279852216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f927019c8e0 con 0x7f92700ffc60 2024-09-13T16:31:06.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.630+0000 7f926dffb640 1 -- 172.21.15.16:0/2279852216 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c018730 con 0x7f92700ffc60 2024-09-13T16:31:06.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.630+0000 7f9275f9d640 1 -- 172.21.15.16:0/2279852216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f927019ce40 con 0x7f92700ffc60 2024-09-13T16:31:06.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.631+0000 7f9275f9d640 1 -- 172.21.15.16:0/2279852216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9270105e10 con 0x7f92700ffc60 2024-09-13T16:31:06.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.632+0000 7f926dffb640 1 -- 172.21.15.16:0/2279852216 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f925c00f450 con 0x7f92700ffc60 2024-09-13T16:31:06.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.633+0000 7f926dffb640 1 --2- 172.21.15.16:0/2279852216 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f92600a11b0 0x7f92600a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:06.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.633+0000 7f9274f9b640 1 --2- 172.21.15.16:0/2279852216 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f92600a11b0 0x7f92600a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:06.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.633+0000 7f926dffb640 1 -- 172.21.15.16:0/2279852216 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f925c0ccd90 con 0x7f92700ffc60 2024-09-13T16:31:06.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.633+0000 7f9274f9b640 1 --2- 172.21.15.16:0/2279852216 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f92600a11b0 0x7f92600a3670 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f92700ffac0 tx=0x7f9264005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:06.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.637+0000 7f926dffb640 1 -- 172.21.15.16:0/2279852216 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f925c096170 con 0x7f92700ffc60 2024-09-13T16:31:06.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.834+0000 7f9275f9d640 1 -- 172.21.15.16:0/2279852216 --> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) v1 -- 0x7f92700608f0 con 0x7f92600a11b0 2024-09-13T16:31:06.845 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:06 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1060390976' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-13T16:31:06.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:06 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1060390976' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-13T16:31:06.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.854+0000 7f926dffb640 1 -- 172.21.15.16:0/2279852216 <== mgr.14150 v2:172.21.15.16:6800/3739009683 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3 (secure 0 0 0) 0x7f92700608f0 con 0x7f92600a11b0 2024-09-13T16:31:06.857 INFO:teuthology.orchestra.run.smithi016.stdout:{} 2024-09-13T16:31:06.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.856+0000 7f9275f9d640 1 -- 172.21.15.16:0/2279852216 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f92600a11b0 msgr2=0x7f92600a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:06.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.856+0000 7f9275f9d640 1 --2- 172.21.15.16:0/2279852216 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f92600a11b0 0x7f92600a3670 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f92700ffac0 tx=0x7f9264005f90 comp rx=0 tx=0).stop 2024-09-13T16:31:06.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.856+0000 7f9275f9d640 1 -- 172.21.15.16:0/2279852216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92700ffc60 msgr2=0x7f9270194d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:06.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.856+0000 7f9275f9d640 1 --2- 172.21.15.16:0/2279852216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92700ffc60 0x7f9270194d20 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f925c00c920 tx=0x7f925c00cde0 comp rx=0 tx=0).stop 2024-09-13T16:31:06.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.857+0000 7f9275f9d640 1 -- 172.21.15.16:0/2279852216 shutdown_connections 2024-09-13T16:31:06.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.857+0000 7f9275f9d640 1 --2- 172.21.15.16:0/2279852216 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f92600a11b0 0x7f92600a3670 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:06.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.857+0000 7f9275f9d640 1 --2- 172.21.15.16:0/2279852216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9270100620 0x7f927019bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:06.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.857+0000 7f9275f9d640 1 --2- 172.21.15.16:0/2279852216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92700ffc60 0x7f9270194d20 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:06.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.857+0000 7f9275f9d640 1 --2- 172.21.15.16:0/2279852216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92700fea70 0x7f92701947e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:06.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.857+0000 7f9275f9d640 1 -- 172.21.15.16:0/2279852216 >> 172.21.15.16:0/2279852216 conn(0x7f92700fa1f0 msgr2=0x7f9270102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:06.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.857+0000 7f9275f9d640 1 -- 172.21.15.16:0/2279852216 shutdown_connections 2024-09-13T16:31:06.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:31:06.857+0000 7f9275f9d640 1 -- 172.21.15.16:0/2279852216 wait complete. 2024-09-13T16:31:06.869 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-13T16:31:06.890 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-13T16:31:06.903 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-09-13T16:31:06.903 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-13T16:31:06.906 INFO:teuthology.run_tasks:Running task workunit... 2024-09-13T16:31:06.940 INFO:tasks.workunit:Pulling workunits from ref 50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T16:31:06.940 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-13T16:31:06.941 INFO:tasks.workunit:timeout=3h 2024-09-13T16:31:06.941 INFO:tasks.workunit:cleanup=True 2024-09-13T16:31:06.941 DEBUG:teuthology.orchestra.run.smithi016:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-13T16:31:06.942 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:06 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1060390976' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-13T16:31:06.960 INFO:teuthology.orchestra.run.smithi016.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-13T16:31:06.960 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-13T16:31:06.960 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 78h/120d Inode: 1 Links: 2 2024-09-13T16:31:06.960 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T16:31:06.960 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:fusefs_t:s0 2024-09-13T16:31:06.960 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-13 16:29:06.988562310 +0000 2024-09-13T16:31:06.960 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-13 16:29:06.988562310 +0000 2024-09-13T16:31:06.960 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-13 16:31:06.093150145 +0000 2024-09-13T16:31:06.961 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: - 2024-09-13T16:31:06.961 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-13T16:31:06.961 DEBUG:teuthology.orchestra.run.smithi016:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-13T16:31:07.035 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T16:31:07.092 INFO:tasks.workunit.client.0.smithi016.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-13T16:31:07.104 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:31:06 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:31:06.838+0000 7fa29c8c8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T16:31:07.105 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:31:06 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:31:06.838+0000 7fa29c8c8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T16:31:07.105 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:31:06 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:31:06.838+0000 7fa29c8c8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T16:31:07.105 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:31:06 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:31:06.838+0000 7fa29c8c8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T16:31:07.105 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:31:06 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:31:06.838+0000 7fa29c8c8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T16:31:07.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:07 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1060390976' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-13T16:31:07.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:07 smithi107 ceph-mon[34139]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T16:31:07.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:07 smithi107 ceph-mon[34139]: pgmap v354: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T16:31:07.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:07 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1060390976' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-13T16:31:07.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:07 smithi016 ceph-mon[31187]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T16:31:07.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:07 smithi016 ceph-mon[31187]: pgmap v354: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T16:31:07.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:07 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1060390976' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-13T16:31:07.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:07 smithi111 ceph-mon[34600]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T16:31:07.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:07 smithi111 ceph-mon[34600]: pgmap v354: 129 pgs: 129 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T16:31:08.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:08 smithi107 ceph-mon[34139]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:31:08.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:08 smithi016 ceph-mon[31187]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:31:08.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:08 smithi111 ceph-mon[34600]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T16:31:09.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:09 smithi107 ceph-mon[34139]: pgmap v355: 129 pgs: 129 active+clean; 649 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-13T16:31:09.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:09 smithi016 ceph-mon[31187]: pgmap v355: 129 pgs: 129 active+clean; 649 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-13T16:31:09.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:09 smithi111 ceph-mon[34600]: pgmap v355: 129 pgs: 129 active+clean; 649 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-13T16:31:10.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:10 smithi107 ceph-mon[34139]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-13T16:31:10.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:10 smithi016 ceph-mon[31187]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-13T16:31:10.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:10 smithi111 ceph-mon[34600]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-13T16:31:11.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:11 smithi107 ceph-mon[34139]: pgmap v356: 129 pgs: 129 active+clean; 649 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-13T16:31:11.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:11 smithi016 ceph-mon[31187]: pgmap v356: 129 pgs: 129 active+clean; 649 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-13T16:31:11.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:11 smithi111 ceph-mon[34600]: pgmap v356: 129 pgs: 129 active+clean; 649 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-13T16:31:12.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:12 smithi111 ceph-mon[34600]: pgmap v357: 129 pgs: 129 active+clean; 658 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-09-13T16:31:13.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:12 smithi107 ceph-mon[34139]: pgmap v357: 129 pgs: 129 active+clean; 658 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-09-13T16:31:13.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:12 smithi016 ceph-mon[31187]: pgmap v357: 129 pgs: 129 active+clean; 658 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-09-13T16:31:13.924 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:13 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:31:13.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:13 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:31:13.943 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:13 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:31:15.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:15 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:31:15.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:15 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:31:15.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:15 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:31:15.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:15 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:31:15.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:15 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:31:15.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:15 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:31:15.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:15 smithi107 ceph-mon[34139]: pgmap v358: 129 pgs: 129 active+clean; 658 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-13T16:31:15.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:15 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:31:15.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:15 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:31:15.850 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:15 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:31:15.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:15 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:31:15.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:15 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:31:15.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:15 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:31:15.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:15 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:31:15.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:15 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:31:15.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:15 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:31:15.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:15 smithi016 ceph-mon[31187]: pgmap v358: 129 pgs: 129 active+clean; 658 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-13T16:31:15.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:15 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:31:15.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:15 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:31:15.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:15 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:31:15.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:15 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:31:15.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:15 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:31:15.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:15 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:31:15.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:15 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:31:15.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:15 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:31:15.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:15 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:31:15.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:15 smithi111 ceph-mon[34600]: pgmap v358: 129 pgs: 129 active+clean; 658 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-13T16:31:15.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:15 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:31:15.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:15 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:31:15.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:15 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:31:16.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:16 smithi111 ceph-mon[34600]: pgmap v359: 129 pgs: 129 active+clean; 658 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-13T16:31:17.058 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:16 smithi107 ceph-mon[34139]: pgmap v359: 129 pgs: 129 active+clean; 658 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-13T16:31:17.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:16 smithi016 ceph-mon[31187]: pgmap v359: 129 pgs: 129 active+clean; 658 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-13T16:31:18.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:18 smithi111 ceph-mon[34600]: pgmap v360: 129 pgs: 129 active+clean; 668 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.3 KiB/s wr, 15 op/s 2024-09-13T16:31:19.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:18 smithi107 ceph-mon[34139]: pgmap v360: 129 pgs: 129 active+clean; 668 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.3 KiB/s wr, 15 op/s 2024-09-13T16:31:19.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:18 smithi016 ceph-mon[31187]: pgmap v360: 129 pgs: 129 active+clean; 668 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.3 KiB/s wr, 15 op/s 2024-09-13T16:31:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.045+0000 7f824a118640 1 -- 172.21.15.16:0/3198616777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8244107bc0 msgr2=0x7f824410a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.045+0000 7f824a118640 1 --2- 172.21.15.16:0/3198616777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8244107bc0 0x7f824410a050 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f823c009ca0 tx=0x7f823c02a950 comp rx=0 tx=0).stop 2024-09-13T16:31:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.045+0000 7f824a118640 1 -- 172.21.15.16:0/3198616777 shutdown_connections 2024-09-13T16:31:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.045+0000 7f824a118640 1 --2- 172.21.15.16:0/3198616777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8244107bc0 0x7f824410a050 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.045+0000 7f824a118640 1 --2- 172.21.15.16:0/3198616777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8244072e90 0x7f82440732f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.045+0000 7f824a118640 1 --2- 172.21.15.16:0/3198616777 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8244074470 0x7f82440728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.045+0000 7f824a118640 1 -- 172.21.15.16:0/3198616777 >> 172.21.15.16:0/3198616777 conn(0x7f82440fb4b0 msgr2=0x7f82440fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.045+0000 7f824a118640 1 -- 172.21.15.16:0/3198616777 shutdown_connections 2024-09-13T16:31:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.045+0000 7f824a118640 1 -- 172.21.15.16:0/3198616777 wait complete. 2024-09-13T16:31:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.046+0000 7f824a118640 1 Processor -- start 2024-09-13T16:31:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.046+0000 7f824a118640 1 -- start start 2024-09-13T16:31:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.047+0000 7f824a118640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8244072e90 0x7f8244197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.047+0000 7f824a118640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8244074470 0x7f8244198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.047+0000 7f824a118640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8244107bc0 0x7f824419f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.047+0000 7f824a118640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824419f8b0 con 0x7f8244072e90 2024-09-13T16:31:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.047+0000 7f824a118640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824419fa20 con 0x7f8244107bc0 2024-09-13T16:31:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.047+0000 7f824a118640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824419fbe0 con 0x7f8244074470 2024-09-13T16:31:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.047+0000 7f82437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8244072e90 0x7f8244197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.047+0000 7f82437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8244072e90 0x7f8244197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59064/0 (socket says 172.21.15.16:59064) 2024-09-13T16:31:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.047+0000 7f82437fe640 1 -- 172.21.15.16:0/1685880410 learned_addr learned my addr 172.21.15.16:0/1685880410 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.047+0000 7f8242ffd640 1 --2- 172.21.15.16:0/1685880410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8244074470 0x7f8244198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.047+0000 7f8243fff640 1 --2- 172.21.15.16:0/1685880410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8244107bc0 0x7f824419f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.047+0000 7f82437fe640 1 -- 172.21.15.16:0/1685880410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8244074470 msgr2=0x7f8244198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.047+0000 7f82437fe640 1 --2- 172.21.15.16:0/1685880410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8244074470 0x7f8244198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.047+0000 7f82437fe640 1 -- 172.21.15.16:0/1685880410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8244107bc0 msgr2=0x7f824419f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.047+0000 7f82437fe640 1 --2- 172.21.15.16:0/1685880410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8244107bc0 0x7f824419f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.047+0000 7f82437fe640 1 -- 172.21.15.16:0/1685880410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823c009900 con 0x7f8244072e90 2024-09-13T16:31:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.047+0000 7f8242ffd640 1 --2- 172.21.15.16:0/1685880410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8244074470 0x7f8244198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:31:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.047+0000 7f8243fff640 1 --2- 172.21.15.16:0/1685880410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8244107bc0 0x7f824419f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:31:20.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.047+0000 7f82437fe640 1 --2- 172.21.15.16:0/1685880410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8244072e90 0x7f8244197dc0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f823400d950 tx=0x7f823400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:20.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.048+0000 7f8240ff9640 1 -- 172.21.15.16:0/1685880410 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8234009c30 con 0x7f8244072e90 2024-09-13T16:31:20.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.048+0000 7f824a118640 1 -- 172.21.15.16:0/1685880410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f824419fec0 con 0x7f8244072e90 2024-09-13T16:31:20.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.048+0000 7f8240ff9640 1 -- 172.21.15.16:0/1685880410 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8234009d90 con 0x7f8244072e90 2024-09-13T16:31:20.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.048+0000 7f8240ff9640 1 -- 172.21.15.16:0/1685880410 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823400f620 con 0x7f8244072e90 2024-09-13T16:31:20.052 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.048+0000 7f824a118640 1 -- 172.21.15.16:0/1685880410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82441a02f0 con 0x7f8244072e90 2024-09-13T16:31:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.051+0000 7f8240ff9640 1 -- 172.21.15.16:0/1685880410 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f82340109f0 con 0x7f8244072e90 2024-09-13T16:31:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.051+0000 7f824ce3a740 1 Processor -- start 2024-09-13T16:31:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.051+0000 7f824ce3a740 1 -- start start 2024-09-13T16:31:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.051+0000 7f824ce3a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d807811d0 0x561d807815d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.051+0000 7f824ce3a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d80781ba0 0x561d807866c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.051+0000 7f824ce3a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d80786d30 0x561d807891c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.051+0000 7f824ce3a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d806ebc30 con 0x561d807811d0 2024-09-13T16:31:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.051+0000 7f824ce3a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d806e93c0 con 0x561d80786d30 2024-09-13T16:31:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.051+0000 7f824ce3a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d806e4cb0 con 0x561d80781ba0 2024-09-13T16:31:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.051+0000 7f8243fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d807811d0 0x561d807815d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.051+0000 7f8243fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d807811d0 0x561d807815d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59074/0 (socket says 172.21.15.16:59074) 2024-09-13T16:31:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.051+0000 7f8243fff640 1 -- 172.21.15.16:0/3129550045 learned_addr learned my addr 172.21.15.16:0/3129550045 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.051+0000 7f8242ffd640 1 --2- 172.21.15.16:0/3129550045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d80786d30 0x561d807891c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.052+0000 7f82437fe640 1 --2- 172.21.15.16:0/3129550045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d80781ba0 0x561d807866c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.052+0000 7f8240ff9640 1 --2- 172.21.15.16:0/1685880410 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f822c0a1120 0x7f822c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.052+0000 7f8243fff640 1 -- 172.21.15.16:0/3129550045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d80781ba0 msgr2=0x561d807866c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.052+0000 7f8243fff640 1 --2- 172.21.15.16:0/3129550045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d80781ba0 0x561d807866c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.052+0000 7f8243fff640 1 -- 172.21.15.16:0/3129550045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d80786d30 msgr2=0x561d807891c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.052+0000 7f8243fff640 1 --2- 172.21.15.16:0/3129550045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d80786d30 0x561d807891c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.052+0000 7f8243fff640 1 -- 172.21.15.16:0/3129550045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d8068b910 con 0x561d807811d0 2024-09-13T16:31:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.052+0000 7f8240ff9640 1 -- 172.21.15.16:0/1685880410 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f82340cca20 con 0x7f8244072e90 2024-09-13T16:31:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.052+0000 7f8243fff640 1 --2- 172.21.15.16:0/1685880410 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f822c0a1120 0x7f822c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.052+0000 7f8243fff640 1 --2- 172.21.15.16:0/3129550045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d807811d0 0x561d807815d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f823c009ca0 tx=0x7f823c009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ffd7c8535ff1407 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.052+0000 7f8229ffb640 1 -- 172.21.15.16:0/3129550045 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823c040070 con 0x561d807811d0 2024-09-13T16:31:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.052+0000 7f8229ffb640 1 -- 172.21.15.16:0/3129550045 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f823c003e00 con 0x561d807811d0 2024-09-13T16:31:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.052+0000 7f8229ffb640 1 -- 172.21.15.16:0/3129550045 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823c0042a0 con 0x561d807811d0 2024-09-13T16:31:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.053+0000 7f824ce3a740 1 -- 172.21.15.16:0/3129550045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d807811d0 msgr2=0x561d807815d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.053+0000 7f824ce3a740 1 --2- 172.21.15.16:0/3129550045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d807811d0 0x561d807815d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f823c009ca0 tx=0x7f823c009360 comp rx=0 tx=0).stop 2024-09-13T16:31:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.053+0000 7f8243fff640 1 --2- 172.21.15.16:0/1685880410 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f822c0a1120 0x7f822c0a35e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f823c047000 tx=0x7f823c0044e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.053+0000 7f824ce3a740 1 -- 172.21.15.16:0/3129550045 shutdown_connections 2024-09-13T16:31:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.053+0000 7f824ce3a740 1 --2- 172.21.15.16:0/3129550045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d80786d30 0x561d807891c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.053+0000 7f824ce3a740 1 --2- 172.21.15.16:0/3129550045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d80781ba0 0x561d807866c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.053+0000 7f824ce3a740 1 --2- 172.21.15.16:0/3129550045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d807811d0 0x561d807815d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.053+0000 7f824ce3a740 1 -- 172.21.15.16:0/3129550045 >> 172.21.15.16:0/3129550045 conn(0x561d8077c8c0 msgr2=0x561d8077ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.053+0000 7f824ce3a740 1 -- 172.21.15.16:0/3129550045 shutdown_connections 2024-09-13T16:31:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.053+0000 7f824ce3a740 1 -- 172.21.15.16:0/3129550045 wait complete. 2024-09-13T16:31:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.053+0000 7f824ce3a740 1 -- start start 2024-09-13T16:31:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.054+0000 7f824ce3a740 1 Processor -- start 2024-09-13T16:31:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.054+0000 7f824ce3a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d807811d0 0x561d8078aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.054+0000 7f824ce3a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d80781ba0 0x561d8078b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.054+0000 7f824ce3a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d80786d30 0x561d80792430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.054+0000 7f824ce3a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d806a2e80 con 0x561d80781ba0 2024-09-13T16:31:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.054+0000 7f824ce3a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d806a2b20 con 0x561d80786d30 2024-09-13T16:31:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.054+0000 7f824ce3a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d80668260 con 0x561d807811d0 2024-09-13T16:31:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.054+0000 7f824ce3a740 10 client.0 _ll_register_callbacks cb 0x561d8069ddf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:31:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.054+0000 7f8242ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d80781ba0 0x561d8078b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.054+0000 7f8242ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d80781ba0 0x561d8078b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59076/0 (socket says 172.21.15.16:59076) 2024-09-13T16:31:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.054+0000 7f8242ffd640 1 -- 172.21.15.16:0/2633926046 learned_addr learned my addr 172.21.15.16:0/2633926046 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.054+0000 7f82437fe640 1 --2- 172.21.15.16:0/2633926046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d807811d0 0x561d8078aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.054+0000 7f8243fff640 1 --2- 172.21.15.16:0/2633926046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d80786d30 0x561d80792430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.054+0000 7f8242ffd640 1 -- 172.21.15.16:0/2633926046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d807811d0 msgr2=0x561d8078aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.054+0000 7f8242ffd640 1 --2- 172.21.15.16:0/2633926046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d807811d0 0x561d8078aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.054+0000 7f8242ffd640 1 -- 172.21.15.16:0/2633926046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d80786d30 msgr2=0x561d80792430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.054+0000 7f8242ffd640 1 --2- 172.21.15.16:0/2633926046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d80786d30 0x561d80792430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.054+0000 7f8242ffd640 1 -- 172.21.15.16:0/2633926046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f824410ac30 con 0x561d80781ba0 2024-09-13T16:31:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.054+0000 7f8243fff640 1 --2- 172.21.15.16:0/2633926046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d80786d30 0x561d80792430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:31:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.055+0000 7f8242ffd640 1 --2- 172.21.15.16:0/2633926046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d80781ba0 0x561d8078b3f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8238011740 tx=0x7f8238011c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=898cd0f851a5a852 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.055+0000 7f8222ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:31:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.055+0000 7f8222ffd640 1 -- 172.21.15.16:0/2633926046 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8238016070 con 0x561d80781ba0 2024-09-13T16:31:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.055+0000 7f8222ffd640 1 -- 172.21.15.16:0/2633926046 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f8238010ba0 con 0x561d80781ba0 2024-09-13T16:31:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.055+0000 7f824ce3a740 1 -- 172.21.15.16:0/2633926046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561d8070d680 con 0x561d80781ba0 2024-09-13T16:31:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.055+0000 7f8222ffd640 1 -- 172.21.15.16:0/2633926046 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8238010d60 con 0x561d80781ba0 2024-09-13T16:31:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.055+0000 7f824ce3a740 10 client.15540 fetch_fsmap learned FSMap version 29 2024-09-13T16:31:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.055+0000 7f824ce3a740 1 -- 172.21.15.16:0/2633926046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561d806e3330 con 0x561d80781ba0 2024-09-13T16:31:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.055+0000 7f8222ffd640 1 -- 172.21.15.16:0/2633926046 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561d8070d680 con 0x561d80781ba0 2024-09-13T16:31:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.055+0000 7f8222ffd640 1 -- 172.21.15.16:0/2633926046 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f823800f3c0 con 0x561d80781ba0 2024-09-13T16:31:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.055+0000 7f824ce3a740 10 client.15540 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T16:31:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.055+0000 7f824ce3a740 10 client.15540 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T16:31:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.056+0000 7f824ce3a740 20 client.15540 populate_metadata read hostname 'smithi016' 2024-09-13T16:31:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.056+0000 7f824ce3a740 1 --2- 172.21.15.16:0/2633926046 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x561d80796b90 0x561d80798fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.056+0000 7f824ce3a740 4 client.15540 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T16:31:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.056+0000 7f824ce3a740 1 -- 172.21.15.16:0/2633926046 --> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d806d85d0 con 0x561d80796b90 2024-09-13T16:31:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.056+0000 7f82437fe640 1 --2- 172.21.15.16:0/2633926046 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x561d80796b90 0x561d80798fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.056+0000 7f82437fe640 1 --2- 172.21.15.16:0/2633926046 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x561d80796b90 0x561d80798fb0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63c4b99a81daeeb server_cookie=ebf43830ad5cd3af in_seq=0 out_seq=0 2024-09-13T16:31:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.056+0000 7f8222ffd640 10 client.15540 ms_handle_connect on v2:172.21.15.107:6838/1914689806 2024-09-13T16:31:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.057+0000 7f8222ffd640 1 -- 172.21.15.16:0/2633926046 <== mds.0 v2:172.21.15.107:6838/1914689806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561d806d85d0 con 0x561d80796b90 2024-09-13T16:31:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.057+0000 7f8222ffd640 10 client.15540 handle_command_reply: tid=0 2024-09-13T16:31:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.057+0000 7f824ce3a740 1 client.15540 shutdown 2024-09-13T16:31:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.057+0000 7f824ce3a740 1 -- 172.21.15.16:0/2633926046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d80781ba0 msgr2=0x561d8078b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.057+0000 7f824ce3a740 1 --2- 172.21.15.16:0/2633926046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d80781ba0 0x561d8078b3f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8238011740 tx=0x7f8238011c00 comp rx=0 tx=0).stop 2024-09-13T16:31:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.057+0000 7f824ce3a740 1 -- 172.21.15.16:0/2633926046 shutdown_connections 2024-09-13T16:31:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.057+0000 7f824ce3a740 1 --2- 172.21.15.16:0/2633926046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d80786d30 0x561d80792430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.057+0000 7f824ce3a740 1 --2- 172.21.15.16:0/2633926046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d80781ba0 0x561d8078b3f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.057+0000 7f824ce3a740 1 --2- 172.21.15.16:0/2633926046 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x561d80796b90 0x561d80798fb0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.057+0000 7f824ce3a740 1 --2- 172.21.15.16:0/2633926046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d807811d0 0x561d8078aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.057+0000 7f8222ffd640 0 client.15540 ms_handle_reset on v2:172.21.15.107:6838/1914689806 2024-09-13T16:31:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.057+0000 7f824ce3a740 1 -- 172.21.15.16:0/2633926046 >> 172.21.15.16:0/2633926046 conn(0x561d8077c8c0 msgr2=0x561d8077bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.057+0000 7f824ce3a740 1 -- 172.21.15.16:0/2633926046 shutdown_connections 2024-09-13T16:31:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.057+0000 7f824ce3a740 1 -- 172.21.15.16:0/2633926046 wait complete. 2024-09-13T16:31:20.066 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.057+0000 7f824ce3a740 20 client.15540 trim_cache size 0 max 16384 2024-09-13T16:31:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.070+0000 7f824ce3a740 1 Processor -- start 2024-09-13T16:31:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.070+0000 7f824ce3a740 1 -- start start 2024-09-13T16:31:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.070+0000 7f824ce3a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d807811d0 0x561d8077c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.070+0000 7f824ce3a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d80781ba0 0x561d807923f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.070+0000 7f824ce3a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d80786d30 0x561d80782b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.070+0000 7f824ce3a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d8078aac0 con 0x561d80781ba0 2024-09-13T16:31:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.070+0000 7f824ce3a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d80780d50 con 0x561d807811d0 2024-09-13T16:31:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.070+0000 7f824ce3a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d80789f00 con 0x561d80786d30 2024-09-13T16:31:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.070+0000 7f8242ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d80781ba0 0x561d807923f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.070+0000 7f8242ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d80781ba0 0x561d807923f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59086/0 (socket says 172.21.15.16:59086) 2024-09-13T16:31:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.070+0000 7f8242ffd640 1 -- 172.21.15.16:0/722590998 learned_addr learned my addr 172.21.15.16:0/722590998 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.071+0000 7f8242ffd640 1 -- 172.21.15.16:0/722590998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d80786d30 msgr2=0x561d80782b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.071+0000 7f8242ffd640 1 --2- 172.21.15.16:0/722590998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d80786d30 0x561d80782b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.071+0000 7f8242ffd640 1 -- 172.21.15.16:0/722590998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d807811d0 msgr2=0x561d8077c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.071+0000 7f8242ffd640 1 --2- 172.21.15.16:0/722590998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d807811d0 0x561d8077c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.071+0000 7f8242ffd640 1 -- 172.21.15.16:0/722590998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d80685300 con 0x561d80781ba0 2024-09-13T16:31:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.071+0000 7f8242ffd640 1 --2- 172.21.15.16:0/722590998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d80781ba0 0x561d807923f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f823800b620 tx=0x7f823800bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5fc9ae2be49646c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.071+0000 7f822a7fc640 1 -- 172.21.15.16:0/722590998 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8238016070 con 0x561d80781ba0 2024-09-13T16:31:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.071+0000 7f822a7fc640 1 -- 172.21.15.16:0/722590998 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8238005bb0 con 0x561d80781ba0 2024-09-13T16:31:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.071+0000 7f822a7fc640 1 -- 172.21.15.16:0/722590998 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82380042a0 con 0x561d80781ba0 2024-09-13T16:31:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.071+0000 7f824ce3a740 1 -- 172.21.15.16:0/722590998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d80781ba0 msgr2=0x561d807923f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.071+0000 7f824ce3a740 1 --2- 172.21.15.16:0/722590998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d80781ba0 0x561d807923f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f823800b620 tx=0x7f823800bef0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.071+0000 7f824ce3a740 1 -- 172.21.15.16:0/722590998 shutdown_connections 2024-09-13T16:31:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.071+0000 7f824ce3a740 1 --2- 172.21.15.16:0/722590998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d80786d30 0x561d80782b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.071+0000 7f824ce3a740 1 --2- 172.21.15.16:0/722590998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d80781ba0 0x561d807923f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.071+0000 7f824ce3a740 1 --2- 172.21.15.16:0/722590998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d807811d0 0x561d8077c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.071+0000 7f824ce3a740 1 -- 172.21.15.16:0/722590998 >> 172.21.15.16:0/722590998 conn(0x561d8077c8c0 msgr2=0x561d80777a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.071+0000 7f824ce3a740 1 -- 172.21.15.16:0/722590998 shutdown_connections 2024-09-13T16:31:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.072+0000 7f824ce3a740 1 -- 172.21.15.16:0/722590998 wait complete. 2024-09-13T16:31:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.072+0000 7f824ce3a740 1 -- start start 2024-09-13T16:31:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.072+0000 7f824ce3a740 1 Processor -- start 2024-09-13T16:31:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.072+0000 7f824ce3a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d807811d0 0x561d80792130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.072+0000 7f824ce3a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d80781ba0 0x561d80790830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.072+0000 7f824ce3a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d80786d30 0x561d80790d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.072+0000 7f824ce3a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d807912b0 con 0x561d80781ba0 2024-09-13T16:31:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.072+0000 7f824ce3a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d807913f0 con 0x561d807811d0 2024-09-13T16:31:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.072+0000 7f824ce3a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d807915b0 con 0x561d80786d30 2024-09-13T16:31:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.072+0000 7f824ce3a740 10 client.0 _ll_register_callbacks cb 0x7f823c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:31:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.072+0000 7f8242ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d80781ba0 0x561d80790830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.072+0000 7f8242ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d80781ba0 0x561d80790830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59090/0 (socket says 172.21.15.16:59090) 2024-09-13T16:31:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.072+0000 7f8242ffd640 1 -- 172.21.15.16:0/2894399871 learned_addr learned my addr 172.21.15.16:0/2894399871 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.073+0000 7f82437fe640 1 --2- 172.21.15.16:0/2894399871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d807811d0 0x561d80792130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.073+0000 7f8243fff640 1 --2- 172.21.15.16:0/2894399871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d80786d30 0x561d80790d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.073+0000 7f8242ffd640 1 -- 172.21.15.16:0/2894399871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d80786d30 msgr2=0x561d80790d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.073+0000 7f8242ffd640 1 --2- 172.21.15.16:0/2894399871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d80786d30 0x561d80790d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.073+0000 7f8242ffd640 1 -- 172.21.15.16:0/2894399871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d807811d0 msgr2=0x561d80792130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.073+0000 7f8242ffd640 1 --2- 172.21.15.16:0/2894399871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d807811d0 0x561d80792130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.073+0000 7f8242ffd640 1 -- 172.21.15.16:0/2894399871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d806e3330 con 0x561d80781ba0 2024-09-13T16:31:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.073+0000 7f8243fff640 1 --2- 172.21.15.16:0/2894399871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d80786d30 0x561d80790d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:31:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.073+0000 7f8242ffd640 1 --2- 172.21.15.16:0/2894399871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d80781ba0 0x561d80790830 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f823800fc50 tx=0x7f8238011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f30624ec66c4e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.073+0000 7f8222ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:31:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.073+0000 7f8222ffd640 1 -- 172.21.15.16:0/2894399871 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8238016040 con 0x561d80781ba0 2024-09-13T16:31:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.073+0000 7f8222ffd640 1 -- 172.21.15.16:0/2894399871 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f8238010da0 con 0x561d80781ba0 2024-09-13T16:31:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.073+0000 7f824ce3a740 1 -- 172.21.15.16:0/2894399871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f823c003e00 con 0x561d80781ba0 2024-09-13T16:31:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.073+0000 7f8222ffd640 1 -- 172.21.15.16:0/2894399871 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8238004020 con 0x561d80781ba0 2024-09-13T16:31:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.073+0000 7f824ce3a740 10 client.15546 fetch_fsmap learned FSMap version 29 2024-09-13T16:31:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.073+0000 7f824ce3a740 1 -- 172.21.15.16:0/2894399871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561d806aa8e0 con 0x561d80781ba0 2024-09-13T16:31:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.074+0000 7f8222ffd640 1 -- 172.21.15.16:0/2894399871 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f823c003e00 con 0x561d80781ba0 2024-09-13T16:31:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.074+0000 7f8222ffd640 1 -- 172.21.15.16:0/2894399871 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8238004660 con 0x561d80781ba0 2024-09-13T16:31:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.074+0000 7f824ce3a740 10 client.15546 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T16:31:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.074+0000 7f824ce3a740 10 client.15546 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T16:31:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.074+0000 7f824ce3a740 20 client.15546 populate_metadata read hostname 'smithi016' 2024-09-13T16:31:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.074+0000 7f824ce3a740 1 --2- 172.21.15.16:0/2894399871 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x561d807ee370 0x561d807f0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.074+0000 7f824ce3a740 4 client.15546 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T16:31:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.074+0000 7f824ce3a740 1 -- 172.21.15.16:0/2894399871 --> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561d803d8300 con 0x561d807ee370 2024-09-13T16:31:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.074+0000 7f82437fe640 1 --2- 172.21.15.16:0/2894399871 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x561d807ee370 0x561d807f0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.074+0000 7f8222ffd640 10 client.15546 ms_handle_connect on v2:172.21.15.107:6838/1914689806 2024-09-13T16:31:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.074+0000 7f82437fe640 1 --2- 172.21.15.16:0/2894399871 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x561d807ee370 0x561d807f0760 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d675ba4ebc61b44f server_cookie=f8abadb0bb859960 in_seq=0 out_seq=0 2024-09-13T16:31:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.087+0000 7f8222ffd640 1 -- 172.21.15.16:0/2894399871 <== mds.0 v2:172.21.15.107:6838/1914689806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561d803d8300 con 0x561d807ee370 2024-09-13T16:31:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.087+0000 7f8222ffd640 10 client.15546 handle_command_reply: tid=0 2024-09-13T16:31:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.087+0000 7f824ce3a740 1 client.15546 shutdown 2024-09-13T16:31:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.087+0000 7f824ce3a740 1 -- 172.21.15.16:0/2894399871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d80781ba0 msgr2=0x561d80790830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.087+0000 7f824ce3a740 1 --2- 172.21.15.16:0/2894399871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d80781ba0 0x561d80790830 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f823800fc50 tx=0x7f8238011440 comp rx=0 tx=0).stop 2024-09-13T16:31:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.087+0000 7f824ce3a740 1 -- 172.21.15.16:0/2894399871 shutdown_connections 2024-09-13T16:31:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.087+0000 7f824ce3a740 1 --2- 172.21.15.16:0/2894399871 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x561d807ee370 0x561d807f0760 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.087+0000 7f824ce3a740 1 --2- 172.21.15.16:0/2894399871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d80786d30 0x561d80790d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.087+0000 7f824ce3a740 1 --2- 172.21.15.16:0/2894399871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d80781ba0 0x561d80790830 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.087+0000 7f824ce3a740 1 --2- 172.21.15.16:0/2894399871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561d807811d0 0x561d80792130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.087+0000 7f8222ffd640 0 client.15546 ms_handle_reset on v2:172.21.15.107:6838/1914689806 2024-09-13T16:31:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.087+0000 7f824ce3a740 1 -- 172.21.15.16:0/2894399871 >> 172.21.15.16:0/2894399871 conn(0x561d8077c8c0 msgr2=0x561d80782b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.088+0000 7f824ce3a740 1 -- 172.21.15.16:0/2894399871 shutdown_connections 2024-09-13T16:31:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.088+0000 7f824ce3a740 1 -- 172.21.15.16:0/2894399871 wait complete. 2024-09-13T16:31:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.088+0000 7f824ce3a740 20 client.15546 trim_cache size 0 max 16384 2024-09-13T16:31:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-13T16:31:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-13T16:31:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-13T16:31:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-13T16:31:20.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.088+0000 7f824a118640 1 -- 172.21.15.16:0/1685880410 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f822c0a1120 msgr2=0x7f822c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:20.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.088+0000 7f824a118640 1 --2- 172.21.15.16:0/1685880410 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f822c0a1120 0x7f822c0a35e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f823c047000 tx=0x7f823c0044e0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.088+0000 7f824a118640 1 -- 172.21.15.16:0/1685880410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8244072e90 msgr2=0x7f8244197dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:20.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.088+0000 7f824a118640 1 --2- 172.21.15.16:0/1685880410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8244072e90 0x7f8244197dc0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f823400d950 tx=0x7f823400de10 comp rx=0 tx=0).stop 2024-09-13T16:31:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.088+0000 7f824a118640 1 -- 172.21.15.16:0/1685880410 shutdown_connections 2024-09-13T16:31:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.088+0000 7f824a118640 1 --2- 172.21.15.16:0/1685880410 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f822c0a1120 0x7f822c0a35e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.088+0000 7f824a118640 1 --2- 172.21.15.16:0/1685880410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8244107bc0 0x7f824419f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.088+0000 7f824a118640 1 --2- 172.21.15.16:0/1685880410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8244074470 0x7f8244198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.088+0000 7f824a118640 1 --2- 172.21.15.16:0/1685880410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8244072e90 0x7f8244197dc0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.088+0000 7f824a118640 1 -- 172.21.15.16:0/1685880410 >> 172.21.15.16:0/1685880410 conn(0x7f82440fb4b0 msgr2=0x7f82440fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.088+0000 7f824a118640 1 -- 172.21.15.16:0/1685880410 shutdown_connections 2024-09-13T16:31:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:20.088+0000 7f824a118640 1 -- 172.21.15.16:0/1685880410 wait complete. 2024-09-13T16:31:20.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:20 smithi111 ceph-mon[34600]: pgmap v361: 129 pgs: 129 active+clean; 668 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-09-13T16:31:21.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:20 smithi107 ceph-mon[34139]: pgmap v361: 129 pgs: 129 active+clean; 668 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-09-13T16:31:21.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:20 smithi016 ceph-mon[31187]: pgmap v361: 129 pgs: 129 active+clean; 668 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-09-13T16:31:22.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:22 smithi111 ceph-mon[34600]: pgmap v362: 129 pgs: 129 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.7 KiB/s wr, 15 op/s 2024-09-13T16:31:23.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:22 smithi107 ceph-mon[34139]: pgmap v362: 129 pgs: 129 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.7 KiB/s wr, 15 op/s 2024-09-13T16:31:23.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:22 smithi016 ceph-mon[31187]: pgmap v362: 129 pgs: 129 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.7 KiB/s wr, 15 op/s 2024-09-13T16:31:24.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:24 smithi111 ceph-mon[34600]: pgmap v363: 129 pgs: 129 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-09-13T16:31:25.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:24 smithi107 ceph-mon[34139]: pgmap v363: 129 pgs: 129 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-09-13T16:31:25.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:24 smithi016 ceph-mon[31187]: pgmap v363: 129 pgs: 129 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-09-13T16:31:26.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:26 smithi107 ceph-mon[34139]: pgmap v364: 129 pgs: 129 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.6 KiB/s wr, 13 op/s 2024-09-13T16:31:26.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:26 smithi111 ceph-mon[34600]: pgmap v364: 129 pgs: 129 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.6 KiB/s wr, 13 op/s 2024-09-13T16:31:27.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:26 smithi016 ceph-mon[31187]: pgmap v364: 129 pgs: 129 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.6 KiB/s wr, 13 op/s 2024-09-13T16:31:28.925 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:28 smithi016 ceph-mon[31187]: pgmap v365: 129 pgs: 129 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.6 KiB/s wr, 16 op/s 2024-09-13T16:31:28.936 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:28 smithi111 ceph-mon[34600]: pgmap v365: 129 pgs: 129 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.6 KiB/s wr, 16 op/s 2024-09-13T16:31:29.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:28 smithi107 ceph-mon[34139]: pgmap v365: 129 pgs: 129 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.6 KiB/s wr, 16 op/s 2024-09-13T16:31:30.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:30 smithi111 ceph-mon[34600]: pgmap v366: 129 pgs: 129 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-09-13T16:31:31.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:30 smithi107 ceph-mon[34139]: pgmap v366: 129 pgs: 129 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-09-13T16:31:31.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:30 smithi016 ceph-mon[31187]: pgmap v366: 129 pgs: 129 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-09-13T16:31:32.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:32 smithi111 ceph-mon[34600]: pgmap v367: 129 pgs: 129 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 15 op/s 2024-09-13T16:31:33.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:32 smithi107 ceph-mon[34139]: pgmap v367: 129 pgs: 129 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 15 op/s 2024-09-13T16:31:33.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:32 smithi016 ceph-mon[31187]: pgmap v367: 129 pgs: 129 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 15 op/s 2024-09-13T16:31:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.177+0000 7f16077e2640 1 -- 172.21.15.16:0/274359906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1600101350 msgr2=0x7f1600101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.177+0000 7f16077e2640 1 --2- 172.21.15.16:0/274359906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1600101350 0x7f1600101750 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009900 tx=0x7f15fc02a410 comp rx=0 tx=0).stop 2024-09-13T16:31:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.177+0000 7f16077e2640 1 -- 172.21.15.16:0/274359906 shutdown_connections 2024-09-13T16:31:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.177+0000 7f16077e2640 1 --2- 172.21.15.16:0/274359906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16000fe780 0x7f1600108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.177+0000 7f16077e2640 1 --2- 172.21.15.16:0/274359906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16000fddc0 0x7f16000fe240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.177+0000 7f16077e2640 1 --2- 172.21.15.16:0/274359906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1600101350 0x7f1600101750 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.177+0000 7f16077e2640 1 -- 172.21.15.16:0/274359906 >> 172.21.15.16:0/274359906 conn(0x7f16000f9b50 msgr2=0x7f16000fbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.177+0000 7f16077e2640 1 -- 172.21.15.16:0/274359906 shutdown_connections 2024-09-13T16:31:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.177+0000 7f16077e2640 1 -- 172.21.15.16:0/274359906 wait complete. 2024-09-13T16:31:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.178+0000 7f16077e2640 1 Processor -- start 2024-09-13T16:31:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.178+0000 7f16077e2640 1 -- start start 2024-09-13T16:31:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.178+0000 7f16077e2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16000fddc0 0x7f1600100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.178+0000 7f16077e2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16000fe780 0x7f16000ff090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.178+0000 7f16077e2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1600101350 0x7f16000ff5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.178+0000 7f16077e2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16000ffd60 con 0x7f16000fddc0 2024-09-13T16:31:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.178+0000 7f16077e2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16000ffed0 con 0x7f1600101350 2024-09-13T16:31:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.178+0000 7f16077e2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160019fe30 con 0x7f16000fe780 2024-09-13T16:31:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.178+0000 7f160557e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16000fddc0 0x7f1600100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.178+0000 7f160557e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16000fddc0 0x7f1600100a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54970/0 (socket says 172.21.15.16:54970) 2024-09-13T16:31:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.178+0000 7f160557e640 1 -- 172.21.15.16:0/237829430 learned_addr learned my addr 172.21.15.16:0/237829430 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.179+0000 7f1605d7f640 1 --2- 172.21.15.16:0/237829430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1600101350 0x7f16000ff5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.179+0000 7f1604d7d640 1 --2- 172.21.15.16:0/237829430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16000fe780 0x7f16000ff090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.179+0000 7f160557e640 1 -- 172.21.15.16:0/237829430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16000fe780 msgr2=0x7f16000ff090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.179+0000 7f160557e640 1 --2- 172.21.15.16:0/237829430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16000fe780 0x7f16000ff090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.179+0000 7f160557e640 1 -- 172.21.15.16:0/237829430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1600101350 msgr2=0x7f16000ff5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.179+0000 7f160557e640 1 --2- 172.21.15.16:0/237829430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1600101350 0x7f16000ff5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.179+0000 7f160557e640 1 -- 172.21.15.16:0/237829430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15fc009560 con 0x7f16000fddc0 2024-09-13T16:31:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.179+0000 7f1605d7f640 1 --2- 172.21.15.16:0/237829430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1600101350 0x7f16000ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:31:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.179+0000 7f160557e640 1 --2- 172.21.15.16:0/237829430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16000fddc0 0x7f1600100a00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f15fc0055e0 tx=0x7f15fc0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.179+0000 7f15f67fc640 1 -- 172.21.15.16:0/237829430 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15fc0035f0 con 0x7f16000fddc0 2024-09-13T16:31:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.179+0000 7f15f67fc640 1 -- 172.21.15.16:0/237829430 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f15fc003be0 con 0x7f16000fddc0 2024-09-13T16:31:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.179+0000 7f16077e2640 1 -- 172.21.15.16:0/237829430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16001a0030 con 0x7f16000fddc0 2024-09-13T16:31:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.179+0000 7f15f67fc640 1 -- 172.21.15.16:0/237829430 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15fc03cba0 con 0x7f16000fddc0 2024-09-13T16:31:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.179+0000 7f16077e2640 1 -- 172.21.15.16:0/237829430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16001a0560 con 0x7f16000fddc0 2024-09-13T16:31:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.182+0000 7f15f67fc640 1 -- 172.21.15.16:0/237829430 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f15fc003750 con 0x7f16000fddc0 2024-09-13T16:31:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.182+0000 7f15f67fc640 1 --2- 172.21.15.16:0/237829430 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f15ec0a1120 0x7f15ec0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.182+0000 7f1604d7d640 1 --2- 172.21.15.16:0/237829430 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f15ec0a1120 0x7f15ec0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.182+0000 7f15f67fc640 1 -- 172.21.15.16:0/237829430 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f15fc0ee120 con 0x7f16000fddc0 2024-09-13T16:31:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.183+0000 7f1604d7d640 1 --2- 172.21.15.16:0/237829430 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f15ec0a1120 0x7f15ec0a35e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1600100770 tx=0x7f15f0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.183+0000 7f160a5dd740 1 Processor -- start 2024-09-13T16:31:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.183+0000 7f160a5dd740 1 -- start start 2024-09-13T16:31:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.183+0000 7f160a5dd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7bd92a1d0 0x55f7bd92a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.183+0000 7f160a5dd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7bd92aba0 0x55f7bd92f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.183+0000 7f160a5dd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7bd92fd30 0x55f7bd9321c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.183+0000 7f160a5dd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7bd894c30 con 0x55f7bd92aba0 2024-09-13T16:31:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.183+0000 7f160a5dd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7bd8923c0 con 0x55f7bd92a1d0 2024-09-13T16:31:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.183+0000 7f160a5dd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7bd88dcb0 con 0x55f7bd92fd30 2024-09-13T16:31:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.183+0000 7f1604d7d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7bd92aba0 0x55f7bd92f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.183+0000 7f1604d7d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7bd92aba0 0x55f7bd92f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54980/0 (socket says 172.21.15.16:54980) 2024-09-13T16:31:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.183+0000 7f1604d7d640 1 -- 172.21.15.16:0/998413087 learned_addr learned my addr 172.21.15.16:0/998413087 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.184+0000 7f1605d7f640 1 --2- 172.21.15.16:0/998413087 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7bd92fd30 0x55f7bd9321c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.184+0000 7f160557e640 1 --2- 172.21.15.16:0/998413087 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7bd92a1d0 0x55f7bd92a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.184+0000 7f1604d7d640 1 -- 172.21.15.16:0/998413087 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7bd92fd30 msgr2=0x55f7bd9321c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.184+0000 7f1604d7d640 1 --2- 172.21.15.16:0/998413087 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7bd92fd30 0x55f7bd9321c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.184+0000 7f1604d7d640 1 -- 172.21.15.16:0/998413087 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7bd92a1d0 msgr2=0x55f7bd92a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.184+0000 7f1604d7d640 1 --2- 172.21.15.16:0/998413087 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7bd92a1d0 0x55f7bd92a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.184+0000 7f1604d7d640 1 -- 172.21.15.16:0/998413087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7bd834910 con 0x55f7bd92aba0 2024-09-13T16:31:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.184+0000 7f1605d7f640 1 --2- 172.21.15.16:0/998413087 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7bd92fd30 0x55f7bd9321c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:31:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.184+0000 7f1604d7d640 1 --2- 172.21.15.16:0/998413087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7bd92aba0 0x55f7bd92f6c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f15f000d810 tx=0x7f15f0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2077ba12a6fd2ee server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.184+0000 7f15eb7fe640 1 -- 172.21.15.16:0/998413087 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f00149b0 con 0x55f7bd92aba0 2024-09-13T16:31:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.184+0000 7f15eb7fe640 1 -- 172.21.15.16:0/998413087 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f15f001f400 con 0x55f7bd92aba0 2024-09-13T16:31:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.184+0000 7f15eb7fe640 1 -- 172.21.15.16:0/998413087 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f00125e0 con 0x55f7bd92aba0 2024-09-13T16:31:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.185+0000 7f160a5dd740 1 -- 172.21.15.16:0/998413087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7bd92aba0 msgr2=0x55f7bd92f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.185+0000 7f160a5dd740 1 --2- 172.21.15.16:0/998413087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7bd92aba0 0x55f7bd92f6c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f15f000d810 tx=0x7f15f0010000 comp rx=0 tx=0).stop 2024-09-13T16:31:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.185+0000 7f160a5dd740 1 -- 172.21.15.16:0/998413087 shutdown_connections 2024-09-13T16:31:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.185+0000 7f160a5dd740 1 --2- 172.21.15.16:0/998413087 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7bd92fd30 0x55f7bd9321c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.185+0000 7f160a5dd740 1 --2- 172.21.15.16:0/998413087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7bd92aba0 0x55f7bd92f6c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.185+0000 7f160a5dd740 1 --2- 172.21.15.16:0/998413087 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7bd92a1d0 0x55f7bd92a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.185+0000 7f160a5dd740 1 -- 172.21.15.16:0/998413087 >> 172.21.15.16:0/998413087 conn(0x55f7bd9258c0 msgr2=0x55f7bd925cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.185+0000 7f160a5dd740 1 -- 172.21.15.16:0/998413087 shutdown_connections 2024-09-13T16:31:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.185+0000 7f160a5dd740 1 -- 172.21.15.16:0/998413087 wait complete. 2024-09-13T16:31:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.185+0000 7f160a5dd740 1 -- start start 2024-09-13T16:31:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.185+0000 7f160a5dd740 1 Processor -- start 2024-09-13T16:31:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.186+0000 7f160a5dd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7bd92a1d0 0x55f7bd933ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.186+0000 7f160a5dd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7bd92aba0 0x55f7bd934420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.186+0000 7f160a5dd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7bd92fd30 0x55f7bd93b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.186+0000 7f160a5dd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7bd84be80 con 0x55f7bd92fd30 2024-09-13T16:31:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.186+0000 7f160a5dd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7bd84bb20 con 0x55f7bd92a1d0 2024-09-13T16:31:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.186+0000 7f160a5dd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7bd811260 con 0x55f7bd92aba0 2024-09-13T16:31:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.186+0000 7f160a5dd740 10 client.0 _ll_register_callbacks cb 0x55f7bd846df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:31:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.186+0000 7f1605d7f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7bd92fd30 0x55f7bd93b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.186+0000 7f1605d7f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7bd92fd30 0x55f7bd93b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54988/0 (socket says 172.21.15.16:54988) 2024-09-13T16:31:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.186+0000 7f1605d7f640 1 -- 172.21.15.16:0/4288815799 learned_addr learned my addr 172.21.15.16:0/4288815799 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.186+0000 7f160557e640 1 --2- 172.21.15.16:0/4288815799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7bd92a1d0 0x55f7bd933ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.186+0000 7f1604d7d640 1 --2- 172.21.15.16:0/4288815799 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7bd92aba0 0x55f7bd934420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.186+0000 7f1605d7f640 1 -- 172.21.15.16:0/4288815799 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7bd92aba0 msgr2=0x55f7bd934420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.186+0000 7f1605d7f640 1 --2- 172.21.15.16:0/4288815799 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7bd92aba0 0x55f7bd934420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.186+0000 7f1605d7f640 1 -- 172.21.15.16:0/4288815799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7bd92a1d0 msgr2=0x55f7bd933ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.186+0000 7f1605d7f640 1 --2- 172.21.15.16:0/4288815799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7bd92a1d0 0x55f7bd933ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.186+0000 7f1605d7f640 1 -- 172.21.15.16:0/4288815799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f000e040 con 0x55f7bd92fd30 2024-09-13T16:31:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.186+0000 7f1604d7d640 1 --2- 172.21.15.16:0/4288815799 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7bd92aba0 0x55f7bd934420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:31:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.186+0000 7f1605d7f640 1 --2- 172.21.15.16:0/4288815799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7bd92fd30 0x55f7bd93b460 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f15f800ac40 tx=0x7f15f8007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ce75d729fd7954 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.186+0000 7f15e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:31:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.187+0000 7f15e8ff9640 1 -- 172.21.15.16:0/4288815799 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f80165f0 con 0x55f7bd92fd30 2024-09-13T16:31:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.187+0000 7f15e8ff9640 1 -- 172.21.15.16:0/4288815799 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f15f8016750 con 0x55f7bd92fd30 2024-09-13T16:31:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.187+0000 7f160a5dd740 1 -- 172.21.15.16:0/4288815799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f7bd8b6680 con 0x55f7bd92fd30 2024-09-13T16:31:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.187+0000 7f15e8ff9640 1 -- 172.21.15.16:0/4288815799 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15f8016910 con 0x55f7bd92fd30 2024-09-13T16:31:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.187+0000 7f160a5dd740 10 client.15558 fetch_fsmap learned FSMap version 29 2024-09-13T16:31:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.187+0000 7f160a5dd740 1 -- 172.21.15.16:0/4288815799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f7bd88c330 con 0x55f7bd92fd30 2024-09-13T16:31:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.187+0000 7f15e8ff9640 1 -- 172.21.15.16:0/4288815799 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f7bd8b6680 con 0x55f7bd92fd30 2024-09-13T16:31:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.187+0000 7f15e8ff9640 1 -- 172.21.15.16:0/4288815799 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f15f8008040 con 0x55f7bd92fd30 2024-09-13T16:31:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.187+0000 7f160a5dd740 10 client.15558 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T16:31:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.187+0000 7f160a5dd740 10 client.15558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T16:31:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.187+0000 7f160a5dd740 20 client.15558 populate_metadata read hostname 'smithi016' 2024-09-13T16:31:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.187+0000 7f160a5dd740 1 --2- 172.21.15.16:0/4288815799 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55f7bd93fbd0 0x55f7bd941ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.187+0000 7f160a5dd740 4 client.15558 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T16:31:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.187+0000 7f160a5dd740 1 -- 172.21.15.16:0/4288815799 --> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f7bd8815d0 con 0x55f7bd93fbd0 2024-09-13T16:31:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.188+0000 7f160557e640 1 --2- 172.21.15.16:0/4288815799 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55f7bd93fbd0 0x55f7bd941ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.188+0000 7f160557e640 1 --2- 172.21.15.16:0/4288815799 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55f7bd93fbd0 0x55f7bd941ff0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=101d80fe4189d740 server_cookie=b5fa42b2aa7ba899 in_seq=0 out_seq=0 2024-09-13T16:31:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.188+0000 7f15e8ff9640 10 client.15558 ms_handle_connect on v2:172.21.15.107:6838/1914689806 2024-09-13T16:31:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.188+0000 7f15e8ff9640 1 -- 172.21.15.16:0/4288815799 <== mds.0 v2:172.21.15.107:6838/1914689806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f7bd8815d0 con 0x55f7bd93fbd0 2024-09-13T16:31:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.188+0000 7f15e8ff9640 10 client.15558 handle_command_reply: tid=0 2024-09-13T16:31:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.189+0000 7f160a5dd740 1 client.15558 shutdown 2024-09-13T16:31:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.189+0000 7f160a5dd740 1 -- 172.21.15.16:0/4288815799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7bd92fd30 msgr2=0x55f7bd93b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.189+0000 7f160a5dd740 1 --2- 172.21.15.16:0/4288815799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7bd92fd30 0x55f7bd93b460 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f15f800ac40 tx=0x7f15f8007680 comp rx=0 tx=0).stop 2024-09-13T16:31:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.189+0000 7f160a5dd740 1 -- 172.21.15.16:0/4288815799 shutdown_connections 2024-09-13T16:31:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.189+0000 7f160a5dd740 1 --2- 172.21.15.16:0/4288815799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7bd92fd30 0x55f7bd93b460 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.189+0000 7f160a5dd740 1 --2- 172.21.15.16:0/4288815799 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55f7bd93fbd0 0x55f7bd941ff0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.189+0000 7f15e8ff9640 0 client.15558 ms_handle_reset on v2:172.21.15.107:6838/1914689806 2024-09-13T16:31:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.189+0000 7f160a5dd740 1 --2- 172.21.15.16:0/4288815799 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7bd92aba0 0x55f7bd934420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.189+0000 7f160a5dd740 1 --2- 172.21.15.16:0/4288815799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7bd92a1d0 0x55f7bd933ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.189+0000 7f160a5dd740 1 -- 172.21.15.16:0/4288815799 >> 172.21.15.16:0/4288815799 conn(0x55f7bd9258c0 msgr2=0x55f7bd924b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.189+0000 7f160a5dd740 1 -- 172.21.15.16:0/4288815799 shutdown_connections 2024-09-13T16:31:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.189+0000 7f160a5dd740 1 -- 172.21.15.16:0/4288815799 wait complete. 2024-09-13T16:31:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.189+0000 7f160a5dd740 20 client.15558 trim_cache size 0 max 16384 2024-09-13T16:31:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.202+0000 7f160a5dd740 1 Processor -- start 2024-09-13T16:31:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.202+0000 7f160a5dd740 1 -- start start 2024-09-13T16:31:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.202+0000 7f160a5dd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7bd92a1d0 0x55f7bd92bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.202+0000 7f160a5dd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7bd92aba0 0x55f7bd92c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.202+0000 7f160a5dd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7bd92fd30 0x55f7bd93b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.202+0000 7f160a5dd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7bd8b6680 con 0x55f7bd92fd30 2024-09-13T16:31:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.202+0000 7f160a5dd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7bd93e190 con 0x55f7bd92aba0 2024-09-13T16:31:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.202+0000 7f160a5dd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7bd93f740 con 0x55f7bd92a1d0 2024-09-13T16:31:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.202+0000 7f1605d7f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7bd92fd30 0x55f7bd93b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.202+0000 7f1605d7f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7bd92fd30 0x55f7bd93b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55002/0 (socket says 172.21.15.16:55002) 2024-09-13T16:31:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.202+0000 7f1605d7f640 1 -- 172.21.15.16:0/4019716331 learned_addr learned my addr 172.21.15.16:0/4019716331 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.202+0000 7f160557e640 1 --2- 172.21.15.16:0/4019716331 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7bd92a1d0 0x55f7bd92bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.202+0000 7f1604d7d640 1 --2- 172.21.15.16:0/4019716331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7bd92aba0 0x55f7bd92c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.202+0000 7f1605d7f640 1 -- 172.21.15.16:0/4019716331 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7bd92a1d0 msgr2=0x55f7bd92bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.202+0000 7f1605d7f640 1 --2- 172.21.15.16:0/4019716331 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7bd92a1d0 0x55f7bd92bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.202+0000 7f1605d7f640 1 -- 172.21.15.16:0/4019716331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7bd92aba0 msgr2=0x55f7bd92c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.202+0000 7f1605d7f640 1 --2- 172.21.15.16:0/4019716331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7bd92aba0 0x55f7bd92c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.202+0000 7f1605d7f640 1 -- 172.21.15.16:0/4019716331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7bd82e300 con 0x55f7bd92fd30 2024-09-13T16:31:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.202+0000 7f1604d7d640 1 --2- 172.21.15.16:0/4019716331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7bd92aba0 0x55f7bd92c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:31:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.202+0000 7f1605d7f640 1 --2- 172.21.15.16:0/4019716331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7bd92fd30 0x55f7bd93b420 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f15f800afd0 tx=0x7f15f80049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5c457ea03d7806 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.203+0000 7f15ebfff640 1 -- 172.21.15.16:0/4019716331 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f8007b80 con 0x55f7bd92fd30 2024-09-13T16:31:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.203+0000 7f15ebfff640 1 -- 172.21.15.16:0/4019716331 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f15f8012040 con 0x55f7bd92fd30 2024-09-13T16:31:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.203+0000 7f15ebfff640 1 -- 172.21.15.16:0/4019716331 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f80266a0 con 0x55f7bd92fd30 2024-09-13T16:31:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.203+0000 7f160a5dd740 1 -- 172.21.15.16:0/4019716331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7bd92fd30 msgr2=0x55f7bd93b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.203+0000 7f160a5dd740 1 --2- 172.21.15.16:0/4019716331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7bd92fd30 0x55f7bd93b420 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f15f800afd0 tx=0x7f15f80049c0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.203+0000 7f160a5dd740 1 -- 172.21.15.16:0/4019716331 shutdown_connections 2024-09-13T16:31:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.203+0000 7f160a5dd740 1 --2- 172.21.15.16:0/4019716331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7bd92fd30 0x55f7bd93b420 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.203+0000 7f160a5dd740 1 --2- 172.21.15.16:0/4019716331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7bd92aba0 0x55f7bd92c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.203+0000 7f160a5dd740 1 --2- 172.21.15.16:0/4019716331 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7bd92a1d0 0x55f7bd92bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.203+0000 7f160a5dd740 1 -- 172.21.15.16:0/4019716331 >> 172.21.15.16:0/4019716331 conn(0x55f7bd9258c0 msgr2=0x55f7bd92f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.203+0000 7f160a5dd740 1 -- 172.21.15.16:0/4019716331 shutdown_connections 2024-09-13T16:31:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.203+0000 7f160a5dd740 1 -- 172.21.15.16:0/4019716331 wait complete. 2024-09-13T16:31:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.204+0000 7f160a5dd740 1 -- start start 2024-09-13T16:31:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.204+0000 7f160a5dd740 1 Processor -- start 2024-09-13T16:31:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.204+0000 7f160a5dd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7bd92a1d0 0x55f7bd939ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.204+0000 7f160a5dd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7bd92aba0 0x55f7bd93a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.204+0000 7f160a5dd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7bd92fd30 0x55f7bd93a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.204+0000 7f160a5dd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7bd93b760 con 0x55f7bd92fd30 2024-09-13T16:31:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.204+0000 7f160a5dd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7bd996080 con 0x55f7bd92a1d0 2024-09-13T16:31:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.204+0000 7f160a5dd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7bd9961c0 con 0x55f7bd92aba0 2024-09-13T16:31:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.204+0000 7f160a5dd740 10 client.0 _ll_register_callbacks cb 0x7f15f0009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:31:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.204+0000 7f1605d7f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7bd92fd30 0x55f7bd93a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.204+0000 7f1605d7f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7bd92fd30 0x55f7bd93a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55008/0 (socket says 172.21.15.16:55008) 2024-09-13T16:31:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.204+0000 7f1605d7f640 1 -- 172.21.15.16:0/801117171 learned_addr learned my addr 172.21.15.16:0/801117171 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.204+0000 7f1604d7d640 1 --2- 172.21.15.16:0/801117171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7bd92aba0 0x55f7bd93a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.204+0000 7f160557e640 1 --2- 172.21.15.16:0/801117171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7bd92a1d0 0x55f7bd939ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.205+0000 7f1605d7f640 1 -- 172.21.15.16:0/801117171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7bd92aba0 msgr2=0x55f7bd93a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.205+0000 7f1605d7f640 1 --2- 172.21.15.16:0/801117171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7bd92aba0 0x55f7bd93a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.205+0000 7f1605d7f640 1 -- 172.21.15.16:0/801117171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7bd92a1d0 msgr2=0x55f7bd939ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.205+0000 7f1605d7f640 1 --2- 172.21.15.16:0/801117171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7bd92a1d0 0x55f7bd939ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.205+0000 7f1605d7f640 1 -- 172.21.15.16:0/801117171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f7bd88c330 con 0x55f7bd92fd30 2024-09-13T16:31:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.205+0000 7f1605d7f640 1 --2- 172.21.15.16:0/801117171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7bd92fd30 0x55f7bd93a560 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f15f800a530 tx=0x7f15f8007b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844d3a73433e117e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.205+0000 7f15e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:31:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.205+0000 7f15e8ff9640 1 -- 172.21.15.16:0/801117171 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f802f470 con 0x55f7bd92fd30 2024-09-13T16:31:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.205+0000 7f15e8ff9640 1 -- 172.21.15.16:0/801117171 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f15f802f5d0 con 0x55f7bd92fd30 2024-09-13T16:31:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.205+0000 7f160a5dd740 1 -- 172.21.15.16:0/801117171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f15f001f400 con 0x55f7bd92fd30 2024-09-13T16:31:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.205+0000 7f15e8ff9640 1 -- 172.21.15.16:0/801117171 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15f802f790 con 0x55f7bd92fd30 2024-09-13T16:31:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.205+0000 7f160a5dd740 10 client.15564 fetch_fsmap learned FSMap version 29 2024-09-13T16:31:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.205+0000 7f160a5dd740 1 -- 172.21.15.16:0/801117171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f7bd8538e0 con 0x55f7bd92fd30 2024-09-13T16:31:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.206+0000 7f15e8ff9640 1 -- 172.21.15.16:0/801117171 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f15f001f400 con 0x55f7bd92fd30 2024-09-13T16:31:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.206+0000 7f15e8ff9640 1 -- 172.21.15.16:0/801117171 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f15f8037070 con 0x55f7bd92fd30 2024-09-13T16:31:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.206+0000 7f160a5dd740 10 client.15564 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T16:31:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.206+0000 7f160a5dd740 10 client.15564 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T16:31:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.206+0000 7f160a5dd740 20 client.15564 populate_metadata read hostname 'smithi016' 2024-09-13T16:31:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.206+0000 7f160a5dd740 1 --2- 172.21.15.16:0/801117171 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55f7bd9978f0 0x55f7bd999ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.206+0000 7f160a5dd740 4 client.15564 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T16:31:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.206+0000 7f160a5dd740 1 -- 172.21.15.16:0/801117171 --> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f7bd581300 con 0x55f7bd9978f0 2024-09-13T16:31:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.206+0000 7f160557e640 1 --2- 172.21.15.16:0/801117171 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55f7bd9978f0 0x55f7bd999ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.207+0000 7f160557e640 1 --2- 172.21.15.16:0/801117171 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55f7bd9978f0 0x55f7bd999ce0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=233ba00bb222cd43 server_cookie=404754af14f5501e in_seq=0 out_seq=0 2024-09-13T16:31:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.207+0000 7f15e8ff9640 10 client.15564 ms_handle_connect on v2:172.21.15.107:6838/1914689806 2024-09-13T16:31:33.213 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.212+0000 7f15e8ff9640 1 -- 172.21.15.16:0/801117171 <== mds.0 v2:172.21.15.107:6838/1914689806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f7bd581300 con 0x55f7bd9978f0 2024-09-13T16:31:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.212+0000 7f15e8ff9640 10 client.15564 handle_command_reply: tid=0 2024-09-13T16:31:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.212+0000 7f160a5dd740 1 client.15564 shutdown 2024-09-13T16:31:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.212+0000 7f160a5dd740 1 -- 172.21.15.16:0/801117171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7bd92fd30 msgr2=0x55f7bd93a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.212+0000 7f160a5dd740 1 --2- 172.21.15.16:0/801117171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7bd92fd30 0x55f7bd93a560 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f15f800a530 tx=0x7f15f8007b80 comp rx=0 tx=0).stop 2024-09-13T16:31:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.212+0000 7f160a5dd740 1 -- 172.21.15.16:0/801117171 shutdown_connections 2024-09-13T16:31:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.212+0000 7f160a5dd740 1 --2- 172.21.15.16:0/801117171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7bd92fd30 0x55f7bd93a560 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.212+0000 7f160a5dd740 1 --2- 172.21.15.16:0/801117171 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55f7bd9978f0 0x55f7bd999ce0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.212+0000 7f160a5dd740 1 --2- 172.21.15.16:0/801117171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7bd92aba0 0x55f7bd93a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.212+0000 7f160a5dd740 1 --2- 172.21.15.16:0/801117171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7bd92a1d0 0x55f7bd939ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.212+0000 7f15e8ff9640 0 client.15564 ms_handle_reset on v2:172.21.15.107:6838/1914689806 2024-09-13T16:31:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.212+0000 7f160a5dd740 1 -- 172.21.15.16:0/801117171 >> 172.21.15.16:0/801117171 conn(0x55f7bd9258c0 msgr2=0x55f7bd92f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.212+0000 7f160a5dd740 1 -- 172.21.15.16:0/801117171 shutdown_connections 2024-09-13T16:31:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.212+0000 7f160a5dd740 1 -- 172.21.15.16:0/801117171 wait complete. 2024-09-13T16:31:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.213+0000 7f160a5dd740 20 client.15564 trim_cache size 0 max 16384 2024-09-13T16:31:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-13T16:31:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-13T16:31:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-13T16:31:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-13T16:31:33.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.213+0000 7f16077e2640 1 -- 172.21.15.16:0/237829430 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f15ec0a1120 msgr2=0x7f15ec0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:33.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.213+0000 7f16077e2640 1 --2- 172.21.15.16:0/237829430 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f15ec0a1120 0x7f15ec0a35e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1600100770 tx=0x7f15f0008040 comp rx=0 tx=0).stop 2024-09-13T16:31:33.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.213+0000 7f16077e2640 1 -- 172.21.15.16:0/237829430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16000fddc0 msgr2=0x7f1600100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:33.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.213+0000 7f16077e2640 1 --2- 172.21.15.16:0/237829430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16000fddc0 0x7f1600100a00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f15fc0055e0 tx=0x7f15fc0048f0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.213+0000 7f16077e2640 1 -- 172.21.15.16:0/237829430 shutdown_connections 2024-09-13T16:31:33.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.213+0000 7f16077e2640 1 --2- 172.21.15.16:0/237829430 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f15ec0a1120 0x7f15ec0a35e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.213+0000 7f16077e2640 1 --2- 172.21.15.16:0/237829430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1600101350 0x7f16000ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.213+0000 7f16077e2640 1 --2- 172.21.15.16:0/237829430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16000fe780 0x7f16000ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.213+0000 7f16077e2640 1 --2- 172.21.15.16:0/237829430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16000fddc0 0x7f1600100a00 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:33.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.213+0000 7f16077e2640 1 -- 172.21.15.16:0/237829430 >> 172.21.15.16:0/237829430 conn(0x7f16000f9b50 msgr2=0x7f16001045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:33.216 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.213+0000 7f16077e2640 1 -- 172.21.15.16:0/237829430 shutdown_connections 2024-09-13T16:31:33.217 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:33.213+0000 7f16077e2640 1 -- 172.21.15.16:0/237829430 wait complete. 2024-09-13T16:31:34.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:34 smithi111 ceph-mon[34600]: pgmap v368: 129 pgs: 129 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-13T16:31:35.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:34 smithi107 ceph-mon[34139]: pgmap v368: 129 pgs: 129 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-13T16:31:35.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:34 smithi016 ceph-mon[31187]: pgmap v368: 129 pgs: 129 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-13T16:31:36.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:36 smithi111 ceph-mon[34600]: pgmap v369: 129 pgs: 129 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-13T16:31:37.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:36 smithi107 ceph-mon[34139]: pgmap v369: 129 pgs: 129 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-13T16:31:37.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:36 smithi016 ceph-mon[31187]: pgmap v369: 129 pgs: 129 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-13T16:31:38.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:38 smithi111 ceph-mon[34600]: pgmap v370: 129 pgs: 129 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-13T16:31:39.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:38 smithi107 ceph-mon[34139]: pgmap v370: 129 pgs: 129 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-13T16:31:39.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:38 smithi016 ceph-mon[31187]: pgmap v370: 129 pgs: 129 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-13T16:31:40.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:40 smithi111 ceph-mon[34600]: pgmap v371: 129 pgs: 129 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-13T16:31:41.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:40 smithi107 ceph-mon[34139]: pgmap v371: 129 pgs: 129 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-13T16:31:41.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:40 smithi016 ceph-mon[31187]: pgmap v371: 129 pgs: 129 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-13T16:31:42.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:42 smithi111 ceph-mon[34600]: pgmap v372: 129 pgs: 129 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-13T16:31:43.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:42 smithi107 ceph-mon[34139]: pgmap v372: 129 pgs: 129 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-13T16:31:43.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:42 smithi016 ceph-mon[31187]: pgmap v372: 129 pgs: 129 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-13T16:31:44.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:44 smithi111 ceph-mon[34600]: pgmap v373: 129 pgs: 129 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-13T16:31:45.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:44 smithi107 ceph-mon[34139]: pgmap v373: 129 pgs: 129 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-13T16:31:45.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:44 smithi016 ceph-mon[31187]: pgmap v373: 129 pgs: 129 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-13T16:31:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.345+0000 7f6c08ea3640 1 -- 172.21.15.16:0/2702921312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c040ffe20 msgr2=0x7f6c041002a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.345+0000 7f6c08ea3640 1 --2- 172.21.15.16:0/2702921312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c040ffe20 0x7f6c041002a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8009900 tx=0x7f6bf802a410 comp rx=0 tx=0).stop 2024-09-13T16:31:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.345+0000 7f6c08ea3640 1 -- 172.21.15.16:0/2702921312 shutdown_connections 2024-09-13T16:31:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.345+0000 7f6c08ea3640 1 --2- 172.21.15.16:0/2702921312 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c041007e0 0x7f6c04107070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.345+0000 7f6c08ea3640 1 --2- 172.21.15.16:0/2702921312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c040ffe20 0x7f6c041002a0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.345+0000 7f6c08ea3640 1 --2- 172.21.15.16:0/2702921312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c040fec30 0x7f6c040ff030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.345+0000 7f6c08ea3640 1 -- 172.21.15.16:0/2702921312 >> 172.21.15.16:0/2702921312 conn(0x7f6c040fa3b0 msgr2=0x7f6c040fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.345+0000 7f6c08ea3640 1 -- 172.21.15.16:0/2702921312 shutdown_connections 2024-09-13T16:31:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.345+0000 7f6c08ea3640 1 -- 172.21.15.16:0/2702921312 wait complete. 2024-09-13T16:31:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.346+0000 7f6c08ea3640 1 Processor -- start 2024-09-13T16:31:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.346+0000 7f6c08ea3640 1 -- start start 2024-09-13T16:31:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.346+0000 7f6c08ea3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c040fec30 0x7f6c04193a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.346+0000 7f6c08ea3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c040ffe20 0x7f6c04193f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.346+0000 7f6c08ea3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c041007e0 0x7f6c0419afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.347+0000 7f6c0259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c040fec30 0x7f6c04193a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.347+0000 7f6c0259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c040fec30 0x7f6c04193a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36262/0 (socket says 172.21.15.16:36262) 2024-09-13T16:31:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.347+0000 7f6c08ea3640 1 -- 172.21.15.16:0/570218572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0419b520 con 0x7f6c040fec30 2024-09-13T16:31:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.347+0000 7f6c08ea3640 1 -- 172.21.15.16:0/570218572 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0419b690 con 0x7f6c041007e0 2024-09-13T16:31:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.347+0000 7f6c08ea3640 1 -- 172.21.15.16:0/570218572 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0419b850 con 0x7f6c040ffe20 2024-09-13T16:31:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.347+0000 7f6c0259c640 1 -- 172.21.15.16:0/570218572 learned_addr learned my addr 172.21.15.16:0/570218572 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.347+0000 7f6c01d9b640 1 --2- 172.21.15.16:0/570218572 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c040ffe20 0x7f6c04193f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.347+0000 7f6c02d9d640 1 --2- 172.21.15.16:0/570218572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c041007e0 0x7f6c0419afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.347+0000 7f6c0259c640 1 -- 172.21.15.16:0/570218572 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c040ffe20 msgr2=0x7f6c04193f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.347+0000 7f6c0259c640 1 --2- 172.21.15.16:0/570218572 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c040ffe20 0x7f6c04193f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.347+0000 7f6c0259c640 1 -- 172.21.15.16:0/570218572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c041007e0 msgr2=0x7f6c0419afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.347+0000 7f6c0259c640 1 --2- 172.21.15.16:0/570218572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c041007e0 0x7f6c0419afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.347+0000 7f6c0259c640 1 -- 172.21.15.16:0/570218572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf8009560 con 0x7f6c040fec30 2024-09-13T16:31:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.348+0000 7f6c0259c640 1 --2- 172.21.15.16:0/570218572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c040fec30 0x7f6c04193a30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4007830 tx=0x7f6bf4007cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.348+0000 7f6bf37fe640 1 -- 172.21.15.16:0/570218572 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4010040 con 0x7f6c040fec30 2024-09-13T16:31:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.348+0000 7f6bf37fe640 1 -- 172.21.15.16:0/570218572 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6bf40044b0 con 0x7f6c040fec30 2024-09-13T16:31:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.348+0000 7f6c08ea3640 1 -- 172.21.15.16:0/570218572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c0419bb30 con 0x7f6c040fec30 2024-09-13T16:31:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.348+0000 7f6bf37fe640 1 -- 172.21.15.16:0/570218572 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf40038f0 con 0x7f6c040fec30 2024-09-13T16:31:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.348+0000 7f6c08ea3640 1 -- 172.21.15.16:0/570218572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c0419c060 con 0x7f6c040fec30 2024-09-13T16:31:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.351+0000 7f6c0bbcd740 1 Processor -- start 2024-09-13T16:31:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.352+0000 7f6c0bbcd740 1 -- start start 2024-09-13T16:31:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.352+0000 7f6c0bbcd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be8a57c1d0 0x55be8a57c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.352+0000 7f6c0bbcd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8a57cba0 0x55be8a5816c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.352+0000 7f6c0bbcd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55be8a581d30 0x55be8a5841c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.352+0000 7f6c0bbcd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be8a4e6c30 con 0x55be8a57cba0 2024-09-13T16:31:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.352+0000 7f6c0bbcd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be8a4e43c0 con 0x55be8a57c1d0 2024-09-13T16:31:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.352+0000 7f6c0bbcd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be8a4dfcb0 con 0x55be8a581d30 2024-09-13T16:31:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.352+0000 7f6bf37fe640 1 -- 172.21.15.16:0/570218572 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f6bf4003a70 con 0x7f6c040fec30 2024-09-13T16:31:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.352+0000 7f6c01d9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55be8a581d30 0x55be8a5841c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.352+0000 7f6c02d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be8a57c1d0 0x55be8a57c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.352+0000 7f6c01d9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55be8a581d30 0x55be8a5841c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.16:44858/0 (socket says 172.21.15.16:44858) 2024-09-13T16:31:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.352+0000 7f6c01d9b640 1 -- 172.21.15.16:0/3088544883 learned_addr learned my addr 172.21.15.16:0/3088544883 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.353+0000 7f6c02d9d640 1 -- 172.21.15.16:0/3088544883 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55be8a581d30 msgr2=0x55be8a5841c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.353+0000 7f6c02d9d640 1 --2- 172.21.15.16:0/3088544883 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55be8a581d30 0x55be8a5841c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.353+0000 7f6c02d9d640 1 -- 172.21.15.16:0/3088544883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8a57cba0 msgr2=0x55be8a5816c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T16:31:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.353+0000 7f6c0259c640 1 --2- 172.21.15.16:0/3088544883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8a57cba0 0x55be8a5816c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.353+0000 7f6c02d9d640 1 --2- 172.21.15.16:0/3088544883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8a57cba0 0x55be8a5816c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.353+0000 7f6c02d9d640 1 -- 172.21.15.16:0/3088544883 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be8a486910 con 0x55be8a57c1d0 2024-09-13T16:31:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.353+0000 7f6c0259c640 1 --2- 172.21.15.16:0/3088544883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8a57cba0 0x55be8a5816c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:31:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.353+0000 7f6c02d9d640 1 --2- 172.21.15.16:0/3088544883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be8a57c1d0 0x55be8a57c5d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc009930 tx=0x7f6bfc009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8645e6329540430 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.353+0000 7f6bf0ff9640 1 -- 172.21.15.16:0/3088544883 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc014070 con 0x55be8a57c1d0 2024-09-13T16:31:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.354+0000 7f6bf0ff9640 1 -- 172.21.15.16:0/3088544883 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6bfc0046e0 con 0x55be8a57c1d0 2024-09-13T16:31:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.354+0000 7f6bf0ff9640 1 -- 172.21.15.16:0/3088544883 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc005920 con 0x55be8a57c1d0 2024-09-13T16:31:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.354+0000 7f6c0bbcd740 1 -- 172.21.15.16:0/3088544883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be8a57c1d0 msgr2=0x55be8a57c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.354+0000 7f6c0bbcd740 1 --2- 172.21.15.16:0/3088544883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be8a57c1d0 0x55be8a57c5d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc009930 tx=0x7f6bfc009500 comp rx=0 tx=0).stop 2024-09-13T16:31:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.355+0000 7f6c0bbcd740 1 -- 172.21.15.16:0/3088544883 shutdown_connections 2024-09-13T16:31:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.355+0000 7f6c0bbcd740 1 --2- 172.21.15.16:0/3088544883 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55be8a581d30 0x55be8a5841c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.355+0000 7f6c0bbcd740 1 --2- 172.21.15.16:0/3088544883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8a57cba0 0x55be8a5816c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.355+0000 7f6c0bbcd740 1 --2- 172.21.15.16:0/3088544883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be8a57c1d0 0x55be8a57c5d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.355+0000 7f6c0bbcd740 1 -- 172.21.15.16:0/3088544883 >> 172.21.15.16:0/3088544883 conn(0x55be8a5778c0 msgr2=0x55be8a577cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.355+0000 7f6c0bbcd740 1 -- 172.21.15.16:0/3088544883 shutdown_connections 2024-09-13T16:31:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.355+0000 7f6bf37fe640 1 --2- 172.21.15.16:0/570218572 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f6bec0a10b0 0x7f6bec0a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.355+0000 7f6c0bbcd740 1 -- 172.21.15.16:0/3088544883 wait complete. 2024-09-13T16:31:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.355+0000 7f6c01d9b640 1 --2- 172.21.15.16:0/570218572 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f6bec0a10b0 0x7f6bec0a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.355+0000 7f6c0bbcd740 1 -- start start 2024-09-13T16:31:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.355+0000 7f6bf37fe640 1 -- 172.21.15.16:0/570218572 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f6bf40ccc10 con 0x7f6c040fec30 2024-09-13T16:31:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.355+0000 7f6c0bbcd740 1 Processor -- start 2024-09-13T16:31:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.356+0000 7f6c01d9b640 1 --2- 172.21.15.16:0/570218572 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f6bec0a10b0 0x7f6bec0a3570 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x55be8a5786f0 tx=0x7f6bf8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.356+0000 7f6c0bbcd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8a57c1d0 0x55be8a58e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.356+0000 7f6c0bbcd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be8a57cba0 0x55be8a58ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.356+0000 7f6c0bbcd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55be8a581d30 0x55be8a595d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.356+0000 7f6c0bbcd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be8a49de80 con 0x55be8a57c1d0 2024-09-13T16:31:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.356+0000 7f6c0bbcd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be8a49db20 con 0x55be8a57cba0 2024-09-13T16:31:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.356+0000 7f6c0bbcd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be8a463260 con 0x55be8a581d30 2024-09-13T16:31:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.356+0000 7f6c0bbcd740 10 client.0 _ll_register_callbacks cb 0x55be8a498df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:31:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.356+0000 7f6c02d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55be8a581d30 0x55be8a595d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.356+0000 7f6c01d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be8a57cba0 0x55be8a58ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.356+0000 7f6c02d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55be8a581d30 0x55be8a595d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.16:44870/0 (socket says 172.21.15.16:44870) 2024-09-13T16:31:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.356+0000 7f6c0259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8a57c1d0 0x55be8a58e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.356+0000 7f6c02d9d640 1 -- 172.21.15.16:0/2359990842 learned_addr learned my addr 172.21.15.16:0/2359990842 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.356+0000 7f6c02d9d640 1 -- 172.21.15.16:0/2359990842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be8a57cba0 msgr2=0x55be8a58ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.356+0000 7f6c02d9d640 1 --2- 172.21.15.16:0/2359990842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be8a57cba0 0x55be8a58ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.356+0000 7f6c02d9d640 1 -- 172.21.15.16:0/2359990842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8a57c1d0 msgr2=0x55be8a58e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.356+0000 7f6c02d9d640 1 --2- 172.21.15.16:0/2359990842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8a57c1d0 0x55be8a58e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.356+0000 7f6c02d9d640 1 -- 172.21.15.16:0/2359990842 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bfc003d30 con 0x55be8a581d30 2024-09-13T16:31:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.357+0000 7f6c02d9d640 1 --2- 172.21.15.16:0/2359990842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55be8a581d30 0x55be8a595d50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc00a010 tx=0x7f6bfc005910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afa86ded76f31086 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.357+0000 7f6be9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-13T16:31:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.357+0000 7f6be9ffb640 1 -- 172.21.15.16:0/2359990842 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc014030 con 0x55be8a581d30 2024-09-13T16:31:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.357+0000 7f6be9ffb640 1 -- 172.21.15.16:0/2359990842 <== mon.1 v2:172.21.15.111:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f6bfc015070 con 0x55be8a581d30 2024-09-13T16:31:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.357+0000 7f6c0bbcd740 1 -- 172.21.15.16:0/2359990842 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be8a508680 con 0x55be8a581d30 2024-09-13T16:31:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.357+0000 7f6be9ffb640 1 -- 172.21.15.16:0/2359990842 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bfc004850 con 0x55be8a581d30 2024-09-13T16:31:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.357+0000 7f6c0bbcd740 10 client.25153 fetch_fsmap learned FSMap version 29 2024-09-13T16:31:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.357+0000 7f6c0bbcd740 1 -- 172.21.15.16:0/2359990842 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55be8a4de330 con 0x55be8a581d30 2024-09-13T16:31:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.358+0000 7f6be9ffb640 1 -- 172.21.15.16:0/2359990842 <== mon.1 v2:172.21.15.111:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55be8a508680 con 0x55be8a581d30 2024-09-13T16:31:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.359+0000 7f6be9ffb640 1 -- 172.21.15.16:0/2359990842 <== mon.1 v2:172.21.15.111:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6bfc020040 con 0x55be8a581d30 2024-09-13T16:31:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.359+0000 7f6c0bbcd740 10 client.25153 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T16:31:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.359+0000 7f6c0bbcd740 10 client.25153 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T16:31:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.359+0000 7f6c0bbcd740 20 client.25153 populate_metadata read hostname 'smithi016' 2024-09-13T16:31:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.359+0000 7f6c0bbcd740 1 --2- 172.21.15.16:0/2359990842 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55be8a59a2d0 0x55be8a59c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.359+0000 7f6c0bbcd740 4 client.25153 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T16:31:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.359+0000 7f6c0bbcd740 1 -- 172.21.15.16:0/2359990842 --> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be8a4d35d0 con 0x55be8a59a2d0 2024-09-13T16:31:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.359+0000 7f6c0259c640 1 --2- 172.21.15.16:0/2359990842 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55be8a59a2d0 0x55be8a59c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.359+0000 7f6c0259c640 1 --2- 172.21.15.16:0/2359990842 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55be8a59a2d0 0x55be8a59c6c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6405bd21d99c8d6 server_cookie=44767253e6bf1375 in_seq=0 out_seq=0 2024-09-13T16:31:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.359+0000 7f6be9ffb640 10 client.25153 ms_handle_connect on v2:172.21.15.107:6838/1914689806 2024-09-13T16:31:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.360+0000 7f6be9ffb640 1 -- 172.21.15.16:0/2359990842 <== mds.0 v2:172.21.15.107:6838/1914689806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55be8a4d35d0 con 0x55be8a59a2d0 2024-09-13T16:31:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.360+0000 7f6be9ffb640 10 client.25153 handle_command_reply: tid=0 2024-09-13T16:31:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.360+0000 7f6c0bbcd740 1 client.25153 shutdown 2024-09-13T16:31:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.360+0000 7f6c0bbcd740 1 -- 172.21.15.16:0/2359990842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55be8a581d30 msgr2=0x55be8a595d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.360+0000 7f6c0bbcd740 1 --2- 172.21.15.16:0/2359990842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55be8a581d30 0x55be8a595d50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc00a010 tx=0x7f6bfc005910 comp rx=0 tx=0).stop 2024-09-13T16:31:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.360+0000 7f6c0bbcd740 1 -- 172.21.15.16:0/2359990842 shutdown_connections 2024-09-13T16:31:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.360+0000 7f6c0bbcd740 1 --2- 172.21.15.16:0/2359990842 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55be8a59a2d0 0x55be8a59c6c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.360+0000 7f6be9ffb640 0 client.25153 ms_handle_reset on v2:172.21.15.107:6838/1914689806 2024-09-13T16:31:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.360+0000 7f6c0bbcd740 1 --2- 172.21.15.16:0/2359990842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55be8a581d30 0x55be8a595d50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.360+0000 7f6c0bbcd740 1 --2- 172.21.15.16:0/2359990842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be8a57cba0 0x55be8a58ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.360+0000 7f6c0bbcd740 1 --2- 172.21.15.16:0/2359990842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8a57c1d0 0x55be8a58e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.360+0000 7f6c0bbcd740 1 -- 172.21.15.16:0/2359990842 >> 172.21.15.16:0/2359990842 conn(0x55be8a5778c0 msgr2=0x55be8a576b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.360+0000 7f6c0bbcd740 1 -- 172.21.15.16:0/2359990842 shutdown_connections 2024-09-13T16:31:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.360+0000 7f6c0bbcd740 1 -- 172.21.15.16:0/2359990842 wait complete. 2024-09-13T16:31:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.361+0000 7f6c0bbcd740 20 client.25153 trim_cache size 0 max 16384 2024-09-13T16:31:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.385+0000 7f6c0bbcd740 1 Processor -- start 2024-09-13T16:31:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.385+0000 7f6c0bbcd740 1 -- start start 2024-09-13T16:31:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.386+0000 7f6c0bbcd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55be8a57c1d0 0x55be8a577300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.386+0000 7f6c0bbcd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8a57cba0 0x55be8a595cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.386+0000 7f6c01d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8a57cba0 0x55be8a595cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.386+0000 7f6c0259c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55be8a57c1d0 0x55be8a577300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.386+0000 7f6c01d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8a57cba0 0x55be8a595cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36306/0 (socket says 172.21.15.16:36306) 2024-09-13T16:31:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.386+0000 7f6c0259c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55be8a57c1d0 0x55be8a577300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.16:44886/0 (socket says 172.21.15.16:44886) 2024-09-13T16:31:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.386+0000 7f6c01d9b640 1 -- 172.21.15.16:0/504828764 learned_addr learned my addr 172.21.15.16:0/504828764 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.386+0000 7f6c0bbcd740 1 --2- 172.21.15.16:0/504828764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be8a581d30 0x55be8a57a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.386+0000 7f6c0bbcd740 1 -- 172.21.15.16:0/504828764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be8a584880 con 0x55be8a57cba0 2024-09-13T16:31:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.386+0000 7f6c0bbcd740 1 -- 172.21.15.16:0/504828764 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be8a599e70 con 0x55be8a581d30 2024-09-13T16:31:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.386+0000 7f6c0bbcd740 1 -- 172.21.15.16:0/504828764 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be8a596200 con 0x55be8a57c1d0 2024-09-13T16:31:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.387+0000 7f6c02d9d640 1 --2- 172.21.15.16:0/504828764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be8a581d30 0x55be8a57a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.387+0000 7f6c0259c640 1 -- 172.21.15.16:0/504828764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be8a581d30 msgr2=0x55be8a57a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.387+0000 7f6c0259c640 1 --2- 172.21.15.16:0/504828764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be8a581d30 0x55be8a57a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.387+0000 7f6c0259c640 1 -- 172.21.15.16:0/504828764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8a57cba0 msgr2=0x55be8a595cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.387+0000 7f6c0259c640 1 --2- 172.21.15.16:0/504828764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8a57cba0 0x55be8a595cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.387+0000 7f6c0259c640 1 -- 172.21.15.16:0/504828764 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be8a480300 con 0x55be8a57c1d0 2024-09-13T16:31:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.387+0000 7f6c01d9b640 1 --2- 172.21.15.16:0/504828764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8a57cba0 0x55be8a595cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:31:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.387+0000 7f6c0259c640 1 --2- 172.21.15.16:0/504828764 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55be8a57c1d0 0x55be8a577300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6bf40ca270 tx=0x7f6bf40ce040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a767d1c8845ef65d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.387+0000 7f6bf17fa640 1 -- 172.21.15.16:0/504828764 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf40b1070 con 0x55be8a57c1d0 2024-09-13T16:31:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.387+0000 7f6bf17fa640 1 -- 172.21.15.16:0/504828764 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6bf409d9e0 con 0x55be8a57c1d0 2024-09-13T16:31:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.387+0000 7f6bf17fa640 1 -- 172.21.15.16:0/504828764 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf400f920 con 0x55be8a57c1d0 2024-09-13T16:31:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.388+0000 7f6c0bbcd740 1 -- 172.21.15.16:0/504828764 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55be8a57c1d0 msgr2=0x55be8a577300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.388+0000 7f6c0bbcd740 1 --2- 172.21.15.16:0/504828764 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55be8a57c1d0 0x55be8a577300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6bf40ca270 tx=0x7f6bf40ce040 comp rx=0 tx=0).stop 2024-09-13T16:31:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.388+0000 7f6c0bbcd740 1 -- 172.21.15.16:0/504828764 shutdown_connections 2024-09-13T16:31:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.388+0000 7f6c0bbcd740 1 --2- 172.21.15.16:0/504828764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be8a581d30 0x55be8a57a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.388+0000 7f6c0bbcd740 1 --2- 172.21.15.16:0/504828764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8a57cba0 0x55be8a595cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.388+0000 7f6c0bbcd740 1 --2- 172.21.15.16:0/504828764 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55be8a57c1d0 0x55be8a577300 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.388+0000 7f6c0bbcd740 1 -- 172.21.15.16:0/504828764 >> 172.21.15.16:0/504828764 conn(0x55be8a5778c0 msgr2=0x55be8a5a1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.388+0000 7f6c0bbcd740 1 -- 172.21.15.16:0/504828764 shutdown_connections 2024-09-13T16:31:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.388+0000 7f6c0bbcd740 1 -- 172.21.15.16:0/504828764 wait complete. 2024-09-13T16:31:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.389+0000 7f6c0bbcd740 1 -- start start 2024-09-13T16:31:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.389+0000 7f6c0bbcd740 1 Processor -- start 2024-09-13T16:31:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.390+0000 7f6c0bbcd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8a57c1d0 0x55be8a595c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.390+0000 7f6c0bbcd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55be8a57cba0 0x55be8a594210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.390+0000 7f6c0259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8a57c1d0 0x55be8a595c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.390+0000 7f6c0259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8a57c1d0 0x55be8a595c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36308/0 (socket says 172.21.15.16:36308) 2024-09-13T16:31:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.390+0000 7f6c0259c640 1 -- 172.21.15.16:0/2788816243 learned_addr learned my addr 172.21.15.16:0/2788816243 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.390+0000 7f6c01d9b640 1 --2- 172.21.15.16:0/2788816243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55be8a57cba0 0x55be8a594210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.390+0000 7f6c0bbcd740 1 --2- 172.21.15.16:0/2788816243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be8a581d30 0x55be8a5947b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.390+0000 7f6c0bbcd740 1 -- 172.21.15.16:0/2788816243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be8a594eb0 con 0x55be8a57c1d0 2024-09-13T16:31:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.390+0000 7f6c0bbcd740 1 -- 172.21.15.16:0/2788816243 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be8a595020 con 0x55be8a581d30 2024-09-13T16:31:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.390+0000 7f6c0bbcd740 1 -- 172.21.15.16:0/2788816243 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be8a5f07a0 con 0x55be8a57cba0 2024-09-13T16:31:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.390+0000 7f6c02d9d640 1 --2- 172.21.15.16:0/2788816243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be8a581d30 0x55be8a5947b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.390+0000 7f6c0259c640 1 -- 172.21.15.16:0/2788816243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55be8a57cba0 msgr2=0x55be8a594210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.391+0000 7f6c0259c640 1 --2- 172.21.15.16:0/2788816243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55be8a57cba0 0x55be8a594210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.391+0000 7f6c0259c640 1 -- 172.21.15.16:0/2788816243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be8a581d30 msgr2=0x55be8a5947b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.391+0000 7f6c0259c640 1 --2- 172.21.15.16:0/2788816243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be8a581d30 0x55be8a5947b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.391+0000 7f6c0259c640 1 -- 172.21.15.16:0/2788816243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c04107970 con 0x55be8a57c1d0 2024-09-13T16:31:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.391+0000 7f6c0259c640 1 --2- 172.21.15.16:0/2788816243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8a57c1d0 0x55be8a595c00 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6bf40960b0 tx=0x7f6bf4005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f36b9a606d345e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.391+0000 7f6c0bbcd740 10 client.0 _ll_register_callbacks cb 0x7f6bfc003c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:31:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.391+0000 7f6be9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:31:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.391+0000 7f6c0bbcd740 1 -- 172.21.15.16:0/2788816243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6bfc0046e0 con 0x55be8a57c1d0 2024-09-13T16:31:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.391+0000 7f6be9ffb640 1 -- 172.21.15.16:0/2788816243 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf40b1070 con 0x55be8a57c1d0 2024-09-13T16:31:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.391+0000 7f6be9ffb640 1 -- 172.21.15.16:0/2788816243 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f6bf40ca4c0 con 0x55be8a57c1d0 2024-09-13T16:31:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.391+0000 7f6be9ffb640 1 -- 172.21.15.16:0/2788816243 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bf40d0040 con 0x55be8a57c1d0 2024-09-13T16:31:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.393+0000 7f6c0bbcd740 10 client.15579 fetch_fsmap learned FSMap version 29 2024-09-13T16:31:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.393+0000 7f6c0bbcd740 1 -- 172.21.15.16:0/2788816243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x7f6c0419c060 con 0x55be8a57c1d0 2024-09-13T16:31:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.393+0000 7f6be9ffb640 1 -- 172.21.15.16:0/2788816243 <== mon.0 v2:172.21.15.16:3300/0 4 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6bf4013050 con 0x55be8a57c1d0 2024-09-13T16:31:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.394+0000 7f6c0bbcd740 10 client.15579 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T16:31:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.394+0000 7f6c0bbcd740 10 client.15579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T16:31:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.394+0000 7f6c0bbcd740 20 client.15579 populate_metadata read hostname 'smithi016' 2024-09-13T16:31:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.394+0000 7f6c0bbcd740 1 --2- 172.21.15.16:0/2788816243 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55be8a59a2d0 0x55be8a5f3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.394+0000 7f6c0bbcd740 4 client.15579 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T16:31:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.394+0000 7f6c0bbcd740 1 -- 172.21.15.16:0/2788816243 --> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55be8a1d3300 con 0x55be8a59a2d0 2024-09-13T16:31:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.394+0000 7f6c01d9b640 1 --2- 172.21.15.16:0/2788816243 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55be8a59a2d0 0x55be8a5f3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.394+0000 7f6be9ffb640 10 client.15579 ms_handle_connect on v2:172.21.15.107:6838/1914689806 2024-09-13T16:31:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.394+0000 7f6c01d9b640 1 --2- 172.21.15.16:0/2788816243 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55be8a59a2d0 0x55be8a5f3d20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=693a0ed4a060bb1b server_cookie=acb832222ee8a22f in_seq=0 out_seq=0 2024-09-13T16:31:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.398+0000 7f6be9ffb640 1 -- 172.21.15.16:0/2788816243 <== mds.0 v2:172.21.15.107:6838/1914689806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55be8a1d3300 con 0x55be8a59a2d0 2024-09-13T16:31:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.398+0000 7f6be9ffb640 10 client.15579 handle_command_reply: tid=0 2024-09-13T16:31:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.398+0000 7f6c0bbcd740 1 client.15579 shutdown 2024-09-13T16:31:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.398+0000 7f6c0bbcd740 1 -- 172.21.15.16:0/2788816243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8a57c1d0 msgr2=0x55be8a595c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.398+0000 7f6c0bbcd740 1 --2- 172.21.15.16:0/2788816243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8a57c1d0 0x55be8a595c00 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6bf40960b0 tx=0x7f6bf4005ab0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.399+0000 7f6c0bbcd740 1 -- 172.21.15.16:0/2788816243 shutdown_connections 2024-09-13T16:31:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.399+0000 7f6c0bbcd740 1 --2- 172.21.15.16:0/2788816243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be8a581d30 0x55be8a5947b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.399+0000 7f6c0bbcd740 1 --2- 172.21.15.16:0/2788816243 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55be8a59a2d0 0x55be8a5f3d20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.399+0000 7f6c0bbcd740 1 --2- 172.21.15.16:0/2788816243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55be8a57cba0 0x55be8a594210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.399+0000 7f6c0bbcd740 1 --2- 172.21.15.16:0/2788816243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be8a57c1d0 0x55be8a595c00 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.399+0000 7f6c0bbcd740 1 -- 172.21.15.16:0/2788816243 >> 172.21.15.16:0/2788816243 conn(0x55be8a5778c0 msgr2=0x55be8a57a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.399+0000 7f6be9ffb640 0 client.15579 ms_handle_reset on v2:172.21.15.107:6838/1914689806 2024-09-13T16:31:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.399+0000 7f6c0bbcd740 1 -- 172.21.15.16:0/2788816243 shutdown_connections 2024-09-13T16:31:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.399+0000 7f6c0bbcd740 1 -- 172.21.15.16:0/2788816243 wait complete. 2024-09-13T16:31:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.399+0000 7f6c0bbcd740 20 client.15579 trim_cache size 0 max 16384 2024-09-13T16:31:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-13T16:31:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-13T16:31:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-13T16:31:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-13T16:31:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.399+0000 7f6c08ea3640 1 -- 172.21.15.16:0/570218572 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f6bec0a10b0 msgr2=0x7f6bec0a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.400+0000 7f6c08ea3640 1 --2- 172.21.15.16:0/570218572 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f6bec0a10b0 0x7f6bec0a3570 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x55be8a5786f0 tx=0x7f6bf8005a20 comp rx=0 tx=0).stop 2024-09-13T16:31:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.400+0000 7f6c08ea3640 1 -- 172.21.15.16:0/570218572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c040fec30 msgr2=0x7f6c04193a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.400+0000 7f6c08ea3640 1 --2- 172.21.15.16:0/570218572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c040fec30 0x7f6c04193a30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4007830 tx=0x7f6bf4007cf0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.400+0000 7f6c08ea3640 1 -- 172.21.15.16:0/570218572 shutdown_connections 2024-09-13T16:31:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.400+0000 7f6c08ea3640 1 --2- 172.21.15.16:0/570218572 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f6bec0a10b0 0x7f6bec0a3570 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.400+0000 7f6c08ea3640 1 --2- 172.21.15.16:0/570218572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c041007e0 0x7f6c0419afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.400+0000 7f6c08ea3640 1 --2- 172.21.15.16:0/570218572 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c040ffe20 0x7f6c04193f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.400+0000 7f6c08ea3640 1 --2- 172.21.15.16:0/570218572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c040fec30 0x7f6c04193a30 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.400+0000 7f6c08ea3640 1 -- 172.21.15.16:0/570218572 >> 172.21.15.16:0/570218572 conn(0x7f6c040fa3b0 msgr2=0x7f6c04103330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.400+0000 7f6c08ea3640 1 -- 172.21.15.16:0/570218572 shutdown_connections 2024-09-13T16:31:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:46.400+0000 7f6c08ea3640 1 -- 172.21.15.16:0/570218572 wait complete. 2024-09-13T16:31:46.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:46 smithi111 ceph-mon[34600]: pgmap v374: 129 pgs: 129 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-13T16:31:47.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:46 smithi107 ceph-mon[34139]: pgmap v374: 129 pgs: 129 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-13T16:31:47.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:46 smithi016 ceph-mon[31187]: pgmap v374: 129 pgs: 129 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-13T16:31:48.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:48 smithi111 ceph-mon[34600]: pgmap v375: 129 pgs: 129 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-13T16:31:49.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:48 smithi107 ceph-mon[34139]: pgmap v375: 129 pgs: 129 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-13T16:31:49.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:48 smithi016 ceph-mon[31187]: pgmap v375: 129 pgs: 129 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-13T16:31:50.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:50 smithi111 ceph-mon[34600]: pgmap v376: 129 pgs: 129 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-13T16:31:51.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:50 smithi107 ceph-mon[34139]: pgmap v376: 129 pgs: 129 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-13T16:31:51.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:50 smithi016 ceph-mon[31187]: pgmap v376: 129 pgs: 129 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-13T16:31:52.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:52 smithi111 ceph-mon[34600]: pgmap v377: 129 pgs: 129 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-13T16:31:53.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:52 smithi107 ceph-mon[34139]: pgmap v377: 129 pgs: 129 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-13T16:31:53.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:52 smithi016 ceph-mon[31187]: pgmap v377: 129 pgs: 129 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-13T16:31:54.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:54 smithi111 ceph-mon[34600]: pgmap v378: 129 pgs: 129 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-13T16:31:55.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:54 smithi107 ceph-mon[34139]: pgmap v378: 129 pgs: 129 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-13T16:31:55.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:54 smithi016 ceph-mon[31187]: pgmap v378: 129 pgs: 129 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-13T16:31:56.940 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:56 smithi111 ceph-mon[34600]: pgmap v379: 129 pgs: 129 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-13T16:31:57.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:56 smithi107 ceph-mon[34139]: pgmap v379: 129 pgs: 129 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-13T16:31:57.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:56 smithi016 ceph-mon[31187]: pgmap v379: 129 pgs: 129 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-13T16:31:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.533+0000 7f84f954c640 1 -- 172.21.15.16:0/3620341188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84f40ffe20 msgr2=0x7f84f41002a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.533+0000 7f84f954c640 1 --2- 172.21.15.16:0/3620341188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84f40ffe20 0x7f84f41002a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f84e8009900 tx=0x7f84e802a410 comp rx=0 tx=0).stop 2024-09-13T16:31:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.534+0000 7f84f954c640 1 -- 172.21.15.16:0/3620341188 shutdown_connections 2024-09-13T16:31:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.534+0000 7f84f954c640 1 --2- 172.21.15.16:0/3620341188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84f41007e0 0x7f84f4107070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.534+0000 7f84f954c640 1 --2- 172.21.15.16:0/3620341188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84f40ffe20 0x7f84f41002a0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.534+0000 7f84f954c640 1 --2- 172.21.15.16:0/3620341188 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84f40fec30 0x7f84f40ff030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.534+0000 7f84f954c640 1 -- 172.21.15.16:0/3620341188 >> 172.21.15.16:0/3620341188 conn(0x7f84f40fa3b0 msgr2=0x7f84f40fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.534+0000 7f84f954c640 1 -- 172.21.15.16:0/3620341188 shutdown_connections 2024-09-13T16:31:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.534+0000 7f84f954c640 1 -- 172.21.15.16:0/3620341188 wait complete. 2024-09-13T16:31:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.534+0000 7f84f954c640 1 Processor -- start 2024-09-13T16:31:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.534+0000 7f84f954c640 1 -- start start 2024-09-13T16:31:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.535+0000 7f84f954c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84f40fec30 0x7f84f4194ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.535+0000 7f84f954c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84f40ffe20 0x7f84f4195000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.535+0000 7f84f954c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84f41007e0 0x7f84f419c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.535+0000 7f84f954c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f419c5b0 con 0x7f84f41007e0 2024-09-13T16:31:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.535+0000 7f84f954c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f419c720 con 0x7f84f40ffe20 2024-09-13T16:31:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.535+0000 7f84f954c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f419c8e0 con 0x7f84f40fec30 2024-09-13T16:31:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.535+0000 7f84f37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84f41007e0 0x7f84f419c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.535+0000 7f84f37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84f41007e0 0x7f84f419c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45976/0 (socket says 172.21.15.16:45976) 2024-09-13T16:31:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.535+0000 7f84f37fe640 1 -- 172.21.15.16:0/2654751187 learned_addr learned my addr 172.21.15.16:0/2654751187 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.535+0000 7f84f27fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84f40ffe20 0x7f84f4195000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.535+0000 7f84f2ffd640 1 --2- 172.21.15.16:0/2654751187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84f40fec30 0x7f84f4194ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.535+0000 7f84f37fe640 1 -- 172.21.15.16:0/2654751187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84f40fec30 msgr2=0x7f84f4194ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.535+0000 7f84f37fe640 1 --2- 172.21.15.16:0/2654751187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84f40fec30 0x7f84f4194ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.535+0000 7f84f37fe640 1 -- 172.21.15.16:0/2654751187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84f40ffe20 msgr2=0x7f84f4195000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.535+0000 7f84f37fe640 1 --2- 172.21.15.16:0/2654751187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84f40ffe20 0x7f84f4195000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.535+0000 7f84f37fe640 1 -- 172.21.15.16:0/2654751187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e8009560 con 0x7f84f41007e0 2024-09-13T16:31:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.535+0000 7f84f27fc640 1 --2- 172.21.15.16:0/2654751187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84f40ffe20 0x7f84f4195000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:31:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.536+0000 7f84f37fe640 1 --2- 172.21.15.16:0/2654751187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84f41007e0 0x7f84f419c070 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00cad0 tx=0x7f84ec00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.536+0000 7f84dbfff640 1 -- 172.21.15.16:0/2654751187 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec00ae20 con 0x7f84f41007e0 2024-09-13T16:31:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.536+0000 7f84dbfff640 1 -- 172.21.15.16:0/2654751187 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f84ec005020 con 0x7f84f41007e0 2024-09-13T16:31:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.536+0000 7f84f954c640 1 -- 172.21.15.16:0/2654751187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f4067890 con 0x7f84f41007e0 2024-09-13T16:31:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.536+0000 7f84dbfff640 1 -- 172.21.15.16:0/2654751187 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec00f6b0 con 0x7f84f41007e0 2024-09-13T16:31:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.536+0000 7f84f954c640 1 -- 172.21.15.16:0/2654751187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f4067d70 con 0x7f84f41007e0 2024-09-13T16:31:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.539+0000 7f84fc26e740 1 Processor -- start 2024-09-13T16:31:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.539+0000 7f84dbfff640 1 -- 172.21.15.16:0/2654751187 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f84ec005190 con 0x7f84f41007e0 2024-09-13T16:31:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.539+0000 7f84fc26e740 1 -- start start 2024-09-13T16:31:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.540+0000 7f84fc26e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ecf08349e0 0x55ecf0834de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.540+0000 7f84fc26e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf08353b0 0x55ecf0839ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.540+0000 7f84fc26e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecf083a540 0x55ecf083c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.540+0000 7f84f2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf08353b0 0x55ecf0839ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.540+0000 7f84f2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf08353b0 0x55ecf0839ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45990/0 (socket says 172.21.15.16:45990) 2024-09-13T16:31:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.540+0000 7f84f2ffd640 1 -- 172.21.15.16:0/3074704478 learned_addr learned my addr 172.21.15.16:0/3074704478 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.540+0000 7f84f27fc640 1 --2- 172.21.15.16:0/3074704478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ecf08349e0 0x55ecf0834de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.540+0000 7f84f27fc640 1 --2- 172.21.15.16:0/3074704478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecf083a540 0x55ecf083c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.540+0000 7f84dbfff640 1 --2- 172.21.15.16:0/2654751187 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f84dc0a11a0 0x7f84dc0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.540+0000 7f84fc26e740 1 -- 172.21.15.16:0/3074704478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf0796c30 con 0x55ecf08353b0 2024-09-13T16:31:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.540+0000 7f84fc26e740 1 -- 172.21.15.16:0/3074704478 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf07943c0 con 0x55ecf083a540 2024-09-13T16:31:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.540+0000 7f84fc26e740 1 -- 172.21.15.16:0/3074704478 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf078fcb0 con 0x55ecf08349e0 2024-09-13T16:31:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.541+0000 7f84f37fe640 1 --2- 172.21.15.16:0/2654751187 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f84dc0a11a0 0x7f84dc0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.541+0000 7f84dbfff640 1 -- 172.21.15.16:0/2654751187 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f84ec0ccd10 con 0x7f84f41007e0 2024-09-13T16:31:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.541+0000 7f84f2ffd640 1 -- 172.21.15.16:0/3074704478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ecf08349e0 msgr2=0x55ecf0834de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.541+0000 7f84f2ffd640 1 --2- 172.21.15.16:0/3074704478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ecf08349e0 0x55ecf0834de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.541+0000 7f84f2ffd640 1 -- 172.21.15.16:0/3074704478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecf083a540 msgr2=0x55ecf083c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.541+0000 7f84f2ffd640 1 --2- 172.21.15.16:0/3074704478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecf083a540 0x55ecf083c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.541+0000 7f84f2ffd640 1 -- 172.21.15.16:0/3074704478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecf0736910 con 0x55ecf08353b0 2024-09-13T16:31:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.541+0000 7f84f27fc640 1 --2- 172.21.15.16:0/3074704478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecf083a540 0x55ecf083c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:31:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.541+0000 7f84f37fe640 1 --2- 172.21.15.16:0/2654751187 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f84dc0a11a0 0x7f84dc0a3660 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f84ec096260 tx=0x7f84ec08f500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.541+0000 7f84f2ffd640 1 --2- 172.21.15.16:0/3074704478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf08353b0 0x55ecf0839ed0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f84e400ac70 tx=0x7f84e400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d39c2da2193660 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.541+0000 7f84d97fa640 1 -- 172.21.15.16:0/3074704478 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e4005220 con 0x55ecf08353b0 2024-09-13T16:31:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.541+0000 7f84d97fa640 1 -- 172.21.15.16:0/3074704478 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f84e40047a0 con 0x55ecf08353b0 2024-09-13T16:31:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.541+0000 7f84d97fa640 1 -- 172.21.15.16:0/3074704478 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e4018680 con 0x55ecf08353b0 2024-09-13T16:31:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.542+0000 7f84fc26e740 1 -- 172.21.15.16:0/3074704478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf08353b0 msgr2=0x55ecf0839ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.542+0000 7f84fc26e740 1 --2- 172.21.15.16:0/3074704478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf08353b0 0x55ecf0839ed0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f84e400ac70 tx=0x7f84e400f790 comp rx=0 tx=0).stop 2024-09-13T16:31:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.542+0000 7f84fc26e740 1 -- 172.21.15.16:0/3074704478 shutdown_connections 2024-09-13T16:31:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.542+0000 7f84fc26e740 1 --2- 172.21.15.16:0/3074704478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecf083a540 0x55ecf083c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.542+0000 7f84fc26e740 1 --2- 172.21.15.16:0/3074704478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf08353b0 0x55ecf0839ed0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.542+0000 7f84fc26e740 1 --2- 172.21.15.16:0/3074704478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ecf08349e0 0x55ecf0834de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.542+0000 7f84fc26e740 1 -- 172.21.15.16:0/3074704478 >> 172.21.15.16:0/3074704478 conn(0x55ecf08278c0 msgr2=0x55ecf0827cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.542+0000 7f84fc26e740 1 -- 172.21.15.16:0/3074704478 shutdown_connections 2024-09-13T16:31:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.542+0000 7f84fc26e740 1 -- 172.21.15.16:0/3074704478 wait complete. 2024-09-13T16:31:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.543+0000 7f84fc26e740 1 -- start start 2024-09-13T16:31:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.543+0000 7f84fc26e740 1 Processor -- start 2024-09-13T16:31:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.543+0000 7f84fc26e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf08349e0 0x55ecf083e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.543+0000 7f84fc26e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecf08353b0 0x55ecf083ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.543+0000 7f84fc26e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ecf083a540 0x55ecf0845c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.543+0000 7f84fc26e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf074de80 con 0x55ecf08349e0 2024-09-13T16:31:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.543+0000 7f84fc26e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf074db20 con 0x55ecf08353b0 2024-09-13T16:31:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.543+0000 7f84fc26e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf0713260 con 0x55ecf083a540 2024-09-13T16:31:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.543+0000 7f84fc26e740 10 client.0 _ll_register_callbacks cb 0x55ecf0748df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:31:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.543+0000 7f84f27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf08349e0 0x55ecf083e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.543+0000 7f84f27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf08349e0 0x55ecf083e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45992/0 (socket says 172.21.15.16:45992) 2024-09-13T16:31:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.543+0000 7f84f27fc640 1 -- 172.21.15.16:0/3717895847 learned_addr learned my addr 172.21.15.16:0/3717895847 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.544+0000 7f84f2ffd640 1 --2- 172.21.15.16:0/3717895847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecf08353b0 0x55ecf083ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.544+0000 7f84f27fc640 1 -- 172.21.15.16:0/3717895847 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ecf083a540 msgr2=0x55ecf0845c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.544+0000 7f84f27fc640 1 --2- 172.21.15.16:0/3717895847 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ecf083a540 0x55ecf0845c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.544+0000 7f84f27fc640 1 -- 172.21.15.16:0/3717895847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecf08353b0 msgr2=0x55ecf083ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.544+0000 7f84f27fc640 1 --2- 172.21.15.16:0/3717895847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecf08353b0 0x55ecf083ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.544+0000 7f84f27fc640 1 -- 172.21.15.16:0/3717895847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e400a8a0 con 0x55ecf08349e0 2024-09-13T16:31:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.544+0000 7f84f2ffd640 1 --2- 172.21.15.16:0/3717895847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecf08353b0 0x55ecf083ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:31:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.544+0000 7f84f27fc640 1 --2- 172.21.15.16:0/3717895847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf08349e0 0x55ecf083e6b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f84e802a930 tx=0x7f84e802aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5720ea9c07930510 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.544+0000 7f84d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:31:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.544+0000 7f84d6ff5640 1 -- 172.21.15.16:0/3717895847 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e8031b60 con 0x55ecf08349e0 2024-09-13T16:31:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.544+0000 7f84fc26e740 1 -- 172.21.15.16:0/3717895847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ecf07b8680 con 0x55ecf08349e0 2024-09-13T16:31:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.544+0000 7f84d6ff5640 1 -- 172.21.15.16:0/3717895847 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f84e8031cc0 con 0x55ecf08349e0 2024-09-13T16:31:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.545+0000 7f84d6ff5640 1 -- 172.21.15.16:0/3717895847 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84e8031e80 con 0x55ecf08349e0 2024-09-13T16:31:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.545+0000 7f84fc26e740 10 client.15591 fetch_fsmap learned FSMap version 29 2024-09-13T16:31:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.545+0000 7f84fc26e740 1 -- 172.21.15.16:0/3717895847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ecf078e330 con 0x55ecf08349e0 2024-09-13T16:31:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.545+0000 7f84d6ff5640 1 -- 172.21.15.16:0/3717895847 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f84e80324d0 con 0x55ecf08349e0 2024-09-13T16:31:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.545+0000 7f84d6ff5640 1 -- 172.21.15.16:0/3717895847 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f84e8051070 con 0x55ecf08349e0 2024-09-13T16:31:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.545+0000 7f84fc26e740 10 client.15591 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T16:31:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.545+0000 7f84fc26e740 10 client.15591 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T16:31:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.545+0000 7f84fc26e740 20 client.15591 populate_metadata read hostname 'smithi016' 2024-09-13T16:31:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.545+0000 7f84fc26e740 1 --2- 172.21.15.16:0/3717895847 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55ecf084a220 0x55ecf084c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.545+0000 7f84fc26e740 4 client.15591 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T16:31:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.545+0000 7f84fc26e740 1 -- 172.21.15.16:0/3717895847 --> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ecf07b8680 con 0x55ecf084a220 2024-09-13T16:31:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.545+0000 7f84f2ffd640 1 --2- 172.21.15.16:0/3717895847 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55ecf084a220 0x55ecf084c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.546+0000 7f84f2ffd640 1 --2- 172.21.15.16:0/3717895847 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55ecf084a220 0x55ecf084c680 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67d83684286700eb server_cookie=6ead920f4febe0c1 in_seq=0 out_seq=0 2024-09-13T16:31:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.546+0000 7f84d6ff5640 10 client.15591 ms_handle_connect on v2:172.21.15.107:6838/1914689806 2024-09-13T16:31:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.547+0000 7f84d6ff5640 1 -- 172.21.15.16:0/3717895847 <== mds.0 v2:172.21.15.107:6838/1914689806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ecf07b8680 con 0x55ecf084a220 2024-09-13T16:31:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.547+0000 7f84d6ff5640 10 client.15591 handle_command_reply: tid=0 2024-09-13T16:31:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.547+0000 7f84fc26e740 1 client.15591 shutdown 2024-09-13T16:31:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.547+0000 7f84fc26e740 1 -- 172.21.15.16:0/3717895847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf08349e0 msgr2=0x55ecf083e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.547+0000 7f84fc26e740 1 --2- 172.21.15.16:0/3717895847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf08349e0 0x55ecf083e6b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f84e802a930 tx=0x7f84e802aa90 comp rx=0 tx=0).stop 2024-09-13T16:31:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.547+0000 7f84fc26e740 1 -- 172.21.15.16:0/3717895847 shutdown_connections 2024-09-13T16:31:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.547+0000 7f84fc26e740 1 --2- 172.21.15.16:0/3717895847 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55ecf084a220 0x55ecf084c680 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.547+0000 7f84fc26e740 1 --2- 172.21.15.16:0/3717895847 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ecf083a540 0x55ecf0845c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.547+0000 7f84fc26e740 1 --2- 172.21.15.16:0/3717895847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecf08353b0 0x55ecf083ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.547+0000 7f84fc26e740 1 --2- 172.21.15.16:0/3717895847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf08349e0 0x55ecf083e6b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.547+0000 7f84fc26e740 1 -- 172.21.15.16:0/3717895847 >> 172.21.15.16:0/3717895847 conn(0x55ecf08278c0 msgr2=0x55ecf0826b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.547+0000 7f84d6ff5640 0 client.15591 ms_handle_reset on v2:172.21.15.107:6838/1914689806 2024-09-13T16:31:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.547+0000 7f84fc26e740 1 -- 172.21.15.16:0/3717895847 shutdown_connections 2024-09-13T16:31:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.547+0000 7f84fc26e740 1 -- 172.21.15.16:0/3717895847 wait complete. 2024-09-13T16:31:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.547+0000 7f84fc26e740 20 client.15591 trim_cache size 0 max 16384 2024-09-13T16:31:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.572+0000 7f84fc26e740 1 Processor -- start 2024-09-13T16:31:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.572+0000 7f84fc26e740 1 -- start start 2024-09-13T16:31:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.572+0000 7f84fc26e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecf08349e0 0x55ecf0822a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.572+0000 7f84fc26e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf08353b0 0x55ecf0845ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.572+0000 7f84fc26e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ecf083a540 0x55ecf0822f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.572+0000 7f84fc26e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf08285b0 con 0x55ecf08353b0 2024-09-13T16:31:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.572+0000 7f84fc26e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf07b8680 con 0x55ecf08349e0 2024-09-13T16:31:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.572+0000 7f84fc26e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf083f410 con 0x55ecf083a540 2024-09-13T16:31:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.572+0000 7f84f2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf08353b0 0x55ecf0845ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.572+0000 7f84f2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf08353b0 0x55ecf0845ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46004/0 (socket says 172.21.15.16:46004) 2024-09-13T16:31:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.572+0000 7f84f2ffd640 1 -- 172.21.15.16:0/1774635456 learned_addr learned my addr 172.21.15.16:0/1774635456 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.572+0000 7f84f27fc640 1 --2- 172.21.15.16:0/1774635456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecf08349e0 0x55ecf0822a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.572+0000 7f84f27fc640 1 --2- 172.21.15.16:0/1774635456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ecf083a540 0x55ecf0822f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.572+0000 7f84f2ffd640 1 -- 172.21.15.16:0/1774635456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ecf083a540 msgr2=0x55ecf0822f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.572+0000 7f84f2ffd640 1 --2- 172.21.15.16:0/1774635456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ecf083a540 0x55ecf0822f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.572+0000 7f84f2ffd640 1 -- 172.21.15.16:0/1774635456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecf08349e0 msgr2=0x55ecf0822a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.572+0000 7f84f2ffd640 1 --2- 172.21.15.16:0/1774635456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecf08349e0 0x55ecf0822a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.572+0000 7f84f2ffd640 1 -- 172.21.15.16:0/1774635456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecf0730300 con 0x55ecf08353b0 2024-09-13T16:31:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.573+0000 7f84f2ffd640 1 --2- 172.21.15.16:0/1774635456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf08353b0 0x55ecf0845ba0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f84e4000c00 tx=0x7f84e400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33cfd9bc56b8c43 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.573+0000 7f84d9ffb640 1 -- 172.21.15.16:0/1774635456 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e4029040 con 0x55ecf08353b0 2024-09-13T16:31:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.573+0000 7f84d9ffb640 1 -- 172.21.15.16:0/1774635456 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f84e4004b70 con 0x55ecf08353b0 2024-09-13T16:31:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.573+0000 7f84d9ffb640 1 -- 172.21.15.16:0/1774635456 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e402a4a0 con 0x55ecf08353b0 2024-09-13T16:31:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.574+0000 7f84fc26e740 1 -- 172.21.15.16:0/1774635456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf08353b0 msgr2=0x55ecf0845ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.574+0000 7f84fc26e740 1 --2- 172.21.15.16:0/1774635456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf08353b0 0x55ecf0845ba0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f84e4000c00 tx=0x7f84e400f790 comp rx=0 tx=0).stop 2024-09-13T16:31:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.574+0000 7f84fc26e740 1 -- 172.21.15.16:0/1774635456 shutdown_connections 2024-09-13T16:31:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.574+0000 7f84fc26e740 1 --2- 172.21.15.16:0/1774635456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ecf083a540 0x55ecf0822f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.574+0000 7f84fc26e740 1 --2- 172.21.15.16:0/1774635456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf08353b0 0x55ecf0845ba0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.574+0000 7f84fc26e740 1 --2- 172.21.15.16:0/1774635456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecf08349e0 0x55ecf0822a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.574+0000 7f84fc26e740 1 -- 172.21.15.16:0/1774635456 >> 172.21.15.16:0/1774635456 conn(0x55ecf08278c0 msgr2=0x55ecf0836300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.574+0000 7f84fc26e740 1 -- 172.21.15.16:0/1774635456 shutdown_connections 2024-09-13T16:31:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.574+0000 7f84fc26e740 1 -- 172.21.15.16:0/1774635456 wait complete. 2024-09-13T16:31:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.575+0000 7f84fc26e740 1 -- start start 2024-09-13T16:31:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.575+0000 7f84fc26e740 1 Processor -- start 2024-09-13T16:31:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.575+0000 7f84fc26e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecf08349e0 0x55ecf0845600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.575+0000 7f84fc26e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf08353b0 0x55ecf0845b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.575+0000 7f84f2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf08353b0 0x55ecf0845b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.575+0000 7f84f2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf08353b0 0x55ecf0845b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46010/0 (socket says 172.21.15.16:46010) 2024-09-13T16:31:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.575+0000 7f84f2ffd640 1 -- 172.21.15.16:0/425096902 learned_addr learned my addr 172.21.15.16:0/425096902 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:31:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.575+0000 7f84f27fc640 1 --2- 172.21.15.16:0/425096902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecf08349e0 0x55ecf0845600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.575+0000 7f84fc26e740 1 --2- 172.21.15.16:0/425096902 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ecf083a540 0x55ecf0843d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.575+0000 7f84fc26e740 1 -- 172.21.15.16:0/425096902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf0844450 con 0x55ecf08353b0 2024-09-13T16:31:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.575+0000 7f84fc26e740 1 -- 172.21.15.16:0/425096902 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf08445c0 con 0x55ecf08349e0 2024-09-13T16:31:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.575+0000 7f84fc26e740 1 -- 172.21.15.16:0/425096902 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecf0844780 con 0x55ecf083a540 2024-09-13T16:31:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.576+0000 7f84fc26e740 10 client.0 _ll_register_callbacks cb 0x7f84e400a690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:31:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.576+0000 7f84f27fc640 1 -- 172.21.15.16:0/425096902 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ecf083a540 msgr2=0x55ecf0843d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.576+0000 7f84f27fc640 1 --2- 172.21.15.16:0/425096902 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ecf083a540 0x55ecf0843d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.576+0000 7f84f27fc640 1 -- 172.21.15.16:0/425096902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf08353b0 msgr2=0x55ecf0845b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:31:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.576+0000 7f84f27fc640 1 --2- 172.21.15.16:0/425096902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf08353b0 0x55ecf0845b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.576+0000 7f84f27fc640 1 -- 172.21.15.16:0/425096902 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ecf0736910 con 0x55ecf08349e0 2024-09-13T16:31:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.576+0000 7f84f2ffd640 1 --2- 172.21.15.16:0/425096902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf08353b0 0x55ecf0845b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:31:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.576+0000 7f84fc26e740 1 -- 172.21.15.16:0/425096902 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84e40047a0 con 0x55ecf08349e0 2024-09-13T16:31:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.576+0000 7f84f27fc640 1 --2- 172.21.15.16:0/425096902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecf08349e0 0x55ecf0845600 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f84e802aa00 tx=0x7f84e8005e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bda0fa4a30a52eb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:31:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.576+0000 7f84d6ff5640 10 client.25217 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-13T16:31:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.576+0000 7f84d6ff5640 1 -- 172.21.15.16:0/425096902 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e80057f0 con 0x55ecf08349e0 2024-09-13T16:31:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.576+0000 7f84d6ff5640 1 -- 172.21.15.16:0/425096902 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f84e8000e40 con 0x55ecf08349e0 2024-09-13T16:31:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.576+0000 7f84d6ff5640 1 -- 172.21.15.16:0/425096902 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84e8003800 con 0x55ecf08349e0 2024-09-13T16:31:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.577+0000 7f84fc26e740 10 client.25217 fetch_fsmap learned FSMap version 29 2024-09-13T16:31:59.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.577+0000 7f84fc26e740 1 -- 172.21.15.16:0/425096902 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x55ecf07558e0 con 0x55ecf08349e0 2024-09-13T16:31:59.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.577+0000 7f84d6ff5640 1 -- 172.21.15.16:0/425096902 <== mon.2 v2:172.21.15.107:3300/0 4 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f84e8051040 con 0x55ecf08349e0 2024-09-13T16:31:59.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.577+0000 7f84fc26e740 10 client.25217 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T16:31:59.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.577+0000 7f84fc26e740 10 client.25217 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T16:31:59.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.577+0000 7f84fc26e740 20 client.25217 populate_metadata read hostname 'smithi016' 2024-09-13T16:31:59.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.577+0000 7f84fc26e740 1 --2- 172.21.15.16:0/425096902 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55ecf08a1870 0x55ecf08a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:31:59.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.577+0000 7f84fc26e740 4 client.25217 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T16:31:59.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.577+0000 7f84fc26e740 1 -- 172.21.15.16:0/425096902 --> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ecf07835d0 con 0x55ecf08a1870 2024-09-13T16:31:59.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.577+0000 7f84f2ffd640 1 --2- 172.21.15.16:0/425096902 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55ecf08a1870 0x55ecf08a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:31:59.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.578+0000 7f84d6ff5640 10 client.25217 ms_handle_connect on v2:172.21.15.107:6838/1914689806 2024-09-13T16:31:59.586 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.578+0000 7f84f2ffd640 1 --2- 172.21.15.16:0/425096902 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55ecf08a1870 0x55ecf08a3c90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3a7e727d9d50f09 server_cookie=6dc3e8687a90608b in_seq=0 out_seq=0 2024-09-13T16:31:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.581+0000 7f84d6ff5640 1 -- 172.21.15.16:0/425096902 <== mds.0 v2:172.21.15.107:6838/1914689806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ecf07835d0 con 0x55ecf08a1870 2024-09-13T16:31:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.581+0000 7f84d6ff5640 10 client.25217 handle_command_reply: tid=0 2024-09-13T16:31:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.581+0000 7f84fc26e740 1 client.25217 shutdown 2024-09-13T16:31:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.582+0000 7f84fc26e740 1 -- 172.21.15.16:0/425096902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecf08349e0 msgr2=0x55ecf0845600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.582+0000 7f84fc26e740 1 --2- 172.21.15.16:0/425096902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecf08349e0 0x55ecf0845600 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f84e802aa00 tx=0x7f84e8005e30 comp rx=0 tx=0).stop 2024-09-13T16:31:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.582+0000 7f84fc26e740 1 -- 172.21.15.16:0/425096902 shutdown_connections 2024-09-13T16:31:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.582+0000 7f84fc26e740 1 --2- 172.21.15.16:0/425096902 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55ecf08a1870 0x55ecf08a3c90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.582+0000 7f84fc26e740 1 --2- 172.21.15.16:0/425096902 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ecf083a540 0x55ecf0843d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.582+0000 7f84fc26e740 1 --2- 172.21.15.16:0/425096902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf08353b0 0x55ecf0845b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.582+0000 7f84fc26e740 1 --2- 172.21.15.16:0/425096902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecf08349e0 0x55ecf0845600 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.582+0000 7f84fc26e740 1 -- 172.21.15.16:0/425096902 >> 172.21.15.16:0/425096902 conn(0x55ecf08278c0 msgr2=0x55ecf083e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.582+0000 7f84d6ff5640 0 client.25217 ms_handle_reset on v2:172.21.15.107:6838/1914689806 2024-09-13T16:31:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.582+0000 7f84fc26e740 1 -- 172.21.15.16:0/425096902 shutdown_connections 2024-09-13T16:31:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.582+0000 7f84fc26e740 1 -- 172.21.15.16:0/425096902 wait complete. 2024-09-13T16:31:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.583+0000 7f84fc26e740 20 client.25217 trim_cache size 0 max 16384 2024-09-13T16:31:59.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-13T16:31:59.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-13T16:31:59.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-13T16:31:59.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-13T16:31:59.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.583+0000 7f84f954c640 1 -- 172.21.15.16:0/2654751187 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f84dc0a11a0 msgr2=0x7f84dc0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:59.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.583+0000 7f84f954c640 1 --2- 172.21.15.16:0/2654751187 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f84dc0a11a0 0x7f84dc0a3660 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f84ec096260 tx=0x7f84ec08f500 comp rx=0 tx=0).stop 2024-09-13T16:31:59.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.583+0000 7f84f954c640 1 -- 172.21.15.16:0/2654751187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84f41007e0 msgr2=0x7f84f419c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:31:59.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.583+0000 7f84f954c640 1 --2- 172.21.15.16:0/2654751187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84f41007e0 0x7f84f419c070 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00cad0 tx=0x7f84ec00cf90 comp rx=0 tx=0).stop 2024-09-13T16:31:59.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.583+0000 7f84f954c640 1 -- 172.21.15.16:0/2654751187 shutdown_connections 2024-09-13T16:31:59.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.583+0000 7f84f954c640 1 --2- 172.21.15.16:0/2654751187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84f41007e0 0x7f84f419c070 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:59.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.583+0000 7f84f954c640 1 --2- 172.21.15.16:0/2654751187 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f84dc0a11a0 0x7f84dc0a3660 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:59.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.583+0000 7f84f954c640 1 --2- 172.21.15.16:0/2654751187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84f40ffe20 0x7f84f4195000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:59.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.583+0000 7f84f954c640 1 --2- 172.21.15.16:0/2654751187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84f40fec30 0x7f84f4194ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:31:59.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.583+0000 7f84f954c640 1 -- 172.21.15.16:0/2654751187 >> 172.21.15.16:0/2654751187 conn(0x7f84f40fa3b0 msgr2=0x7f84f41032e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:31:59.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.584+0000 7f84f954c640 1 -- 172.21.15.16:0/2654751187 shutdown_connections 2024-09-13T16:31:59.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:31:59.584+0000 7f84f954c640 1 -- 172.21.15.16:0/2654751187 wait complete. 2024-09-13T16:31:59.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:31:59 smithi111 ceph-mon[34600]: pgmap v380: 129 pgs: 129 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-13T16:32:00.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:31:59 smithi107 ceph-mon[34139]: pgmap v380: 129 pgs: 129 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-13T16:32:00.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:31:59 smithi016 ceph-mon[31187]: pgmap v380: 129 pgs: 129 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-13T16:32:01.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:01 smithi111 ceph-mon[34600]: pgmap v381: 129 pgs: 129 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-13T16:32:02.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:01 smithi107 ceph-mon[34139]: pgmap v381: 129 pgs: 129 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-13T16:32:02.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:01 smithi016 ceph-mon[31187]: pgmap v381: 129 pgs: 129 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-09-13T16:32:03.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:03 smithi111 ceph-mon[34600]: pgmap v382: 129 pgs: 129 active+clean; 673 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-13T16:32:04.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:03 smithi107 ceph-mon[34139]: pgmap v382: 129 pgs: 129 active+clean; 673 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-13T16:32:04.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:03 smithi016 ceph-mon[31187]: pgmap v382: 129 pgs: 129 active+clean; 673 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-13T16:32:05.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:05 smithi111 ceph-mon[34600]: pgmap v383: 129 pgs: 129 active+clean; 673 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-13T16:32:06.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:05 smithi107 ceph-mon[34139]: pgmap v383: 129 pgs: 129 active+clean; 673 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-13T16:32:06.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:05 smithi016 ceph-mon[31187]: pgmap v383: 129 pgs: 129 active+clean; 673 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-13T16:32:07.937 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:07 smithi111 ceph-mon[34600]: pgmap v384: 129 pgs: 129 active+clean; 673 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-13T16:32:08.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:07 smithi107 ceph-mon[34139]: pgmap v384: 129 pgs: 129 active+clean; 673 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-13T16:32:08.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:07 smithi016 ceph-mon[31187]: pgmap v384: 129 pgs: 129 active+clean; 673 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-13T16:32:09.027 INFO:tasks.workunit.client.0.smithi016.stderr:Updating files: 92% (11697/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-13T16:32:09.520 INFO:tasks.workunit.client.0.smithi016.stderr:Note: switching to '50bbe7f395997dcd871813949c49addbbf630361'. 2024-09-13T16:32:09.520 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-13T16:32:09.521 INFO:tasks.workunit.client.0.smithi016.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-13T16:32:09.521 INFO:tasks.workunit.client.0.smithi016.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-13T16:32:09.521 INFO:tasks.workunit.client.0.smithi016.stderr:state without impacting any branches by switching back to a branch. 2024-09-13T16:32:09.521 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-13T16:32:09.521 INFO:tasks.workunit.client.0.smithi016.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-13T16:32:09.521 INFO:tasks.workunit.client.0.smithi016.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-13T16:32:09.521 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-13T16:32:09.521 INFO:tasks.workunit.client.0.smithi016.stderr: git switch -c 2024-09-13T16:32:09.521 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-13T16:32:09.521 INFO:tasks.workunit.client.0.smithi016.stderr:Or undo this operation with: 2024-09-13T16:32:09.522 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-13T16:32:09.522 INFO:tasks.workunit.client.0.smithi016.stderr: git switch - 2024-09-13T16:32:09.522 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-13T16:32:09.522 INFO:tasks.workunit.client.0.smithi016.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-13T16:32:09.522 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-13T16:32:09.522 INFO:tasks.workunit.client.0.smithi016.stderr:HEAD is now at 50bbe7f3959 Merge PR #58789 into wip-rishabh-testing-20240910.095512-reef 2024-09-13T16:32:09.526 DEBUG:teuthology.orchestra.run.smithi016:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-13T16:32:09.587 INFO:tasks.workunit.client.0.smithi016.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-13T16:32:09.589 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-13T16:32:09.589 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-13T16:32:10.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:09 smithi107 ceph-mon[34139]: pgmap v385: 129 pgs: 129 active+clean; 673 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-13T16:32:10.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:09 smithi016 ceph-mon[31187]: pgmap v385: 129 pgs: 129 active+clean; 673 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-13T16:32:10.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:09 smithi111 ceph-mon[34600]: pgmap v385: 129 pgs: 129 active+clean; 673 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-13T16:32:10.307 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-13T16:32:10.381 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-13T16:32:10.423 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-13T16:32:10.425 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-13T16:32:10.425 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-13T16:32:10.465 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-13T16:32:10.468 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-13T16:32:10.469 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-13T16:32:10.526 INFO:tasks.workunit:Running workunits matching suites/iozone.sh on client.0... 2024-09-13T16:32:10.527 INFO:tasks.workunit:Running workunit suites/iozone.sh... 2024-09-13T16:32:10.528 DEBUG:teuthology.orchestra.run.smithi016:workunit test suites/iozone.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=50bbe7f395997dcd871813949c49addbbf630361 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh 2024-09-13T16:32:10.592 INFO:tasks.workunit.client.0.smithi016.stderr:+ iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-09-13T16:32:10.669 INFO:tasks.workunit.client.0.smithi016.stdout: Iozone: Performance Test of File I/O 2024-09-13T16:32:10.670 INFO:tasks.workunit.client.0.smithi016.stdout: Version $Revision: 3.494 $ 2024-09-13T16:32:10.670 INFO:tasks.workunit.client.0.smithi016.stdout: Compiled for 64 bit mode. 2024-09-13T16:32:10.670 INFO:tasks.workunit.client.0.smithi016.stdout: Build: linux-AMD64 2024-09-13T16:32:10.670 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-13T16:32:10.670 INFO:tasks.workunit.client.0.smithi016.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-09-13T16:32:10.670 INFO:tasks.workunit.client.0.smithi016.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-09-13T16:32:10.670 INFO:tasks.workunit.client.0.smithi016.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-09-13T16:32:10.670 INFO:tasks.workunit.client.0.smithi016.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-09-13T16:32:10.670 INFO:tasks.workunit.client.0.smithi016.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-09-13T16:32:10.670 INFO:tasks.workunit.client.0.smithi016.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-09-13T16:32:10.670 INFO:tasks.workunit.client.0.smithi016.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-09-13T16:32:10.670 INFO:tasks.workunit.client.0.smithi016.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-09-13T16:32:10.671 INFO:tasks.workunit.client.0.smithi016.stdout: Alexey Skidanov, Sudhir Kumar. 2024-09-13T16:32:10.671 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-13T16:32:10.671 INFO:tasks.workunit.client.0.smithi016.stdout: Run began: Fri Sep 13 16:32:10 2024 2024-09-13T16:32:10.671 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-13T16:32:10.671 INFO:tasks.workunit.client.0.smithi016.stdout: Include close in write timing 2024-09-13T16:32:10.671 INFO:tasks.workunit.client.0.smithi016.stdout: Include fsync in write timing 2024-09-13T16:32:10.671 INFO:tasks.workunit.client.0.smithi016.stdout: File size set to 1048576 kB 2024-09-13T16:32:10.671 INFO:tasks.workunit.client.0.smithi016.stdout: Record Size 16 kB 2024-09-13T16:32:10.671 INFO:tasks.workunit.client.0.smithi016.stdout: Command line used: iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-09-13T16:32:10.671 INFO:tasks.workunit.client.0.smithi016.stdout: Output is in kBytes/sec 2024-09-13T16:32:10.671 INFO:tasks.workunit.client.0.smithi016.stdout: Time Resolution = 0.000001 seconds. 2024-09-13T16:32:10.671 INFO:tasks.workunit.client.0.smithi016.stdout: Processor cache size set to 1024 kBytes. 2024-09-13T16:32:10.671 INFO:tasks.workunit.client.0.smithi016.stdout: Processor cache line size set to 32 bytes. 2024-09-13T16:32:10.671 INFO:tasks.workunit.client.0.smithi016.stdout: File stride size set to 17 * record size. 2024-09-13T16:32:10.672 INFO:tasks.workunit.client.0.smithi016.stdout: Throughput test with 1 process 2024-09-13T16:32:10.672 INFO:tasks.workunit.client.0.smithi016.stdout: Each process writes a 1048576 kByte file in 16 kByte records 2024-09-13T16:32:12.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:11 smithi107 ceph-mon[34139]: pgmap v386: 129 pgs: 129 active+clean; 673 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-13T16:32:12.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:11 smithi016 ceph-mon[31187]: pgmap v386: 129 pgs: 129 active+clean; 673 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-13T16:32:12.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:11 smithi111 ceph-mon[34600]: pgmap v386: 129 pgs: 129 active+clean; 673 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-13T16:32:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.687+0000 7fa9f0fae640 1 -- 172.21.15.16:0/2769846127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9ec1007c0 msgr2=0x7fa9ec107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.687+0000 7fa9f0fae640 1 --2- 172.21.15.16:0/2769846127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9ec1007c0 0x7fa9ec107050 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4009ca0 tx=0x7fa9e402a8c0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.687+0000 7fa9f0fae640 1 -- 172.21.15.16:0/2769846127 shutdown_connections 2024-09-13T16:32:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.687+0000 7fa9f0fae640 1 --2- 172.21.15.16:0/2769846127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9ec1007c0 0x7fa9ec107050 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.687+0000 7fa9f0fae640 1 --2- 172.21.15.16:0/2769846127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9ec0ffe00 0x7fa9ec100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.687+0000 7fa9f0fae640 1 --2- 172.21.15.16:0/2769846127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9ec0fec10 0x7fa9ec0ff010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.687+0000 7fa9f0fae640 1 -- 172.21.15.16:0/2769846127 >> 172.21.15.16:0/2769846127 conn(0x7fa9ec0fa370 msgr2=0x7fa9ec0fc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:32:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.687+0000 7fa9f0fae640 1 -- 172.21.15.16:0/2769846127 shutdown_connections 2024-09-13T16:32:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.687+0000 7fa9f0fae640 1 -- 172.21.15.16:0/2769846127 wait complete. 2024-09-13T16:32:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.688+0000 7fa9f0fae640 1 Processor -- start 2024-09-13T16:32:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.688+0000 7fa9f0fae640 1 -- start start 2024-09-13T16:32:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.689+0000 7fa9f0fae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9ec0fec10 0x7fa9ec069040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.689+0000 7fa9f0fae640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9ec0ffe00 0x7fa9ec19a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:12.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.689+0000 7fa9f0fae640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9ec1007c0 0x7fa9ec19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:12.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.689+0000 7fa9f0fae640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec19d6f0 con 0x7fa9ec0fec10 2024-09-13T16:32:12.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.689+0000 7fa9f0fae640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec19d860 con 0x7fa9ec0ffe00 2024-09-13T16:32:12.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.689+0000 7fa9f0fae640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec19da20 con 0x7fa9ec1007c0 2024-09-13T16:32:12.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.689+0000 7fa9e9d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9ec0ffe00 0x7fa9ec19a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:12.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.689+0000 7fa9ea59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9ec0fec10 0x7fa9ec069040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:12.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.689+0000 7fa9e9d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9ec0ffe00 0x7fa9ec19a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.16:59308/0 (socket says 172.21.15.16:59308) 2024-09-13T16:32:12.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.689+0000 7fa9e9d9b640 1 -- 172.21.15.16:0/3684613783 learned_addr learned my addr 172.21.15.16:0/3684613783 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:32:12.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.689+0000 7fa9ead9d640 1 --2- 172.21.15.16:0/3684613783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9ec1007c0 0x7fa9ec19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:12.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.689+0000 7fa9e9d9b640 1 -- 172.21.15.16:0/3684613783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9ec1007c0 msgr2=0x7fa9ec19cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:12.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.689+0000 7fa9e9d9b640 1 --2- 172.21.15.16:0/3684613783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9ec1007c0 0x7fa9ec19cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.689+0000 7fa9e9d9b640 1 -- 172.21.15.16:0/3684613783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9ec0fec10 msgr2=0x7fa9ec069040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:12.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.689+0000 7fa9e9d9b640 1 --2- 172.21.15.16:0/3684613783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9ec0fec10 0x7fa9ec069040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.689+0000 7fa9e9d9b640 1 -- 172.21.15.16:0/3684613783 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e0009490 con 0x7fa9ec0ffe00 2024-09-13T16:32:12.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.689+0000 7fa9e9d9b640 1 --2- 172.21.15.16:0/3684613783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9ec0ffe00 0x7fa9ec19a8f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa9e000ea60 tx=0x7fa9e000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:32:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.692+0000 7fa9db7fe640 1 -- 172.21.15.16:0/3684613783 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e000cde0 con 0x7fa9ec0ffe00 2024-09-13T16:32:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.692+0000 7fa9db7fe640 1 -- 172.21.15.16:0/3684613783 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa9e00044b0 con 0x7fa9ec0ffe00 2024-09-13T16:32:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.692+0000 7fa9db7fe640 1 -- 172.21.15.16:0/3684613783 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0010430 con 0x7fa9ec0ffe00 2024-09-13T16:32:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.692+0000 7fa9f0fae640 1 -- 172.21.15.16:0/3684613783 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e4009900 con 0x7fa9ec0ffe00 2024-09-13T16:32:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.692+0000 7fa9f0fae640 1 -- 172.21.15.16:0/3684613783 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9ec194aa0 con 0x7fa9ec0ffe00 2024-09-13T16:32:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.694+0000 7fa9db7fe640 1 -- 172.21.15.16:0/3684613783 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fa9e0004020 con 0x7fa9ec0ffe00 2024-09-13T16:32:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.695+0000 7fa9db7fe640 1 --2- 172.21.15.16:0/3684613783 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa9d40a11a0 0x7fa9d40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.695+0000 7fa9ea59c640 1 --2- 172.21.15.16:0/3684613783 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa9d40a11a0 0x7fa9d40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.695+0000 7fa9db7fe640 1 -- 172.21.15.16:0/3684613783 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fa9e0014070 con 0x7fa9ec0ffe00 2024-09-13T16:32:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.695+0000 7fa9f3cd2740 1 Processor -- start 2024-09-13T16:32:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.695+0000 7fa9f3cd2740 1 -- start start 2024-09-13T16:32:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.695+0000 7fa9ea59c640 1 --2- 172.21.15.16:0/3684613783 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa9d40a11a0 0x7fa9d40a3660 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec0ffc60 tx=0x7fa9dc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:32:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.695+0000 7fa9f3cd2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c13a7531d0 0x55c13a7535d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.695+0000 7fa9f3cd2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c13a753ba0 0x55c13a7586c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.695+0000 7fa9f3cd2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c13a758d30 0x55c13a75b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.695+0000 7fa9f3cd2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c13a6bdc30 con 0x55c13a758d30 2024-09-13T16:32:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.695+0000 7fa9f3cd2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c13a6bb3c0 con 0x55c13a7531d0 2024-09-13T16:32:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.695+0000 7fa9f3cd2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c13a6b6cb0 con 0x55c13a753ba0 2024-09-13T16:32:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.696+0000 7fa9ead9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c13a758d30 0x55c13a75b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.696+0000 7fa9ead9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c13a758d30 0x55c13a75b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45490/0 (socket says 172.21.15.16:45490) 2024-09-13T16:32:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.696+0000 7fa9ead9d640 1 -- 172.21.15.16:0/3898953135 learned_addr learned my addr 172.21.15.16:0/3898953135 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:32:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.696+0000 7fa9ea59c640 1 --2- 172.21.15.16:0/3898953135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c13a7531d0 0x55c13a7535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.696+0000 7fa9e9d9b640 1 --2- 172.21.15.16:0/3898953135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c13a753ba0 0x55c13a7586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.696+0000 7fa9ead9d640 1 -- 172.21.15.16:0/3898953135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c13a753ba0 msgr2=0x55c13a7586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:32:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.696+0000 7fa9ead9d640 1 --2- 172.21.15.16:0/3898953135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c13a753ba0 0x55c13a7586c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.696+0000 7fa9ead9d640 1 -- 172.21.15.16:0/3898953135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c13a7531d0 msgr2=0x55c13a7535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:32:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.696+0000 7fa9ead9d640 1 --2- 172.21.15.16:0/3898953135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c13a7531d0 0x55c13a7535d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.696+0000 7fa9ead9d640 1 -- 172.21.15.16:0/3898953135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c13a65d910 con 0x55c13a758d30 2024-09-13T16:32:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.696+0000 7fa9e9d9b640 1 --2- 172.21.15.16:0/3898953135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c13a753ba0 0x55c13a7586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:32:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.696+0000 7fa9ead9d640 1 --2- 172.21.15.16:0/3898953135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c13a758d30 0x55c13a75b1c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4009ca0 tx=0x7fa9e40093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a1c45e083f3503 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:32:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.696+0000 7fa9d8ff9640 1 -- 172.21.15.16:0/3898953135 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4031db0 con 0x55c13a758d30 2024-09-13T16:32:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.696+0000 7fa9d8ff9640 1 -- 172.21.15.16:0/3898953135 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa9e402f310 con 0x55c13a758d30 2024-09-13T16:32:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.696+0000 7fa9d8ff9640 1 -- 172.21.15.16:0/3898953135 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e40307e0 con 0x55c13a758d30 2024-09-13T16:32:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.697+0000 7fa9f3cd2740 1 -- 172.21.15.16:0/3898953135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c13a758d30 msgr2=0x55c13a75b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.697+0000 7fa9f3cd2740 1 --2- 172.21.15.16:0/3898953135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c13a758d30 0x55c13a75b1c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4009ca0 tx=0x7fa9e40093e0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.697+0000 7fa9f3cd2740 1 -- 172.21.15.16:0/3898953135 shutdown_connections 2024-09-13T16:32:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.697+0000 7fa9f3cd2740 1 --2- 172.21.15.16:0/3898953135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c13a758d30 0x55c13a75b1c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.697+0000 7fa9f3cd2740 1 --2- 172.21.15.16:0/3898953135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c13a753ba0 0x55c13a7586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.697+0000 7fa9f3cd2740 1 --2- 172.21.15.16:0/3898953135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c13a7531d0 0x55c13a7535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.697+0000 7fa9f3cd2740 1 -- 172.21.15.16:0/3898953135 >> 172.21.15.16:0/3898953135 conn(0x55c13a74e8c0 msgr2=0x55c13a74ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:32:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.697+0000 7fa9f3cd2740 1 -- 172.21.15.16:0/3898953135 shutdown_connections 2024-09-13T16:32:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.697+0000 7fa9f3cd2740 1 -- 172.21.15.16:0/3898953135 wait complete. 2024-09-13T16:32:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.697+0000 7fa9f3cd2740 1 -- start start 2024-09-13T16:32:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.698+0000 7fa9f3cd2740 1 Processor -- start 2024-09-13T16:32:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.698+0000 7fa9f3cd2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c13a7531d0 0x55c13a75cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.698+0000 7fa9f3cd2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c13a753ba0 0x55c13a75d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.698+0000 7fa9f3cd2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c13a758d30 0x55c13a7643d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.698+0000 7fa9f3cd2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c13a674e80 con 0x55c13a7531d0 2024-09-13T16:32:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.698+0000 7fa9f3cd2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c13a674b20 con 0x55c13a758d30 2024-09-13T16:32:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.698+0000 7fa9f3cd2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c13a63a260 con 0x55c13a753ba0 2024-09-13T16:32:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.698+0000 7fa9f3cd2740 10 client.0 _ll_register_callbacks cb 0x55c13a66fdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:32:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.698+0000 7fa9ea59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c13a7531d0 0x55c13a75cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.698+0000 7fa9ea59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c13a7531d0 0x55c13a75cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45498/0 (socket says 172.21.15.16:45498) 2024-09-13T16:32:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.698+0000 7fa9ea59c640 1 -- 172.21.15.16:0/3137323407 learned_addr learned my addr 172.21.15.16:0/3137323407 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:32:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.698+0000 7fa9e9d9b640 1 --2- 172.21.15.16:0/3137323407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c13a753ba0 0x55c13a75d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.698+0000 7fa9ead9d640 1 --2- 172.21.15.16:0/3137323407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c13a758d30 0x55c13a7643d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.698+0000 7fa9ead9d640 1 -- 172.21.15.16:0/3137323407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c13a753ba0 msgr2=0x55c13a75d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:32:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.698+0000 7fa9ead9d640 1 --2- 172.21.15.16:0/3137323407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c13a753ba0 0x55c13a75d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.698+0000 7fa9ead9d640 1 -- 172.21.15.16:0/3137323407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c13a7531d0 msgr2=0x55c13a75cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:32:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.698+0000 7fa9ead9d640 1 --2- 172.21.15.16:0/3137323407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c13a7531d0 0x55c13a75cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.698+0000 7fa9ead9d640 1 -- 172.21.15.16:0/3137323407 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9ec107950 con 0x55c13a758d30 2024-09-13T16:32:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.698+0000 7fa9e9d9b640 1 --2- 172.21.15.16:0/3137323407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c13a753ba0 0x55c13a75d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:32:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.698+0000 7fa9ea59c640 1 --2- 172.21.15.16:0/3137323407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c13a7531d0 0x55c13a75cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:32:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.699+0000 7fa9ead9d640 1 --2- 172.21.15.16:0/3137323407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c13a758d30 0x55c13a7643d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa9e40319e0 tx=0x7fa9e4003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c8d6e522058877a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:32:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.699+0000 7fa9d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-13T16:32:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.699+0000 7fa9d1ffb640 1 -- 172.21.15.16:0/3137323407 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4052070 con 0x55c13a758d30 2024-09-13T16:32:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.699+0000 7fa9d1ffb640 1 -- 172.21.15.16:0/3137323407 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fa9e40305c0 con 0x55c13a758d30 2024-09-13T16:32:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.699+0000 7fa9f3cd2740 1 -- 172.21.15.16:0/3137323407 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c13a6df680 con 0x55c13a758d30 2024-09-13T16:32:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.699+0000 7fa9d1ffb640 1 -- 172.21.15.16:0/3137323407 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9e4030780 con 0x55c13a758d30 2024-09-13T16:32:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.699+0000 7fa9f3cd2740 10 client.25229 fetch_fsmap learned FSMap version 29 2024-09-13T16:32:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.699+0000 7fa9f3cd2740 1 -- 172.21.15.16:0/3137323407 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c13a6b5330 con 0x55c13a758d30 2024-09-13T16:32:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.699+0000 7fa9d1ffb640 1 -- 172.21.15.16:0/3137323407 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c13a6df680 con 0x55c13a758d30 2024-09-13T16:32:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.699+0000 7fa9d1ffb640 1 -- 172.21.15.16:0/3137323407 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa9e4055020 con 0x55c13a758d30 2024-09-13T16:32:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.699+0000 7fa9f3cd2740 10 client.25229 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T16:32:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.699+0000 7fa9f3cd2740 10 client.25229 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T16:32:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.699+0000 7fa9f3cd2740 20 client.25229 populate_metadata read hostname 'smithi016' 2024-09-13T16:32:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.699+0000 7fa9f3cd2740 1 --2- 172.21.15.16:0/3137323407 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55c13a7688b0 0x55c13a76aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.699+0000 7fa9f3cd2740 4 client.25229 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T16:32:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.699+0000 7fa9f3cd2740 1 -- 172.21.15.16:0/3137323407 --> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c13a6aa5d0 con 0x55c13a7688b0 2024-09-13T16:32:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.700+0000 7fa9ea59c640 1 --2- 172.21.15.16:0/3137323407 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55c13a7688b0 0x55c13a76aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.700+0000 7fa9ea59c640 1 --2- 172.21.15.16:0/3137323407 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55c13a7688b0 0x55c13a76aca0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60be570bdd91b74d server_cookie=5978674ff9ad2f7f in_seq=0 out_seq=0 2024-09-13T16:32:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.700+0000 7fa9d1ffb640 10 client.25229 ms_handle_connect on v2:172.21.15.107:6838/1914689806 2024-09-13T16:32:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.701+0000 7fa9d1ffb640 1 -- 172.21.15.16:0/3137323407 <== mds.0 v2:172.21.15.107:6838/1914689806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c13a6aa5d0 con 0x55c13a7688b0 2024-09-13T16:32:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.701+0000 7fa9d1ffb640 10 client.25229 handle_command_reply: tid=0 2024-09-13T16:32:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.701+0000 7fa9f3cd2740 1 client.25229 shutdown 2024-09-13T16:32:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.701+0000 7fa9f3cd2740 1 -- 172.21.15.16:0/3137323407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c13a758d30 msgr2=0x55c13a7643d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.701+0000 7fa9f3cd2740 1 --2- 172.21.15.16:0/3137323407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c13a758d30 0x55c13a7643d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa9e40319e0 tx=0x7fa9e4003c10 comp rx=0 tx=0).stop 2024-09-13T16:32:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.701+0000 7fa9f3cd2740 1 -- 172.21.15.16:0/3137323407 shutdown_connections 2024-09-13T16:32:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.701+0000 7fa9f3cd2740 1 --2- 172.21.15.16:0/3137323407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c13a758d30 0x55c13a7643d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.701+0000 7fa9f3cd2740 1 --2- 172.21.15.16:0/3137323407 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55c13a7688b0 0x55c13a76aca0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.701+0000 7fa9f3cd2740 1 --2- 172.21.15.16:0/3137323407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c13a753ba0 0x55c13a75d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.701+0000 7fa9f3cd2740 1 --2- 172.21.15.16:0/3137323407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c13a7531d0 0x55c13a75cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.701+0000 7fa9d1ffb640 0 client.25229 ms_handle_reset on v2:172.21.15.107:6838/1914689806 2024-09-13T16:32:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.701+0000 7fa9f3cd2740 1 -- 172.21.15.16:0/3137323407 >> 172.21.15.16:0/3137323407 conn(0x55c13a74e8c0 msgr2=0x55c13a74db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:32:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.701+0000 7fa9f3cd2740 1 -- 172.21.15.16:0/3137323407 shutdown_connections 2024-09-13T16:32:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.701+0000 7fa9f3cd2740 1 -- 172.21.15.16:0/3137323407 wait complete. 2024-09-13T16:32:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.701+0000 7fa9f3cd2740 20 client.25229 trim_cache size 0 max 16384 2024-09-13T16:32:12.722 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.714+0000 7fa9f3cd2740 1 Processor -- start 2024-09-13T16:32:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.714+0000 7fa9f3cd2740 1 -- start start 2024-09-13T16:32:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.714+0000 7fa9f3cd2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c13a7531d0 0x55c13a74e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.714+0000 7fa9f3cd2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c13a753ba0 0x55c13a764390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.714+0000 7fa9f3cd2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c13a758d30 0x55c13a74eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.714+0000 7fa9f3cd2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c13a7543e0 con 0x55c13a758d30 2024-09-13T16:32:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.714+0000 7fa9f3cd2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c13a66fd10 con 0x55c13a753ba0 2024-09-13T16:32:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.714+0000 7fa9f3cd2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c13a75ca80 con 0x55c13a7531d0 2024-09-13T16:32:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.714+0000 7fa9ead9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c13a758d30 0x55c13a74eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.714+0000 7fa9e9d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c13a753ba0 0x55c13a764390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.714+0000 7fa9ead9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c13a758d30 0x55c13a74eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45510/0 (socket says 172.21.15.16:45510) 2024-09-13T16:32:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.714+0000 7fa9ead9d640 1 -- 172.21.15.16:0/1474396161 learned_addr learned my addr 172.21.15.16:0/1474396161 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:32:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.714+0000 7fa9ead9d640 1 -- 172.21.15.16:0/1474396161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c13a7531d0 msgr2=0x55c13a74e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:32:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.714+0000 7fa9ead9d640 1 --2- 172.21.15.16:0/1474396161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c13a7531d0 0x55c13a74e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.714+0000 7fa9ead9d640 1 -- 172.21.15.16:0/1474396161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c13a753ba0 msgr2=0x55c13a764390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:32:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.714+0000 7fa9ead9d640 1 --2- 172.21.15.16:0/1474396161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c13a753ba0 0x55c13a764390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.715+0000 7fa9ead9d640 1 -- 172.21.15.16:0/1474396161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c13a657300 con 0x55c13a758d30 2024-09-13T16:32:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.715+0000 7fa9e9d9b640 1 --2- 172.21.15.16:0/1474396161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c13a753ba0 0x55c13a764390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:32:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.715+0000 7fa9ead9d640 1 --2- 172.21.15.16:0/1474396161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c13a758d30 0x55c13a74eca0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4000f80 tx=0x7fa9e402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ebec185002d6dd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:32:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.715+0000 7fa9d97fa640 1 -- 172.21.15.16:0/1474396161 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4052070 con 0x55c13a758d30 2024-09-13T16:32:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.715+0000 7fa9d97fa640 1 -- 172.21.15.16:0/1474396161 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa9e4005cf0 con 0x55c13a758d30 2024-09-13T16:32:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.715+0000 7fa9d97fa640 1 -- 172.21.15.16:0/1474396161 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4004290 con 0x55c13a758d30 2024-09-13T16:32:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.716+0000 7fa9f3cd2740 1 -- 172.21.15.16:0/1474396161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c13a758d30 msgr2=0x55c13a74eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.716+0000 7fa9f3cd2740 1 --2- 172.21.15.16:0/1474396161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c13a758d30 0x55c13a74eca0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4000f80 tx=0x7fa9e402a8c0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.716+0000 7fa9f3cd2740 1 -- 172.21.15.16:0/1474396161 shutdown_connections 2024-09-13T16:32:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.716+0000 7fa9f3cd2740 1 --2- 172.21.15.16:0/1474396161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c13a758d30 0x55c13a74eca0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.716+0000 7fa9f3cd2740 1 --2- 172.21.15.16:0/1474396161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c13a753ba0 0x55c13a764390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.716+0000 7fa9f3cd2740 1 --2- 172.21.15.16:0/1474396161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c13a7531d0 0x55c13a74e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.716+0000 7fa9f3cd2740 1 -- 172.21.15.16:0/1474396161 >> 172.21.15.16:0/1474396161 conn(0x55c13a74e8c0 msgr2=0x55c13a758170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:32:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.716+0000 7fa9f3cd2740 1 -- 172.21.15.16:0/1474396161 shutdown_connections 2024-09-13T16:32:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.716+0000 7fa9f3cd2740 1 -- 172.21.15.16:0/1474396161 wait complete. 2024-09-13T16:32:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.716+0000 7fa9f3cd2740 1 -- start start 2024-09-13T16:32:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.717+0000 7fa9f3cd2740 1 Processor -- start 2024-09-13T16:32:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.717+0000 7fa9f3cd2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c13a7531d0 0x55c13a763cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.717+0000 7fa9f3cd2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c13a753ba0 0x55c13a764200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.717+0000 7fa9f3cd2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c13a758d30 0x55c13a762320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.717+0000 7fa9f3cd2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c13a762a20 con 0x55c13a758d30 2024-09-13T16:32:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.717+0000 7fa9f3cd2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c13a762b90 con 0x55c13a7531d0 2024-09-13T16:32:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.717+0000 7fa9f3cd2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c13a762d50 con 0x55c13a753ba0 2024-09-13T16:32:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.717+0000 7fa9f3cd2740 10 client.0 _ll_register_callbacks cb 0x7fa9e0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:32:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.717+0000 7fa9ea59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c13a7531d0 0x55c13a763cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.717+0000 7fa9ea59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c13a7531d0 0x55c13a763cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.16:59350/0 (socket says 172.21.15.16:59350) 2024-09-13T16:32:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.717+0000 7fa9ea59c640 1 -- 172.21.15.16:0/2021738282 learned_addr learned my addr 172.21.15.16:0/2021738282 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:32:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.717+0000 7fa9e9d9b640 1 --2- 172.21.15.16:0/2021738282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c13a753ba0 0x55c13a764200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.717+0000 7fa9ead9d640 1 --2- 172.21.15.16:0/2021738282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c13a758d30 0x55c13a762320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.718+0000 7fa9ea59c640 1 -- 172.21.15.16:0/2021738282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c13a753ba0 msgr2=0x55c13a764200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:32:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.718+0000 7fa9ea59c640 1 --2- 172.21.15.16:0/2021738282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c13a753ba0 0x55c13a764200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.718+0000 7fa9ea59c640 1 -- 172.21.15.16:0/2021738282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c13a758d30 msgr2=0x55c13a762320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:32:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.718+0000 7fa9ea59c640 1 --2- 172.21.15.16:0/2021738282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c13a758d30 0x55c13a762320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.718+0000 7fa9ea59c640 1 -- 172.21.15.16:0/2021738282 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9ec194aa0 con 0x55c13a7531d0 2024-09-13T16:32:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.718+0000 7fa9e9d9b640 1 --2- 172.21.15.16:0/2021738282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c13a753ba0 0x55c13a764200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:32:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.718+0000 7fa9ea59c640 1 --2- 172.21.15.16:0/2021738282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c13a7531d0 0x55c13a763cc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x55c13a76b520 tx=0x7fa9dc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1258c041b809ad8d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:32:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.718+0000 7fa9d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-13T16:32:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.718+0000 7fa9d1ffb640 1 -- 172.21.15.16:0/2021738282 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc022040 con 0x55c13a7531d0 2024-09-13T16:32:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.718+0000 7fa9d1ffb640 1 -- 172.21.15.16:0/2021738282 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fa9dc01dc70 con 0x55c13a7531d0 2024-09-13T16:32:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.718+0000 7fa9f3cd2740 1 -- 172.21.15.16:0/2021738282 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9e402f310 con 0x55c13a7531d0 2024-09-13T16:32:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.718+0000 7fa9d1ffb640 1 -- 172.21.15.16:0/2021738282 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9dc01de30 con 0x55c13a7531d0 2024-09-13T16:32:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.718+0000 7fa9f3cd2740 10 client.25235 fetch_fsmap learned FSMap version 29 2024-09-13T16:32:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.718+0000 7fa9f3cd2740 1 -- 172.21.15.16:0/2021738282 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c13a6b5330 con 0x55c13a7531d0 2024-09-13T16:32:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.719+0000 7fa9d1ffb640 1 -- 172.21.15.16:0/2021738282 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa9e402f310 con 0x55c13a7531d0 2024-09-13T16:32:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.719+0000 7fa9d1ffb640 1 -- 172.21.15.16:0/2021738282 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa9dc034690 con 0x55c13a7531d0 2024-09-13T16:32:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.719+0000 7fa9f3cd2740 10 client.25235 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T16:32:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.719+0000 7fa9f3cd2740 10 client.25235 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T16:32:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.719+0000 7fa9f3cd2740 20 client.25235 populate_metadata read hostname 'smithi016' 2024-09-13T16:32:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.719+0000 7fa9f3cd2740 1 --2- 172.21.15.16:0/2021738282 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55c13a7c0200 0x55c13a7c2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.719+0000 7fa9f3cd2740 4 client.25235 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T16:32:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.719+0000 7fa9f3cd2740 1 -- 172.21.15.16:0/2021738282 --> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c13a3aa300 con 0x55c13a7c0200 2024-09-13T16:32:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.719+0000 7fa9e9d9b640 1 --2- 172.21.15.16:0/2021738282 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55c13a7c0200 0x55c13a7c2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.719+0000 7fa9d1ffb640 10 client.25235 ms_handle_connect on v2:172.21.15.107:6838/1914689806 2024-09-13T16:32:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.719+0000 7fa9e9d9b640 1 --2- 172.21.15.16:0/2021738282 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55c13a7c0200 0x55c13a7c2620 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efb429ea7b8ae54c server_cookie=1bb35cf44a6a5a3e in_seq=0 out_seq=0 2024-09-13T16:32:12.729 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.726+0000 7fa9d1ffb640 1 -- 172.21.15.16:0/2021738282 <== mds.0 v2:172.21.15.107:6838/1914689806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c13a3aa300 con 0x55c13a7c0200 2024-09-13T16:32:12.729 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.726+0000 7fa9d1ffb640 10 client.25235 handle_command_reply: tid=0 2024-09-13T16:32:12.729 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.726+0000 7fa9f3cd2740 1 client.25235 shutdown 2024-09-13T16:32:12.729 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.727+0000 7fa9f3cd2740 1 -- 172.21.15.16:0/2021738282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c13a7531d0 msgr2=0x55c13a763cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:12.729 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.727+0000 7fa9f3cd2740 1 --2- 172.21.15.16:0/2021738282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c13a7531d0 0x55c13a763cc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x55c13a76b520 tx=0x7fa9dc0046c0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.727+0000 7fa9f3cd2740 1 -- 172.21.15.16:0/2021738282 shutdown_connections 2024-09-13T16:32:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.727+0000 7fa9f3cd2740 1 --2- 172.21.15.16:0/2021738282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c13a758d30 0x55c13a762320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.727+0000 7fa9f3cd2740 1 --2- 172.21.15.16:0/2021738282 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55c13a7c0200 0x55c13a7c2620 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.727+0000 7fa9f3cd2740 1 --2- 172.21.15.16:0/2021738282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c13a753ba0 0x55c13a764200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.727+0000 7fa9f3cd2740 1 --2- 172.21.15.16:0/2021738282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c13a7531d0 0x55c13a763cc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.727+0000 7fa9d1ffb640 0 client.25235 ms_handle_reset on v2:172.21.15.107:6838/1914689806 2024-09-13T16:32:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.727+0000 7fa9f3cd2740 1 -- 172.21.15.16:0/2021738282 >> 172.21.15.16:0/2021738282 conn(0x55c13a74e8c0 msgr2=0x55c13a754b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:32:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.727+0000 7fa9f3cd2740 1 -- 172.21.15.16:0/2021738282 shutdown_connections 2024-09-13T16:32:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.727+0000 7fa9f3cd2740 1 -- 172.21.15.16:0/2021738282 wait complete. 2024-09-13T16:32:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.727+0000 7fa9f3cd2740 20 client.25235 trim_cache size 0 max 16384 2024-09-13T16:32:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-13T16:32:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-13T16:32:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-13T16:32:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-13T16:32:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.727+0000 7fa9f0fae640 1 -- 172.21.15.16:0/3684613783 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa9d40a11a0 msgr2=0x7fa9d40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.727+0000 7fa9f0fae640 1 --2- 172.21.15.16:0/3684613783 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa9d40a11a0 0x7fa9d40a3660 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec0ffc60 tx=0x7fa9dc005f90 comp rx=0 tx=0).stop 2024-09-13T16:32:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.727+0000 7fa9f0fae640 1 -- 172.21.15.16:0/3684613783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9ec0ffe00 msgr2=0x7fa9ec19a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.727+0000 7fa9f0fae640 1 --2- 172.21.15.16:0/3684613783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9ec0ffe00 0x7fa9ec19a8f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa9e000ea60 tx=0x7fa9e000ef20 comp rx=0 tx=0).stop 2024-09-13T16:32:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.727+0000 7fa9f0fae640 1 -- 172.21.15.16:0/3684613783 shutdown_connections 2024-09-13T16:32:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.728+0000 7fa9f0fae640 1 --2- 172.21.15.16:0/3684613783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9ec1007c0 0x7fa9ec19cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.728+0000 7fa9f0fae640 1 --2- 172.21.15.16:0/3684613783 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa9d40a11a0 0x7fa9d40a3660 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.728+0000 7fa9f0fae640 1 --2- 172.21.15.16:0/3684613783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9ec0ffe00 0x7fa9ec19a8f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.728+0000 7fa9f0fae640 1 --2- 172.21.15.16:0/3684613783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9ec0fec10 0x7fa9ec069040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.728+0000 7fa9f0fae640 1 -- 172.21.15.16:0/3684613783 >> 172.21.15.16:0/3684613783 conn(0x7fa9ec0fa370 msgr2=0x7fa9ec1032c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:32:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.728+0000 7fa9f0fae640 1 -- 172.21.15.16:0/3684613783 shutdown_connections 2024-09-13T16:32:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:12.728+0000 7fa9f0fae640 1 -- 172.21.15.16:0/3684613783 wait complete. 2024-09-13T16:32:14.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:13 smithi107 ceph-mon[34139]: pgmap v387: 129 pgs: 129 active+clean; 41 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.3 MiB/s wr, 12 op/s 2024-09-13T16:32:14.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:13 smithi016 ceph-mon[31187]: pgmap v387: 129 pgs: 129 active+clean; 41 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.3 MiB/s wr, 12 op/s 2024-09-13T16:32:14.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:13 smithi111 ceph-mon[34600]: pgmap v387: 129 pgs: 129 active+clean; 41 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.3 MiB/s wr, 12 op/s 2024-09-13T16:32:16.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:15 smithi107 ceph-mon[34139]: pgmap v388: 129 pgs: 129 active+clean; 153 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 13 MiB/s wr, 13 op/s 2024-09-13T16:32:16.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:15 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:32:16.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:15 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:32:16.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:15 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:32:16.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:15 smithi016 ceph-mon[31187]: pgmap v388: 129 pgs: 129 active+clean; 153 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 13 MiB/s wr, 13 op/s 2024-09-13T16:32:16.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:15 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:32:16.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:15 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:32:16.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:15 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:32:16.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:15 smithi111 ceph-mon[34600]: pgmap v388: 129 pgs: 129 active+clean; 153 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 13 MiB/s wr, 13 op/s 2024-09-13T16:32:16.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:15 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:32:16.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:15 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:32:16.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:15 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:32:17.000 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:16 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:32:17.000 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:16 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:32:17.000 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:16 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:32:17.088 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:16 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:32:17.088 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:16 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:32:17.088 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:16 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:32:17.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:16 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:32:17.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:16 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:32:17.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:16 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:32:18.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:17 smithi107 ceph-mon[34139]: pgmap v389: 129 pgs: 129 active+clean; 369 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 31 MiB/s wr, 18 op/s 2024-09-13T16:32:18.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:17 smithi016 ceph-mon[31187]: pgmap v389: 129 pgs: 129 active+clean; 369 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 31 MiB/s wr, 18 op/s 2024-09-13T16:32:18.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:17 smithi111 ceph-mon[34600]: pgmap v389: 129 pgs: 129 active+clean; 369 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 31 MiB/s wr, 18 op/s 2024-09-13T16:32:20.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:19 smithi107 ceph-mon[34139]: pgmap v390: 129 pgs: 129 active+clean; 609 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 51 MiB/s wr, 28 op/s 2024-09-13T16:32:20.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:19 smithi016 ceph-mon[31187]: pgmap v390: 129 pgs: 129 active+clean; 609 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 51 MiB/s wr, 28 op/s 2024-09-13T16:32:20.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:19 smithi111 ceph-mon[34600]: pgmap v390: 129 pgs: 129 active+clean; 609 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 51 MiB/s wr, 28 op/s 2024-09-13T16:32:20.891 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-13T16:32:20.891 INFO:tasks.workunit.client.0.smithi016.stdout: Children see throughput for 1 initial writers = 102613.40 kB/sec 2024-09-13T16:32:20.891 INFO:tasks.workunit.client.0.smithi016.stdout: Parent sees throughput for 1 initial writers = 102611.47 kB/sec 2024-09-13T16:32:20.891 INFO:tasks.workunit.client.0.smithi016.stdout: Min throughput per process = 102613.40 kB/sec 2024-09-13T16:32:20.892 INFO:tasks.workunit.client.0.smithi016.stdout: Max throughput per process = 102613.40 kB/sec 2024-09-13T16:32:20.892 INFO:tasks.workunit.client.0.smithi016.stdout: Avg throughput per process = 102613.40 kB/sec 2024-09-13T16:32:20.892 INFO:tasks.workunit.client.0.smithi016.stdout: Min xfer = 1048576.00 kB 2024-09-13T16:32:22.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:21 smithi107 ceph-mon[34139]: pgmap v391: 129 pgs: 129 active+clean; 757 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 63 MiB/s wr, 28 op/s 2024-09-13T16:32:22.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:21 smithi016 ceph-mon[31187]: pgmap v391: 129 pgs: 129 active+clean; 757 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 63 MiB/s wr, 28 op/s 2024-09-13T16:32:22.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:21 smithi111 ceph-mon[34600]: pgmap v391: 129 pgs: 129 active+clean; 757 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 63 MiB/s wr, 28 op/s 2024-09-13T16:32:24.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:23 smithi107 ceph-mon[34139]: pgmap v392: 129 pgs: 129 active+clean; 913 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 76 MiB/s wr, 34 op/s 2024-09-13T16:32:24.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:23 smithi016 ceph-mon[31187]: pgmap v392: 129 pgs: 129 active+clean; 913 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 76 MiB/s wr, 34 op/s 2024-09-13T16:32:24.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:23 smithi111 ceph-mon[34600]: pgmap v392: 129 pgs: 129 active+clean; 913 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 76 MiB/s wr, 34 op/s 2024-09-13T16:32:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.830+0000 7f4908fe2640 1 -- 172.21.15.16:0/875750335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49040fec30 msgr2=0x7f49040ff030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.830+0000 7f4908fe2640 1 --2- 172.21.15.16:0/875750335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49040fec30 0x7f49040ff030 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f48f4009900 tx=0x7f48f402a410 comp rx=0 tx=0).stop 2024-09-13T16:32:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.830+0000 7f4908fe2640 1 -- 172.21.15.16:0/875750335 shutdown_connections 2024-09-13T16:32:25.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.830+0000 7f4908fe2640 1 --2- 172.21.15.16:0/875750335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49041007e0 0x7f4904107070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:25.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.830+0000 7f4908fe2640 1 --2- 172.21.15.16:0/875750335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49040ffe20 0x7f49041002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:25.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.830+0000 7f4908fe2640 1 --2- 172.21.15.16:0/875750335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49040fec30 0x7f49040ff030 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:25.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.830+0000 7f4908fe2640 1 -- 172.21.15.16:0/875750335 >> 172.21.15.16:0/875750335 conn(0x7f49040fa3b0 msgr2=0x7f49040fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:32:25.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.830+0000 7f4908fe2640 1 -- 172.21.15.16:0/875750335 shutdown_connections 2024-09-13T16:32:25.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.830+0000 7f4908fe2640 1 -- 172.21.15.16:0/875750335 wait complete. 2024-09-13T16:32:25.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.831+0000 7f4908fe2640 1 Processor -- start 2024-09-13T16:32:25.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.831+0000 7f4908fe2640 1 -- start start 2024-09-13T16:32:25.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.831+0000 7f4908fe2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49040fec30 0x7f4904195c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:25.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.831+0000 7f4908fe2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49040ffe20 0x7f49041961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:25.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.831+0000 7f4908fe2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49041007e0 0x7f490419d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:25.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.831+0000 7f4908fe2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490419d740 con 0x7f49040ffe20 2024-09-13T16:32:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.831+0000 7f4908fe2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490419d8b0 con 0x7f49041007e0 2024-09-13T16:32:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.831+0000 7f4908fe2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490419da70 con 0x7f49040fec30 2024-09-13T16:32:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.831+0000 7f48fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49040ffe20 0x7f49041961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.831+0000 7f48fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49040ffe20 0x7f49041961a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47912/0 (socket says 172.21.15.16:47912) 2024-09-13T16:32:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.831+0000 7f48fbfff640 1 -- 172.21.15.16:0/1135745359 learned_addr learned my addr 172.21.15.16:0/1135745359 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:32:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.831+0000 7f4902d9d640 1 --2- 172.21.15.16:0/1135745359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49041007e0 0x7f490419d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.831+0000 7f490259c640 1 --2- 172.21.15.16:0/1135745359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49040fec30 0x7f4904195c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.832+0000 7f48fbfff640 1 -- 172.21.15.16:0/1135745359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49040fec30 msgr2=0x7f4904195c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.832+0000 7f48fbfff640 1 --2- 172.21.15.16:0/1135745359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49040fec30 0x7f4904195c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.832+0000 7f48fbfff640 1 -- 172.21.15.16:0/1135745359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49041007e0 msgr2=0x7f490419d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.832+0000 7f48fbfff640 1 --2- 172.21.15.16:0/1135745359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49041007e0 0x7f490419d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.832+0000 7f48fbfff640 1 -- 172.21.15.16:0/1135745359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f4009560 con 0x7f49040ffe20 2024-09-13T16:32:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.832+0000 7f48fbfff640 1 --2- 172.21.15.16:0/1135745359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49040ffe20 0x7f49041961a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f48ec00d950 tx=0x7f48ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:32:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.832+0000 7f48fb7fe640 1 -- 172.21.15.16:0/1135745359 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec009c30 con 0x7f49040ffe20 2024-09-13T16:32:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.832+0000 7f4908fe2640 1 -- 172.21.15.16:0/1135745359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f490419dd50 con 0x7f49040ffe20 2024-09-13T16:32:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.833+0000 7f48fb7fe640 1 -- 172.21.15.16:0/1135745359 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f48ec009d90 con 0x7f49040ffe20 2024-09-13T16:32:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.833+0000 7f48fb7fe640 1 -- 172.21.15.16:0/1135745359 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec00f620 con 0x7f49040ffe20 2024-09-13T16:32:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.833+0000 7f4908fe2640 1 -- 172.21.15.16:0/1135745359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490419e1d0 con 0x7f49040ffe20 2024-09-13T16:32:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.835+0000 7f48fb7fe640 1 -- 172.21.15.16:0/1135745359 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f48ec0109f0 con 0x7f49040ffe20 2024-09-13T16:32:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.835+0000 7f490bdde740 1 Processor -- start 2024-09-13T16:32:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.835+0000 7f490bdde740 1 -- start start 2024-09-13T16:32:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.836+0000 7f490bdde740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55db7f8a65e0 0x55db7f8a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.836+0000 7f490bdde740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db7f8a6fb0 0x55db7f8abad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.836+0000 7f490bdde740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db7f8ac140 0x55db7f8ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.836+0000 7f490bdde740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7f80cc30 con 0x55db7f8a6fb0 2024-09-13T16:32:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.836+0000 7f490bdde740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7f80a3c0 con 0x55db7f8ac140 2024-09-13T16:32:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.836+0000 7f490bdde740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7f805cb0 con 0x55db7f8a65e0 2024-09-13T16:32:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.836+0000 7f48fb7fe640 1 --2- 172.21.15.16:0/1135745359 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f48f00a11a0 0x7f48f00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.836+0000 7f490259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db7f8a6fb0 0x55db7f8abad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.836+0000 7f490259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db7f8a6fb0 0x55db7f8abad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47914/0 (socket says 172.21.15.16:47914) 2024-09-13T16:32:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.836+0000 7f490259c640 1 -- 172.21.15.16:0/475797133 learned_addr learned my addr 172.21.15.16:0/475797133 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:32:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.836+0000 7f48fbfff640 1 --2- 172.21.15.16:0/475797133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db7f8ac140 0x55db7f8ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.836+0000 7f4902d9d640 1 --2- 172.21.15.16:0/475797133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55db7f8a65e0 0x55db7f8a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.836+0000 7f4902d9d640 1 --2- 172.21.15.16:0/1135745359 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f48f00a11a0 0x7f48f00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.836+0000 7f48fb7fe640 1 -- 172.21.15.16:0/1135745359 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f48ec0cc980 con 0x7f49040ffe20 2024-09-13T16:32:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.836+0000 7f490259c640 1 -- 172.21.15.16:0/475797133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55db7f8a65e0 msgr2=0x55db7f8a69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:32:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.836+0000 7f490259c640 1 --2- 172.21.15.16:0/475797133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55db7f8a65e0 0x55db7f8a69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.836+0000 7f490259c640 1 -- 172.21.15.16:0/475797133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db7f8ac140 msgr2=0x55db7f8ae5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:32:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.836+0000 7f490259c640 1 --2- 172.21.15.16:0/475797133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db7f8ac140 0x55db7f8ae5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.836+0000 7f490259c640 1 -- 172.21.15.16:0/475797133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db7f7ac910 con 0x55db7f8a6fb0 2024-09-13T16:32:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.837+0000 7f490259c640 1 --2- 172.21.15.16:0/475797133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db7f8a6fb0 0x55db7f8abad0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f48f4009900 tx=0x7f48f40315a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c1983bc4dc9f8e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:32:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.837+0000 7f4902d9d640 1 --2- 172.21.15.16:0/1135745359 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f48f00a11a0 0x7f48f00a3660 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55db7f8a7830 tx=0x7f48fc009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:32:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.837+0000 7f48f8ff9640 1 -- 172.21.15.16:0/475797133 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f4004230 con 0x55db7f8a6fb0 2024-09-13T16:32:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.837+0000 7f48f8ff9640 1 -- 172.21.15.16:0/475797133 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f48f4031790 con 0x55db7f8a6fb0 2024-09-13T16:32:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.837+0000 7f48f8ff9640 1 -- 172.21.15.16:0/475797133 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f4003b50 con 0x55db7f8a6fb0 2024-09-13T16:32:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.837+0000 7f490bdde740 1 -- 172.21.15.16:0/475797133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db7f8a6fb0 msgr2=0x55db7f8abad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.837+0000 7f490bdde740 1 --2- 172.21.15.16:0/475797133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db7f8a6fb0 0x55db7f8abad0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f48f4009900 tx=0x7f48f40315a0 comp rx=0 tx=0).stop 2024-09-13T16:32:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.838+0000 7f490bdde740 1 -- 172.21.15.16:0/475797133 shutdown_connections 2024-09-13T16:32:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.838+0000 7f490bdde740 1 --2- 172.21.15.16:0/475797133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db7f8ac140 0x55db7f8ae5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.838+0000 7f490bdde740 1 --2- 172.21.15.16:0/475797133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db7f8a6fb0 0x55db7f8abad0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.838+0000 7f490bdde740 1 --2- 172.21.15.16:0/475797133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55db7f8a65e0 0x55db7f8a69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.838+0000 7f490bdde740 1 -- 172.21.15.16:0/475797133 >> 172.21.15.16:0/475797133 conn(0x55db7f89d8c0 msgr2=0x55db7f89dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:32:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.838+0000 7f490bdde740 1 -- 172.21.15.16:0/475797133 shutdown_connections 2024-09-13T16:32:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.838+0000 7f490bdde740 1 -- 172.21.15.16:0/475797133 wait complete. 2024-09-13T16:32:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.838+0000 7f490bdde740 1 -- start start 2024-09-13T16:32:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.838+0000 7f490bdde740 1 Processor -- start 2024-09-13T16:32:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.838+0000 7f490bdde740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db7f8a65e0 0x55db7f8b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.838+0000 7f490bdde740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db7f8a6fb0 0x55db7f8b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.838+0000 7f490bdde740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55db7f8ac140 0x55db7f8b7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.838+0000 7f490bdde740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7f7c3e80 con 0x55db7f8a6fb0 2024-09-13T16:32:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.838+0000 7f490bdde740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7f7c3b20 con 0x55db7f8a65e0 2024-09-13T16:32:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.838+0000 7f490bdde740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7f789260 con 0x55db7f8ac140 2024-09-13T16:32:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.838+0000 7f490bdde740 10 client.0 _ll_register_callbacks cb 0x55db7f7bedf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:32:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.839+0000 7f48fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db7f8a6fb0 0x55db7f8b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.839+0000 7f48fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db7f8a6fb0 0x55db7f8b07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47918/0 (socket says 172.21.15.16:47918) 2024-09-13T16:32:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.839+0000 7f48fbfff640 1 -- 172.21.15.16:0/910285795 learned_addr learned my addr 172.21.15.16:0/910285795 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:32:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.839+0000 7f490259c640 1 --2- 172.21.15.16:0/910285795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db7f8a65e0 0x55db7f8b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.839+0000 7f4902d9d640 1 --2- 172.21.15.16:0/910285795 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55db7f8ac140 0x55db7f8b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.839+0000 7f48fbfff640 1 -- 172.21.15.16:0/910285795 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55db7f8ac140 msgr2=0x55db7f8b7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:32:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.839+0000 7f48fbfff640 1 --2- 172.21.15.16:0/910285795 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55db7f8ac140 0x55db7f8b7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.839+0000 7f48fbfff640 1 -- 172.21.15.16:0/910285795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db7f8a65e0 msgr2=0x55db7f8b02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:32:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.839+0000 7f48fbfff640 1 --2- 172.21.15.16:0/910285795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db7f8a65e0 0x55db7f8b02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.839+0000 7f48fbfff640 1 -- 172.21.15.16:0/910285795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4904107970 con 0x55db7f8a6fb0 2024-09-13T16:32:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.839+0000 7f48fbfff640 1 --2- 172.21.15.16:0/910285795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db7f8a6fb0 0x55db7f8b07f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0ca010 tx=0x7f48ec06ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9edea16cd8f7df03 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:32:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.839+0000 7f48e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:32:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.839+0000 7f48e9ffb640 1 -- 172.21.15.16:0/910285795 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec0a1b10 con 0x55db7f8a6fb0 2024-09-13T16:32:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.839+0000 7f48e9ffb640 1 -- 172.21.15.16:0/910285795 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f48ec084040 con 0x55db7f8a6fb0 2024-09-13T16:32:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.839+0000 7f490bdde740 1 -- 172.21.15.16:0/910285795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db7f7f95d0 con 0x55db7f8a6fb0 2024-09-13T16:32:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.839+0000 7f48e9ffb640 1 -- 172.21.15.16:0/910285795 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48ec09d750 con 0x55db7f8a6fb0 2024-09-13T16:32:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.839+0000 7f490bdde740 10 client.15627 fetch_fsmap learned FSMap version 29 2024-09-13T16:32:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.839+0000 7f490bdde740 1 -- 172.21.15.16:0/910285795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55db7f804330 con 0x55db7f8a6fb0 2024-09-13T16:32:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.840+0000 7f48e9ffb640 1 -- 172.21.15.16:0/910285795 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55db7f7f95d0 con 0x55db7f8a6fb0 2024-09-13T16:32:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.840+0000 7f48e9ffb640 1 -- 172.21.15.16:0/910285795 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f48ec095730 con 0x55db7f8a6fb0 2024-09-13T16:32:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.840+0000 7f490bdde740 10 client.15627 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T16:32:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.840+0000 7f490bdde740 10 client.15627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T16:32:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.840+0000 7f490bdde740 20 client.15627 populate_metadata read hostname 'smithi016' 2024-09-13T16:32:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.840+0000 7f490bdde740 1 --2- 172.21.15.16:0/910285795 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55db7f8bbf10 0x55db7f8be300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.840+0000 7f490bdde740 4 client.15627 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T16:32:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.840+0000 7f490bdde740 1 -- 172.21.15.16:0/910285795 --> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db7f4f9300 con 0x55db7f8bbf10 2024-09-13T16:32:25.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.840+0000 7f490259c640 1 --2- 172.21.15.16:0/910285795 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55db7f8bbf10 0x55db7f8be300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:25.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.841+0000 7f48e9ffb640 10 client.15627 ms_handle_connect on v2:172.21.15.107:6838/1914689806 2024-09-13T16:32:25.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.841+0000 7f490259c640 1 --2- 172.21.15.16:0/910285795 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55db7f8bbf10 0x55db7f8be300 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba33cf0655ecf791 server_cookie=acfcf0f970701341 in_seq=0 out_seq=0 2024-09-13T16:32:25.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.841+0000 7f48e9ffb640 1 -- 172.21.15.16:0/910285795 <== mds.0 v2:172.21.15.107:6838/1914689806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55db7f4f9300 con 0x55db7f8bbf10 2024-09-13T16:32:25.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.841+0000 7f48e9ffb640 10 client.15627 handle_command_reply: tid=0 2024-09-13T16:32:25.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.841+0000 7f490bdde740 1 client.15627 shutdown 2024-09-13T16:32:25.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.841+0000 7f490bdde740 1 -- 172.21.15.16:0/910285795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db7f8a6fb0 msgr2=0x55db7f8b07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:25.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.842+0000 7f490bdde740 1 --2- 172.21.15.16:0/910285795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db7f8a6fb0 0x55db7f8b07f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0ca010 tx=0x7f48ec06ba70 comp rx=0 tx=0).stop 2024-09-13T16:32:25.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.842+0000 7f490bdde740 1 -- 172.21.15.16:0/910285795 shutdown_connections 2024-09-13T16:32:25.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.842+0000 7f490bdde740 1 --2- 172.21.15.16:0/910285795 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55db7f8bbf10 0x55db7f8be300 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:25.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.842+0000 7f490bdde740 1 --2- 172.21.15.16:0/910285795 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55db7f8ac140 0x55db7f8b7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:25.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.842+0000 7f490bdde740 1 --2- 172.21.15.16:0/910285795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db7f8a6fb0 0x55db7f8b07f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:25.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.842+0000 7f48e9ffb640 0 client.15627 ms_handle_reset on v2:172.21.15.107:6838/1914689806 2024-09-13T16:32:25.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.842+0000 7f490bdde740 1 --2- 172.21.15.16:0/910285795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db7f8a65e0 0x55db7f8b02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:25.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.842+0000 7f490bdde740 1 -- 172.21.15.16:0/910285795 >> 172.21.15.16:0/910285795 conn(0x55db7f89d8c0 msgr2=0x55db7f89cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:32:25.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.842+0000 7f490bdde740 1 -- 172.21.15.16:0/910285795 shutdown_connections 2024-09-13T16:32:25.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.842+0000 7f490bdde740 1 -- 172.21.15.16:0/910285795 wait complete. 2024-09-13T16:32:25.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.842+0000 7f490bdde740 20 client.15627 trim_cache size 0 max 16384 2024-09-13T16:32:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.854+0000 7f490bdde740 1 Processor -- start 2024-09-13T16:32:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.854+0000 7f490bdde740 1 -- start start 2024-09-13T16:32:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.855+0000 7f490bdde740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55db7f8a65e0 0x55db7f8b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.855+0000 7f490bdde740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db7f8a6fb0 0x55db7f8b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.855+0000 7f490bdde740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db7f8ac140 0x55db7f8b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.855+0000 7f490bdde740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7f8ba560 con 0x55db7f8a6fb0 2024-09-13T16:32:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.855+0000 7f490bdde740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7f89d730 con 0x55db7f8ac140 2024-09-13T16:32:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.855+0000 7f490bdde740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7f8af2e0 con 0x55db7f8a65e0 2024-09-13T16:32:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.855+0000 7f48fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db7f8a6fb0 0x55db7f8b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.855+0000 7f48fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db7f8a6fb0 0x55db7f8b77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47922/0 (socket says 172.21.15.16:47922) 2024-09-13T16:32:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.855+0000 7f48fbfff640 1 -- 172.21.15.16:0/4100247057 learned_addr learned my addr 172.21.15.16:0/4100247057 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:32:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.855+0000 7f490259c640 1 --2- 172.21.15.16:0/4100247057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55db7f8a65e0 0x55db7f8b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.855+0000 7f4902d9d640 1 --2- 172.21.15.16:0/4100247057 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db7f8ac140 0x55db7f8b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.855+0000 7f48fbfff640 1 -- 172.21.15.16:0/4100247057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55db7f8a65e0 msgr2=0x55db7f8b02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:32:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.855+0000 7f48fbfff640 1 --2- 172.21.15.16:0/4100247057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55db7f8a65e0 0x55db7f8b02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.855+0000 7f48fbfff640 1 -- 172.21.15.16:0/4100247057 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db7f8ac140 msgr2=0x55db7f8b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:32:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.855+0000 7f48fbfff640 1 --2- 172.21.15.16:0/4100247057 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db7f8ac140 0x55db7f8b07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.855+0000 7f48fbfff640 1 -- 172.21.15.16:0/4100247057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db7f7a6300 con 0x55db7f8a6fb0 2024-09-13T16:32:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.855+0000 7f490259c640 1 --2- 172.21.15.16:0/4100247057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55db7f8a65e0 0x55db7f8b02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:32:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.855+0000 7f48fbfff640 1 --2- 172.21.15.16:0/4100247057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db7f8a6fb0 0x55db7f8b77f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0963b0 tx=0x7f48ec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd4222882d05e45 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:32:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.855+0000 7f48f97fa640 1 -- 172.21.15.16:0/4100247057 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec0ccc00 con 0x55db7f8a6fb0 2024-09-13T16:32:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.855+0000 7f48f97fa640 1 -- 172.21.15.16:0/4100247057 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f48ec097bb0 con 0x55db7f8a6fb0 2024-09-13T16:32:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.855+0000 7f48f97fa640 1 -- 172.21.15.16:0/4100247057 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec095730 con 0x55db7f8a6fb0 2024-09-13T16:32:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.856+0000 7f490bdde740 1 -- 172.21.15.16:0/4100247057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db7f8a6fb0 msgr2=0x55db7f8b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.856+0000 7f490bdde740 1 --2- 172.21.15.16:0/4100247057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db7f8a6fb0 0x55db7f8b77f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0963b0 tx=0x7f48ec005880 comp rx=0 tx=0).stop 2024-09-13T16:32:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.856+0000 7f490bdde740 1 -- 172.21.15.16:0/4100247057 shutdown_connections 2024-09-13T16:32:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.856+0000 7f490bdde740 1 --2- 172.21.15.16:0/4100247057 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db7f8ac140 0x55db7f8b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.856+0000 7f490bdde740 1 --2- 172.21.15.16:0/4100247057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db7f8a6fb0 0x55db7f8b77f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.856+0000 7f490bdde740 1 --2- 172.21.15.16:0/4100247057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55db7f8a65e0 0x55db7f8b02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.856+0000 7f490bdde740 1 -- 172.21.15.16:0/4100247057 >> 172.21.15.16:0/4100247057 conn(0x55db7f89d8c0 msgr2=0x55db7f898a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:32:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.856+0000 7f490bdde740 1 -- 172.21.15.16:0/4100247057 shutdown_connections 2024-09-13T16:32:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.856+0000 7f490bdde740 1 -- 172.21.15.16:0/4100247057 wait complete. 2024-09-13T16:32:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.856+0000 7f490bdde740 1 -- start start 2024-09-13T16:32:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.856+0000 7f490bdde740 1 Processor -- start 2024-09-13T16:32:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.857+0000 7f490bdde740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db7f8a65e0 0x55db7f8b74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.857+0000 7f490bdde740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55db7f8a6fb0 0x55db7f8b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.857+0000 7f490bdde740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db7f8ac140 0x55db7f8b6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.857+0000 7f490bdde740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7f8b65c0 con 0x55db7f8a65e0 2024-09-13T16:32:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.857+0000 7f490bdde740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7f8b6700 con 0x55db7f8ac140 2024-09-13T16:32:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.857+0000 7f490bdde740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7f8b68c0 con 0x55db7f8a6fb0 2024-09-13T16:32:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.857+0000 7f490bdde740 10 client.0 _ll_register_callbacks cb 0x7f48f4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:32:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.857+0000 7f490259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db7f8a65e0 0x55db7f8b74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.857+0000 7f490259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db7f8a65e0 0x55db7f8b74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47938/0 (socket says 172.21.15.16:47938) 2024-09-13T16:32:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.857+0000 7f490259c640 1 -- 172.21.15.16:0/1205383618 learned_addr learned my addr 172.21.15.16:0/1205383618 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:32:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.857+0000 7f4902d9d640 1 --2- 172.21.15.16:0/1205383618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db7f8ac140 0x55db7f8b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.857+0000 7f48fbfff640 1 --2- 172.21.15.16:0/1205383618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55db7f8a6fb0 0x55db7f8b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.857+0000 7f490259c640 1 -- 172.21.15.16:0/1205383618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55db7f8a6fb0 msgr2=0x55db7f8b5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:32:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.857+0000 7f490259c640 1 --2- 172.21.15.16:0/1205383618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55db7f8a6fb0 0x55db7f8b5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.857+0000 7f490259c640 1 -- 172.21.15.16:0/1205383618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db7f8ac140 msgr2=0x55db7f8b6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:32:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.857+0000 7f490259c640 1 --2- 172.21.15.16:0/1205383618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db7f8ac140 0x55db7f8b6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.857+0000 7f490259c640 1 -- 172.21.15.16:0/1205383618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55db7f804330 con 0x55db7f8a65e0 2024-09-13T16:32:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.857+0000 7f490259c640 1 --2- 172.21.15.16:0/1205383618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db7f8a65e0 0x55db7f8b74e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x55db7f8a73d0 tx=0x7f48f4004030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfccff29d7d71212 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:32:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.857+0000 7f48e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:32:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.857+0000 7f48e9ffb640 1 -- 172.21.15.16:0/1205383618 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f4049040 con 0x55db7f8a65e0 2024-09-13T16:32:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.858+0000 7f48e9ffb640 1 -- 172.21.15.16:0/1205383618 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f48f4031960 con 0x55db7f8a65e0 2024-09-13T16:32:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.858+0000 7f490bdde740 1 -- 172.21.15.16:0/1205383618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db7f82e680 con 0x55db7f8a65e0 2024-09-13T16:32:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.858+0000 7f48e9ffb640 1 -- 172.21.15.16:0/1205383618 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48f4031b20 con 0x55db7f8a65e0 2024-09-13T16:32:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.858+0000 7f490bdde740 10 client.15633 fetch_fsmap learned FSMap version 29 2024-09-13T16:32:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.858+0000 7f490bdde740 1 -- 172.21.15.16:0/1205383618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55db7f7cb8e0 con 0x55db7f8a65e0 2024-09-13T16:32:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.858+0000 7f48e9ffb640 1 -- 172.21.15.16:0/1205383618 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55db7f82e680 con 0x55db7f8a65e0 2024-09-13T16:32:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.858+0000 7f48e9ffb640 1 -- 172.21.15.16:0/1205383618 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f48f4051190 con 0x55db7f8a65e0 2024-09-13T16:32:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.858+0000 7f490bdde740 10 client.15633 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T16:32:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.858+0000 7f490bdde740 10 client.15633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T16:32:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.858+0000 7f490bdde740 20 client.15633 populate_metadata read hostname 'smithi016' 2024-09-13T16:32:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.858+0000 7f490bdde740 1 --2- 172.21.15.16:0/1205383618 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55db7f913880 0x55db7f915ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.858+0000 7f490bdde740 4 client.15633 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T16:32:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.858+0000 7f490bdde740 1 -- 172.21.15.16:0/1205383618 --> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55db7f50a600 con 0x55db7f913880 2024-09-13T16:32:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.858+0000 7f48fbfff640 1 --2- 172.21.15.16:0/1205383618 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55db7f913880 0x55db7f915ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.859+0000 7f48e9ffb640 10 client.15633 ms_handle_connect on v2:172.21.15.107:6838/1914689806 2024-09-13T16:32:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.859+0000 7f48fbfff640 1 --2- 172.21.15.16:0/1205383618 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55db7f913880 0x55db7f915ca0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=689cd1b5fca67429 server_cookie=2c3f08d90bc04a69 in_seq=0 out_seq=0 2024-09-13T16:32:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.869+0000 7f48e9ffb640 1 -- 172.21.15.16:0/1205383618 <== mds.0 v2:172.21.15.107:6838/1914689806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55db7f50a600 con 0x55db7f913880 2024-09-13T16:32:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.869+0000 7f48e9ffb640 10 client.15633 handle_command_reply: tid=0 2024-09-13T16:32:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.869+0000 7f490bdde740 1 client.15633 shutdown 2024-09-13T16:32:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.869+0000 7f490bdde740 1 -- 172.21.15.16:0/1205383618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db7f8a65e0 msgr2=0x55db7f8b74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.869+0000 7f490bdde740 1 --2- 172.21.15.16:0/1205383618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db7f8a65e0 0x55db7f8b74e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x55db7f8a73d0 tx=0x7f48f4004030 comp rx=0 tx=0).stop 2024-09-13T16:32:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.869+0000 7f490bdde740 1 -- 172.21.15.16:0/1205383618 shutdown_connections 2024-09-13T16:32:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.869+0000 7f490bdde740 1 --2- 172.21.15.16:0/1205383618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55db7f8ac140 0x55db7f8b6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.869+0000 7f490bdde740 1 --2- 172.21.15.16:0/1205383618 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55db7f913880 0x55db7f915ca0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.869+0000 7f490bdde740 1 --2- 172.21.15.16:0/1205383618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55db7f8a6fb0 0x55db7f8b5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.869+0000 7f48e9ffb640 0 client.15633 ms_handle_reset on v2:172.21.15.107:6838/1914689806 2024-09-13T16:32:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.869+0000 7f490bdde740 1 --2- 172.21.15.16:0/1205383618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db7f8a65e0 0x55db7f8b74e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.869+0000 7f490bdde740 1 -- 172.21.15.16:0/1205383618 >> 172.21.15.16:0/1205383618 conn(0x55db7f89d8c0 msgr2=0x55db7f8a7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:32:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.869+0000 7f490bdde740 1 -- 172.21.15.16:0/1205383618 shutdown_connections 2024-09-13T16:32:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.869+0000 7f490bdde740 1 -- 172.21.15.16:0/1205383618 wait complete. 2024-09-13T16:32:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.869+0000 7f490bdde740 20 client.15633 trim_cache size 0 max 16384 2024-09-13T16:32:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-13T16:32:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-13T16:32:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-13T16:32:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-13T16:32:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.870+0000 7f4908fe2640 1 -- 172.21.15.16:0/1135745359 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f48f00a11a0 msgr2=0x7f48f00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.870+0000 7f4908fe2640 1 --2- 172.21.15.16:0/1135745359 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f48f00a11a0 0x7f48f00a3660 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55db7f8a7830 tx=0x7f48fc009500 comp rx=0 tx=0).stop 2024-09-13T16:32:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.870+0000 7f4908fe2640 1 -- 172.21.15.16:0/1135745359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49040ffe20 msgr2=0x7f49041961a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.870+0000 7f4908fe2640 1 --2- 172.21.15.16:0/1135745359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49040ffe20 0x7f49041961a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f48ec00d950 tx=0x7f48ec00de10 comp rx=0 tx=0).stop 2024-09-13T16:32:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.870+0000 7f4908fe2640 1 -- 172.21.15.16:0/1135745359 shutdown_connections 2024-09-13T16:32:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.870+0000 7f4908fe2640 1 --2- 172.21.15.16:0/1135745359 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f48f00a11a0 0x7f48f00a3660 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.870+0000 7f4908fe2640 1 --2- 172.21.15.16:0/1135745359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49041007e0 0x7f490419d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.870+0000 7f4908fe2640 1 --2- 172.21.15.16:0/1135745359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49040ffe20 0x7f49041961a0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.870+0000 7f4908fe2640 1 --2- 172.21.15.16:0/1135745359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49040fec30 0x7f4904195c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.870+0000 7f4908fe2640 1 -- 172.21.15.16:0/1135745359 >> 172.21.15.16:0/1135745359 conn(0x7f49040fa3b0 msgr2=0x7f4904067830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:32:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.870+0000 7f4908fe2640 1 -- 172.21.15.16:0/1135745359 shutdown_connections 2024-09-13T16:32:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:25.870+0000 7f4908fe2640 1 -- 172.21.15.16:0/1135745359 wait complete. 2024-09-13T16:32:26.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:25 smithi107 ceph-mon[34139]: pgmap v393: 129 pgs: 129 active+clean; 1017 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 81 MiB/s wr, 34 op/s 2024-09-13T16:32:26.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:25 smithi016 ceph-mon[31187]: pgmap v393: 129 pgs: 129 active+clean; 1017 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 81 MiB/s wr, 34 op/s 2024-09-13T16:32:26.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:25 smithi111 ceph-mon[34600]: pgmap v393: 129 pgs: 129 active+clean; 1017 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 81 MiB/s wr, 34 op/s 2024-09-13T16:32:28.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:27 smithi107 ceph-mon[34139]: pgmap v394: 129 pgs: 129 active+clean; 1.0 GiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 73 MiB/s wr, 31 op/s 2024-09-13T16:32:28.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:27 smithi016 ceph-mon[31187]: pgmap v394: 129 pgs: 129 active+clean; 1.0 GiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 73 MiB/s wr, 31 op/s 2024-09-13T16:32:28.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:27 smithi111 ceph-mon[34600]: pgmap v394: 129 pgs: 129 active+clean; 1.0 GiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 73 MiB/s wr, 31 op/s 2024-09-13T16:32:30.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:29 smithi107 ceph-mon[34139]: pgmap v395: 129 pgs: 129 active+clean; 1.0 GiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 61 MiB/s wr, 32 op/s 2024-09-13T16:32:30.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:29 smithi016 ceph-mon[31187]: pgmap v395: 129 pgs: 129 active+clean; 1.0 GiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 61 MiB/s wr, 32 op/s 2024-09-13T16:32:30.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:29 smithi111 ceph-mon[34600]: pgmap v395: 129 pgs: 129 active+clean; 1.0 GiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 61 MiB/s wr, 32 op/s 2024-09-13T16:32:32.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:31 smithi107 ceph-mon[34139]: pgmap v396: 129 pgs: 129 active+clean; 1.0 GiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 49 MiB/s wr, 24 op/s 2024-09-13T16:32:32.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:31 smithi016 ceph-mon[31187]: pgmap v396: 129 pgs: 129 active+clean; 1.0 GiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 49 MiB/s wr, 24 op/s 2024-09-13T16:32:32.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:31 smithi111 ceph-mon[34600]: pgmap v396: 129 pgs: 129 active+clean; 1.0 GiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 49 MiB/s wr, 24 op/s 2024-09-13T16:32:34.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:33 smithi107 ceph-mon[34139]: pgmap v397: 129 pgs: 129 active+clean; 1.0 GiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 54 MiB/s wr, 28 op/s 2024-09-13T16:32:34.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:33 smithi016 ceph-mon[31187]: pgmap v397: 129 pgs: 129 active+clean; 1.0 GiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 54 MiB/s wr, 28 op/s 2024-09-13T16:32:34.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:33 smithi111 ceph-mon[34600]: pgmap v397: 129 pgs: 129 active+clean; 1.0 GiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 54 MiB/s wr, 28 op/s 2024-09-13T16:32:35.941 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-13T16:32:35.942 INFO:tasks.workunit.client.0.smithi016.stdout: Children see throughput for 1 rewriters = 114686.66 kB/sec 2024-09-13T16:32:35.942 INFO:tasks.workunit.client.0.smithi016.stdout: Parent sees throughput for 1 rewriters = 114683.56 kB/sec 2024-09-13T16:32:35.942 INFO:tasks.workunit.client.0.smithi016.stdout: Min throughput per process = 114686.66 kB/sec 2024-09-13T16:32:35.942 INFO:tasks.workunit.client.0.smithi016.stdout: Max throughput per process = 114686.66 kB/sec 2024-09-13T16:32:35.942 INFO:tasks.workunit.client.0.smithi016.stdout: Avg throughput per process = 114686.66 kB/sec 2024-09-13T16:32:35.942 INFO:tasks.workunit.client.0.smithi016.stdout: Min xfer = 1048576.00 kB 2024-09-13T16:32:36.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:35 smithi107 ceph-mon[34139]: pgmap v398: 129 pgs: 129 active+clean; 1.0 GiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 60 MiB/s wr, 27 op/s 2024-09-13T16:32:36.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:35 smithi107 ceph-mon[34139]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-13T16:32:36.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:35 smithi016 ceph-mon[31187]: pgmap v398: 129 pgs: 129 active+clean; 1.0 GiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 60 MiB/s wr, 27 op/s 2024-09-13T16:32:36.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:35 smithi016 ceph-mon[31187]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-13T16:32:36.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:35 smithi111 ceph-mon[34600]: pgmap v398: 129 pgs: 129 active+clean; 1.0 GiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 60 MiB/s wr, 27 op/s 2024-09-13T16:32:36.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:35 smithi111 ceph-mon[34600]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-13T16:32:38.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:37 smithi107 ceph-mon[34139]: pgmap v399: 129 pgs: 129 active+clean; 1.0 GiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 68 MiB/s wr, 29 op/s 2024-09-13T16:32:38.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:37 smithi016 ceph-mon[31187]: pgmap v399: 129 pgs: 129 active+clean; 1.0 GiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 68 MiB/s wr, 29 op/s 2024-09-13T16:32:38.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:37 smithi111 ceph-mon[34600]: pgmap v399: 129 pgs: 129 active+clean; 1.0 GiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 68 MiB/s wr, 29 op/s 2024-09-13T16:32:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.963+0000 7f199c6f7640 1 -- 172.21.15.16:0/2220071935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19940fec60 msgr2=0x7f19940ff060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.963+0000 7f199c6f7640 1 --2- 172.21.15.16:0/2220071935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19940fec60 0x7f19940ff060 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1988009900 tx=0x7f198802a410 comp rx=0 tx=0).stop 2024-09-13T16:32:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.964+0000 7f199c6f7640 1 -- 172.21.15.16:0/2220071935 shutdown_connections 2024-09-13T16:32:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.964+0000 7f199c6f7640 1 --2- 172.21.15.16:0/2220071935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1994100810 0x7f19941070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.964+0000 7f199c6f7640 1 --2- 172.21.15.16:0/2220071935 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19940ffe50 0x7f19941002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.964+0000 7f199c6f7640 1 --2- 172.21.15.16:0/2220071935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19940fec60 0x7f19940ff060 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.964+0000 7f199c6f7640 1 -- 172.21.15.16:0/2220071935 >> 172.21.15.16:0/2220071935 conn(0x7f19940fa3c0 msgr2=0x7f19940fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:32:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.964+0000 7f199c6f7640 1 -- 172.21.15.16:0/2220071935 shutdown_connections 2024-09-13T16:32:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.964+0000 7f199c6f7640 1 -- 172.21.15.16:0/2220071935 wait complete. 2024-09-13T16:32:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.964+0000 7f199c6f7640 1 Processor -- start 2024-09-13T16:32:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.964+0000 7f199c6f7640 1 -- start start 2024-09-13T16:32:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.964+0000 7f199c6f7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19940fec60 0x7f1994197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.964+0000 7f199c6f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19940ffe50 0x7f1994198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.964+0000 7f199c6f7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1994100810 0x7f199419f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.964+0000 7f199c6f7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199419f9e0 con 0x7f19940ffe50 2024-09-13T16:32:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.965+0000 7f199c6f7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199419fb50 con 0x7f19940fec60 2024-09-13T16:32:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.965+0000 7f199c6f7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199419fd10 con 0x7f1994100810 2024-09-13T16:32:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.965+0000 7f1999c92640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19940ffe50 0x7f1994198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.965+0000 7f1999c92640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19940ffe50 0x7f1994198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58250/0 (socket says 172.21.15.16:58250) 2024-09-13T16:32:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.965+0000 7f1999c92640 1 -- 172.21.15.16:0/3023111596 learned_addr learned my addr 172.21.15.16:0/3023111596 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:32:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.965+0000 7f199a493640 1 --2- 172.21.15.16:0/3023111596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19940fec60 0x7f1994197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.965+0000 7f199ac94640 1 --2- 172.21.15.16:0/3023111596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1994100810 0x7f199419f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.965+0000 7f1999c92640 1 -- 172.21.15.16:0/3023111596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1994100810 msgr2=0x7f199419f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.965+0000 7f1999c92640 1 --2- 172.21.15.16:0/3023111596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1994100810 0x7f199419f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.965+0000 7f1999c92640 1 -- 172.21.15.16:0/3023111596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19940fec60 msgr2=0x7f1994197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.965+0000 7f1999c92640 1 --2- 172.21.15.16:0/3023111596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19940fec60 0x7f1994197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.965+0000 7f1999c92640 1 -- 172.21.15.16:0/3023111596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1988009560 con 0x7f19940ffe50 2024-09-13T16:32:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.965+0000 7f1999c92640 1 --2- 172.21.15.16:0/3023111596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19940ffe50 0x7f1994198430 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f199000d950 tx=0x7f199000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:32:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.965+0000 7f19877fe640 1 -- 172.21.15.16:0/3023111596 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1990009c30 con 0x7f19940ffe50 2024-09-13T16:32:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.965+0000 7f19877fe640 1 -- 172.21.15.16:0/3023111596 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1990009d90 con 0x7f19940ffe50 2024-09-13T16:32:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.965+0000 7f199c6f7640 1 -- 172.21.15.16:0/3023111596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f199419fff0 con 0x7f19940ffe50 2024-09-13T16:32:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.965+0000 7f19877fe640 1 -- 172.21.15.16:0/3023111596 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199000f620 con 0x7f19940ffe50 2024-09-13T16:32:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.966+0000 7f199c6f7640 1 -- 172.21.15.16:0/3023111596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19941a0470 con 0x7f19940ffe50 2024-09-13T16:32:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.968+0000 7f19877fe640 1 -- 172.21.15.16:0/3023111596 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f19900109f0 con 0x7f19940ffe50 2024-09-13T16:32:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.968+0000 7f199f41a740 1 Processor -- start 2024-09-13T16:32:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.968+0000 7f199f41a740 1 -- start start 2024-09-13T16:32:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.968+0000 7f199f41a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfd63d1d0 0x55fdfd63d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.968+0000 7f199f41a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdfd63dba0 0x55fdfd6426c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.968+0000 7f199f41a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdfd642d30 0x55fdfd6451c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.968+0000 7f199f41a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdfd5a7c30 con 0x55fdfd63d1d0 2024-09-13T16:32:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.968+0000 7f199f41a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdfd5a53c0 con 0x55fdfd63dba0 2024-09-13T16:32:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.968+0000 7f199f41a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdfd5a0cb0 con 0x55fdfd642d30 2024-09-13T16:32:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.969+0000 7f199ac94640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfd63d1d0 0x55fdfd63d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.969+0000 7f19877fe640 1 --2- 172.21.15.16:0/3023111596 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f19800a1090 0x7f19800a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.969+0000 7f199ac94640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfd63d1d0 0x55fdfd63d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58266/0 (socket says 172.21.15.16:58266) 2024-09-13T16:32:38.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.969+0000 7f199ac94640 1 -- 172.21.15.16:0/17232419 learned_addr learned my addr 172.21.15.16:0/17232419 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:32:38.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.969+0000 7f1999c92640 1 --2- 172.21.15.16:0/17232419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdfd642d30 0x55fdfd6451c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:38.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.969+0000 7f199a493640 1 --2- 172.21.15.16:0/17232419 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdfd63dba0 0x55fdfd6426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:38.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.969+0000 7f199ac94640 1 --2- 172.21.15.16:0/3023111596 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f19800a1090 0x7f19800a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:38.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.969+0000 7f199ac94640 1 -- 172.21.15.16:0/17232419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdfd642d30 msgr2=0x55fdfd6451c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:32:38.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.969+0000 7f199ac94640 1 --2- 172.21.15.16:0/17232419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdfd642d30 0x55fdfd6451c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:38.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.969+0000 7f19877fe640 1 -- 172.21.15.16:0/3023111596 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f19900cc860 con 0x7f19940ffe50 2024-09-13T16:32:38.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.969+0000 7f199ac94640 1 -- 172.21.15.16:0/17232419 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdfd63dba0 msgr2=0x55fdfd6426c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:32:38.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.969+0000 7f199ac94640 1 --2- 172.21.15.16:0/17232419 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdfd63dba0 0x55fdfd6426c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:38.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.969+0000 7f199ac94640 1 -- 172.21.15.16:0/17232419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdfd547910 con 0x55fdfd63d1d0 2024-09-13T16:32:38.979 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.969+0000 7f199ac94640 1 --2- 172.21.15.16:0/17232419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfd63d1d0 0x55fdfd63d5d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f198c0149c0 tx=0x7f198c014e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da6743bff990fe1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:32:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.969+0000 7f1984ff9640 1 -- 172.21.15.16:0/17232419 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198c01eb40 con 0x55fdfd63d1d0 2024-09-13T16:32:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.969+0000 7f1984ff9640 1 -- 172.21.15.16:0/17232419 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f198c01a070 con 0x55fdfd63d1d0 2024-09-13T16:32:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.970+0000 7f1984ff9640 1 -- 172.21.15.16:0/17232419 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198c013890 con 0x55fdfd63d1d0 2024-09-13T16:32:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.970+0000 7f199f41a740 1 -- 172.21.15.16:0/17232419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfd63d1d0 msgr2=0x55fdfd63d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.970+0000 7f199f41a740 1 --2- 172.21.15.16:0/17232419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfd63d1d0 0x55fdfd63d5d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f198c0149c0 tx=0x7f198c014e80 comp rx=0 tx=0).stop 2024-09-13T16:32:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.970+0000 7f199ac94640 1 --2- 172.21.15.16:0/3023111596 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f19800a1090 0x7f19800a3550 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f198c022000 tx=0x7f198c020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:32:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.970+0000 7f199f41a740 1 -- 172.21.15.16:0/17232419 shutdown_connections 2024-09-13T16:32:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.970+0000 7f199f41a740 1 --2- 172.21.15.16:0/17232419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdfd642d30 0x55fdfd6451c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.970+0000 7f199f41a740 1 --2- 172.21.15.16:0/17232419 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdfd63dba0 0x55fdfd6426c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.970+0000 7f199f41a740 1 --2- 172.21.15.16:0/17232419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfd63d1d0 0x55fdfd63d5d0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.970+0000 7f199f41a740 1 -- 172.21.15.16:0/17232419 >> 172.21.15.16:0/17232419 conn(0x55fdfd6388c0 msgr2=0x55fdfd638cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:32:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.970+0000 7f199f41a740 1 -- 172.21.15.16:0/17232419 shutdown_connections 2024-09-13T16:32:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.970+0000 7f199f41a740 1 -- 172.21.15.16:0/17232419 wait complete. 2024-09-13T16:32:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.971+0000 7f199f41a740 1 -- start start 2024-09-13T16:32:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.971+0000 7f199f41a740 1 Processor -- start 2024-09-13T16:32:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.971+0000 7f199f41a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdfd63d1d0 0x55fdfd646ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.971+0000 7f199f41a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfd63dba0 0x55fdfd6473e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.971+0000 7f199f41a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdfd642d30 0x55fdfd64e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.971+0000 7f199f41a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdfd55ee80 con 0x55fdfd63dba0 2024-09-13T16:32:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.971+0000 7f199f41a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdfd55eb20 con 0x55fdfd642d30 2024-09-13T16:32:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.971+0000 7f199f41a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdfd524260 con 0x55fdfd63d1d0 2024-09-13T16:32:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.971+0000 7f199f41a740 10 client.0 _ll_register_callbacks cb 0x55fdfd559df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:32:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.971+0000 7f1999c92640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfd63dba0 0x55fdfd6473e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.971+0000 7f1999c92640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfd63dba0 0x55fdfd6473e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58274/0 (socket says 172.21.15.16:58274) 2024-09-13T16:32:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.971+0000 7f1999c92640 1 -- 172.21.15.16:0/3624577933 learned_addr learned my addr 172.21.15.16:0/3624577933 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:32:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.971+0000 7f199a493640 1 --2- 172.21.15.16:0/3624577933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdfd63d1d0 0x55fdfd646ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.971+0000 7f199ac94640 1 --2- 172.21.15.16:0/3624577933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdfd642d30 0x55fdfd64e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.971+0000 7f1999c92640 1 -- 172.21.15.16:0/3624577933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdfd63d1d0 msgr2=0x55fdfd646ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:32:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.971+0000 7f1999c92640 1 --2- 172.21.15.16:0/3624577933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdfd63d1d0 0x55fdfd646ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.971+0000 7f1999c92640 1 -- 172.21.15.16:0/3624577933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdfd642d30 msgr2=0x55fdfd64e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:32:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.971+0000 7f1999c92640 1 --2- 172.21.15.16:0/3624577933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdfd642d30 0x55fdfd64e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.971+0000 7f1999c92640 1 -- 172.21.15.16:0/3624577933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f198c014630 con 0x55fdfd63dba0 2024-09-13T16:32:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.971+0000 7f199a493640 1 --2- 172.21.15.16:0/3624577933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdfd63d1d0 0x55fdfd646ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:32:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.972+0000 7f1999c92640 1 --2- 172.21.15.16:0/3624577933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfd63dba0 0x55fdfd6473e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f19900ca010 tx=0x7f19900994a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1f87d819a48fc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:32:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.972+0000 7f197dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:32:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.972+0000 7f197dffb640 1 -- 172.21.15.16:0/3624577933 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1990099960 con 0x55fdfd63dba0 2024-09-13T16:32:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.972+0000 7f197dffb640 1 -- 172.21.15.16:0/3624577933 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f199009d790 con 0x55fdfd63dba0 2024-09-13T16:32:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.972+0000 7f199f41a740 1 -- 172.21.15.16:0/3624577933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fdfd5c9680 con 0x55fdfd63dba0 2024-09-13T16:32:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.972+0000 7f197dffb640 1 -- 172.21.15.16:0/3624577933 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19900a1de0 con 0x55fdfd63dba0 2024-09-13T16:32:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.972+0000 7f199f41a740 10 client.15645 fetch_fsmap learned FSMap version 29 2024-09-13T16:32:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.972+0000 7f199f41a740 1 -- 172.21.15.16:0/3624577933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fdfd59f330 con 0x55fdfd63dba0 2024-09-13T16:32:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.972+0000 7f197dffb640 1 -- 172.21.15.16:0/3624577933 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fdfd5c9680 con 0x55fdfd63dba0 2024-09-13T16:32:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.972+0000 7f197dffb640 1 -- 172.21.15.16:0/3624577933 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1990095750 con 0x55fdfd63dba0 2024-09-13T16:32:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.972+0000 7f199f41a740 10 client.15645 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T16:32:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.972+0000 7f199f41a740 10 client.15645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T16:32:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.972+0000 7f199f41a740 20 client.15645 populate_metadata read hostname 'smithi016' 2024-09-13T16:32:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.972+0000 7f199f41a740 1 --2- 172.21.15.16:0/3624577933 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55fdfd652b00 0x55fdfd654ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.972+0000 7f199f41a740 4 client.15645 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T16:32:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.972+0000 7f199f41a740 1 -- 172.21.15.16:0/3624577933 --> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fdfd5945d0 con 0x55fdfd652b00 2024-09-13T16:32:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.973+0000 7f199a493640 1 --2- 172.21.15.16:0/3624577933 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55fdfd652b00 0x55fdfd654ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.973+0000 7f199a493640 1 --2- 172.21.15.16:0/3624577933 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55fdfd652b00 0x55fdfd654ef0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52f6f6ce5bb05ee4 server_cookie=97b64b86235a0f8b in_seq=0 out_seq=0 2024-09-13T16:32:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.973+0000 7f197dffb640 10 client.15645 ms_handle_connect on v2:172.21.15.107:6838/1914689806 2024-09-13T16:32:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.974+0000 7f197dffb640 1 -- 172.21.15.16:0/3624577933 <== mds.0 v2:172.21.15.107:6838/1914689806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fdfd5945d0 con 0x55fdfd652b00 2024-09-13T16:32:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.974+0000 7f197dffb640 10 client.15645 handle_command_reply: tid=0 2024-09-13T16:32:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.974+0000 7f199f41a740 1 client.15645 shutdown 2024-09-13T16:32:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.974+0000 7f199f41a740 1 -- 172.21.15.16:0/3624577933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfd63dba0 msgr2=0x55fdfd6473e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.974+0000 7f199f41a740 1 --2- 172.21.15.16:0/3624577933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfd63dba0 0x55fdfd6473e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f19900ca010 tx=0x7f19900994a0 comp rx=0 tx=0).stop 2024-09-13T16:32:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.974+0000 7f199f41a740 1 -- 172.21.15.16:0/3624577933 shutdown_connections 2024-09-13T16:32:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.974+0000 7f199f41a740 1 --2- 172.21.15.16:0/3624577933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdfd642d30 0x55fdfd64e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.974+0000 7f199f41a740 1 --2- 172.21.15.16:0/3624577933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfd63dba0 0x55fdfd6473e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.974+0000 7f199f41a740 1 --2- 172.21.15.16:0/3624577933 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55fdfd652b00 0x55fdfd654ef0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.974+0000 7f199f41a740 1 --2- 172.21.15.16:0/3624577933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdfd63d1d0 0x55fdfd646ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.974+0000 7f199f41a740 1 -- 172.21.15.16:0/3624577933 >> 172.21.15.16:0/3624577933 conn(0x55fdfd6388c0 msgr2=0x55fdfd637b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:32:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.974+0000 7f197dffb640 0 client.15645 ms_handle_reset on v2:172.21.15.107:6838/1914689806 2024-09-13T16:32:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.974+0000 7f199f41a740 1 -- 172.21.15.16:0/3624577933 shutdown_connections 2024-09-13T16:32:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.974+0000 7f199f41a740 1 -- 172.21.15.16:0/3624577933 wait complete. 2024-09-13T16:32:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.974+0000 7f199f41a740 20 client.15645 trim_cache size 0 max 16384 2024-09-13T16:32:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.987+0000 7f199f41a740 1 Processor -- start 2024-09-13T16:32:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.987+0000 7f199f41a740 1 -- start start 2024-09-13T16:32:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.987+0000 7f199f41a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfd63d1d0 0x55fdfd647500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.987+0000 7f199f41a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdfd63dba0 0x55fdfd64e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.987+0000 7f199f41a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdfd642d30 0x55fdfd63acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.987+0000 7f199f41a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdfd632fd0 con 0x55fdfd63d1d0 2024-09-13T16:32:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.987+0000 7f199f41a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdfd64f5a0 con 0x55fdfd642d30 2024-09-13T16:32:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.987+0000 7f199f41a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdfd646a80 con 0x55fdfd63dba0 2024-09-13T16:32:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.987+0000 7f199a493640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfd63d1d0 0x55fdfd647500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.987+0000 7f199a493640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfd63d1d0 0x55fdfd647500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58278/0 (socket says 172.21.15.16:58278) 2024-09-13T16:32:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.987+0000 7f199a493640 1 -- 172.21.15.16:0/3427889211 learned_addr learned my addr 172.21.15.16:0/3427889211 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:32:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.987+0000 7f1999c92640 1 --2- 172.21.15.16:0/3427889211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdfd63dba0 0x55fdfd64e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.988+0000 7f199a493640 1 -- 172.21.15.16:0/3427889211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdfd63dba0 msgr2=0x55fdfd64e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:32:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.988+0000 7f199a493640 1 --2- 172.21.15.16:0/3427889211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdfd63dba0 0x55fdfd64e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.988+0000 7f199a493640 1 -- 172.21.15.16:0/3427889211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdfd642d30 msgr2=0x55fdfd63acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:32:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.988+0000 7f199ac94640 1 --2- 172.21.15.16:0/3427889211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdfd642d30 0x55fdfd63acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.988+0000 7f199a493640 1 --2- 172.21.15.16:0/3427889211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdfd642d30 0x55fdfd63acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.988+0000 7f199a493640 1 -- 172.21.15.16:0/3427889211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdfd541300 con 0x55fdfd63d1d0 2024-09-13T16:32:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.988+0000 7f199ac94640 1 --2- 172.21.15.16:0/3427889211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdfd642d30 0x55fdfd63acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:32:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.988+0000 7f199a493640 1 --2- 172.21.15.16:0/3427889211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfd63d1d0 0x55fdfd647500 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f198802aa00 tx=0x7f1988004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c1675fabcf71c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:32:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.988+0000 7f19857fa640 1 -- 172.21.15.16:0/3427889211 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1988042040 con 0x55fdfd63d1d0 2024-09-13T16:32:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.988+0000 7f19857fa640 1 -- 172.21.15.16:0/3427889211 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1988004490 con 0x55fdfd63d1d0 2024-09-13T16:32:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.988+0000 7f19857fa640 1 -- 172.21.15.16:0/3427889211 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1988003850 con 0x55fdfd63d1d0 2024-09-13T16:32:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.989+0000 7f199f41a740 1 -- 172.21.15.16:0/3427889211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfd63d1d0 msgr2=0x55fdfd647500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.989+0000 7f199f41a740 1 --2- 172.21.15.16:0/3427889211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfd63d1d0 0x55fdfd647500 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f198802aa00 tx=0x7f1988004320 comp rx=0 tx=0).stop 2024-09-13T16:32:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.989+0000 7f199f41a740 1 -- 172.21.15.16:0/3427889211 shutdown_connections 2024-09-13T16:32:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.989+0000 7f199f41a740 1 --2- 172.21.15.16:0/3427889211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdfd642d30 0x55fdfd63acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.989+0000 7f199f41a740 1 --2- 172.21.15.16:0/3427889211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdfd63dba0 0x55fdfd64e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.989+0000 7f199f41a740 1 --2- 172.21.15.16:0/3427889211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfd63d1d0 0x55fdfd647500 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.989+0000 7f199f41a740 1 -- 172.21.15.16:0/3427889211 >> 172.21.15.16:0/3427889211 conn(0x55fdfd6388c0 msgr2=0x55fdfd6420e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:32:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.989+0000 7f199f41a740 1 -- 172.21.15.16:0/3427889211 shutdown_connections 2024-09-13T16:32:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.989+0000 7f199f41a740 1 -- 172.21.15.16:0/3427889211 wait complete. 2024-09-13T16:32:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.989+0000 7f199f41a740 1 -- start start 2024-09-13T16:32:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.989+0000 7f199f41a740 1 Processor -- start 2024-09-13T16:32:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.989+0000 7f199f41a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfd63d1d0 0x55fdfd64cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.989+0000 7f199f41a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdfd63dba0 0x55fdfd64d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.989+0000 7f199f41a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdfd642d30 0x55fdfd64d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.989+0000 7f199f41a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdfd64e6b0 con 0x55fdfd63d1d0 2024-09-13T16:32:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.989+0000 7f199f41a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdfd6a8f90 con 0x55fdfd642d30 2024-09-13T16:32:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.989+0000 7f199f41a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdfd6a90d0 con 0x55fdfd63dba0 2024-09-13T16:32:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.989+0000 7f199f41a740 10 client.0 _ll_register_callbacks cb 0x7f198c009760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:32:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.989+0000 7f199a493640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfd63d1d0 0x55fdfd64cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.989+0000 7f199a493640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfd63d1d0 0x55fdfd64cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58280/0 (socket says 172.21.15.16:58280) 2024-09-13T16:32:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.989+0000 7f199a493640 1 -- 172.21.15.16:0/3134718979 learned_addr learned my addr 172.21.15.16:0/3134718979 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:32:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.989+0000 7f1999c92640 1 --2- 172.21.15.16:0/3134718979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdfd63dba0 0x55fdfd64d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.989+0000 7f199ac94640 1 --2- 172.21.15.16:0/3134718979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdfd642d30 0x55fdfd64d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.990+0000 7f199a493640 1 -- 172.21.15.16:0/3134718979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdfd63dba0 msgr2=0x55fdfd64d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:32:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.990+0000 7f199a493640 1 --2- 172.21.15.16:0/3134718979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdfd63dba0 0x55fdfd64d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.990+0000 7f199a493640 1 -- 172.21.15.16:0/3134718979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdfd642d30 msgr2=0x55fdfd64d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:32:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.990+0000 7f199a493640 1 --2- 172.21.15.16:0/3134718979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdfd642d30 0x55fdfd64d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.990+0000 7f199a493640 1 -- 172.21.15.16:0/3134718979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19941079a0 con 0x55fdfd63d1d0 2024-09-13T16:32:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.990+0000 7f199ac94640 1 --2- 172.21.15.16:0/3134718979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdfd642d30 0x55fdfd64d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:32:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.990+0000 7f199a493640 1 --2- 172.21.15.16:0/3134718979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfd63d1d0 0x55fdfd64cb20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1988005ab0 tx=0x7f1988005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf2d4b7c10d1b8a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:32:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.990+0000 7f197dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:32:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.990+0000 7f197dffb640 1 -- 172.21.15.16:0/3134718979 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1988042040 con 0x55fdfd63d1d0 2024-09-13T16:32:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.990+0000 7f197dffb640 1 -- 172.21.15.16:0/3134718979 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f19880475a0 con 0x55fdfd63d1d0 2024-09-13T16:32:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.990+0000 7f199f41a740 1 -- 172.21.15.16:0/3134718979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f198c01a070 con 0x55fdfd63d1d0 2024-09-13T16:32:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.990+0000 7f197dffb640 1 -- 172.21.15.16:0/3134718979 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1988047760 con 0x55fdfd63d1d0 2024-09-13T16:32:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.990+0000 7f199f41a740 10 client.15651 fetch_fsmap learned FSMap version 29 2024-09-13T16:32:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.990+0000 7f199f41a740 1 -- 172.21.15.16:0/3134718979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fdfd5668e0 con 0x55fdfd63d1d0 2024-09-13T16:32:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.990+0000 7f197dffb640 1 -- 172.21.15.16:0/3134718979 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f198c01a070 con 0x55fdfd63d1d0 2024-09-13T16:32:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.991+0000 7f197dffb640 1 -- 172.21.15.16:0/3134718979 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f198804d020 con 0x55fdfd63d1d0 2024-09-13T16:32:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.991+0000 7f199f41a740 10 client.15651 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T16:32:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.991+0000 7f199f41a740 10 client.15651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T16:32:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.991+0000 7f199f41a740 20 client.15651 populate_metadata read hostname 'smithi016' 2024-09-13T16:32:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.991+0000 7f199f41a740 1 --2- 172.21.15.16:0/3134718979 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55fdfd6aa680 0x55fdfd6acaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.991+0000 7f199f41a740 4 client.15651 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T16:32:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.991+0000 7f199f41a740 1 -- 172.21.15.16:0/3134718979 --> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fdfd294300 con 0x55fdfd6aa680 2024-09-13T16:32:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.991+0000 7f1999c92640 1 --2- 172.21.15.16:0/3134718979 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55fdfd6aa680 0x55fdfd6acaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.991+0000 7f197dffb640 10 client.15651 ms_handle_connect on v2:172.21.15.107:6838/1914689806 2024-09-13T16:32:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:38.991+0000 7f1999c92640 1 --2- 172.21.15.16:0/3134718979 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55fdfd6aa680 0x55fdfd6acaa0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4018c33d3311eefa server_cookie=88cfc77adf25f26f in_seq=0 out_seq=0 2024-09-13T16:32:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:39.001+0000 7f197dffb640 1 -- 172.21.15.16:0/3134718979 <== mds.0 v2:172.21.15.107:6838/1914689806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fdfd294300 con 0x55fdfd6aa680 2024-09-13T16:32:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:39.001+0000 7f197dffb640 10 client.15651 handle_command_reply: tid=0 2024-09-13T16:32:39.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:39.001+0000 7f199f41a740 1 client.15651 shutdown 2024-09-13T16:32:39.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:39.001+0000 7f199f41a740 1 -- 172.21.15.16:0/3134718979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfd63d1d0 msgr2=0x55fdfd64cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:39.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:39.001+0000 7f199f41a740 1 --2- 172.21.15.16:0/3134718979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfd63d1d0 0x55fdfd64cb20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1988005ab0 tx=0x7f1988005fd0 comp rx=0 tx=0).stop 2024-09-13T16:32:39.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:39.001+0000 7f199f41a740 1 -- 172.21.15.16:0/3134718979 shutdown_connections 2024-09-13T16:32:39.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:39.001+0000 7f199f41a740 1 --2- 172.21.15.16:0/3134718979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdfd642d30 0x55fdfd64d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:39.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:39.001+0000 7f199f41a740 1 --2- 172.21.15.16:0/3134718979 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55fdfd6aa680 0x55fdfd6acaa0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:39.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:39.001+0000 7f199f41a740 1 --2- 172.21.15.16:0/3134718979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdfd63dba0 0x55fdfd64d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:39.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:39.001+0000 7f199f41a740 1 --2- 172.21.15.16:0/3134718979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fdfd63d1d0 0x55fdfd64cb20 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:39.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:39.001+0000 7f197dffb640 0 client.15651 ms_handle_reset on v2:172.21.15.107:6838/1914689806 2024-09-13T16:32:39.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:39.001+0000 7f199f41a740 1 -- 172.21.15.16:0/3134718979 >> 172.21.15.16:0/3134718979 conn(0x55fdfd6388c0 msgr2=0x55fdfd6372f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:32:39.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:39.001+0000 7f199f41a740 1 -- 172.21.15.16:0/3134718979 shutdown_connections 2024-09-13T16:32:39.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:39.001+0000 7f199f41a740 1 -- 172.21.15.16:0/3134718979 wait complete. 2024-09-13T16:32:39.003 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:39.001+0000 7f199f41a740 20 client.15651 trim_cache size 0 max 16384 2024-09-13T16:32:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-13T16:32:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-13T16:32:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-13T16:32:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-13T16:32:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:39.002+0000 7f199c6f7640 1 -- 172.21.15.16:0/3023111596 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f19800a1090 msgr2=0x7f19800a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:39.002+0000 7f199c6f7640 1 --2- 172.21.15.16:0/3023111596 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f19800a1090 0x7f19800a3550 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f198c022000 tx=0x7f198c020000 comp rx=0 tx=0).stop 2024-09-13T16:32:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:39.002+0000 7f199c6f7640 1 -- 172.21.15.16:0/3023111596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19940ffe50 msgr2=0x7f1994198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:39.002+0000 7f199c6f7640 1 --2- 172.21.15.16:0/3023111596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19940ffe50 0x7f1994198430 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f199000d950 tx=0x7f199000de10 comp rx=0 tx=0).stop 2024-09-13T16:32:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:39.002+0000 7f199c6f7640 1 -- 172.21.15.16:0/3023111596 shutdown_connections 2024-09-13T16:32:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:39.002+0000 7f199c6f7640 1 --2- 172.21.15.16:0/3023111596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1994100810 0x7f199419f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:39.002+0000 7f199c6f7640 1 --2- 172.21.15.16:0/3023111596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19940ffe50 0x7f1994198430 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:39.002+0000 7f199c6f7640 1 --2- 172.21.15.16:0/3023111596 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f19800a1090 0x7f19800a3550 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:39.002+0000 7f199c6f7640 1 --2- 172.21.15.16:0/3023111596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19940fec60 0x7f1994197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:39.002+0000 7f199c6f7640 1 -- 172.21.15.16:0/3023111596 >> 172.21.15.16:0/3023111596 conn(0x7f19940fa3c0 msgr2=0x7f19940685e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:32:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:39.002+0000 7f199c6f7640 1 -- 172.21.15.16:0/3023111596 shutdown_connections 2024-09-13T16:32:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:39.002+0000 7f199c6f7640 1 -- 172.21.15.16:0/3023111596 wait complete. 2024-09-13T16:32:40.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:39 smithi107 ceph-mon[34139]: pgmap v400: 129 pgs: 129 active+clean; 1.0 GiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 82 MiB/s wr, 36 op/s 2024-09-13T16:32:40.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:39 smithi016 ceph-mon[31187]: pgmap v400: 129 pgs: 129 active+clean; 1.0 GiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 82 MiB/s wr, 36 op/s 2024-09-13T16:32:40.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:39 smithi111 ceph-mon[34600]: pgmap v400: 129 pgs: 129 active+clean; 1.0 GiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 82 MiB/s wr, 36 op/s 2024-09-13T16:32:40.259 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-13T16:32:40.259 INFO:tasks.workunit.client.0.smithi016.stdout: Children see throughput for 1 readers = 7081621.00 kB/sec 2024-09-13T16:32:40.259 INFO:tasks.workunit.client.0.smithi016.stdout: Parent sees throughput for 1 readers = 7072886.09 kB/sec 2024-09-13T16:32:40.259 INFO:tasks.workunit.client.0.smithi016.stdout: Min throughput per process = 7081621.00 kB/sec 2024-09-13T16:32:40.259 INFO:tasks.workunit.client.0.smithi016.stdout: Max throughput per process = 7081621.00 kB/sec 2024-09-13T16:32:40.260 INFO:tasks.workunit.client.0.smithi016.stdout: Avg throughput per process = 7081621.00 kB/sec 2024-09-13T16:32:40.260 INFO:tasks.workunit.client.0.smithi016.stdout: Min xfer = 1048576.00 kB 2024-09-13T16:32:42.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:41 smithi107 ceph-mon[34139]: pgmap v401: 129 pgs: 129 active+clean; 1.0 GiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 79 MiB/s wr, 30 op/s 2024-09-13T16:32:42.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:41 smithi016 ceph-mon[31187]: pgmap v401: 129 pgs: 129 active+clean; 1.0 GiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 79 MiB/s wr, 30 op/s 2024-09-13T16:32:42.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:41 smithi111 ceph-mon[34600]: pgmap v401: 129 pgs: 129 active+clean; 1.0 GiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 79 MiB/s wr, 30 op/s 2024-09-13T16:32:42.500 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-13T16:32:42.500 INFO:tasks.workunit.client.0.smithi016.stdout: Children see throughput for 1 re-readers = 7098205.00 kB/sec 2024-09-13T16:32:42.500 INFO:tasks.workunit.client.0.smithi016.stdout: Parent sees throughput for 1 re-readers = 7090434.93 kB/sec 2024-09-13T16:32:42.500 INFO:tasks.workunit.client.0.smithi016.stdout: Min throughput per process = 7098205.00 kB/sec 2024-09-13T16:32:42.500 INFO:tasks.workunit.client.0.smithi016.stdout: Max throughput per process = 7098205.00 kB/sec 2024-09-13T16:32:42.500 INFO:tasks.workunit.client.0.smithi016.stdout: Avg throughput per process = 7098205.00 kB/sec 2024-09-13T16:32:42.501 INFO:tasks.workunit.client.0.smithi016.stdout: Min xfer = 1048576.00 kB 2024-09-13T16:32:44.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:43 smithi107 ceph-mon[34139]: pgmap v402: 129 pgs: 129 active+clean; 1.0 GiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 71 MiB/s wr, 32 op/s 2024-09-13T16:32:44.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:43 smithi016 ceph-mon[31187]: pgmap v402: 129 pgs: 129 active+clean; 1.0 GiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 71 MiB/s wr, 32 op/s 2024-09-13T16:32:44.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:43 smithi111 ceph-mon[34600]: pgmap v402: 129 pgs: 129 active+clean; 1.0 GiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 71 MiB/s wr, 32 op/s 2024-09-13T16:32:46.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:45 smithi107 ceph-mon[34139]: pgmap v403: 129 pgs: 129 active+clean; 1.0 GiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 53 MiB/s wr, 26 op/s 2024-09-13T16:32:46.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:45 smithi016 ceph-mon[31187]: pgmap v403: 129 pgs: 129 active+clean; 1.0 GiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 53 MiB/s wr, 26 op/s 2024-09-13T16:32:46.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:45 smithi111 ceph-mon[34600]: pgmap v403: 129 pgs: 129 active+clean; 1.0 GiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 53 MiB/s wr, 26 op/s 2024-09-13T16:32:48.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:47 smithi107 ceph-mon[34139]: pgmap v404: 129 pgs: 129 active+clean; 1.0 GiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 35 MiB/s wr, 21 op/s 2024-09-13T16:32:48.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:47 smithi016 ceph-mon[31187]: pgmap v404: 129 pgs: 129 active+clean; 1.0 GiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 35 MiB/s wr, 21 op/s 2024-09-13T16:32:48.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:47 smithi111 ceph-mon[34600]: pgmap v404: 129 pgs: 129 active+clean; 1.0 GiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 35 MiB/s wr, 21 op/s 2024-09-13T16:32:48.794 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-13T16:32:48.795 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-13T16:32:48.795 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-13T16:32:48.795 INFO:tasks.workunit.client.0.smithi016.stdout:iozone test complete. 2024-09-13T16:32:48.795 INFO:tasks.workunit.client.0.smithi016.stderr:+ iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-09-13T16:32:48.863 INFO:tasks.workunit.client.0.smithi016.stdout: Iozone: Performance Test of File I/O 2024-09-13T16:32:48.863 INFO:tasks.workunit.client.0.smithi016.stdout: Version $Revision: 3.494 $ 2024-09-13T16:32:48.863 INFO:tasks.workunit.client.0.smithi016.stdout: Compiled for 64 bit mode. 2024-09-13T16:32:48.863 INFO:tasks.workunit.client.0.smithi016.stdout: Build: linux-AMD64 2024-09-13T16:32:48.863 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-13T16:32:48.863 INFO:tasks.workunit.client.0.smithi016.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-09-13T16:32:48.863 INFO:tasks.workunit.client.0.smithi016.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-09-13T16:32:48.863 INFO:tasks.workunit.client.0.smithi016.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-09-13T16:32:48.863 INFO:tasks.workunit.client.0.smithi016.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-09-13T16:32:48.863 INFO:tasks.workunit.client.0.smithi016.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-09-13T16:32:48.864 INFO:tasks.workunit.client.0.smithi016.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-09-13T16:32:48.864 INFO:tasks.workunit.client.0.smithi016.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-09-13T16:32:48.864 INFO:tasks.workunit.client.0.smithi016.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-09-13T16:32:48.864 INFO:tasks.workunit.client.0.smithi016.stdout: Alexey Skidanov, Sudhir Kumar. 2024-09-13T16:32:48.864 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-13T16:32:48.864 INFO:tasks.workunit.client.0.smithi016.stdout: Run began: Fri Sep 13 16:32:48 2024 2024-09-13T16:32:48.864 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-13T16:32:48.864 INFO:tasks.workunit.client.0.smithi016.stdout: Include close in write timing 2024-09-13T16:32:48.864 INFO:tasks.workunit.client.0.smithi016.stdout: Include fsync in write timing 2024-09-13T16:32:48.864 INFO:tasks.workunit.client.0.smithi016.stdout: File size set to 1048576 kB 2024-09-13T16:32:48.864 INFO:tasks.workunit.client.0.smithi016.stdout: Record Size 1024 kB 2024-09-13T16:32:48.864 INFO:tasks.workunit.client.0.smithi016.stdout: Command line used: iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-09-13T16:32:48.864 INFO:tasks.workunit.client.0.smithi016.stdout: Output is in kBytes/sec 2024-09-13T16:32:48.864 INFO:tasks.workunit.client.0.smithi016.stdout: Time Resolution = 0.000001 seconds. 2024-09-13T16:32:48.865 INFO:tasks.workunit.client.0.smithi016.stdout: Processor cache size set to 1024 kBytes. 2024-09-13T16:32:48.865 INFO:tasks.workunit.client.0.smithi016.stdout: Processor cache line size set to 32 bytes. 2024-09-13T16:32:48.865 INFO:tasks.workunit.client.0.smithi016.stdout: File stride size set to 17 * record size. 2024-09-13T16:32:48.865 INFO:tasks.workunit.client.0.smithi016.stdout: Throughput test with 1 process 2024-09-13T16:32:48.865 INFO:tasks.workunit.client.0.smithi016.stdout: Each process writes a 1048576 kByte file in 1024 kByte records 2024-09-13T16:32:50.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:49 smithi107 ceph-mon[34139]: pgmap v405: 129 pgs: 129 active+clean; 1.0 GiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 18 MiB/s wr, 20 op/s 2024-09-13T16:32:50.105 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:49 smithi016 ceph-mon[31187]: pgmap v405: 129 pgs: 129 active+clean; 1.0 GiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 18 MiB/s wr, 20 op/s 2024-09-13T16:32:50.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:49 smithi111 ceph-mon[34600]: pgmap v405: 129 pgs: 129 active+clean; 1.0 GiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 18 MiB/s wr, 20 op/s 2024-09-13T16:32:50.534 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-13T16:32:50.534 INFO:tasks.workunit.client.0.smithi016.stdout: Children see throughput for 1 initial writers = 641143.44 kB/sec 2024-09-13T16:32:50.534 INFO:tasks.workunit.client.0.smithi016.stdout: Parent sees throughput for 1 initial writers = 641043.46 kB/sec 2024-09-13T16:32:50.534 INFO:tasks.workunit.client.0.smithi016.stdout: Min throughput per process = 641143.44 kB/sec 2024-09-13T16:32:50.534 INFO:tasks.workunit.client.0.smithi016.stdout: Max throughput per process = 641143.44 kB/sec 2024-09-13T16:32:50.534 INFO:tasks.workunit.client.0.smithi016.stdout: Avg throughput per process = 641143.44 kB/sec 2024-09-13T16:32:50.534 INFO:tasks.workunit.client.0.smithi016.stdout: Min xfer = 1048576.00 kB 2024-09-13T16:32:52.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:51 smithi107 ceph-mon[34139]: pgmap v406: 129 pgs: 129 active+clean; 1.2 GiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 19 MiB/s wr, 16 op/s 2024-09-13T16:32:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.101+0000 7f122ec44640 1 -- 172.21.15.16:0/2411741040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12281007b0 msgr2=0x7f1228102ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.101+0000 7f122ec44640 1 --2- 172.21.15.16:0/2411741040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12281007b0 0x7f1228102ba0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f121c009900 tx=0x7f121c02a410 comp rx=0 tx=0).stop 2024-09-13T16:32:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.101+0000 7f122ec44640 1 -- 172.21.15.16:0/2411741040 shutdown_connections 2024-09-13T16:32:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.101+0000 7f122ec44640 1 --2- 172.21.15.16:0/2411741040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1228105a10 0x7f1228107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.101+0000 7f122ec44640 1 --2- 172.21.15.16:0/2411741040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12281030e0 0x7f12281054d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.101+0000 7f122ec44640 1 --2- 172.21.15.16:0/2411741040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12281007b0 0x7f1228102ba0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.101+0000 7f122ec44640 1 -- 172.21.15.16:0/2411741040 >> 172.21.15.16:0/2411741040 conn(0x7f12280fa390 msgr2=0x7f12280fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:32:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.101+0000 7f122ec44640 1 -- 172.21.15.16:0/2411741040 shutdown_connections 2024-09-13T16:32:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.101+0000 7f122ec44640 1 -- 172.21.15.16:0/2411741040 wait complete. 2024-09-13T16:32:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.101+0000 7f122ec44640 1 Processor -- start 2024-09-13T16:32:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.101+0000 7f122ec44640 1 -- start start 2024-09-13T16:32:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.102+0000 7f122ec44640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12281007b0 0x7f12281000d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.102+0000 7f122ec44640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12281030e0 0x7f12280fe760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.102+0000 7f122ec44640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1228105a10 0x7f12280feca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.102+0000 7f122ec44640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12280ff430 con 0x7f1228105a10 2024-09-13T16:32:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.102+0000 7f122ec44640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12280ff5a0 con 0x7f12281030e0 2024-09-13T16:32:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.102+0000 7f122ec44640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122819fd80 con 0x7f12281007b0 2024-09-13T16:32:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.102+0000 7f122d1e1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1228105a10 0x7f12280feca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.102+0000 7f122d1e1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1228105a10 0x7f12280feca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33708/0 (socket says 172.21.15.16:33708) 2024-09-13T16:32:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.102+0000 7f122d1e1640 1 -- 172.21.15.16:0/2199305222 learned_addr learned my addr 172.21.15.16:0/2199305222 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:32:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.102+0000 7f1227fff640 1 --2- 172.21.15.16:0/2199305222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12281030e0 0x7f12280fe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.102+0000 7f122c9e0640 1 --2- 172.21.15.16:0/2199305222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12281007b0 0x7f12281000d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.102+0000 7f122d1e1640 1 -- 172.21.15.16:0/2199305222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12281007b0 msgr2=0x7f12281000d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.102+0000 7f122d1e1640 1 --2- 172.21.15.16:0/2199305222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12281007b0 0x7f12281000d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.102+0000 7f122d1e1640 1 -- 172.21.15.16:0/2199305222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12281030e0 msgr2=0x7f12280fe760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.102+0000 7f122d1e1640 1 --2- 172.21.15.16:0/2199305222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12281030e0 0x7f12280fe760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.102+0000 7f122d1e1640 1 -- 172.21.15.16:0/2199305222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f121c009560 con 0x7f1228105a10 2024-09-13T16:32:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.102+0000 7f122d1e1640 1 --2- 172.21.15.16:0/2199305222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1228105a10 0x7f12280feca0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f122000eb70 tx=0x7f122000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:32:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.102+0000 7f1225ffb640 1 -- 172.21.15.16:0/2199305222 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1220004d20 con 0x7f1228105a10 2024-09-13T16:32:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.103+0000 7f1225ffb640 1 -- 172.21.15.16:0/2199305222 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1220004e80 con 0x7f1228105a10 2024-09-13T16:32:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.103+0000 7f1225ffb640 1 -- 172.21.15.16:0/2199305222 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1220010690 con 0x7f1228105a10 2024-09-13T16:32:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.103+0000 7f122ec44640 1 -- 172.21.15.16:0/2199305222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f122819ffe0 con 0x7f1228105a10 2024-09-13T16:32:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.103+0000 7f122ec44640 1 -- 172.21.15.16:0/2199305222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12281a0410 con 0x7f1228105a10 2024-09-13T16:32:52.106 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:51 smithi016 ceph-mon[31187]: pgmap v406: 129 pgs: 129 active+clean; 1.2 GiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 19 MiB/s wr, 16 op/s 2024-09-13T16:32:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.105+0000 7f1225ffb640 1 -- 172.21.15.16:0/2199305222 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1220005020 con 0x7f1228105a10 2024-09-13T16:32:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.105+0000 7f1231a39740 1 Processor -- start 2024-09-13T16:32:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.106+0000 7f1231a39740 1 -- start start 2024-09-13T16:32:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.106+0000 7f1231a39740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c52f7c1d0 0x560c52f7c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.106+0000 7f1231a39740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c52f7cba0 0x560c52f816c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.106+0000 7f1231a39740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c52f81d30 0x560c52f841c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.106+0000 7f1231a39740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c52ee6c30 con 0x560c52f7c1d0 2024-09-13T16:32:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.106+0000 7f1231a39740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c52ee43c0 con 0x560c52f81d30 2024-09-13T16:32:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.106+0000 7f1231a39740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c52edfcb0 con 0x560c52f7cba0 2024-09-13T16:32:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.106+0000 7f1225ffb640 1 --2- 172.21.15.16:0/2199305222 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f12180a1110 0x7f12180a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.106+0000 7f1227fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c52f7c1d0 0x560c52f7c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.106+0000 7f1227fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c52f7c1d0 0x560c52f7c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33716/0 (socket says 172.21.15.16:33716) 2024-09-13T16:32:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.106+0000 7f1227fff640 1 -- 172.21.15.16:0/2184931140 learned_addr learned my addr 172.21.15.16:0/2184931140 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:32:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.106+0000 7f122c9e0640 1 --2- 172.21.15.16:0/2184931140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c52f7cba0 0x560c52f816c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.106+0000 7f122d1e1640 1 --2- 172.21.15.16:0/2199305222 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f12180a1110 0x7f12180a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.106+0000 7f1225ffb640 1 -- 172.21.15.16:0/2199305222 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f1220014070 con 0x7f1228105a10 2024-09-13T16:32:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.106+0000 7f1227fff640 1 --2- 172.21.15.16:0/2184931140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c52f81d30 0x560c52f841c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.106+0000 7f1227fff640 1 -- 172.21.15.16:0/2184931140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c52f7cba0 msgr2=0x560c52f816c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:32:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.106+0000 7f1227fff640 1 --2- 172.21.15.16:0/2184931140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c52f7cba0 0x560c52f816c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.106+0000 7f1227fff640 1 -- 172.21.15.16:0/2184931140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c52f81d30 msgr2=0x560c52f841c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:32:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.106+0000 7f1227fff640 1 --2- 172.21.15.16:0/2184931140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c52f81d30 0x560c52f841c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.106+0000 7f1227fff640 1 -- 172.21.15.16:0/2184931140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c52e86910 con 0x560c52f7c1d0 2024-09-13T16:32:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.107+0000 7f122d1e1640 1 --2- 172.21.15.16:0/2199305222 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f12180a1110 0x7f12180a35d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f12200a2470 tx=0x7f1220085040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:32:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.107+0000 7f1227fff640 1 --2- 172.21.15.16:0/2184931140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c52f7c1d0 0x560c52f7c5d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f12140137c0 tx=0x7f1214009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b571dfe9b8e3b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:32:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.107+0000 7f1212ffd640 1 -- 172.21.15.16:0/2184931140 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1214013d40 con 0x560c52f7c1d0 2024-09-13T16:32:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.107+0000 7f1212ffd640 1 -- 172.21.15.16:0/2184931140 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f12140044c0 con 0x560c52f7c1d0 2024-09-13T16:32:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.107+0000 7f1212ffd640 1 -- 172.21.15.16:0/2184931140 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1214003850 con 0x560c52f7c1d0 2024-09-13T16:32:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.108+0000 7f1231a39740 1 -- 172.21.15.16:0/2184931140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c52f7c1d0 msgr2=0x560c52f7c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.108+0000 7f1231a39740 1 --2- 172.21.15.16:0/2184931140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c52f7c1d0 0x560c52f7c5d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f12140137c0 tx=0x7f1214009780 comp rx=0 tx=0).stop 2024-09-13T16:32:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.108+0000 7f1231a39740 1 -- 172.21.15.16:0/2184931140 shutdown_connections 2024-09-13T16:32:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.108+0000 7f1231a39740 1 --2- 172.21.15.16:0/2184931140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c52f81d30 0x560c52f841c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.108+0000 7f1231a39740 1 --2- 172.21.15.16:0/2184931140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c52f7cba0 0x560c52f816c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.108+0000 7f1231a39740 1 --2- 172.21.15.16:0/2184931140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c52f7c1d0 0x560c52f7c5d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.108+0000 7f1231a39740 1 -- 172.21.15.16:0/2184931140 >> 172.21.15.16:0/2184931140 conn(0x560c52f778c0 msgr2=0x560c52f77cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:32:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.108+0000 7f1231a39740 1 -- 172.21.15.16:0/2184931140 shutdown_connections 2024-09-13T16:32:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.108+0000 7f1231a39740 1 -- 172.21.15.16:0/2184931140 wait complete. 2024-09-13T16:32:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.108+0000 7f1231a39740 1 -- start start 2024-09-13T16:32:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.108+0000 7f1231a39740 1 Processor -- start 2024-09-13T16:32:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.109+0000 7f1231a39740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c52f7c1d0 0x560c52f85f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.109+0000 7f1231a39740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c52f7cba0 0x560c52f86450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.109+0000 7f1231a39740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c52f81d30 0x560c52f8d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.109+0000 7f1231a39740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c52e9de80 con 0x560c52f7cba0 2024-09-13T16:32:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.109+0000 7f1231a39740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c52e9db20 con 0x560c52f81d30 2024-09-13T16:32:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.109+0000 7f1231a39740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c52e63260 con 0x560c52f7c1d0 2024-09-13T16:32:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.109+0000 7f1231a39740 10 client.0 _ll_register_callbacks cb 0x560c52e98df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:32:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.109+0000 7f122c9e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c52f7cba0 0x560c52f86450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.109+0000 7f122c9e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c52f7cba0 0x560c52f86450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33730/0 (socket says 172.21.15.16:33730) 2024-09-13T16:32:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.109+0000 7f122c9e0640 1 -- 172.21.15.16:0/3226488116 learned_addr learned my addr 172.21.15.16:0/3226488116 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:32:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.109+0000 7f1227fff640 1 --2- 172.21.15.16:0/3226488116 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c52f7c1d0 0x560c52f85f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.109+0000 7f1227fff640 1 --2- 172.21.15.16:0/3226488116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c52f81d30 0x560c52f8d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.109+0000 7f122c9e0640 1 -- 172.21.15.16:0/3226488116 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c52f7c1d0 msgr2=0x560c52f85f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:32:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.109+0000 7f122c9e0640 1 --2- 172.21.15.16:0/3226488116 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c52f7c1d0 0x560c52f85f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.109+0000 7f122c9e0640 1 -- 172.21.15.16:0/3226488116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c52f81d30 msgr2=0x560c52f8d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:32:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.109+0000 7f122c9e0640 1 --2- 172.21.15.16:0/3226488116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c52f81d30 0x560c52f8d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.109+0000 7f122c9e0640 1 -- 172.21.15.16:0/3226488116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f121401b040 con 0x560c52f7cba0 2024-09-13T16:32:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.109+0000 7f1227fff640 1 --2- 172.21.15.16:0/3226488116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c52f81d30 0x560c52f8d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:32:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.109+0000 7f122c9e0640 1 --2- 172.21.15.16:0/3226488116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c52f7cba0 0x560c52f86450 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f121c009900 tx=0x7f121c004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e8b1527e01c6a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:32:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.109+0000 7f1207fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:32:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.109+0000 7f1207fff640 1 -- 172.21.15.16:0/3226488116 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c0037f0 con 0x560c52f7cba0 2024-09-13T16:32:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.109+0000 7f1207fff640 1 -- 172.21.15.16:0/3226488116 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f121c003950 con 0x560c52f7cba0 2024-09-13T16:32:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.109+0000 7f1231a39740 1 -- 172.21.15.16:0/3226488116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c52f08680 con 0x560c52f7cba0 2024-09-13T16:32:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.110+0000 7f1207fff640 1 -- 172.21.15.16:0/3226488116 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f121c003b10 con 0x560c52f7cba0 2024-09-13T16:32:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.110+0000 7f1231a39740 10 client.15663 fetch_fsmap learned FSMap version 29 2024-09-13T16:32:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.110+0000 7f1231a39740 1 -- 172.21.15.16:0/3226488116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560c52ede330 con 0x560c52f7cba0 2024-09-13T16:32:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.110+0000 7f1207fff640 1 -- 172.21.15.16:0/3226488116 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560c52f08680 con 0x560c52f7cba0 2024-09-13T16:32:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.110+0000 7f1207fff640 1 -- 172.21.15.16:0/3226488116 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f121c03c450 con 0x560c52f7cba0 2024-09-13T16:32:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.110+0000 7f1231a39740 10 client.15663 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T16:32:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.110+0000 7f1231a39740 10 client.15663 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T16:32:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.110+0000 7f1231a39740 20 client.15663 populate_metadata read hostname 'smithi016' 2024-09-13T16:32:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.110+0000 7f1231a39740 1 --2- 172.21.15.16:0/3226488116 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x560c52f91bf0 0x560c52f94010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.110+0000 7f1231a39740 4 client.15663 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T16:32:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.110+0000 7f1231a39740 1 -- 172.21.15.16:0/3226488116 --> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c52ed35d0 con 0x560c52f91bf0 2024-09-13T16:32:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.111+0000 7f1227fff640 1 --2- 172.21.15.16:0/3226488116 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x560c52f91bf0 0x560c52f94010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.111+0000 7f1227fff640 1 --2- 172.21.15.16:0/3226488116 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x560c52f91bf0 0x560c52f94010 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=144ec2522c374a6 server_cookie=84f6bfa43366a556 in_seq=0 out_seq=0 2024-09-13T16:32:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.111+0000 7f1207fff640 10 client.15663 ms_handle_connect on v2:172.21.15.107:6838/1914689806 2024-09-13T16:32:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.112+0000 7f1207fff640 1 -- 172.21.15.16:0/3226488116 <== mds.0 v2:172.21.15.107:6838/1914689806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560c52ed35d0 con 0x560c52f91bf0 2024-09-13T16:32:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.112+0000 7f1207fff640 10 client.15663 handle_command_reply: tid=0 2024-09-13T16:32:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.112+0000 7f1231a39740 1 client.15663 shutdown 2024-09-13T16:32:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.112+0000 7f1231a39740 1 -- 172.21.15.16:0/3226488116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c52f7cba0 msgr2=0x560c52f86450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.112+0000 7f1231a39740 1 --2- 172.21.15.16:0/3226488116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c52f7cba0 0x560c52f86450 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f121c009900 tx=0x7f121c004440 comp rx=0 tx=0).stop 2024-09-13T16:32:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.112+0000 7f1231a39740 1 -- 172.21.15.16:0/3226488116 shutdown_connections 2024-09-13T16:32:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.112+0000 7f1231a39740 1 --2- 172.21.15.16:0/3226488116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c52f81d30 0x560c52f8d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.112+0000 7f1231a39740 1 --2- 172.21.15.16:0/3226488116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c52f7cba0 0x560c52f86450 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.112+0000 7f1231a39740 1 --2- 172.21.15.16:0/3226488116 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x560c52f91bf0 0x560c52f94010 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.112+0000 7f1231a39740 1 --2- 172.21.15.16:0/3226488116 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c52f7c1d0 0x560c52f85f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.112+0000 7f1207fff640 0 client.15663 ms_handle_reset on v2:172.21.15.107:6838/1914689806 2024-09-13T16:32:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.112+0000 7f1231a39740 1 -- 172.21.15.16:0/3226488116 >> 172.21.15.16:0/3226488116 conn(0x560c52f778c0 msgr2=0x560c52f76b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:32:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.112+0000 7f1231a39740 1 -- 172.21.15.16:0/3226488116 shutdown_connections 2024-09-13T16:32:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.112+0000 7f1231a39740 1 -- 172.21.15.16:0/3226488116 wait complete. 2024-09-13T16:32:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.112+0000 7f1231a39740 20 client.15663 trim_cache size 0 max 16384 2024-09-13T16:32:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.124+0000 7f1231a39740 1 Processor -- start 2024-09-13T16:32:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.125+0000 7f1231a39740 1 -- start start 2024-09-13T16:32:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.125+0000 7f1231a39740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c52f7c1d0 0x560c52f79cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.125+0000 7f1231a39740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c52f7cba0 0x560c52f8d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.125+0000 7f1231a39740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c52f81d30 0x560c52f7a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.125+0000 7f1231a39740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c52f08680 con 0x560c52f81d30 2024-09-13T16:32:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.125+0000 7f1231a39740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c52f7d2b0 con 0x560c52f7cba0 2024-09-13T16:32:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.125+0000 7f1231a39740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c52f84680 con 0x560c52f7c1d0 2024-09-13T16:32:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.125+0000 7f1227fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c52f81d30 0x560c52f7a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.125+0000 7f1227fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c52f81d30 0x560c52f7a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33742/0 (socket says 172.21.15.16:33742) 2024-09-13T16:32:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.125+0000 7f1227fff640 1 -- 172.21.15.16:0/2489767830 learned_addr learned my addr 172.21.15.16:0/2489767830 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:32:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.125+0000 7f1227fff640 1 --2- 172.21.15.16:0/2489767830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c52f7c1d0 0x560c52f79cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.125+0000 7f122c9e0640 1 --2- 172.21.15.16:0/2489767830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c52f7cba0 0x560c52f8d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.125+0000 7f1227fff640 1 -- 172.21.15.16:0/2489767830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c52f7c1d0 msgr2=0x560c52f79cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:32:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.125+0000 7f1227fff640 1 --2- 172.21.15.16:0/2489767830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c52f7c1d0 0x560c52f79cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.125+0000 7f1227fff640 1 -- 172.21.15.16:0/2489767830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c52f7cba0 msgr2=0x560c52f8d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:32:52.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.125+0000 7f1227fff640 1 --2- 172.21.15.16:0/2489767830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c52f7cba0 0x560c52f8d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.125+0000 7f1227fff640 1 -- 172.21.15.16:0/2489767830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c52e80300 con 0x560c52f81d30 2024-09-13T16:32:52.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.125+0000 7f122c9e0640 1 --2- 172.21.15.16:0/2489767830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c52f7cba0 0x560c52f8d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:32:52.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.125+0000 7f1227fff640 1 --2- 172.21.15.16:0/2489767830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c52f81d30 0x560c52f7a1f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f12140058c0 tx=0x7f1214005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52189157cd7b663f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:32:52.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.125+0000 7f12137fe640 1 -- 172.21.15.16:0/2489767830 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1214025040 con 0x560c52f81d30 2024-09-13T16:32:52.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.126+0000 7f12137fe640 1 -- 172.21.15.16:0/2489767830 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1214011c80 con 0x560c52f81d30 2024-09-13T16:32:52.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.126+0000 7f12137fe640 1 -- 172.21.15.16:0/2489767830 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121402d600 con 0x560c52f81d30 2024-09-13T16:32:52.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.126+0000 7f1231a39740 1 -- 172.21.15.16:0/2489767830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c52f81d30 msgr2=0x560c52f7a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:52.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.126+0000 7f1231a39740 1 --2- 172.21.15.16:0/2489767830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c52f81d30 0x560c52f7a1f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f12140058c0 tx=0x7f1214005f90 comp rx=0 tx=0).stop 2024-09-13T16:32:52.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.126+0000 7f1231a39740 1 -- 172.21.15.16:0/2489767830 shutdown_connections 2024-09-13T16:32:52.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.126+0000 7f1231a39740 1 --2- 172.21.15.16:0/2489767830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c52f81d30 0x560c52f7a1f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.126+0000 7f1231a39740 1 --2- 172.21.15.16:0/2489767830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c52f7cba0 0x560c52f8d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.126+0000 7f1231a39740 1 --2- 172.21.15.16:0/2489767830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c52f7c1d0 0x560c52f79cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.126+0000 7f1231a39740 1 -- 172.21.15.16:0/2489767830 >> 172.21.15.16:0/2489767830 conn(0x560c52f778c0 msgr2=0x560c52f7db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:32:52.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.126+0000 7f1231a39740 1 -- 172.21.15.16:0/2489767830 shutdown_connections 2024-09-13T16:32:52.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.126+0000 7f1231a39740 1 -- 172.21.15.16:0/2489767830 wait complete. 2024-09-13T16:32:52.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.126+0000 7f1231a39740 1 -- start start 2024-09-13T16:32:52.130 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.126+0000 7f1231a39740 1 Processor -- start 2024-09-13T16:32:52.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.127+0000 7f1231a39740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c52f7c1d0 0x560c52f8bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:52.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.127+0000 7f1231a39740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c52f7cba0 0x560c52f8c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:52.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.127+0000 7f1231a39740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c52f81d30 0x560c52f8c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:52.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.127+0000 7f1231a39740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c52f8d7f0 con 0x560c52f81d30 2024-09-13T16:32:52.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.127+0000 7f1231a39740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c52fe80e0 con 0x560c52f7c1d0 2024-09-13T16:32:52.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.127+0000 7f1231a39740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c52fe8270 con 0x560c52f7cba0 2024-09-13T16:32:52.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.127+0000 7f1231a39740 10 client.0 _ll_register_callbacks cb 0x7f1214009a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:32:52.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.127+0000 7f1227fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c52f81d30 0x560c52f8c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:52.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.127+0000 7f1227fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c52f81d30 0x560c52f8c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33758/0 (socket says 172.21.15.16:33758) 2024-09-13T16:32:52.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.127+0000 7f1227fff640 1 -- 172.21.15.16:0/2241755793 learned_addr learned my addr 172.21.15.16:0/2241755793 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:32:52.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.127+0000 7f1227fff640 1 --2- 172.21.15.16:0/2241755793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c52f7c1d0 0x560c52f8bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:52.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.127+0000 7f122c9e0640 1 --2- 172.21.15.16:0/2241755793 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c52f7cba0 0x560c52f8c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:52.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.127+0000 7f1227fff640 1 -- 172.21.15.16:0/2241755793 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c52f7cba0 msgr2=0x560c52f8c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:32:52.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.127+0000 7f1227fff640 1 --2- 172.21.15.16:0/2241755793 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c52f7cba0 0x560c52f8c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.127+0000 7f1227fff640 1 -- 172.21.15.16:0/2241755793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c52f7c1d0 msgr2=0x560c52f8bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:32:52.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.127+0000 7f1227fff640 1 --2- 172.21.15.16:0/2241755793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c52f7c1d0 0x560c52f8bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.127+0000 7f1227fff640 1 -- 172.21.15.16:0/2241755793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1214028040 con 0x560c52f81d30 2024-09-13T16:32:52.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.128+0000 7f1227fff640 1 --2- 172.21.15.16:0/2241755793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c52f81d30 0x560c52f8c5f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f121402d340 tx=0x7f1214013ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b500b297ada2639 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:32:52.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.128+0000 7f1207fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:32:52.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.128+0000 7f1207fff640 1 -- 172.21.15.16:0/2241755793 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1214025070 con 0x560c52f81d30 2024-09-13T16:32:52.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.128+0000 7f1207fff640 1 -- 172.21.15.16:0/2241755793 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f12140131a0 con 0x560c52f81d30 2024-09-13T16:32:52.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.128+0000 7f1231a39740 1 -- 172.21.15.16:0/2241755793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f12140044c0 con 0x560c52f81d30 2024-09-13T16:32:52.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.128+0000 7f1207fff640 1 -- 172.21.15.16:0/2241755793 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1214013360 con 0x560c52f81d30 2024-09-13T16:32:52.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.128+0000 7f1231a39740 10 client.15669 fetch_fsmap learned FSMap version 29 2024-09-13T16:32:52.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.128+0000 7f1231a39740 1 -- 172.21.15.16:0/2241755793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560c52ea58e0 con 0x560c52f81d30 2024-09-13T16:32:52.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.128+0000 7f1207fff640 1 -- 172.21.15.16:0/2241755793 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f12140044c0 con 0x560c52f81d30 2024-09-13T16:32:52.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.128+0000 7f1207fff640 1 -- 172.21.15.16:0/2241755793 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1214004630 con 0x560c52f81d30 2024-09-13T16:32:52.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.128+0000 7f1231a39740 10 client.15669 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T16:32:52.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.128+0000 7f1231a39740 10 client.15669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T16:32:52.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.128+0000 7f1231a39740 20 client.15669 populate_metadata read hostname 'smithi016' 2024-09-13T16:32:52.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.128+0000 7f1231a39740 1 --2- 172.21.15.16:0/2241755793 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x560c52fe98c0 0x560c52febce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:32:52.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.128+0000 7f1231a39740 4 client.15669 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T16:32:52.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.128+0000 7f1231a39740 1 -- 172.21.15.16:0/2241755793 --> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560c52bd3300 con 0x560c52fe98c0 2024-09-13T16:32:52.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.129+0000 7f122c9e0640 1 --2- 172.21.15.16:0/2241755793 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x560c52fe98c0 0x560c52febce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:32:52.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.129+0000 7f122c9e0640 1 --2- 172.21.15.16:0/2241755793 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x560c52fe98c0 0x560c52febce0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7dbfd56183cf63b server_cookie=73215b09593beb96 in_seq=0 out_seq=0 2024-09-13T16:32:52.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.129+0000 7f1207fff640 10 client.15669 ms_handle_connect on v2:172.21.15.107:6838/1914689806 2024-09-13T16:32:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.140+0000 7f1207fff640 1 -- 172.21.15.16:0/2241755793 <== mds.0 v2:172.21.15.107:6838/1914689806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560c52bd3300 con 0x560c52fe98c0 2024-09-13T16:32:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.140+0000 7f1207fff640 10 client.15669 handle_command_reply: tid=0 2024-09-13T16:32:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.140+0000 7f1231a39740 1 client.15669 shutdown 2024-09-13T16:32:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.140+0000 7f1231a39740 1 -- 172.21.15.16:0/2241755793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c52f81d30 msgr2=0x560c52f8c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.140+0000 7f1231a39740 1 --2- 172.21.15.16:0/2241755793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c52f81d30 0x560c52f8c5f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f121402d340 tx=0x7f1214013ee0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.140+0000 7f1231a39740 1 -- 172.21.15.16:0/2241755793 shutdown_connections 2024-09-13T16:32:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.140+0000 7f1231a39740 1 --2- 172.21.15.16:0/2241755793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c52f81d30 0x560c52f8c5f0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.140+0000 7f1231a39740 1 --2- 172.21.15.16:0/2241755793 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x560c52fe98c0 0x560c52febce0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.140+0000 7f1231a39740 1 --2- 172.21.15.16:0/2241755793 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c52f7cba0 0x560c52f8c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.140+0000 7f1231a39740 1 --2- 172.21.15.16:0/2241755793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c52f7c1d0 0x560c52f8bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.140+0000 7f1207fff640 0 client.15669 ms_handle_reset on v2:172.21.15.107:6838/1914689806 2024-09-13T16:32:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.140+0000 7f1231a39740 1 -- 172.21.15.16:0/2241755793 >> 172.21.15.16:0/2241755793 conn(0x560c52f778c0 msgr2=0x560c52f762f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:32:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.140+0000 7f1231a39740 1 -- 172.21.15.16:0/2241755793 shutdown_connections 2024-09-13T16:32:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.140+0000 7f1231a39740 1 -- 172.21.15.16:0/2241755793 wait complete. 2024-09-13T16:32:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.140+0000 7f1231a39740 20 client.15669 trim_cache size 0 max 16384 2024-09-13T16:32:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-13T16:32:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-13T16:32:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-13T16:32:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-13T16:32:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.141+0000 7f122ec44640 1 -- 172.21.15.16:0/2199305222 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f12180a1110 msgr2=0x7f12180a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.141+0000 7f122ec44640 1 --2- 172.21.15.16:0/2199305222 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f12180a1110 0x7f12180a35d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f12200a2470 tx=0x7f1220085040 comp rx=0 tx=0).stop 2024-09-13T16:32:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.141+0000 7f122ec44640 1 -- 172.21.15.16:0/2199305222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1228105a10 msgr2=0x7f12280feca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:32:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.141+0000 7f122ec44640 1 --2- 172.21.15.16:0/2199305222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1228105a10 0x7f12280feca0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f122000eb70 tx=0x7f122000c6a0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.141+0000 7f122ec44640 1 -- 172.21.15.16:0/2199305222 shutdown_connections 2024-09-13T16:32:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.141+0000 7f122ec44640 1 --2- 172.21.15.16:0/2199305222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1228105a10 0x7f12280feca0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.141+0000 7f122ec44640 1 --2- 172.21.15.16:0/2199305222 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f12180a1110 0x7f12180a35d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.141+0000 7f122ec44640 1 --2- 172.21.15.16:0/2199305222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12281030e0 0x7f12280fe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.141+0000 7f122ec44640 1 --2- 172.21.15.16:0/2199305222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12281007b0 0x7f12281000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:32:52.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.141+0000 7f122ec44640 1 -- 172.21.15.16:0/2199305222 >> 172.21.15.16:0/2199305222 conn(0x7f12280fa390 msgr2=0x7f1228101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:32:52.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.141+0000 7f122ec44640 1 -- 172.21.15.16:0/2199305222 shutdown_connections 2024-09-13T16:32:52.145 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:32:52.141+0000 7f122ec44640 1 -- 172.21.15.16:0/2199305222 wait complete. 2024-09-13T16:32:52.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:51 smithi111 ceph-mon[34600]: pgmap v406: 129 pgs: 129 active+clean; 1.2 GiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 19 MiB/s wr, 16 op/s 2024-09-13T16:32:54.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:53 smithi107 ceph-mon[34139]: pgmap v407: 129 pgs: 129 active+clean; 1.3 GiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 MiB/s wr, 33 op/s 2024-09-13T16:32:54.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:53 smithi016 ceph-mon[31187]: pgmap v407: 129 pgs: 129 active+clean; 1.3 GiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 MiB/s wr, 33 op/s 2024-09-13T16:32:54.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:53 smithi111 ceph-mon[34600]: pgmap v407: 129 pgs: 129 active+clean; 1.3 GiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 MiB/s wr, 33 op/s 2024-09-13T16:32:54.638 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-13T16:32:54.638 INFO:tasks.workunit.client.0.smithi016.stdout: Children see throughput for 1 rewriters = 718571.31 kB/sec 2024-09-13T16:32:54.638 INFO:tasks.workunit.client.0.smithi016.stdout: Parent sees throughput for 1 rewriters = 718447.24 kB/sec 2024-09-13T16:32:54.638 INFO:tasks.workunit.client.0.smithi016.stdout: Min throughput per process = 718571.31 kB/sec 2024-09-13T16:32:54.638 INFO:tasks.workunit.client.0.smithi016.stdout: Max throughput per process = 718571.31 kB/sec 2024-09-13T16:32:54.639 INFO:tasks.workunit.client.0.smithi016.stdout: Avg throughput per process = 718571.31 kB/sec 2024-09-13T16:32:54.639 INFO:tasks.workunit.client.0.smithi016.stdout: Min xfer = 1048576.00 kB 2024-09-13T16:32:56.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:55 smithi107 ceph-mon[34139]: pgmap v408: 129 pgs: 129 active+clean; 1.3 GiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 93 MiB/s wr, 51 op/s 2024-09-13T16:32:56.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:55 smithi016 ceph-mon[31187]: pgmap v408: 129 pgs: 129 active+clean; 1.3 GiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 93 MiB/s wr, 51 op/s 2024-09-13T16:32:56.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:55 smithi111 ceph-mon[34600]: pgmap v408: 129 pgs: 129 active+clean; 1.3 GiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 93 MiB/s wr, 51 op/s 2024-09-13T16:32:57.999 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-13T16:32:57.999 INFO:tasks.workunit.client.0.smithi016.stdout: Children see throughput for 1 readers = 7648313.50 kB/sec 2024-09-13T16:32:57.999 INFO:tasks.workunit.client.0.smithi016.stdout: Parent sees throughput for 1 readers = 7634678.20 kB/sec 2024-09-13T16:32:58.000 INFO:tasks.workunit.client.0.smithi016.stdout: Min throughput per process = 7648313.50 kB/sec 2024-09-13T16:32:58.000 INFO:tasks.workunit.client.0.smithi016.stdout: Max throughput per process = 7648313.50 kB/sec 2024-09-13T16:32:58.000 INFO:tasks.workunit.client.0.smithi016.stdout: Avg throughput per process = 7648313.50 kB/sec 2024-09-13T16:32:58.000 INFO:tasks.workunit.client.0.smithi016.stdout: Min xfer = 1048576.00 kB 2024-09-13T16:32:58.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:57 smithi107 ceph-mon[34139]: pgmap v409: 129 pgs: 129 active+clean; 1.0 GiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 127 MiB/s wr, 67 op/s 2024-09-13T16:32:58.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:57 smithi016 ceph-mon[31187]: pgmap v409: 129 pgs: 129 active+clean; 1.0 GiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 127 MiB/s wr, 67 op/s 2024-09-13T16:32:58.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:57 smithi111 ceph-mon[34600]: pgmap v409: 129 pgs: 129 active+clean; 1.0 GiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 127 MiB/s wr, 67 op/s 2024-09-13T16:33:00.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:32:59 smithi107 ceph-mon[34139]: pgmap v410: 129 pgs: 129 active+clean; 1.0 GiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 169 MiB/s wr, 81 op/s 2024-09-13T16:33:00.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:32:59 smithi016 ceph-mon[31187]: pgmap v410: 129 pgs: 129 active+clean; 1.0 GiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 169 MiB/s wr, 81 op/s 2024-09-13T16:33:00.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:32:59 smithi111 ceph-mon[34600]: pgmap v410: 129 pgs: 129 active+clean; 1.0 GiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 169 MiB/s wr, 81 op/s 2024-09-13T16:33:00.216 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-13T16:33:00.216 INFO:tasks.workunit.client.0.smithi016.stdout: Children see throughput for 1 re-readers = 7696080.00 kB/sec 2024-09-13T16:33:00.216 INFO:tasks.workunit.client.0.smithi016.stdout: Parent sees throughput for 1 re-readers = 7682998.54 kB/sec 2024-09-13T16:33:00.216 INFO:tasks.workunit.client.0.smithi016.stdout: Min throughput per process = 7696080.00 kB/sec 2024-09-13T16:33:00.216 INFO:tasks.workunit.client.0.smithi016.stdout: Max throughput per process = 7696080.00 kB/sec 2024-09-13T16:33:00.216 INFO:tasks.workunit.client.0.smithi016.stdout: Avg throughput per process = 7696080.00 kB/sec 2024-09-13T16:33:00.216 INFO:tasks.workunit.client.0.smithi016.stdout: Min xfer = 1048576.00 kB 2024-09-13T16:33:02.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:01 smithi107 ceph-mon[34139]: pgmap v411: 129 pgs: 129 active+clean; 1.0 GiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 171 MiB/s wr, 78 op/s 2024-09-13T16:33:02.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:01 smithi016 ceph-mon[31187]: pgmap v411: 129 pgs: 129 active+clean; 1.0 GiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 171 MiB/s wr, 78 op/s 2024-09-13T16:33:02.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:01 smithi111 ceph-mon[34600]: pgmap v411: 129 pgs: 129 active+clean; 1.0 GiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 171 MiB/s wr, 78 op/s 2024-09-13T16:33:04.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:03 smithi107 ceph-mon[34139]: pgmap v412: 129 pgs: 129 active+clean; 1.0 GiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 154 MiB/s wr, 76 op/s 2024-09-13T16:33:04.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:03 smithi016 ceph-mon[31187]: pgmap v412: 129 pgs: 129 active+clean; 1.0 GiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 154 MiB/s wr, 76 op/s 2024-09-13T16:33:04.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:03 smithi111 ceph-mon[34600]: pgmap v412: 129 pgs: 129 active+clean; 1.0 GiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 154 MiB/s wr, 76 op/s 2024-09-13T16:33:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.239+0000 7f74da0b1640 1 -- 172.21.15.16:0/135429985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d40fe580 msgr2=0x7f74d40fe980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.239+0000 7f74da0b1640 1 --2- 172.21.15.16:0/135429985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d40fe580 0x7f74d40fe980 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f74c4009900 tx=0x7f74c402a3f0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.239+0000 7f74da0b1640 1 -- 172.21.15.16:0/135429985 shutdown_connections 2024-09-13T16:33:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.239+0000 7f74da0b1640 1 --2- 172.21.15.16:0/135429985 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74d40ff860 0x7f74d40ffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.239+0000 7f74da0b1640 1 --2- 172.21.15.16:0/135429985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74d40feec0 0x7f74d40ff320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.239+0000 7f74da0b1640 1 --2- 172.21.15.16:0/135429985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d40fe580 0x7f74d40fe980 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.239+0000 7f74da0b1640 1 -- 172.21.15.16:0/135429985 >> 172.21.15.16:0/135429985 conn(0x7f74d40fa370 msgr2=0x7f74d40fc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:33:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.239+0000 7f74da0b1640 1 -- 172.21.15.16:0/135429985 shutdown_connections 2024-09-13T16:33:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.239+0000 7f74da0b1640 1 -- 172.21.15.16:0/135429985 wait complete. 2024-09-13T16:33:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.240+0000 7f74da0b1640 1 Processor -- start 2024-09-13T16:33:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.240+0000 7f74da0b1640 1 -- start start 2024-09-13T16:33:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.240+0000 7f74da0b1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74d40fe580 0x7f74d4197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.240+0000 7f74da0b1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d40feec0 0x7f74d41982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.240+0000 7f74da0b1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74d40ff860 0x7f74d419f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.240+0000 7f74da0b1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d419f880 con 0x7f74d40feec0 2024-09-13T16:33:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.240+0000 7f74da0b1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d419f9f0 con 0x7f74d40fe580 2024-09-13T16:33:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.240+0000 7f74da0b1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d419fbb0 con 0x7f74d40ff860 2024-09-13T16:33:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.240+0000 7f74d2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d40feec0 0x7f74d41982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.241+0000 7f74d2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d40feec0 0x7f74d41982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42254/0 (socket says 172.21.15.16:42254) 2024-09-13T16:33:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.241+0000 7f74d2ffd640 1 -- 172.21.15.16:0/3592239591 learned_addr learned my addr 172.21.15.16:0/3592239591 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:33:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.241+0000 7f74d37fe640 1 --2- 172.21.15.16:0/3592239591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74d40fe580 0x7f74d4197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.241+0000 7f74d3fff640 1 --2- 172.21.15.16:0/3592239591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74d40ff860 0x7f74d419f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:05.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.241+0000 7f74d2ffd640 1 -- 172.21.15.16:0/3592239591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74d40ff860 msgr2=0x7f74d419f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:05.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.241+0000 7f74d2ffd640 1 --2- 172.21.15.16:0/3592239591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74d40ff860 0x7f74d419f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.241+0000 7f74d2ffd640 1 -- 172.21.15.16:0/3592239591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74d40fe580 msgr2=0x7f74d4197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:05.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.241+0000 7f74d2ffd640 1 --2- 172.21.15.16:0/3592239591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74d40fe580 0x7f74d4197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.241+0000 7f74d2ffd640 1 -- 172.21.15.16:0/3592239591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c4009560 con 0x7f74d40feec0 2024-09-13T16:33:05.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.241+0000 7f74d37fe640 1 --2- 172.21.15.16:0/3592239591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74d40fe580 0x7f74d4197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:33:05.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.241+0000 7f74d2ffd640 1 --2- 172.21.15.16:0/3592239591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d40feec0 0x7f74d41982e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f74c800c920 tx=0x7f74c800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:33:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.241+0000 7f74d0ff9640 1 -- 172.21.15.16:0/3592239591 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c8007c70 con 0x7f74d40feec0 2024-09-13T16:33:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.241+0000 7f74d0ff9640 1 -- 172.21.15.16:0/3592239591 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74c8007dd0 con 0x7f74d40feec0 2024-09-13T16:33:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.241+0000 7f74da0b1640 1 -- 172.21.15.16:0/3592239591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74d419fe90 con 0x7f74d40feec0 2024-09-13T16:33:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.241+0000 7f74d0ff9640 1 -- 172.21.15.16:0/3592239591 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c8018730 con 0x7f74d40feec0 2024-09-13T16:33:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.241+0000 7f74da0b1640 1 -- 172.21.15.16:0/3592239591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74d41a02b0 con 0x7f74d40feec0 2024-09-13T16:33:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.244+0000 7f74d0ff9640 1 -- 172.21.15.16:0/3592239591 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f74c800f450 con 0x7f74d40feec0 2024-09-13T16:33:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.244+0000 7f74dcdd6740 1 Processor -- start 2024-09-13T16:33:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.244+0000 7f74dcdd6740 1 -- start start 2024-09-13T16:33:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.244+0000 7f74dcdd6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c4f2f21d0 0x559c4f2f25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.244+0000 7f74dcdd6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c4f2f2ba0 0x559c4f2f76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.244+0000 7f74dcdd6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c4f2f7d30 0x559c4f2fa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.244+0000 7f74dcdd6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c4f25cc30 con 0x559c4f2f2ba0 2024-09-13T16:33:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.244+0000 7f74dcdd6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c4f25a3c0 con 0x559c4f2f7d30 2024-09-13T16:33:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.244+0000 7f74dcdd6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c4f255cb0 con 0x559c4f2f21d0 2024-09-13T16:33:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.245+0000 7f74d37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c4f2f2ba0 0x559c4f2f76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.245+0000 7f74d37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c4f2f2ba0 0x559c4f2f76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42266/0 (socket says 172.21.15.16:42266) 2024-09-13T16:33:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.245+0000 7f74d37fe640 1 -- 172.21.15.16:0/1027554345 learned_addr learned my addr 172.21.15.16:0/1027554345 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:33:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.245+0000 7f74d3fff640 1 --2- 172.21.15.16:0/1027554345 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c4f2f21d0 0x559c4f2f25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.245+0000 7f74d0ff9640 1 --2- 172.21.15.16:0/3592239591 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f74bc0a1090 0x7f74bc0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.245+0000 7f74d2ffd640 1 --2- 172.21.15.16:0/1027554345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c4f2f7d30 0x559c4f2fa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.245+0000 7f74d37fe640 1 -- 172.21.15.16:0/1027554345 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c4f2f21d0 msgr2=0x559c4f2f25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.245+0000 7f74d37fe640 1 --2- 172.21.15.16:0/1027554345 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c4f2f21d0 0x559c4f2f25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.245+0000 7f74d3fff640 1 --2- 172.21.15.16:0/3592239591 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f74bc0a1090 0x7f74bc0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.245+0000 7f74d37fe640 1 -- 172.21.15.16:0/1027554345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c4f2f7d30 msgr2=0x559c4f2fa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.245+0000 7f74d37fe640 1 --2- 172.21.15.16:0/1027554345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c4f2f7d30 0x559c4f2fa1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.245+0000 7f74d37fe640 1 -- 172.21.15.16:0/1027554345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c4f1fc910 con 0x559c4f2f2ba0 2024-09-13T16:33:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.245+0000 7f74d0ff9640 1 -- 172.21.15.16:0/3592239591 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f74c80ccee0 con 0x7f74d40feec0 2024-09-13T16:33:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.245+0000 7f74d2ffd640 1 --2- 172.21.15.16:0/1027554345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c4f2f7d30 0x559c4f2fa1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:33:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.245+0000 7f74d37fe640 1 --2- 172.21.15.16:0/1027554345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c4f2f2ba0 0x559c4f2f76c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f74c402a910 tx=0x7f74c40316b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aae0cbe86d5ea0e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:33:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.245+0000 7f74d3fff640 1 --2- 172.21.15.16:0/3592239591 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f74bc0a1090 0x7f74bc0a3550 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x559c4f2f3420 tx=0x7f74cc004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:33:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.245+0000 7f74b9ffb640 1 -- 172.21.15.16:0/1027554345 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c4004300 con 0x559c4f2f2ba0 2024-09-13T16:33:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.245+0000 7f74b9ffb640 1 -- 172.21.15.16:0/1027554345 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74c4031df0 con 0x559c4f2f2ba0 2024-09-13T16:33:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.245+0000 7f74b9ffb640 1 -- 172.21.15.16:0/1027554345 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c4003d00 con 0x559c4f2f2ba0 2024-09-13T16:33:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.246+0000 7f74dcdd6740 1 -- 172.21.15.16:0/1027554345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c4f2f2ba0 msgr2=0x559c4f2f76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:05.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.246+0000 7f74dcdd6740 1 --2- 172.21.15.16:0/1027554345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c4f2f2ba0 0x559c4f2f76c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f74c402a910 tx=0x7f74c40316b0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.246+0000 7f74dcdd6740 1 -- 172.21.15.16:0/1027554345 shutdown_connections 2024-09-13T16:33:05.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.246+0000 7f74dcdd6740 1 --2- 172.21.15.16:0/1027554345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c4f2f7d30 0x559c4f2fa1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.246+0000 7f74dcdd6740 1 --2- 172.21.15.16:0/1027554345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c4f2f2ba0 0x559c4f2f76c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.246+0000 7f74dcdd6740 1 --2- 172.21.15.16:0/1027554345 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c4f2f21d0 0x559c4f2f25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.246+0000 7f74dcdd6740 1 -- 172.21.15.16:0/1027554345 >> 172.21.15.16:0/1027554345 conn(0x559c4f2ed8c0 msgr2=0x559c4f2edcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:33:05.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.246+0000 7f74dcdd6740 1 -- 172.21.15.16:0/1027554345 shutdown_connections 2024-09-13T16:33:05.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.246+0000 7f74dcdd6740 1 -- 172.21.15.16:0/1027554345 wait complete. 2024-09-13T16:33:05.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.247+0000 7f74dcdd6740 1 -- start start 2024-09-13T16:33:05.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.247+0000 7f74dcdd6740 1 Processor -- start 2024-09-13T16:33:05.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.247+0000 7f74dcdd6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c4f2f21d0 0x559c4f2fbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:05.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.247+0000 7f74dcdd6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c4f2f2ba0 0x559c4f2fc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.247+0000 7f74dcdd6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c4f2f7d30 0x559c4f3033f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.247+0000 7f74dcdd6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c4f213e80 con 0x559c4f2f21d0 2024-09-13T16:33:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.247+0000 7f74dcdd6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c4f213b20 con 0x559c4f2f2ba0 2024-09-13T16:33:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.247+0000 7f74dcdd6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c4f1d9260 con 0x559c4f2f7d30 2024-09-13T16:33:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.247+0000 7f74dcdd6740 10 client.0 _ll_register_callbacks cb 0x559c4f20edf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:33:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.247+0000 7f74d37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c4f2f21d0 0x559c4f2fbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.247+0000 7f74d37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c4f2f21d0 0x559c4f2fbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42282/0 (socket says 172.21.15.16:42282) 2024-09-13T16:33:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.247+0000 7f74d37fe640 1 -- 172.21.15.16:0/3419978312 learned_addr learned my addr 172.21.15.16:0/3419978312 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:33:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.247+0000 7f74d3fff640 1 --2- 172.21.15.16:0/3419978312 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c4f2f7d30 0x559c4f3033f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.247+0000 7f74d2ffd640 1 --2- 172.21.15.16:0/3419978312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c4f2f2ba0 0x559c4f2fc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.247+0000 7f74d37fe640 1 -- 172.21.15.16:0/3419978312 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c4f2f7d30 msgr2=0x559c4f3033f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.248+0000 7f74d37fe640 1 --2- 172.21.15.16:0/3419978312 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c4f2f7d30 0x559c4f3033f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.248+0000 7f74d37fe640 1 -- 172.21.15.16:0/3419978312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c4f2f2ba0 msgr2=0x559c4f2fc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.248+0000 7f74d37fe640 1 --2- 172.21.15.16:0/3419978312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c4f2f2ba0 0x559c4f2fc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.248+0000 7f74d37fe640 1 -- 172.21.15.16:0/3419978312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74d410ac50 con 0x559c4f2f21d0 2024-09-13T16:33:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.248+0000 7f74d37fe640 1 --2- 172.21.15.16:0/3419978312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c4f2f21d0 0x559c4f2fbe70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f74c4006010 tx=0x7f74c4031b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2b3fd5b22d5f8b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:33:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.248+0000 7f74b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:33:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.248+0000 7f74b2ffd640 1 -- 172.21.15.16:0/3419978312 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c404b070 con 0x559c4f2f21d0 2024-09-13T16:33:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.248+0000 7f74b2ffd640 1 -- 172.21.15.16:0/3419978312 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f74c4004310 con 0x559c4f2f21d0 2024-09-13T16:33:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.248+0000 7f74dcdd6740 1 -- 172.21.15.16:0/3419978312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c4f27e680 con 0x559c4f2f21d0 2024-09-13T16:33:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.248+0000 7f74b2ffd640 1 -- 172.21.15.16:0/3419978312 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74c40044d0 con 0x559c4f2f21d0 2024-09-13T16:33:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.248+0000 7f74dcdd6740 10 client.15681 fetch_fsmap learned FSMap version 29 2024-09-13T16:33:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.248+0000 7f74dcdd6740 1 -- 172.21.15.16:0/3419978312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559c4f254330 con 0x559c4f2f21d0 2024-09-13T16:33:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.248+0000 7f74b2ffd640 1 -- 172.21.15.16:0/3419978312 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559c4f27e680 con 0x559c4f2f21d0 2024-09-13T16:33:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.248+0000 7f74b2ffd640 1 -- 172.21.15.16:0/3419978312 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f74c4043020 con 0x559c4f2f21d0 2024-09-13T16:33:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.249+0000 7f74dcdd6740 10 client.15681 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T16:33:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.249+0000 7f74dcdd6740 10 client.15681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T16:33:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.249+0000 7f74dcdd6740 20 client.15681 populate_metadata read hostname 'smithi016' 2024-09-13T16:33:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.249+0000 7f74dcdd6740 1 --2- 172.21.15.16:0/3419978312 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x559c4f3077f0 0x559c4f309be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.249+0000 7f74dcdd6740 4 client.15681 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T16:33:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.249+0000 7f74dcdd6740 1 -- 172.21.15.16:0/3419978312 --> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c4f2495d0 con 0x559c4f3077f0 2024-09-13T16:33:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.249+0000 7f74d2ffd640 1 --2- 172.21.15.16:0/3419978312 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x559c4f3077f0 0x559c4f309be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.249+0000 7f74d2ffd640 1 --2- 172.21.15.16:0/3419978312 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x559c4f3077f0 0x559c4f309be0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dbebd60940c75bd server_cookie=ad0476984bfd103f in_seq=0 out_seq=0 2024-09-13T16:33:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.249+0000 7f74b2ffd640 10 client.15681 ms_handle_connect on v2:172.21.15.107:6838/1914689806 2024-09-13T16:33:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.250+0000 7f74b2ffd640 1 -- 172.21.15.16:0/3419978312 <== mds.0 v2:172.21.15.107:6838/1914689806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559c4f2495d0 con 0x559c4f3077f0 2024-09-13T16:33:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.250+0000 7f74b2ffd640 10 client.15681 handle_command_reply: tid=0 2024-09-13T16:33:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.250+0000 7f74dcdd6740 1 client.15681 shutdown 2024-09-13T16:33:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.250+0000 7f74dcdd6740 1 -- 172.21.15.16:0/3419978312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c4f2f21d0 msgr2=0x559c4f2fbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.250+0000 7f74dcdd6740 1 --2- 172.21.15.16:0/3419978312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c4f2f21d0 0x559c4f2fbe70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f74c4006010 tx=0x7f74c4031b90 comp rx=0 tx=0).stop 2024-09-13T16:33:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.250+0000 7f74dcdd6740 1 -- 172.21.15.16:0/3419978312 shutdown_connections 2024-09-13T16:33:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.250+0000 7f74dcdd6740 1 --2- 172.21.15.16:0/3419978312 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x559c4f3077f0 0x559c4f309be0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.250+0000 7f74dcdd6740 1 --2- 172.21.15.16:0/3419978312 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c4f2f7d30 0x559c4f3033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.250+0000 7f74dcdd6740 1 --2- 172.21.15.16:0/3419978312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c4f2f2ba0 0x559c4f2fc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.251+0000 7f74dcdd6740 1 --2- 172.21.15.16:0/3419978312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c4f2f21d0 0x559c4f2fbe70 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.250+0000 7f74b2ffd640 0 client.15681 ms_handle_reset on v2:172.21.15.107:6838/1914689806 2024-09-13T16:33:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.251+0000 7f74dcdd6740 1 -- 172.21.15.16:0/3419978312 >> 172.21.15.16:0/3419978312 conn(0x559c4f2ed8c0 msgr2=0x559c4f2ecb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:33:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.251+0000 7f74dcdd6740 1 -- 172.21.15.16:0/3419978312 shutdown_connections 2024-09-13T16:33:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.251+0000 7f74dcdd6740 1 -- 172.21.15.16:0/3419978312 wait complete. 2024-09-13T16:33:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.251+0000 7f74dcdd6740 20 client.15681 trim_cache size 0 max 16384 2024-09-13T16:33:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.263+0000 7f74dcdd6740 1 Processor -- start 2024-09-13T16:33:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.263+0000 7f74dcdd6740 1 -- start start 2024-09-13T16:33:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.263+0000 7f74dcdd6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c4f2f21d0 0x559c4f2fbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.263+0000 7f74dcdd6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c4f2f2ba0 0x559c4f3033b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.263+0000 7f74dcdd6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c4f2f7d30 0x559c4f2fc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.263+0000 7f74dcdd6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c4f27e680 con 0x559c4f2f2ba0 2024-09-13T16:33:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.263+0000 7f74dcdd6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c4f2f32b0 con 0x559c4f2f21d0 2024-09-13T16:33:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.263+0000 7f74dcdd6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c4f2fcbd0 con 0x559c4f2f7d30 2024-09-13T16:33:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.264+0000 7f74d2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c4f2f2ba0 0x559c4f3033b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.264+0000 7f74d2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c4f2f2ba0 0x559c4f3033b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42288/0 (socket says 172.21.15.16:42288) 2024-09-13T16:33:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.264+0000 7f74d2ffd640 1 -- 172.21.15.16:0/10106150 learned_addr learned my addr 172.21.15.16:0/10106150 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:33:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.264+0000 7f74d3fff640 1 --2- 172.21.15.16:0/10106150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c4f2f7d30 0x559c4f2fc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.264+0000 7f74d37fe640 1 --2- 172.21.15.16:0/10106150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c4f2f21d0 0x559c4f2fbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.264+0000 7f74d2ffd640 1 -- 172.21.15.16:0/10106150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c4f2f7d30 msgr2=0x559c4f2fc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.264+0000 7f74d2ffd640 1 --2- 172.21.15.16:0/10106150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c4f2f7d30 0x559c4f2fc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.264+0000 7f74d2ffd640 1 -- 172.21.15.16:0/10106150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c4f2f21d0 msgr2=0x559c4f2fbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.264+0000 7f74d2ffd640 1 --2- 172.21.15.16:0/10106150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c4f2f21d0 0x559c4f2fbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.264+0000 7f74d2ffd640 1 -- 172.21.15.16:0/10106150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c4f1f6300 con 0x559c4f2f2ba0 2024-09-13T16:33:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.264+0000 7f74d2ffd640 1 --2- 172.21.15.16:0/10106150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c4f2f2ba0 0x559c4f3033b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f74c80969c0 tx=0x7f74c806d130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f19eb92d90a799a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:33:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.264+0000 7f74ba7fc640 1 -- 172.21.15.16:0/10106150 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c80981c0 con 0x559c4f2f2ba0 2024-09-13T16:33:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.264+0000 7f74ba7fc640 1 -- 172.21.15.16:0/10106150 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74c800f9e0 con 0x559c4f2f2ba0 2024-09-13T16:33:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.264+0000 7f74ba7fc640 1 -- 172.21.15.16:0/10106150 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c809de10 con 0x559c4f2f2ba0 2024-09-13T16:33:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.265+0000 7f74dcdd6740 1 -- 172.21.15.16:0/10106150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c4f2f2ba0 msgr2=0x559c4f3033b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.265+0000 7f74dcdd6740 1 --2- 172.21.15.16:0/10106150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c4f2f2ba0 0x559c4f3033b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f74c80969c0 tx=0x7f74c806d130 comp rx=0 tx=0).stop 2024-09-13T16:33:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.265+0000 7f74dcdd6740 1 -- 172.21.15.16:0/10106150 shutdown_connections 2024-09-13T16:33:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.265+0000 7f74dcdd6740 1 --2- 172.21.15.16:0/10106150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c4f2f7d30 0x559c4f2fc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.265+0000 7f74dcdd6740 1 --2- 172.21.15.16:0/10106150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c4f2f2ba0 0x559c4f3033b0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.265+0000 7f74dcdd6740 1 --2- 172.21.15.16:0/10106150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c4f2f21d0 0x559c4f2fbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.265+0000 7f74dcdd6740 1 -- 172.21.15.16:0/10106150 >> 172.21.15.16:0/10106150 conn(0x559c4f2ed8c0 msgr2=0x559c4f2f3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:33:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.265+0000 7f74dcdd6740 1 -- 172.21.15.16:0/10106150 shutdown_connections 2024-09-13T16:33:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.265+0000 7f74dcdd6740 1 -- 172.21.15.16:0/10106150 wait complete. 2024-09-13T16:33:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.265+0000 7f74dcdd6740 1 -- start start 2024-09-13T16:33:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.265+0000 7f74dcdd6740 1 Processor -- start 2024-09-13T16:33:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.266+0000 7f74dcdd6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c4f2f21d0 0x559c4f302e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.266+0000 7f74dcdd6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c4f2f2ba0 0x559c4f303360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.266+0000 7f74dcdd6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c4f2f7d30 0x559c4f301570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.266+0000 7f74dcdd6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c4f301c40 con 0x559c4f2f21d0 2024-09-13T16:33:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.266+0000 7f74dcdd6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c4f301db0 con 0x559c4f2f7d30 2024-09-13T16:33:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.266+0000 7f74dcdd6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c4f301f70 con 0x559c4f2f2ba0 2024-09-13T16:33:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.266+0000 7f74dcdd6740 10 client.0 _ll_register_callbacks cb 0x7f74c4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:33:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.266+0000 7f74d37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c4f2f21d0 0x559c4f302e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.266+0000 7f74d37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c4f2f21d0 0x559c4f302e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42292/0 (socket says 172.21.15.16:42292) 2024-09-13T16:33:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.266+0000 7f74d37fe640 1 -- 172.21.15.16:0/2767463518 learned_addr learned my addr 172.21.15.16:0/2767463518 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:33:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.266+0000 7f74d3fff640 1 --2- 172.21.15.16:0/2767463518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c4f2f7d30 0x559c4f301570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.266+0000 7f74d2ffd640 1 --2- 172.21.15.16:0/2767463518 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c4f2f2ba0 0x559c4f303360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.266+0000 7f74d37fe640 1 -- 172.21.15.16:0/2767463518 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c4f2f2ba0 msgr2=0x559c4f303360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.266+0000 7f74d37fe640 1 --2- 172.21.15.16:0/2767463518 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c4f2f2ba0 0x559c4f303360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.266+0000 7f74d37fe640 1 -- 172.21.15.16:0/2767463518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c4f2f7d30 msgr2=0x559c4f301570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.266+0000 7f74d37fe640 1 --2- 172.21.15.16:0/2767463518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c4f2f7d30 0x559c4f301570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.266+0000 7f74d37fe640 1 -- 172.21.15.16:0/2767463518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74d41a02b0 con 0x559c4f2f21d0 2024-09-13T16:33:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.266+0000 7f74d37fe640 1 --2- 172.21.15.16:0/2767463518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c4f2f21d0 0x559c4f302e20 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f74c403efd0 tx=0x7f74c403d680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3606b9670f08650 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:33:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.266+0000 7f74b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:33:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.266+0000 7f74b2ffd640 1 -- 172.21.15.16:0/2767463518 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c404b070 con 0x559c4f2f21d0 2024-09-13T16:33:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.266+0000 7f74b2ffd640 1 -- 172.21.15.16:0/2767463518 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f74c4000e40 con 0x559c4f2f21d0 2024-09-13T16:33:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.266+0000 7f74dcdd6740 1 -- 172.21.15.16:0/2767463518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74c4031df0 con 0x559c4f2f21d0 2024-09-13T16:33:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.267+0000 7f74b2ffd640 1 -- 172.21.15.16:0/2767463518 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74c4031580 con 0x559c4f2f21d0 2024-09-13T16:33:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.267+0000 7f74dcdd6740 10 client.15687 fetch_fsmap learned FSMap version 29 2024-09-13T16:33:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.267+0000 7f74dcdd6740 1 -- 172.21.15.16:0/2767463518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559c4f21b8e0 con 0x559c4f2f21d0 2024-09-13T16:33:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.267+0000 7f74b2ffd640 1 -- 172.21.15.16:0/2767463518 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74c4031df0 con 0x559c4f2f21d0 2024-09-13T16:33:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.267+0000 7f74b2ffd640 1 -- 172.21.15.16:0/2767463518 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f74c4043040 con 0x559c4f2f21d0 2024-09-13T16:33:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.267+0000 7f74dcdd6740 10 client.15687 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T16:33:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.267+0000 7f74dcdd6740 10 client.15687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T16:33:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.267+0000 7f74dcdd6740 20 client.15687 populate_metadata read hostname 'smithi016' 2024-09-13T16:33:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.267+0000 7f74dcdd6740 1 --2- 172.21.15.16:0/2767463518 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x559c4f35eee0 0x559c4f361300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.267+0000 7f74dcdd6740 4 client.15687 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T16:33:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.267+0000 7f74dcdd6740 1 -- 172.21.15.16:0/2767463518 --> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559c4ef49300 con 0x559c4f35eee0 2024-09-13T16:33:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.267+0000 7f74d2ffd640 1 --2- 172.21.15.16:0/2767463518 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x559c4f35eee0 0x559c4f361300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.268+0000 7f74b2ffd640 10 client.15687 ms_handle_connect on v2:172.21.15.107:6838/1914689806 2024-09-13T16:33:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.268+0000 7f74d2ffd640 1 --2- 172.21.15.16:0/2767463518 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x559c4f35eee0 0x559c4f361300 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdcfc6c150c7dc09 server_cookie=e514cf9cde44d5da in_seq=0 out_seq=0 2024-09-13T16:33:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.278+0000 7f74b2ffd640 1 -- 172.21.15.16:0/2767463518 <== mds.0 v2:172.21.15.107:6838/1914689806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559c4ef49300 con 0x559c4f35eee0 2024-09-13T16:33:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.278+0000 7f74b2ffd640 10 client.15687 handle_command_reply: tid=0 2024-09-13T16:33:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.278+0000 7f74dcdd6740 1 client.15687 shutdown 2024-09-13T16:33:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.278+0000 7f74dcdd6740 1 -- 172.21.15.16:0/2767463518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c4f2f21d0 msgr2=0x559c4f302e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.278+0000 7f74dcdd6740 1 --2- 172.21.15.16:0/2767463518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c4f2f21d0 0x559c4f302e20 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f74c403efd0 tx=0x7f74c403d680 comp rx=0 tx=0).stop 2024-09-13T16:33:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.278+0000 7f74dcdd6740 1 -- 172.21.15.16:0/2767463518 shutdown_connections 2024-09-13T16:33:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.278+0000 7f74dcdd6740 1 --2- 172.21.15.16:0/2767463518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c4f2f7d30 0x559c4f301570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.278+0000 7f74dcdd6740 1 --2- 172.21.15.16:0/2767463518 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x559c4f35eee0 0x559c4f361300 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.278+0000 7f74dcdd6740 1 --2- 172.21.15.16:0/2767463518 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c4f2f2ba0 0x559c4f303360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.278+0000 7f74dcdd6740 1 --2- 172.21.15.16:0/2767463518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c4f2f21d0 0x559c4f302e20 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.278+0000 7f74b2ffd640 0 client.15687 ms_handle_reset on v2:172.21.15.107:6838/1914689806 2024-09-13T16:33:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.278+0000 7f74dcdd6740 1 -- 172.21.15.16:0/2767463518 >> 172.21.15.16:0/2767463518 conn(0x559c4f2ed8c0 msgr2=0x559c4f2fbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:33:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.278+0000 7f74dcdd6740 1 -- 172.21.15.16:0/2767463518 shutdown_connections 2024-09-13T16:33:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.278+0000 7f74dcdd6740 1 -- 172.21.15.16:0/2767463518 wait complete. 2024-09-13T16:33:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.278+0000 7f74dcdd6740 20 client.15687 trim_cache size 0 max 16384 2024-09-13T16:33:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-13T16:33:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-13T16:33:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-13T16:33:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-13T16:33:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.279+0000 7f74da0b1640 1 -- 172.21.15.16:0/3592239591 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f74bc0a1090 msgr2=0x7f74bc0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.279+0000 7f74da0b1640 1 --2- 172.21.15.16:0/3592239591 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f74bc0a1090 0x7f74bc0a3550 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x559c4f2f3420 tx=0x7f74cc004ab0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.279+0000 7f74da0b1640 1 -- 172.21.15.16:0/3592239591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d40feec0 msgr2=0x7f74d41982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.279+0000 7f74da0b1640 1 --2- 172.21.15.16:0/3592239591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d40feec0 0x7f74d41982e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f74c800c920 tx=0x7f74c800cde0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.279+0000 7f74da0b1640 1 -- 172.21.15.16:0/3592239591 shutdown_connections 2024-09-13T16:33:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.279+0000 7f74da0b1640 1 --2- 172.21.15.16:0/3592239591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74d40ff860 0x7f74d419f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.279+0000 7f74da0b1640 1 --2- 172.21.15.16:0/3592239591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d40feec0 0x7f74d41982e0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.279+0000 7f74da0b1640 1 --2- 172.21.15.16:0/3592239591 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f74bc0a1090 0x7f74bc0a3550 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.279+0000 7f74da0b1640 1 --2- 172.21.15.16:0/3592239591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74d40fe580 0x7f74d4197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.279+0000 7f74da0b1640 1 -- 172.21.15.16:0/3592239591 >> 172.21.15.16:0/3592239591 conn(0x7f74d40fa370 msgr2=0x7f74d40fbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:33:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.279+0000 7f74da0b1640 1 -- 172.21.15.16:0/3592239591 shutdown_connections 2024-09-13T16:33:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:05.279+0000 7f74da0b1640 1 -- 172.21.15.16:0/3592239591 wait complete. 2024-09-13T16:33:06.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:05 smithi016 ceph-mon[31187]: pgmap v413: 129 pgs: 129 active+clean; 1.0 GiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 126 MiB/s wr, 59 op/s 2024-09-13T16:33:06.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:05 smithi111 ceph-mon[34600]: pgmap v413: 129 pgs: 129 active+clean; 1.0 GiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 126 MiB/s wr, 59 op/s 2024-09-13T16:33:06.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:05 smithi107 ceph-mon[34139]: pgmap v413: 129 pgs: 129 active+clean; 1.0 GiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 126 MiB/s wr, 59 op/s 2024-09-13T16:33:06.513 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-13T16:33:06.513 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-13T16:33:06.513 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-13T16:33:06.513 INFO:tasks.workunit.client.0.smithi016.stdout:iozone test complete. 2024-09-13T16:33:06.514 INFO:tasks.workunit.client.0.smithi016.stderr:+ iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-09-13T16:33:06.581 INFO:tasks.workunit.client.0.smithi016.stdout: Iozone: Performance Test of File I/O 2024-09-13T16:33:06.581 INFO:tasks.workunit.client.0.smithi016.stdout: Version $Revision: 3.494 $ 2024-09-13T16:33:06.581 INFO:tasks.workunit.client.0.smithi016.stdout: Compiled for 64 bit mode. 2024-09-13T16:33:06.581 INFO:tasks.workunit.client.0.smithi016.stdout: Build: linux-AMD64 2024-09-13T16:33:06.581 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-13T16:33:06.581 INFO:tasks.workunit.client.0.smithi016.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-09-13T16:33:06.581 INFO:tasks.workunit.client.0.smithi016.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-09-13T16:33:06.581 INFO:tasks.workunit.client.0.smithi016.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-09-13T16:33:06.581 INFO:tasks.workunit.client.0.smithi016.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-09-13T16:33:06.581 INFO:tasks.workunit.client.0.smithi016.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-09-13T16:33:06.582 INFO:tasks.workunit.client.0.smithi016.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-09-13T16:33:06.582 INFO:tasks.workunit.client.0.smithi016.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-09-13T16:33:06.582 INFO:tasks.workunit.client.0.smithi016.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-09-13T16:33:06.582 INFO:tasks.workunit.client.0.smithi016.stdout: Alexey Skidanov, Sudhir Kumar. 2024-09-13T16:33:06.582 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-13T16:33:06.582 INFO:tasks.workunit.client.0.smithi016.stdout: Run began: Fri Sep 13 16:33:06 2024 2024-09-13T16:33:06.582 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-13T16:33:06.582 INFO:tasks.workunit.client.0.smithi016.stdout: Include close in write timing 2024-09-13T16:33:06.582 INFO:tasks.workunit.client.0.smithi016.stdout: Include fsync in write timing 2024-09-13T16:33:06.582 INFO:tasks.workunit.client.0.smithi016.stdout: File size set to 10485760 kB 2024-09-13T16:33:06.582 INFO:tasks.workunit.client.0.smithi016.stdout: Record Size 1024 kB 2024-09-13T16:33:06.582 INFO:tasks.workunit.client.0.smithi016.stdout: Command line used: iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-09-13T16:33:06.582 INFO:tasks.workunit.client.0.smithi016.stdout: Output is in kBytes/sec 2024-09-13T16:33:06.583 INFO:tasks.workunit.client.0.smithi016.stdout: Time Resolution = 0.000001 seconds. 2024-09-13T16:33:06.583 INFO:tasks.workunit.client.0.smithi016.stdout: Processor cache size set to 1024 kBytes. 2024-09-13T16:33:06.583 INFO:tasks.workunit.client.0.smithi016.stdout: Processor cache line size set to 32 bytes. 2024-09-13T16:33:06.583 INFO:tasks.workunit.client.0.smithi016.stdout: File stride size set to 17 * record size. 2024-09-13T16:33:06.583 INFO:tasks.workunit.client.0.smithi016.stdout: Throughput test with 1 process 2024-09-13T16:33:06.583 INFO:tasks.workunit.client.0.smithi016.stdout: Each process writes a 10485760 kByte file in 1024 kByte records 2024-09-13T16:33:08.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:07 smithi016 ceph-mon[31187]: pgmap v414: 129 pgs: 129 active+clean; 1.0 GiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 78 MiB/s wr, 39 op/s 2024-09-13T16:33:08.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:07 smithi111 ceph-mon[34600]: pgmap v414: 129 pgs: 129 active+clean; 1.0 GiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 78 MiB/s wr, 39 op/s 2024-09-13T16:33:08.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:07 smithi107 ceph-mon[34139]: pgmap v414: 129 pgs: 129 active+clean; 1.0 GiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 78 MiB/s wr, 39 op/s 2024-09-13T16:33:10.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:09 smithi111 ceph-mon[34600]: pgmap v415: 129 pgs: 129 active+clean; 977 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 78 MiB/s wr, 46 op/s 2024-09-13T16:33:10.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:09 smithi107 ceph-mon[34139]: pgmap v415: 129 pgs: 129 active+clean; 977 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 78 MiB/s wr, 46 op/s 2024-09-13T16:33:10.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:09 smithi016 ceph-mon[31187]: pgmap v415: 129 pgs: 129 active+clean; 977 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 78 MiB/s wr, 46 op/s 2024-09-13T16:33:12.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:11 smithi111 ceph-mon[34600]: pgmap v416: 129 pgs: 129 active+clean; 1.2 GiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 79 MiB/s wr, 48 op/s 2024-09-13T16:33:12.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:11 smithi107 ceph-mon[34139]: pgmap v416: 129 pgs: 129 active+clean; 1.2 GiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 79 MiB/s wr, 48 op/s 2024-09-13T16:33:12.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:11 smithi016 ceph-mon[31187]: pgmap v416: 129 pgs: 129 active+clean; 1.2 GiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 79 MiB/s wr, 48 op/s 2024-09-13T16:33:14.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:13 smithi111 ceph-mon[34600]: pgmap v417: 129 pgs: 129 active+clean; 2.0 GiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 169 MiB/s wr, 80 op/s 2024-09-13T16:33:14.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:13 smithi107 ceph-mon[34139]: pgmap v417: 129 pgs: 129 active+clean; 2.0 GiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 169 MiB/s wr, 80 op/s 2024-09-13T16:33:14.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:13 smithi016 ceph-mon[31187]: pgmap v417: 129 pgs: 129 active+clean; 2.0 GiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 169 MiB/s wr, 80 op/s 2024-09-13T16:33:16.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:15 smithi111 ceph-mon[34600]: pgmap v418: 129 pgs: 129 active+clean; 3.4 GiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 289 MiB/s wr, 109 op/s 2024-09-13T16:33:16.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:15 smithi107 ceph-mon[34139]: pgmap v418: 129 pgs: 129 active+clean; 3.4 GiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 289 MiB/s wr, 109 op/s 2024-09-13T16:33:16.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:15 smithi016 ceph-mon[31187]: pgmap v418: 129 pgs: 129 active+clean; 3.4 GiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 289 MiB/s wr, 109 op/s 2024-09-13T16:33:17.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:16 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:33:17.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:16 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:33:17.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:16 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:33:17.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:16 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:33:17.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:16 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:33:17.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:16 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:33:17.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:16 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:33:17.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:16 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:33:17.250 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:16 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:33:17.250 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:16 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:33:17.250 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:16 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:33:17.250 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:16 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:33:17.250 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:16 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:33:17.251 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:16 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:33:17.251 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:16 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:33:17.251 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:16 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:33:17.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:16 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:33:17.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:16 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:33:17.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:16 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:33:17.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:16 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:33:17.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:16 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:33:17.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:16 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:33:17.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:16 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:33:17.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:16 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:33:18.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:17 smithi111 ceph-mon[34600]: pgmap v419: 129 pgs: 129 active+clean; 4.5 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 388 MiB/s wr, 133 op/s 2024-09-13T16:33:18.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:17 smithi107 ceph-mon[34139]: pgmap v419: 129 pgs: 129 active+clean; 4.5 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 388 MiB/s wr, 133 op/s 2024-09-13T16:33:18.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:17 smithi016 ceph-mon[31187]: pgmap v419: 129 pgs: 129 active+clean; 4.5 GiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 388 MiB/s wr, 133 op/s 2024-09-13T16:33:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.415+0000 7f9d62872640 1 -- 172.21.15.16:0/1907139592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d5c0fec30 msgr2=0x7f9d5c0ff030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.415+0000 7f9d62872640 1 --2- 172.21.15.16:0/1907139592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d5c0fec30 0x7f9d5c0ff030 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c009860 tx=0x7f9d4c02a340 comp rx=0 tx=0).stop 2024-09-13T16:33:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.415+0000 7f9d62872640 1 -- 172.21.15.16:0/1907139592 shutdown_connections 2024-09-13T16:33:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.415+0000 7f9d62872640 1 --2- 172.21.15.16:0/1907139592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d5c1007e0 0x7f9d5c107070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.415+0000 7f9d62872640 1 --2- 172.21.15.16:0/1907139592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d5c0ffe20 0x7f9d5c1002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.415+0000 7f9d62872640 1 --2- 172.21.15.16:0/1907139592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d5c0fec30 0x7f9d5c0ff030 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.415+0000 7f9d62872640 1 -- 172.21.15.16:0/1907139592 >> 172.21.15.16:0/1907139592 conn(0x7f9d5c0fa3b0 msgr2=0x7f9d5c0fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:33:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.415+0000 7f9d62872640 1 -- 172.21.15.16:0/1907139592 shutdown_connections 2024-09-13T16:33:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.415+0000 7f9d62872640 1 -- 172.21.15.16:0/1907139592 wait complete. 2024-09-13T16:33:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.416+0000 7f9d62872640 1 Processor -- start 2024-09-13T16:33:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.416+0000 7f9d62872640 1 -- start start 2024-09-13T16:33:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.416+0000 7f9d62872640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d5c0fec30 0x7f9d5c195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.416+0000 7f9d62872640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d5c0ffe20 0x7f9d5c196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.416+0000 7f9d62872640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d5c1007e0 0x7f9d5c19d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.416+0000 7f9d62872640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c19d7f0 con 0x7f9d5c0fec30 2024-09-13T16:33:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.416+0000 7f9d62872640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c19d960 con 0x7f9d5c1007e0 2024-09-13T16:33:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.416+0000 7f9d62872640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c19db20 con 0x7f9d5c0ffe20 2024-09-13T16:33:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.416+0000 7f9d60e0f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d5c1007e0 0x7f9d5c19d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.416+0000 7f9d5bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d5c0fec30 0x7f9d5c195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.416+0000 7f9d60e0f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d5c1007e0 0x7f9d5c19d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.16:48556/0 (socket says 172.21.15.16:48556) 2024-09-13T16:33:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.416+0000 7f9d60e0f640 1 -- 172.21.15.16:0/2416725181 learned_addr learned my addr 172.21.15.16:0/2416725181 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:33:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.417+0000 7f9d60e0f640 1 -- 172.21.15.16:0/2416725181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d5c0ffe20 msgr2=0x7f9d5c196250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.417+0000 7f9d60e0f640 1 --2- 172.21.15.16:0/2416725181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d5c0ffe20 0x7f9d5c196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.417+0000 7f9d60e0f640 1 -- 172.21.15.16:0/2416725181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d5c0fec30 msgr2=0x7f9d5c195d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.417+0000 7f9d60e0f640 1 --2- 172.21.15.16:0/2416725181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d5c0fec30 0x7f9d5c195d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.417+0000 7f9d60e0f640 1 -- 172.21.15.16:0/2416725181 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d44009560 con 0x7f9d5c1007e0 2024-09-13T16:33:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.417+0000 7f9d60e0f640 1 --2- 172.21.15.16:0/2416725181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d5c1007e0 0x7f9d5c19d280 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400efd0 tx=0x7f9d5400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:33:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.417+0000 7f9d59ffb640 1 -- 172.21.15.16:0/2416725181 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d54009b40 con 0x7f9d5c1007e0 2024-09-13T16:33:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.417+0000 7f9d59ffb640 1 -- 172.21.15.16:0/2416725181 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d54004020 con 0x7f9d5c1007e0 2024-09-13T16:33:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.417+0000 7f9d59ffb640 1 -- 172.21.15.16:0/2416725181 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d54010740 con 0x7f9d5c1007e0 2024-09-13T16:33:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.417+0000 7f9d62872640 1 -- 172.21.15.16:0/2416725181 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d54009840 con 0x7f9d5c1007e0 2024-09-13T16:33:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.417+0000 7f9d62872640 1 -- 172.21.15.16:0/2416725181 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d4c009490 con 0x7f9d5c1007e0 2024-09-13T16:33:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.421+0000 7f9d65670740 1 Processor -- start 2024-09-13T16:33:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.422+0000 7f9d59ffb640 1 -- 172.21.15.16:0/2416725181 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f9d54003b70 con 0x7f9d5c1007e0 2024-09-13T16:33:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.422+0000 7f9d65670740 1 -- start start 2024-09-13T16:33:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.422+0000 7f9d65670740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df393e11d0 0x55df393e15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.422+0000 7f9d65670740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df393e1ba0 0x55df393e66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.422+0000 7f9d65670740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df393e6d30 0x55df393e91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.422+0000 7f9d65670740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3934bc30 con 0x55df393e11d0 2024-09-13T16:33:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.422+0000 7f9d65670740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df393493c0 con 0x55df393e1ba0 2024-09-13T16:33:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.422+0000 7f9d65670740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df39344cb0 con 0x55df393e6d30 2024-09-13T16:33:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.423+0000 7f9d53fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df393e11d0 0x55df393e15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.423+0000 7f9d53fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df393e11d0 0x55df393e15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42456/0 (socket says 172.21.15.16:42456) 2024-09-13T16:33:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.423+0000 7f9d53fff640 1 -- 172.21.15.16:0/1995893901 learned_addr learned my addr 172.21.15.16:0/1995893901 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:33:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.423+0000 7f9d53fff640 1 -- 172.21.15.16:0/1995893901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df393e6d30 msgr2=0x55df393e91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.423+0000 7f9d53fff640 1 --2- 172.21.15.16:0/1995893901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df393e6d30 0x55df393e91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.423+0000 7f9d53fff640 1 -- 172.21.15.16:0/1995893901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df393e1ba0 msgr2=0x55df393e66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.423+0000 7f9d53fff640 1 --2- 172.21.15.16:0/1995893901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df393e1ba0 0x55df393e66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.423+0000 7f9d53fff640 1 -- 172.21.15.16:0/1995893901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df392eb910 con 0x55df393e11d0 2024-09-13T16:33:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.423+0000 7f9d53fff640 1 --2- 172.21.15.16:0/1995893901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df393e11d0 0x55df393e15d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9d44009e10 tx=0x7f9d44010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563bd2e38a36bf50 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:33:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.423+0000 7f9d527fc640 1 -- 172.21.15.16:0/1995893901 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d44004230 con 0x55df393e11d0 2024-09-13T16:33:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.424+0000 7f9d527fc640 1 -- 172.21.15.16:0/1995893901 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d44010e90 con 0x55df393e11d0 2024-09-13T16:33:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.423+0000 7f9d59ffb640 1 --2- 172.21.15.16:0/2416725181 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f9d480a1090 0x7f9d480a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.424+0000 7f9d5bfff640 1 --2- 172.21.15.16:0/2416725181 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f9d480a1090 0x7f9d480a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.424+0000 7f9d59ffb640 1 -- 172.21.15.16:0/2416725181 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f9d54014070 con 0x7f9d5c1007e0 2024-09-13T16:33:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.424+0000 7f9d65670740 1 -- 172.21.15.16:0/1995893901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df393e11d0 msgr2=0x55df393e15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.424+0000 7f9d65670740 1 --2- 172.21.15.16:0/1995893901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df393e11d0 0x55df393e15d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9d44009e10 tx=0x7f9d44010740 comp rx=0 tx=0).stop 2024-09-13T16:33:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.424+0000 7f9d65670740 1 -- 172.21.15.16:0/1995893901 shutdown_connections 2024-09-13T16:33:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.424+0000 7f9d65670740 1 --2- 172.21.15.16:0/1995893901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df393e6d30 0x55df393e91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.424+0000 7f9d65670740 1 --2- 172.21.15.16:0/1995893901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df393e1ba0 0x55df393e66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.424+0000 7f9d65670740 1 --2- 172.21.15.16:0/1995893901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df393e11d0 0x55df393e15d0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.424+0000 7f9d65670740 1 -- 172.21.15.16:0/1995893901 >> 172.21.15.16:0/1995893901 conn(0x55df393dc8c0 msgr2=0x55df393dccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:33:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.425+0000 7f9d65670740 1 -- 172.21.15.16:0/1995893901 shutdown_connections 2024-09-13T16:33:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.425+0000 7f9d5bfff640 1 --2- 172.21.15.16:0/2416725181 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f9d480a1090 0x7f9d480a3550 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c1971e0 tx=0x7f9d4c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:33:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.425+0000 7f9d65670740 1 -- 172.21.15.16:0/1995893901 wait complete. 2024-09-13T16:33:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.425+0000 7f9d65670740 1 -- start start 2024-09-13T16:33:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.426+0000 7f9d65670740 1 Processor -- start 2024-09-13T16:33:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.426+0000 7f9d65670740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df393e11d0 0x55df393eae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.426+0000 7f9d65670740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df393e1ba0 0x55df393eb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.426+0000 7f9d65670740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df393e6d30 0x55df393f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.426+0000 7f9d65670740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df39302e80 con 0x55df393e11d0 2024-09-13T16:33:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.426+0000 7f9d65670740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df39302b20 con 0x55df393e1ba0 2024-09-13T16:33:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.426+0000 7f9d65670740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df392c8260 con 0x55df393e6d30 2024-09-13T16:33:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.426+0000 7f9d65670740 10 client.0 _ll_register_callbacks cb 0x55df392fddf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:33:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.426+0000 7f9d5bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df393e11d0 0x55df393eae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.426+0000 7f9d5bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df393e11d0 0x55df393eae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42464/0 (socket says 172.21.15.16:42464) 2024-09-13T16:33:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.426+0000 7f9d5bfff640 1 -- 172.21.15.16:0/1762113865 learned_addr learned my addr 172.21.15.16:0/1762113865 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:33:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.426+0000 7f9d53fff640 1 --2- 172.21.15.16:0/1762113865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df393e1ba0 0x55df393eb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.427+0000 7f9d60e0f640 1 --2- 172.21.15.16:0/1762113865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df393e6d30 0x55df393f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.427+0000 7f9d53fff640 1 -- 172.21.15.16:0/1762113865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df393e6d30 msgr2=0x55df393f23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.427+0000 7f9d53fff640 1 --2- 172.21.15.16:0/1762113865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df393e6d30 0x55df393f23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.427+0000 7f9d53fff640 1 -- 172.21.15.16:0/1762113865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df393e11d0 msgr2=0x55df393eae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.427+0000 7f9d53fff640 1 --2- 172.21.15.16:0/1762113865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df393e11d0 0x55df393eae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.427+0000 7f9d53fff640 1 -- 172.21.15.16:0/1762113865 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d44009ac0 con 0x55df393e1ba0 2024-09-13T16:33:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.427+0000 7f9d53fff640 1 --2- 172.21.15.16:0/1762113865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df393e1ba0 0x55df393eb360 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55df393e2420 tx=0x7f9d44013bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e96e69dafa2fbb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:33:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.428+0000 7f9d3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-13T16:33:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.428+0000 7f9d3b7fe640 1 -- 172.21.15.16:0/1762113865 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d440040d0 con 0x55df393e1ba0 2024-09-13T16:33:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.428+0000 7f9d65670740 1 -- 172.21.15.16:0/1762113865 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df3936d680 con 0x55df393e1ba0 2024-09-13T16:33:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.428+0000 7f9d3b7fe640 1 -- 172.21.15.16:0/1762113865 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f9d44012680 con 0x55df393e1ba0 2024-09-13T16:33:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.428+0000 7f9d3b7fe640 1 -- 172.21.15.16:0/1762113865 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d44009940 con 0x55df393e1ba0 2024-09-13T16:33:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.428+0000 7f9d65670740 10 client.25298 fetch_fsmap learned FSMap version 29 2024-09-13T16:33:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.428+0000 7f9d65670740 1 -- 172.21.15.16:0/1762113865 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55df39343330 con 0x55df393e1ba0 2024-09-13T16:33:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.428+0000 7f9d3b7fe640 1 -- 172.21.15.16:0/1762113865 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55df3936d680 con 0x55df393e1ba0 2024-09-13T16:33:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.428+0000 7f9d3b7fe640 1 -- 172.21.15.16:0/1762113865 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9d4401bba0 con 0x55df393e1ba0 2024-09-13T16:33:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.429+0000 7f9d65670740 10 client.25298 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T16:33:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.429+0000 7f9d65670740 10 client.25298 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T16:33:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.429+0000 7f9d65670740 20 client.25298 populate_metadata read hostname 'smithi016' 2024-09-13T16:33:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.429+0000 7f9d65670740 1 --2- 172.21.15.16:0/1762113865 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55df393ff380 0x55df394017a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.429+0000 7f9d65670740 4 client.25298 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T16:33:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.429+0000 7f9d65670740 1 -- 172.21.15.16:0/1762113865 --> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df393385d0 con 0x55df393ff380 2024-09-13T16:33:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.429+0000 7f9d5bfff640 1 --2- 172.21.15.16:0/1762113865 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55df393ff380 0x55df394017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.430+0000 7f9d5bfff640 1 --2- 172.21.15.16:0/1762113865 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55df393ff380 0x55df394017a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a85019f5247299a8 server_cookie=c40b2d1fe355832f in_seq=0 out_seq=0 2024-09-13T16:33:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.430+0000 7f9d3b7fe640 10 client.25298 ms_handle_connect on v2:172.21.15.107:6838/1914689806 2024-09-13T16:33:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.432+0000 7f9d3b7fe640 1 -- 172.21.15.16:0/1762113865 <== mds.0 v2:172.21.15.107:6838/1914689806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55df393385d0 con 0x55df393ff380 2024-09-13T16:33:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.432+0000 7f9d3b7fe640 10 client.25298 handle_command_reply: tid=0 2024-09-13T16:33:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.432+0000 7f9d65670740 1 client.25298 shutdown 2024-09-13T16:33:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.433+0000 7f9d65670740 1 -- 172.21.15.16:0/1762113865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df393e1ba0 msgr2=0x55df393eb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.433+0000 7f9d65670740 1 --2- 172.21.15.16:0/1762113865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df393e1ba0 0x55df393eb360 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55df393e2420 tx=0x7f9d44013bc0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.433+0000 7f9d65670740 1 -- 172.21.15.16:0/1762113865 shutdown_connections 2024-09-13T16:33:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.433+0000 7f9d65670740 1 --2- 172.21.15.16:0/1762113865 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55df393ff380 0x55df394017a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.433+0000 7f9d65670740 1 --2- 172.21.15.16:0/1762113865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df393e6d30 0x55df393f23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.433+0000 7f9d3b7fe640 0 client.25298 ms_handle_reset on v2:172.21.15.107:6838/1914689806 2024-09-13T16:33:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.433+0000 7f9d65670740 1 --2- 172.21.15.16:0/1762113865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df393e1ba0 0x55df393eb360 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.433+0000 7f9d65670740 1 --2- 172.21.15.16:0/1762113865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df393e11d0 0x55df393eae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.433+0000 7f9d65670740 1 -- 172.21.15.16:0/1762113865 >> 172.21.15.16:0/1762113865 conn(0x55df393dc8c0 msgr2=0x55df393dbb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:33:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.434+0000 7f9d65670740 1 -- 172.21.15.16:0/1762113865 shutdown_connections 2024-09-13T16:33:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.434+0000 7f9d65670740 1 -- 172.21.15.16:0/1762113865 wait complete. 2024-09-13T16:33:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.435+0000 7f9d65670740 20 client.25298 trim_cache size 0 max 16384 2024-09-13T16:33:18.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.456+0000 7f9d65670740 1 Processor -- start 2024-09-13T16:33:18.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.456+0000 7f9d65670740 1 -- start start 2024-09-13T16:33:18.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.456+0000 7f9d65670740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df393e11d0 0x55df393eae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:18.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.456+0000 7f9d65670740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df393e1ba0 0x55df393f2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.456+0000 7f9d65670740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df393e6d30 0x55df393eb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.456+0000 7f9d65670740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3936d680 con 0x55df393e11d0 2024-09-13T16:33:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.456+0000 7f9d65670740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df393dc750 con 0x55df393e1ba0 2024-09-13T16:33:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.456+0000 7f9d65670740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df393e22b0 con 0x55df393e6d30 2024-09-13T16:33:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.456+0000 7f9d5bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df393e11d0 0x55df393eae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.456+0000 7f9d5bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df393e11d0 0x55df393eae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42468/0 (socket says 172.21.15.16:42468) 2024-09-13T16:33:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.456+0000 7f9d5bfff640 1 -- 172.21.15.16:0/325573696 learned_addr learned my addr 172.21.15.16:0/325573696 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:33:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.457+0000 7f9d5bfff640 1 -- 172.21.15.16:0/325573696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df393e6d30 msgr2=0x55df393eb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.457+0000 7f9d5bfff640 1 --2- 172.21.15.16:0/325573696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df393e6d30 0x55df393eb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.457+0000 7f9d5bfff640 1 -- 172.21.15.16:0/325573696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df393e1ba0 msgr2=0x55df393f2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.457+0000 7f9d5bfff640 1 --2- 172.21.15.16:0/325573696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df393e1ba0 0x55df393f2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.457+0000 7f9d5bfff640 1 -- 172.21.15.16:0/325573696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df392e5300 con 0x55df393e11d0 2024-09-13T16:33:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.457+0000 7f9d5bfff640 1 --2- 172.21.15.16:0/325573696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df393e11d0 0x55df393eae20 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0057c0 tx=0x7f9d4c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646383f0f14a25e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:33:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.457+0000 7f9d52ffd640 1 -- 172.21.15.16:0/325573696 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c040070 con 0x55df393e11d0 2024-09-13T16:33:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.457+0000 7f9d52ffd640 1 -- 172.21.15.16:0/325573696 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d4c032a40 con 0x55df393e11d0 2024-09-13T16:33:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.457+0000 7f9d52ffd640 1 -- 172.21.15.16:0/325573696 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c0044d0 con 0x55df393e11d0 2024-09-13T16:33:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.458+0000 7f9d65670740 1 -- 172.21.15.16:0/325573696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df393e11d0 msgr2=0x55df393eae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.458+0000 7f9d65670740 1 --2- 172.21.15.16:0/325573696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df393e11d0 0x55df393eae20 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0057c0 tx=0x7f9d4c02c000 comp rx=0 tx=0).stop 2024-09-13T16:33:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.458+0000 7f9d65670740 1 -- 172.21.15.16:0/325573696 shutdown_connections 2024-09-13T16:33:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.458+0000 7f9d65670740 1 --2- 172.21.15.16:0/325573696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df393e6d30 0x55df393eb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.458+0000 7f9d65670740 1 --2- 172.21.15.16:0/325573696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df393e1ba0 0x55df393f2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.458+0000 7f9d65670740 1 --2- 172.21.15.16:0/325573696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df393e11d0 0x55df393eae20 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.458+0000 7f9d65670740 1 -- 172.21.15.16:0/325573696 >> 172.21.15.16:0/325573696 conn(0x55df393dc8c0 msgr2=0x55df393e2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:33:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.458+0000 7f9d65670740 1 -- 172.21.15.16:0/325573696 shutdown_connections 2024-09-13T16:33:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.458+0000 7f9d65670740 1 -- 172.21.15.16:0/325573696 wait complete. 2024-09-13T16:33:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.459+0000 7f9d65670740 1 -- start start 2024-09-13T16:33:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.460+0000 7f9d65670740 1 Processor -- start 2024-09-13T16:33:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.460+0000 7f9d65670740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df393e11d0 0x55df393f21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.460+0000 7f9d65670740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df393e1ba0 0x55df393f07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.460+0000 7f9d65670740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df393e6d30 0x55df393f0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.460+0000 7f9d65670740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df393f1340 con 0x55df393e11d0 2024-09-13T16:33:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.460+0000 7f9d65670740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df393f14b0 con 0x55df393e6d30 2024-09-13T16:33:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.460+0000 7f9d65670740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df393f1670 con 0x55df393e1ba0 2024-09-13T16:33:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.460+0000 7f9d65670740 10 client.0 _ll_register_callbacks cb 0x7f9d44009820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:33:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.461+0000 7f9d5bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df393e11d0 0x55df393f21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.461+0000 7f9d5bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df393e11d0 0x55df393f21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42482/0 (socket says 172.21.15.16:42482) 2024-09-13T16:33:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.461+0000 7f9d5bfff640 1 -- 172.21.15.16:0/1950461866 learned_addr learned my addr 172.21.15.16:0/1950461866 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:33:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.461+0000 7f9d60e0f640 1 --2- 172.21.15.16:0/1950461866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df393e6d30 0x55df393f0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.461+0000 7f9d5bfff640 1 -- 172.21.15.16:0/1950461866 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df393e1ba0 msgr2=0x55df393f07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.461+0000 7f9d5bfff640 1 --2- 172.21.15.16:0/1950461866 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df393e1ba0 0x55df393f07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.461+0000 7f9d5bfff640 1 -- 172.21.15.16:0/1950461866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df393e6d30 msgr2=0x55df393f0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.461+0000 7f9d5bfff640 1 --2- 172.21.15.16:0/1950461866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df393e6d30 0x55df393f0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.461+0000 7f9d5bfff640 1 -- 172.21.15.16:0/1950461866 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d5c107970 con 0x55df393e11d0 2024-09-13T16:33:18.467 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.461+0000 7f9d5bfff640 1 --2- 172.21.15.16:0/1950461866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df393e11d0 0x55df393f21e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0328a0 tx=0x7f9d4c032550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bcc96587a137fb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:33:18.467 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.461+0000 7f9d3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:33:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.462+0000 7f9d3b7fe640 1 -- 172.21.15.16:0/1950461866 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c040070 con 0x55df393e11d0 2024-09-13T16:33:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.462+0000 7f9d3b7fe640 1 -- 172.21.15.16:0/1950461866 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f9d4c032db0 con 0x55df393e11d0 2024-09-13T16:33:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.462+0000 7f9d65670740 1 -- 172.21.15.16:0/1950461866 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9d44010e90 con 0x55df393e11d0 2024-09-13T16:33:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.462+0000 7f9d3b7fe640 1 -- 172.21.15.16:0/1950461866 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d4c0035f0 con 0x55df393e11d0 2024-09-13T16:33:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.462+0000 7f9d65670740 10 client.15705 fetch_fsmap learned FSMap version 29 2024-09-13T16:33:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.462+0000 7f9d65670740 1 -- 172.21.15.16:0/1950461866 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55df3930a8e0 con 0x55df393e11d0 2024-09-13T16:33:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.462+0000 7f9d3b7fe640 1 -- 172.21.15.16:0/1950461866 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d44010e90 con 0x55df393e11d0 2024-09-13T16:33:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.462+0000 7f9d3b7fe640 1 -- 172.21.15.16:0/1950461866 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9d4c055760 con 0x55df393e11d0 2024-09-13T16:33:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.463+0000 7f9d65670740 10 client.15705 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T16:33:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.463+0000 7f9d65670740 10 client.15705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T16:33:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.463+0000 7f9d65670740 20 client.15705 populate_metadata read hostname 'smithi016' 2024-09-13T16:33:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.463+0000 7f9d65670740 1 --2- 172.21.15.16:0/1950461866 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55df393ff380 0x55df39458ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.463+0000 7f9d65670740 4 client.15705 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T16:33:18.469 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.463+0000 7f9d65670740 1 -- 172.21.15.16:0/1950461866 --> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55df39038300 con 0x55df393ff380 2024-09-13T16:33:18.469 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.463+0000 7f9d53fff640 1 --2- 172.21.15.16:0/1950461866 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55df393ff380 0x55df39458ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:18.469 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.464+0000 7f9d53fff640 1 --2- 172.21.15.16:0/1950461866 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55df393ff380 0x55df39458ff0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90ccdba82fa0d6a2 server_cookie=a267df1b9934443a in_seq=0 out_seq=0 2024-09-13T16:33:18.469 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.465+0000 7f9d3b7fe640 10 client.15705 ms_handle_connect on v2:172.21.15.107:6838/1914689806 2024-09-13T16:33:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.474+0000 7f9d3b7fe640 1 -- 172.21.15.16:0/1950461866 <== mds.0 v2:172.21.15.107:6838/1914689806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55df39038300 con 0x55df393ff380 2024-09-13T16:33:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.474+0000 7f9d3b7fe640 10 client.15705 handle_command_reply: tid=0 2024-09-13T16:33:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.474+0000 7f9d65670740 1 client.15705 shutdown 2024-09-13T16:33:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.474+0000 7f9d65670740 1 -- 172.21.15.16:0/1950461866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df393e11d0 msgr2=0x55df393f21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.474+0000 7f9d65670740 1 --2- 172.21.15.16:0/1950461866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df393e11d0 0x55df393f21e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0328a0 tx=0x7f9d4c032550 comp rx=0 tx=0).stop 2024-09-13T16:33:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.474+0000 7f9d65670740 1 -- 172.21.15.16:0/1950461866 shutdown_connections 2024-09-13T16:33:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.474+0000 7f9d65670740 1 --2- 172.21.15.16:0/1950461866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df393e6d30 0x55df393f0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.474+0000 7f9d65670740 1 --2- 172.21.15.16:0/1950461866 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55df393ff380 0x55df39458ff0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.474+0000 7f9d65670740 1 --2- 172.21.15.16:0/1950461866 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df393e1ba0 0x55df393f07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.474+0000 7f9d65670740 1 --2- 172.21.15.16:0/1950461866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df393e11d0 0x55df393f21e0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.474+0000 7f9d3b7fe640 0 client.15705 ms_handle_reset on v2:172.21.15.107:6838/1914689806 2024-09-13T16:33:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.474+0000 7f9d65670740 1 -- 172.21.15.16:0/1950461866 >> 172.21.15.16:0/1950461866 conn(0x55df393dc8c0 msgr2=0x55df393e2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:33:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.474+0000 7f9d65670740 1 -- 172.21.15.16:0/1950461866 shutdown_connections 2024-09-13T16:33:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.474+0000 7f9d65670740 1 -- 172.21.15.16:0/1950461866 wait complete. 2024-09-13T16:33:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.475+0000 7f9d65670740 20 client.15705 trim_cache size 0 max 16384 2024-09-13T16:33:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-13T16:33:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-13T16:33:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-13T16:33:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-13T16:33:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.475+0000 7f9d62872640 1 -- 172.21.15.16:0/2416725181 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f9d480a1090 msgr2=0x7f9d480a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.475+0000 7f9d62872640 1 --2- 172.21.15.16:0/2416725181 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f9d480a1090 0x7f9d480a3550 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c1971e0 tx=0x7f9d4c000bc0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.475+0000 7f9d62872640 1 -- 172.21.15.16:0/2416725181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d5c1007e0 msgr2=0x7f9d5c19d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.475+0000 7f9d62872640 1 --2- 172.21.15.16:0/2416725181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d5c1007e0 0x7f9d5c19d280 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400efd0 tx=0x7f9d5400c6a0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.475+0000 7f9d62872640 1 -- 172.21.15.16:0/2416725181 shutdown_connections 2024-09-13T16:33:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.475+0000 7f9d62872640 1 --2- 172.21.15.16:0/2416725181 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f9d480a1090 0x7f9d480a3550 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.475+0000 7f9d62872640 1 --2- 172.21.15.16:0/2416725181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d5c1007e0 0x7f9d5c19d280 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.475+0000 7f9d62872640 1 --2- 172.21.15.16:0/2416725181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d5c0ffe20 0x7f9d5c196250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.475+0000 7f9d62872640 1 --2- 172.21.15.16:0/2416725181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d5c0fec30 0x7f9d5c195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.475+0000 7f9d62872640 1 -- 172.21.15.16:0/2416725181 >> 172.21.15.16:0/2416725181 conn(0x7f9d5c0fa3b0 msgr2=0x7f9d5c0fc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:33:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.475+0000 7f9d62872640 1 -- 172.21.15.16:0/2416725181 shutdown_connections 2024-09-13T16:33:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:18.475+0000 7f9d62872640 1 -- 172.21.15.16:0/2416725181 wait complete. 2024-09-13T16:33:20.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:19 smithi111 ceph-mon[34600]: pgmap v420: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 533 MiB/s wr, 172 op/s 2024-09-13T16:33:20.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:19 smithi107 ceph-mon[34139]: pgmap v420: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 533 MiB/s wr, 172 op/s 2024-09-13T16:33:20.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:19 smithi016 ceph-mon[31187]: pgmap v420: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 533 MiB/s wr, 172 op/s 2024-09-13T16:33:22.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:21 smithi111 ceph-mon[34600]: pgmap v421: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 563 MiB/s wr, 164 op/s 2024-09-13T16:33:22.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:21 smithi107 ceph-mon[34139]: pgmap v421: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 563 MiB/s wr, 164 op/s 2024-09-13T16:33:22.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:21 smithi016 ceph-mon[31187]: pgmap v421: 129 pgs: 129 active+clean; 7.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 563 MiB/s wr, 164 op/s 2024-09-13T16:33:22.841 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-13T16:33:22.841 INFO:tasks.workunit.client.0.smithi016.stdout: Children see throughput for 1 initial writers = 646289.62 kB/sec 2024-09-13T16:33:22.841 INFO:tasks.workunit.client.0.smithi016.stdout: Parent sees throughput for 1 initial writers = 646276.59 kB/sec 2024-09-13T16:33:22.841 INFO:tasks.workunit.client.0.smithi016.stdout: Min throughput per process = 646289.62 kB/sec 2024-09-13T16:33:22.841 INFO:tasks.workunit.client.0.smithi016.stdout: Max throughput per process = 646289.62 kB/sec 2024-09-13T16:33:22.842 INFO:tasks.workunit.client.0.smithi016.stdout: Avg throughput per process = 646289.62 kB/sec 2024-09-13T16:33:22.842 INFO:tasks.workunit.client.0.smithi016.stdout: Min xfer = 10485760.00 kB 2024-09-13T16:33:24.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:23 smithi111 ceph-mon[34600]: pgmap v422: 129 pgs: 129 active+clean; 8.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 618 MiB/s wr, 176 op/s 2024-09-13T16:33:24.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:23 smithi107 ceph-mon[34139]: pgmap v422: 129 pgs: 129 active+clean; 8.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 618 MiB/s wr, 176 op/s 2024-09-13T16:33:24.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:23 smithi016 ceph-mon[31187]: pgmap v422: 129 pgs: 129 active+clean; 8.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 618 MiB/s wr, 176 op/s 2024-09-13T16:33:26.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:25 smithi111 ceph-mon[34600]: pgmap v423: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 633 MiB/s wr, 171 op/s 2024-09-13T16:33:26.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:25 smithi107 ceph-mon[34139]: pgmap v423: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 633 MiB/s wr, 171 op/s 2024-09-13T16:33:26.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:25 smithi016 ceph-mon[31187]: pgmap v423: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 633 MiB/s wr, 171 op/s 2024-09-13T16:33:28.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:27 smithi111 ceph-mon[34600]: pgmap v424: 129 pgs: 129 active+clean; 9.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 558 MiB/s wr, 152 op/s 2024-09-13T16:33:28.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:27 smithi107 ceph-mon[34139]: pgmap v424: 129 pgs: 129 active+clean; 9.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 558 MiB/s wr, 152 op/s 2024-09-13T16:33:28.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:27 smithi016 ceph-mon[31187]: pgmap v424: 129 pgs: 129 active+clean; 9.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 558 MiB/s wr, 152 op/s 2024-09-13T16:33:30.156 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:29 smithi016 ceph-mon[31187]: pgmap v425: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 465 MiB/s wr, 132 op/s 2024-09-13T16:33:30.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:29 smithi111 ceph-mon[34600]: pgmap v425: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 465 MiB/s wr, 132 op/s 2024-09-13T16:33:30.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:29 smithi107 ceph-mon[34139]: pgmap v425: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 465 MiB/s wr, 132 op/s 2024-09-13T16:33:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.633+0000 7f166e318640 1 -- 172.21.15.16:0/18490406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16681007e0 msgr2=0x7f1668107070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.633+0000 7f166e318640 1 --2- 172.21.15.16:0/18490406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16681007e0 0x7f1668107070 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1660009ca0 tx=0x7f166002a8c0 comp rx=0 tx=0).stop 2024-09-13T16:33:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.633+0000 7f166e318640 1 -- 172.21.15.16:0/18490406 shutdown_connections 2024-09-13T16:33:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.633+0000 7f166e318640 1 --2- 172.21.15.16:0/18490406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16681007e0 0x7f1668107070 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.633+0000 7f166e318640 1 --2- 172.21.15.16:0/18490406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16680ffe20 0x7f16681002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.633+0000 7f166e318640 1 --2- 172.21.15.16:0/18490406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16680fec30 0x7f16680ff030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.633+0000 7f166e318640 1 -- 172.21.15.16:0/18490406 >> 172.21.15.16:0/18490406 conn(0x7f16680fa3b0 msgr2=0x7f16680fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:33:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.633+0000 7f166e318640 1 -- 172.21.15.16:0/18490406 shutdown_connections 2024-09-13T16:33:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.634+0000 7f166e318640 1 -- 172.21.15.16:0/18490406 wait complete. 2024-09-13T16:33:31.636 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.634+0000 7f166e318640 1 Processor -- start 2024-09-13T16:33:31.636 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.635+0000 7f166e318640 1 -- start start 2024-09-13T16:33:31.636 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.635+0000 7f166e318640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16680fec30 0x7f16680683a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:31.636 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.635+0000 7f166e318640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16680ffe20 0x7f166819a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:31.636 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.635+0000 7f166e318640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16681007e0 0x7f166819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:31.636 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.635+0000 7f166e318640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166819d5e0 con 0x7f16681007e0 2024-09-13T16:33:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.635+0000 7f166e318640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166819d750 con 0x7f16680ffe20 2024-09-13T16:33:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.635+0000 7f166e318640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166819d910 con 0x7f16680fec30 2024-09-13T16:33:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.635+0000 7f166c8b5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16681007e0 0x7f166819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.635+0000 7f166c8b5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16681007e0 0x7f166819ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36778/0 (socket says 172.21.15.16:36778) 2024-09-13T16:33:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.635+0000 7f166c8b5640 1 -- 172.21.15.16:0/1845575831 learned_addr learned my addr 172.21.15.16:0/1845575831 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:33:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.636+0000 7f16677fe640 1 --2- 172.21.15.16:0/1845575831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16680ffe20 0x7f166819a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.636+0000 7f166c8b5640 1 -- 172.21.15.16:0/1845575831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16680fec30 msgr2=0x7f16680683a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.636+0000 7f166c8b5640 1 --2- 172.21.15.16:0/1845575831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16680fec30 0x7f16680683a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.636+0000 7f166c8b5640 1 -- 172.21.15.16:0/1845575831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16680ffe20 msgr2=0x7f166819a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.636+0000 7f166c8b5640 1 --2- 172.21.15.16:0/1845575831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16680ffe20 0x7f166819a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.636+0000 7f166c8b5640 1 -- 172.21.15.16:0/1845575831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1660009900 con 0x7f16681007e0 2024-09-13T16:33:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.636+0000 7f166c8b5640 1 --2- 172.21.15.16:0/1845575831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16681007e0 0x7f166819ce00 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1660005cc0 tx=0x7f1660003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:33:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.637+0000 7f16657fa640 1 -- 172.21.15.16:0/1845575831 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16600307c0 con 0x7f16681007e0 2024-09-13T16:33:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.637+0000 7f16657fa640 1 -- 172.21.15.16:0/1845575831 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1660040070 con 0x7f16681007e0 2024-09-13T16:33:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.637+0000 7f166e318640 1 -- 172.21.15.16:0/1845575831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f166819db30 con 0x7f16681007e0 2024-09-13T16:33:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.637+0000 7f16657fa640 1 -- 172.21.15.16:0/1845575831 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1660004650 con 0x7f16681007e0 2024-09-13T16:33:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.637+0000 7f166e318640 1 -- 172.21.15.16:0/1845575831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f166819e0c0 con 0x7f16681007e0 2024-09-13T16:33:31.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.641+0000 7f16657fa640 1 -- 172.21.15.16:0/1845575831 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1660042030 con 0x7f16681007e0 2024-09-13T16:33:31.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.642+0000 7f16657fa640 1 --2- 172.21.15.16:0/1845575831 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f16500a11e0 0x7f16500a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:31.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.642+0000 7f16657fa640 1 -- 172.21.15.16:0/1845575831 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f16600ef2f0 con 0x7f16681007e0 2024-09-13T16:33:31.644 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.642+0000 7f1667fff640 1 --2- 172.21.15.16:0/1845575831 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f16500a11e0 0x7f16500a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:31.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.642+0000 7f167103a740 1 Processor -- start 2024-09-13T16:33:31.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.643+0000 7f167103a740 1 -- start start 2024-09-13T16:33:31.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.643+0000 7f1667fff640 1 --2- 172.21.15.16:0/1845575831 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f16500a11e0 0x7f16500a36a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f16680ffc80 tx=0x7f1658006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:33:31.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.643+0000 7f167103a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55656528c090 0x55656528e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:31.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.643+0000 7f167103a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55656528ea50 0x556565290e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:31.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.643+0000 7f1667fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55656528c090 0x55656528e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:31.645 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.643+0000 7f1667fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55656528c090 0x55656528e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36788/0 (socket says 172.21.15.16:36788) 2024-09-13T16:33:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.644+0000 7f1667fff640 1 -- 172.21.15.16:0/1360350867 learned_addr learned my addr 172.21.15.16:0/1360350867 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:33:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.644+0000 7f167103a740 1 --2- 172.21.15.16:0/1360350867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565652914f0 0x5565652939b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.644+0000 7f167103a740 1 -- 172.21.15.16:0/1360350867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565651edc30 con 0x55656528c090 2024-09-13T16:33:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.644+0000 7f167103a740 1 -- 172.21.15.16:0/1360350867 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565651eb3c0 con 0x5565652914f0 2024-09-13T16:33:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.644+0000 7f167103a740 1 -- 172.21.15.16:0/1360350867 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565651e6cb0 con 0x55656528ea50 2024-09-13T16:33:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.645+0000 7f16677fe640 1 --2- 172.21.15.16:0/1360350867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55656528ea50 0x556565290e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:31.646 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.645+0000 7f166c8b5640 1 --2- 172.21.15.16:0/1360350867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565652914f0 0x5565652939b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:31.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.646+0000 7f1667fff640 1 -- 172.21.15.16:0/1360350867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55656528ea50 msgr2=0x556565290e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:31.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.646+0000 7f1667fff640 1 --2- 172.21.15.16:0/1360350867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55656528ea50 0x556565290e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:31.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.646+0000 7f1667fff640 1 -- 172.21.15.16:0/1360350867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565652914f0 msgr2=0x5565652939b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:31.647 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.646+0000 7f1667fff640 1 --2- 172.21.15.16:0/1360350867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565652914f0 0x5565652939b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:31.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.646+0000 7f1667fff640 1 -- 172.21.15.16:0/1360350867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55656518d910 con 0x55656528c090 2024-09-13T16:33:31.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.646+0000 7f1667fff640 1 --2- 172.21.15.16:0/1360350867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55656528c090 0x55656528e480 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f16580158c0 tx=0x7f165800e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510e152622e5e8b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:33:31.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.646+0000 7f164e7fc640 1 -- 172.21.15.16:0/1360350867 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165801a070 con 0x55656528c090 2024-09-13T16:33:31.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.646+0000 7f164e7fc640 1 -- 172.21.15.16:0/1360350867 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f165800bdb0 con 0x55656528c090 2024-09-13T16:33:31.648 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.646+0000 7f164e7fc640 1 -- 172.21.15.16:0/1360350867 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1658013a00 con 0x55656528c090 2024-09-13T16:33:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.647+0000 7f167103a740 1 -- 172.21.15.16:0/1360350867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55656528c090 msgr2=0x55656528e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.647+0000 7f167103a740 1 --2- 172.21.15.16:0/1360350867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55656528c090 0x55656528e480 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f16580158c0 tx=0x7f165800e000 comp rx=0 tx=0).stop 2024-09-13T16:33:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.647+0000 7f167103a740 1 -- 172.21.15.16:0/1360350867 shutdown_connections 2024-09-13T16:33:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.647+0000 7f167103a740 1 --2- 172.21.15.16:0/1360350867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565652914f0 0x5565652939b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.647+0000 7f167103a740 1 --2- 172.21.15.16:0/1360350867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55656528ea50 0x556565290e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.647+0000 7f167103a740 1 --2- 172.21.15.16:0/1360350867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55656528c090 0x55656528e480 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.647+0000 7f167103a740 1 -- 172.21.15.16:0/1360350867 >> 172.21.15.16:0/1360350867 conn(0x55656527e820 msgr2=0x55656527ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:33:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.647+0000 7f167103a740 1 -- 172.21.15.16:0/1360350867 shutdown_connections 2024-09-13T16:33:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.647+0000 7f167103a740 1 -- 172.21.15.16:0/1360350867 wait complete. 2024-09-13T16:33:31.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.647+0000 7f167103a740 1 -- start start 2024-09-13T16:33:31.650 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.648+0000 7f167103a740 1 Processor -- start 2024-09-13T16:33:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.649+0000 7f167103a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656528c090 0x5565652956d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.649+0000 7f167103a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55656528ea50 0x556565295c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.649+0000 7f167103a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565652914f0 0x55656529cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.649+0000 7f1667fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656528c090 0x5565652956d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.649+0000 7f1667fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656528c090 0x5565652956d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.16:47864/0 (socket says 172.21.15.16:47864) 2024-09-13T16:33:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.649+0000 7f1667fff640 1 -- 172.21.15.16:0/3391635351 learned_addr learned my addr 172.21.15.16:0/3391635351 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:33:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.649+0000 7f167103a740 1 -- 172.21.15.16:0/3391635351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565651a4e80 con 0x5565652914f0 2024-09-13T16:33:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.649+0000 7f167103a740 1 -- 172.21.15.16:0/3391635351 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565651a4b20 con 0x55656528c090 2024-09-13T16:33:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.649+0000 7f167103a740 1 -- 172.21.15.16:0/3391635351 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656516a260 con 0x55656528ea50 2024-09-13T16:33:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.649+0000 7f166c8b5640 1 --2- 172.21.15.16:0/3391635351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565652914f0 0x55656529cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.650+0000 7f16677fe640 1 --2- 172.21.15.16:0/3391635351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55656528ea50 0x556565295c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.650+0000 7f167103a740 10 client.0 _ll_register_callbacks cb 0x55656519fdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:33:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.650+0000 7f1667fff640 1 -- 172.21.15.16:0/3391635351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55656528ea50 msgr2=0x556565295c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.650+0000 7f1667fff640 1 --2- 172.21.15.16:0/3391635351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55656528ea50 0x556565295c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.650+0000 7f1667fff640 1 -- 172.21.15.16:0/3391635351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565652914f0 msgr2=0x55656529cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.650+0000 7f1667fff640 1 --2- 172.21.15.16:0/3391635351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565652914f0 0x55656529cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.650+0000 7f1667fff640 1 -- 172.21.15.16:0/3391635351 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16580155a0 con 0x55656528c090 2024-09-13T16:33:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.650+0000 7f166c8b5640 1 --2- 172.21.15.16:0/3391635351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565652914f0 0x55656529cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:33:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.650+0000 7f1667fff640 1 --2- 172.21.15.16:0/3391635351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656528c090 0x5565652956d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1658006eb0 tx=0x7f165800bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e85e569171b1706 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:33:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.650+0000 7f16477fe640 10 client.25310 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-13T16:33:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.650+0000 7f167103a740 1 -- 172.21.15.16:0/3391635351 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55656520f680 con 0x55656528c090 2024-09-13T16:33:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.650+0000 7f16477fe640 1 -- 172.21.15.16:0/3391635351 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165801a040 con 0x55656528c090 2024-09-13T16:33:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.650+0000 7f16477fe640 1 -- 172.21.15.16:0/3391635351 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f165800bb20 con 0x55656528c090 2024-09-13T16:33:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.651+0000 7f16477fe640 1 -- 172.21.15.16:0/3391635351 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1658030440 con 0x55656528c090 2024-09-13T16:33:31.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.651+0000 7f167103a740 10 client.25310 fetch_fsmap learned FSMap version 29 2024-09-13T16:33:31.654 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.651+0000 7f167103a740 1 -- 172.21.15.16:0/3391635351 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x5565651e5330 con 0x55656528c090 2024-09-13T16:33:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.652+0000 7f16477fe640 1 -- 172.21.15.16:0/3391635351 <== mon.2 v2:172.21.15.107:3300/0 4 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f16580305a0 con 0x55656528c090 2024-09-13T16:33:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.653+0000 7f167103a740 10 client.25310 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T16:33:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.653+0000 7f167103a740 10 client.25310 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T16:33:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.653+0000 7f167103a740 20 client.25310 populate_metadata read hostname 'smithi016' 2024-09-13T16:33:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.653+0000 7f167103a740 1 --2- 172.21.15.16:0/3391635351 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x5565652a1170 0x5565652a3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.653+0000 7f167103a740 4 client.25310 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T16:33:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.653+0000 7f167103a740 1 -- 172.21.15.16:0/3391635351 --> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565651da5d0 con 0x5565652a1170 2024-09-13T16:33:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.654+0000 7f16677fe640 1 --2- 172.21.15.16:0/3391635351 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x5565652a1170 0x5565652a3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.656+0000 7f16677fe640 1 --2- 172.21.15.16:0/3391635351 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x5565652a1170 0x5565652a3590 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d947aee87d3cf812 server_cookie=e2cc959a3fb4c74a in_seq=0 out_seq=0 2024-09-13T16:33:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.656+0000 7f16477fe640 10 client.25310 ms_handle_connect on v2:172.21.15.107:6838/1914689806 2024-09-13T16:33:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.657+0000 7f16477fe640 1 -- 172.21.15.16:0/3391635351 <== mds.0 v2:172.21.15.107:6838/1914689806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5565651da5d0 con 0x5565652a1170 2024-09-13T16:33:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.657+0000 7f16477fe640 10 client.25310 handle_command_reply: tid=0 2024-09-13T16:33:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.657+0000 7f167103a740 1 client.25310 shutdown 2024-09-13T16:33:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.657+0000 7f167103a740 1 -- 172.21.15.16:0/3391635351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656528c090 msgr2=0x5565652956d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.657+0000 7f167103a740 1 --2- 172.21.15.16:0/3391635351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656528c090 0x5565652956d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1658006eb0 tx=0x7f165800bf20 comp rx=0 tx=0).stop 2024-09-13T16:33:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.657+0000 7f167103a740 1 -- 172.21.15.16:0/3391635351 shutdown_connections 2024-09-13T16:33:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.657+0000 7f167103a740 1 --2- 172.21.15.16:0/3391635351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565652914f0 0x55656529cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.657+0000 7f167103a740 1 --2- 172.21.15.16:0/3391635351 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x5565652a1170 0x5565652a3590 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.657+0000 7f167103a740 1 --2- 172.21.15.16:0/3391635351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55656528ea50 0x556565295c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.657+0000 7f167103a740 1 --2- 172.21.15.16:0/3391635351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656528c090 0x5565652956d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.657+0000 7f16477fe640 0 client.25310 ms_handle_reset on v2:172.21.15.107:6838/1914689806 2024-09-13T16:33:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.657+0000 7f167103a740 1 -- 172.21.15.16:0/3391635351 >> 172.21.15.16:0/3391635351 conn(0x55656527e820 msgr2=0x55656527db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:33:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.657+0000 7f167103a740 1 -- 172.21.15.16:0/3391635351 shutdown_connections 2024-09-13T16:33:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.658+0000 7f167103a740 1 -- 172.21.15.16:0/3391635351 wait complete. 2024-09-13T16:33:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.658+0000 7f167103a740 20 client.25310 trim_cache size 0 max 16384 2024-09-13T16:33:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.686+0000 7f167103a740 1 Processor -- start 2024-09-13T16:33:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.686+0000 7f167103a740 1 -- start start 2024-09-13T16:33:31.688 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.686+0000 7f167103a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55656528c090 0x5565652923c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.686+0000 7f167103a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656528ea50 0x556565292900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.686+0000 7f167103a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565652914f0 0x55656529cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.686+0000 7f167103a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656529d0b0 con 0x5565652914f0 2024-09-13T16:33:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.686+0000 7f167103a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656527e650 con 0x55656528ea50 2024-09-13T16:33:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.686+0000 7f167103a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565652950c0 con 0x55656528c090 2024-09-13T16:33:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.687+0000 7f166c8b5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565652914f0 0x55656529cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.687+0000 7f166c8b5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565652914f0 0x55656529cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36796/0 (socket says 172.21.15.16:36796) 2024-09-13T16:33:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.687+0000 7f166c8b5640 1 -- 172.21.15.16:0/4066482873 learned_addr learned my addr 172.21.15.16:0/4066482873 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:33:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.687+0000 7f16677fe640 1 --2- 172.21.15.16:0/4066482873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656528ea50 0x556565292900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.687+0000 7f166c8b5640 1 -- 172.21.15.16:0/4066482873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55656528c090 msgr2=0x5565652923c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.687+0000 7f166c8b5640 1 --2- 172.21.15.16:0/4066482873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55656528c090 0x5565652923c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.687+0000 7f166c8b5640 1 -- 172.21.15.16:0/4066482873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656528ea50 msgr2=0x556565292900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.687+0000 7f166c8b5640 1 --2- 172.21.15.16:0/4066482873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656528ea50 0x556565292900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.687+0000 7f166c8b5640 1 -- 172.21.15.16:0/4066482873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556565187300 con 0x5565652914f0 2024-09-13T16:33:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.688+0000 7f166c8b5640 1 --2- 172.21.15.16:0/4066482873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565652914f0 0x55656529cb70 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f16600eca40 tx=0x7f16600ef040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12cf5cdd8ca8c8a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:33:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.688+0000 7f164effd640 1 -- 172.21.15.16:0/4066482873 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16600d8070 con 0x5565652914f0 2024-09-13T16:33:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.688+0000 7f164effd640 1 -- 172.21.15.16:0/4066482873 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f16600bcaf0 con 0x5565652914f0 2024-09-13T16:33:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.688+0000 7f164effd640 1 -- 172.21.15.16:0/4066482873 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16600ba600 con 0x5565652914f0 2024-09-13T16:33:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.689+0000 7f167103a740 1 -- 172.21.15.16:0/4066482873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565652914f0 msgr2=0x55656529cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.689+0000 7f167103a740 1 --2- 172.21.15.16:0/4066482873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565652914f0 0x55656529cb70 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f16600eca40 tx=0x7f16600ef040 comp rx=0 tx=0).stop 2024-09-13T16:33:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.690+0000 7f167103a740 1 -- 172.21.15.16:0/4066482873 shutdown_connections 2024-09-13T16:33:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.690+0000 7f167103a740 1 --2- 172.21.15.16:0/4066482873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565652914f0 0x55656529cb70 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.690+0000 7f167103a740 1 --2- 172.21.15.16:0/4066482873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656528ea50 0x556565292900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.690+0000 7f167103a740 1 --2- 172.21.15.16:0/4066482873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55656528c090 0x5565652923c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.690+0000 7f167103a740 1 -- 172.21.15.16:0/4066482873 >> 172.21.15.16:0/4066482873 conn(0x55656527e820 msgr2=0x5565652a7ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:33:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.690+0000 7f167103a740 1 -- 172.21.15.16:0/4066482873 shutdown_connections 2024-09-13T16:33:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.690+0000 7f167103a740 1 -- 172.21.15.16:0/4066482873 wait complete. 2024-09-13T16:33:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.690+0000 7f167103a740 1 -- start start 2024-09-13T16:33:31.695 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.690+0000 7f167103a740 1 Processor -- start 2024-09-13T16:33:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.691+0000 7f167103a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55656528c090 0x55656529a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.691+0000 7f167103a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55656528ea50 0x55656529ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.691+0000 7f167103a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565652914f0 0x55656529be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.691+0000 7f167103a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656529c380 con 0x55656528ea50 2024-09-13T16:33:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.691+0000 7f167103a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656529c4f0 con 0x5565652914f0 2024-09-13T16:33:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.691+0000 7f167103a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656529c6b0 con 0x55656528c090 2024-09-13T16:33:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.691+0000 7f167103a740 10 client.0 _ll_register_callbacks cb 0x7f1658009440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:33:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.691+0000 7f16677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55656528ea50 0x55656529ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.691+0000 7f166c8b5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565652914f0 0x55656529be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.691+0000 7f166c8b5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565652914f0 0x55656529be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.16:47880/0 (socket says 172.21.15.16:47880) 2024-09-13T16:33:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.691+0000 7f166c8b5640 1 -- 172.21.15.16:0/256988406 learned_addr learned my addr 172.21.15.16:0/256988406 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:33:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.691+0000 7f1667fff640 1 --2- 172.21.15.16:0/256988406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55656528c090 0x55656529a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.692+0000 7f166c8b5640 1 -- 172.21.15.16:0/256988406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55656528c090 msgr2=0x55656529a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.692+0000 7f166c8b5640 1 --2- 172.21.15.16:0/256988406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55656528c090 0x55656529a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.692+0000 7f166c8b5640 1 -- 172.21.15.16:0/256988406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55656528ea50 msgr2=0x55656529ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.692+0000 7f166c8b5640 1 --2- 172.21.15.16:0/256988406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55656528ea50 0x55656529ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.692+0000 7f166c8b5640 1 -- 172.21.15.16:0/256988406 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f166819e0c0 con 0x5565652914f0 2024-09-13T16:33:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.692+0000 7f166c8b5640 1 --2- 172.21.15.16:0/256988406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565652914f0 0x55656529be40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f16600eca70 tx=0x7f16600c0150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7b07b7f483868e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:33:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.692+0000 7f16477fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-13T16:33:31.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.692+0000 7f16477fe640 1 -- 172.21.15.16:0/256988406 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16600d8070 con 0x5565652914f0 2024-09-13T16:33:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.692+0000 7f16477fe640 1 -- 172.21.15.16:0/256988406 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f16600c0220 con 0x5565652914f0 2024-09-13T16:33:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.692+0000 7f167103a740 1 -- 172.21.15.16:0/256988406 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f165800bdb0 con 0x5565652914f0 2024-09-13T16:33:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.692+0000 7f16477fe640 1 -- 172.21.15.16:0/256988406 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16600ba5a0 con 0x5565652914f0 2024-09-13T16:33:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.693+0000 7f167103a740 10 client.25316 fetch_fsmap learned FSMap version 29 2024-09-13T16:33:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.693+0000 7f167103a740 1 -- 172.21.15.16:0/256988406 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5565651ac8e0 con 0x5565652914f0 2024-09-13T16:33:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.693+0000 7f16477fe640 1 -- 172.21.15.16:0/256988406 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f16600ba790 con 0x5565652914f0 2024-09-13T16:33:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.693+0000 7f16477fe640 1 -- 172.21.15.16:0/256988406 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f166004c060 con 0x5565652914f0 2024-09-13T16:33:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.693+0000 7f167103a740 10 client.25316 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T16:33:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.693+0000 7f167103a740 10 client.25316 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T16:33:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.693+0000 7f167103a740 20 client.25316 populate_metadata read hostname 'smithi016' 2024-09-13T16:33:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.693+0000 7f167103a740 1 --2- 172.21.15.16:0/256988406 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x5565652f8200 0x5565652fa620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.693+0000 7f167103a740 4 client.25316 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T16:33:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.693+0000 7f167103a740 1 -- 172.21.15.16:0/256988406 --> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f165800bdb0 con 0x5565652f8200 2024-09-13T16:33:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.694+0000 7f1667fff640 1 --2- 172.21.15.16:0/256988406 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x5565652f8200 0x5565652fa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.695+0000 7f1667fff640 1 --2- 172.21.15.16:0/256988406 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x5565652f8200 0x5565652fa620 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=157202e58a709e7d server_cookie=2455612cb16884ee in_seq=0 out_seq=0 2024-09-13T16:33:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.695+0000 7f16477fe640 10 client.25316 ms_handle_connect on v2:172.21.15.107:6838/1914689806 2024-09-13T16:33:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.705+0000 7f16477fe640 1 -- 172.21.15.16:0/256988406 <== mds.0 v2:172.21.15.107:6838/1914689806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f165800bdb0 con 0x5565652f8200 2024-09-13T16:33:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.705+0000 7f16477fe640 10 client.25316 handle_command_reply: tid=0 2024-09-13T16:33:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.705+0000 7f167103a740 1 client.25316 shutdown 2024-09-13T16:33:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.706+0000 7f167103a740 1 -- 172.21.15.16:0/256988406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565652914f0 msgr2=0x55656529be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.706+0000 7f167103a740 1 --2- 172.21.15.16:0/256988406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565652914f0 0x55656529be40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f16600eca70 tx=0x7f16600c0150 comp rx=0 tx=0).stop 2024-09-13T16:33:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.707+0000 7f167103a740 1 -- 172.21.15.16:0/256988406 shutdown_connections 2024-09-13T16:33:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.707+0000 7f167103a740 1 --2- 172.21.15.16:0/256988406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5565652914f0 0x55656529be40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.707+0000 7f167103a740 1 --2- 172.21.15.16:0/256988406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55656528ea50 0x55656529ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.707+0000 7f167103a740 1 --2- 172.21.15.16:0/256988406 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x5565652f8200 0x5565652fa620 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.707+0000 7f167103a740 1 --2- 172.21.15.16:0/256988406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55656528c090 0x55656529a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.707+0000 7f16477fe640 0 client.25316 ms_handle_reset on v2:172.21.15.107:6838/1914689806 2024-09-13T16:33:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.707+0000 7f167103a740 1 -- 172.21.15.16:0/256988406 >> 172.21.15.16:0/256988406 conn(0x55656527e820 msgr2=0x5565652923c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:33:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.707+0000 7f167103a740 1 -- 172.21.15.16:0/256988406 shutdown_connections 2024-09-13T16:33:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.707+0000 7f167103a740 1 -- 172.21.15.16:0/256988406 wait complete. 2024-09-13T16:33:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.707+0000 7f167103a740 20 client.25316 trim_cache size 0 max 16384 2024-09-13T16:33:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-13T16:33:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-13T16:33:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-13T16:33:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-13T16:33:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.708+0000 7f166e318640 1 -- 172.21.15.16:0/1845575831 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f16500a11e0 msgr2=0x7f16500a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.708+0000 7f166e318640 1 --2- 172.21.15.16:0/1845575831 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f16500a11e0 0x7f16500a36a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f16680ffc80 tx=0x7f1658006c90 comp rx=0 tx=0).stop 2024-09-13T16:33:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.708+0000 7f166e318640 1 -- 172.21.15.16:0/1845575831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16681007e0 msgr2=0x7f166819ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.708+0000 7f166e318640 1 --2- 172.21.15.16:0/1845575831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16681007e0 0x7f166819ce00 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1660005cc0 tx=0x7f1660003c20 comp rx=0 tx=0).stop 2024-09-13T16:33:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.708+0000 7f166e318640 1 -- 172.21.15.16:0/1845575831 shutdown_connections 2024-09-13T16:33:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.708+0000 7f166e318640 1 --2- 172.21.15.16:0/1845575831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16681007e0 0x7f166819ce00 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.708+0000 7f166e318640 1 --2- 172.21.15.16:0/1845575831 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f16500a11e0 0x7f16500a36a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.708+0000 7f166e318640 1 --2- 172.21.15.16:0/1845575831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16680ffe20 0x7f166819a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.708+0000 7f166e318640 1 --2- 172.21.15.16:0/1845575831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16680fec30 0x7f16680683a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.708+0000 7f166e318640 1 -- 172.21.15.16:0/1845575831 >> 172.21.15.16:0/1845575831 conn(0x7f16680fa3b0 msgr2=0x7f1668103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:33:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.708+0000 7f166e318640 1 -- 172.21.15.16:0/1845575831 shutdown_connections 2024-09-13T16:33:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:31.708+0000 7f166e318640 1 -- 172.21.15.16:0/1845575831 wait complete. 2024-09-13T16:33:32.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:31 smithi111 ceph-mon[34600]: pgmap v426: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 338 MiB/s wr, 96 op/s 2024-09-13T16:33:32.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:31 smithi107 ceph-mon[34139]: pgmap v426: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 338 MiB/s wr, 96 op/s 2024-09-13T16:33:32.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:31 smithi016 ceph-mon[31187]: pgmap v426: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 338 MiB/s wr, 96 op/s 2024-09-13T16:33:34.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:33 smithi111 ceph-mon[34600]: pgmap v427: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 334 MiB/s wr, 99 op/s 2024-09-13T16:33:34.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:33 smithi107 ceph-mon[34139]: pgmap v427: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 334 MiB/s wr, 99 op/s 2024-09-13T16:33:34.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:33 smithi016 ceph-mon[31187]: pgmap v427: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 334 MiB/s wr, 99 op/s 2024-09-13T16:33:36.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:35 smithi111 ceph-mon[34600]: pgmap v428: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 351 MiB/s wr, 99 op/s 2024-09-13T16:33:36.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:35 smithi107 ceph-mon[34139]: pgmap v428: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 351 MiB/s wr, 99 op/s 2024-09-13T16:33:36.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:35 smithi016 ceph-mon[31187]: pgmap v428: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 351 MiB/s wr, 99 op/s 2024-09-13T16:33:38.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:37 smithi111 ceph-mon[34600]: pgmap v429: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 363 MiB/s wr, 102 op/s 2024-09-13T16:33:38.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:37 smithi107 ceph-mon[34139]: pgmap v429: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 363 MiB/s wr, 102 op/s 2024-09-13T16:33:38.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:37 smithi016 ceph-mon[31187]: pgmap v429: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 363 MiB/s wr, 102 op/s 2024-09-13T16:33:40.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:39 smithi111 ceph-mon[34600]: pgmap v430: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 476 MiB/s wr, 133 op/s 2024-09-13T16:33:40.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:39 smithi107 ceph-mon[34139]: pgmap v430: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 476 MiB/s wr, 133 op/s 2024-09-13T16:33:40.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:39 smithi016 ceph-mon[31187]: pgmap v430: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 476 MiB/s wr, 133 op/s 2024-09-13T16:33:42.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:41 smithi111 ceph-mon[34600]: pgmap v431: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 544 MiB/s wr, 147 op/s 2024-09-13T16:33:42.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:41 smithi107 ceph-mon[34139]: pgmap v431: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 544 MiB/s wr, 147 op/s 2024-09-13T16:33:42.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:41 smithi016 ceph-mon[31187]: pgmap v431: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 544 MiB/s wr, 147 op/s 2024-09-13T16:33:43.312 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-13T16:33:43.312 INFO:tasks.workunit.client.0.smithi016.stdout: Children see throughput for 1 rewriters = 723739.81 kB/sec 2024-09-13T16:33:43.313 INFO:tasks.workunit.client.0.smithi016.stdout: Parent sees throughput for 1 rewriters = 723725.50 kB/sec 2024-09-13T16:33:43.313 INFO:tasks.workunit.client.0.smithi016.stdout: Min throughput per process = 723739.81 kB/sec 2024-09-13T16:33:43.313 INFO:tasks.workunit.client.0.smithi016.stdout: Max throughput per process = 723739.81 kB/sec 2024-09-13T16:33:43.313 INFO:tasks.workunit.client.0.smithi016.stdout: Avg throughput per process = 723739.81 kB/sec 2024-09-13T16:33:43.313 INFO:tasks.workunit.client.0.smithi016.stdout: Min xfer = 10485760.00 kB 2024-09-13T16:33:44.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:43 smithi111 ceph-mon[34600]: pgmap v432: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 636 MiB/s wr, 172 op/s 2024-09-13T16:33:44.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:43 smithi107 ceph-mon[34139]: pgmap v432: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 636 MiB/s wr, 172 op/s 2024-09-13T16:33:44.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:43 smithi016 ceph-mon[31187]: pgmap v432: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 636 MiB/s wr, 172 op/s 2024-09-13T16:33:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.815+0000 7f09dd6f0640 1 -- 172.21.15.16:0/4039726465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09d80ffd10 msgr2=0x7f09d8100110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.815+0000 7f09dd6f0640 1 --2- 172.21.15.16:0/4039726465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09d80ffd10 0x7f09d8100110 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f09c8009900 tx=0x7f09c802a410 comp rx=0 tx=0).stop 2024-09-13T16:33:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.815+0000 7f09dd6f0640 1 -- 172.21.15.16:0/4039726465 shutdown_connections 2024-09-13T16:33:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.815+0000 7f09dd6f0640 1 --2- 172.21.15.16:0/4039726465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09d81018c0 0x7f09d8108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.815+0000 7f09dd6f0640 1 --2- 172.21.15.16:0/4039726465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09d8100f00 0x7f09d8101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.815+0000 7f09dd6f0640 1 --2- 172.21.15.16:0/4039726465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09d80ffd10 0x7f09d8100110 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.815+0000 7f09dd6f0640 1 -- 172.21.15.16:0/4039726465 >> 172.21.15.16:0/4039726465 conn(0x7f09d80fb4b0 msgr2=0x7f09d80fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:33:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.815+0000 7f09dd6f0640 1 -- 172.21.15.16:0/4039726465 shutdown_connections 2024-09-13T16:33:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.815+0000 7f09dd6f0640 1 -- 172.21.15.16:0/4039726465 wait complete. 2024-09-13T16:33:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.816+0000 7f09dd6f0640 1 Processor -- start 2024-09-13T16:33:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.816+0000 7f09dd6f0640 1 -- start start 2024-09-13T16:33:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.816+0000 7f09dd6f0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09d80ffd10 0x7f09d8195ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.816+0000 7f09dd6f0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09d8100f00 0x7f09d8196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.816+0000 7f09dd6f0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09d81018c0 0x7f09d819d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.816+0000 7f09dd6f0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d819d7a0 con 0x7f09d80ffd10 2024-09-13T16:33:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.816+0000 7f09dd6f0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d819d910 con 0x7f09d8100f00 2024-09-13T16:33:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.816+0000 7f09dd6f0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d819dad0 con 0x7f09d81018c0 2024-09-13T16:33:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.816+0000 7f09d6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09d80ffd10 0x7f09d8195ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.816+0000 7f09d6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09d80ffd10 0x7f09d8195ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53040/0 (socket says 172.21.15.16:53040) 2024-09-13T16:33:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.816+0000 7f09d6ffd640 1 -- 172.21.15.16:0/289056271 learned_addr learned my addr 172.21.15.16:0/289056271 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:33:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.817+0000 7f09d77fe640 1 --2- 172.21.15.16:0/289056271 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09d81018c0 0x7f09d819d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.817+0000 7f09d67fc640 1 --2- 172.21.15.16:0/289056271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09d8100f00 0x7f09d8196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.817+0000 7f09d6ffd640 1 -- 172.21.15.16:0/289056271 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09d81018c0 msgr2=0x7f09d819d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.817+0000 7f09d6ffd640 1 --2- 172.21.15.16:0/289056271 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09d81018c0 0x7f09d819d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.817+0000 7f09d6ffd640 1 -- 172.21.15.16:0/289056271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09d8100f00 msgr2=0x7f09d8196220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.817+0000 7f09d6ffd640 1 --2- 172.21.15.16:0/289056271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09d8100f00 0x7f09d8196220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.817+0000 7f09d6ffd640 1 -- 172.21.15.16:0/289056271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c8009560 con 0x7f09d80ffd10 2024-09-13T16:33:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.817+0000 7f09d6ffd640 1 --2- 172.21.15.16:0/289056271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09d80ffd10 0x7f09d8195ce0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f09c8005ab0 tx=0x7f09c80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:33:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.817+0000 7f09bffff640 1 -- 172.21.15.16:0/289056271 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c8031d70 con 0x7f09d80ffd10 2024-09-13T16:33:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.817+0000 7f09dd6f0640 1 -- 172.21.15.16:0/289056271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d819dd50 con 0x7f09d80ffd10 2024-09-13T16:33:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.817+0000 7f09bffff640 1 -- 172.21.15.16:0/289056271 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f09c80044e0 con 0x7f09d80ffd10 2024-09-13T16:33:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.817+0000 7f09bffff640 1 -- 172.21.15.16:0/289056271 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c8003a00 con 0x7f09d80ffd10 2024-09-13T16:33:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.817+0000 7f09dd6f0640 1 -- 172.21.15.16:0/289056271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d819e1a0 con 0x7f09d80ffd10 2024-09-13T16:33:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.820+0000 7f09bffff640 1 -- 172.21.15.16:0/289056271 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f09c8003b80 con 0x7f09d80ffd10 2024-09-13T16:33:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.820+0000 7f09e0414740 1 Processor -- start 2024-09-13T16:33:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.820+0000 7f09e0414740 1 -- start start 2024-09-13T16:33:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.820+0000 7f09e0414740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612c9bf31d0 0x5612c9bf35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.820+0000 7f09e0414740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612c9bf3ba0 0x5612c9bf86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.820+0000 7f09e0414740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c9bf8d30 0x5612c9bfb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.820+0000 7f09bffff640 1 --2- 172.21.15.16:0/289056271 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f09c00a1090 0x7f09c00a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.820+0000 7f09e0414740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c9b5dc30 con 0x5612c9bf3ba0 2024-09-13T16:33:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.820+0000 7f09e0414740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c9b5b3c0 con 0x5612c9bf31d0 2024-09-13T16:33:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.820+0000 7f09e0414740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c9b56cb0 con 0x5612c9bf8d30 2024-09-13T16:33:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.820+0000 7f09d6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612c9bf3ba0 0x5612c9bf86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.820+0000 7f09d77fe640 1 --2- 172.21.15.16:0/289056271 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f09c00a1090 0x7f09c00a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.820+0000 7f09d67fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c9bf8d30 0x5612c9bfb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.820+0000 7f09d67fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c9bf8d30 0x5612c9bfb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.16:55574/0 (socket says 172.21.15.16:55574) 2024-09-13T16:33:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.820+0000 7f09d67fc640 1 -- 172.21.15.16:0/253365938 learned_addr learned my addr 172.21.15.16:0/253365938 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:33:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.820+0000 7f09d6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612c9bf3ba0 0x5612c9bf86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53048/0 (socket says 172.21.15.16:53048) 2024-09-13T16:33:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.821+0000 7f09bffff640 1 -- 172.21.15.16:0/289056271 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f09c80edda0 con 0x7f09d80ffd10 2024-09-13T16:33:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.821+0000 7f09d77fe640 1 --2- 172.21.15.16:0/253365938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612c9bf31d0 0x5612c9bf35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.821+0000 7f09d67fc640 1 -- 172.21.15.16:0/253365938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612c9bf31d0 msgr2=0x5612c9bf35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.821+0000 7f09d67fc640 1 --2- 172.21.15.16:0/253365938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612c9bf31d0 0x5612c9bf35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.821+0000 7f09d67fc640 1 -- 172.21.15.16:0/253365938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612c9bf3ba0 msgr2=0x5612c9bf86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.821+0000 7f09d67fc640 1 --2- 172.21.15.16:0/253365938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612c9bf3ba0 0x5612c9bf86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.821+0000 7f09d67fc640 1 -- 172.21.15.16:0/253365938 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612c9afd910 con 0x5612c9bf8d30 2024-09-13T16:33:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.821+0000 7f09d6ffd640 1 --2- 172.21.15.16:0/253365938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612c9bf3ba0 0x5612c9bf86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:33:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.821+0000 7f09d77fe640 1 --2- 172.21.15.16:0/289056271 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f09c00a1090 0x7f09c00a3550 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f09d0005fd0 tx=0x7f09d0009de0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:33:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.821+0000 7f09d67fc640 1 --2- 172.21.15.16:0/253365938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c9bf8d30 0x5612c9bfb1c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f09cc009c00 tx=0x7f09cc010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dde5275c67652a56 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:33:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.821+0000 7f09bd7fa640 1 -- 172.21.15.16:0/253365938 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc009700 con 0x5612c9bf8d30 2024-09-13T16:33:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.821+0000 7f09bd7fa640 1 -- 172.21.15.16:0/253365938 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f09cc01b910 con 0x5612c9bf8d30 2024-09-13T16:33:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.821+0000 7f09bd7fa640 1 -- 172.21.15.16:0/253365938 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc012880 con 0x5612c9bf8d30 2024-09-13T16:33:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.822+0000 7f09e0414740 1 -- 172.21.15.16:0/253365938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c9bf8d30 msgr2=0x5612c9bfb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.822+0000 7f09e0414740 1 --2- 172.21.15.16:0/253365938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c9bf8d30 0x5612c9bfb1c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f09cc009c00 tx=0x7f09cc010740 comp rx=0 tx=0).stop 2024-09-13T16:33:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.822+0000 7f09e0414740 1 -- 172.21.15.16:0/253365938 shutdown_connections 2024-09-13T16:33:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.822+0000 7f09e0414740 1 --2- 172.21.15.16:0/253365938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c9bf8d30 0x5612c9bfb1c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.822+0000 7f09e0414740 1 --2- 172.21.15.16:0/253365938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612c9bf3ba0 0x5612c9bf86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.822+0000 7f09e0414740 1 --2- 172.21.15.16:0/253365938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612c9bf31d0 0x5612c9bf35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.822+0000 7f09e0414740 1 -- 172.21.15.16:0/253365938 >> 172.21.15.16:0/253365938 conn(0x5612c9bee8c0 msgr2=0x5612c9beecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:33:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.822+0000 7f09e0414740 1 -- 172.21.15.16:0/253365938 shutdown_connections 2024-09-13T16:33:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.822+0000 7f09e0414740 1 -- 172.21.15.16:0/253365938 wait complete. 2024-09-13T16:33:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.823+0000 7f09e0414740 1 -- start start 2024-09-13T16:33:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.823+0000 7f09e0414740 1 Processor -- start 2024-09-13T16:33:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.823+0000 7f09e0414740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612c9bf31d0 0x5612c9bfcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.823+0000 7f09e0414740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612c9bf3ba0 0x5612c9bfd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.823+0000 7f09e0414740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c9bf8d30 0x5612c9c04480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.823+0000 7f09e0414740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c9b14e80 con 0x5612c9bf3ba0 2024-09-13T16:33:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.823+0000 7f09e0414740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c9b14b20 con 0x5612c9bf31d0 2024-09-13T16:33:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.823+0000 7f09e0414740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c9ada260 con 0x5612c9bf8d30 2024-09-13T16:33:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.823+0000 7f09e0414740 10 client.0 _ll_register_callbacks cb 0x5612c9b0fdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:33:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.823+0000 7f09d67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612c9bf3ba0 0x5612c9bfd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.823+0000 7f09d67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612c9bf3ba0 0x5612c9bfd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53060/0 (socket says 172.21.15.16:53060) 2024-09-13T16:33:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.823+0000 7f09d67fc640 1 -- 172.21.15.16:0/3660442086 learned_addr learned my addr 172.21.15.16:0/3660442086 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:33:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.823+0000 7f09d77fe640 1 --2- 172.21.15.16:0/3660442086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c9bf8d30 0x5612c9c04480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.823+0000 7f09d6ffd640 1 --2- 172.21.15.16:0/3660442086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612c9bf31d0 0x5612c9bfcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.823+0000 7f09d67fc640 1 -- 172.21.15.16:0/3660442086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c9bf8d30 msgr2=0x5612c9c04480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.823+0000 7f09d67fc640 1 --2- 172.21.15.16:0/3660442086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c9bf8d30 0x5612c9c04480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.823+0000 7f09d67fc640 1 -- 172.21.15.16:0/3660442086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612c9bf31d0 msgr2=0x5612c9bfcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.823+0000 7f09d67fc640 1 --2- 172.21.15.16:0/3660442086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612c9bf31d0 0x5612c9bfcf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.823+0000 7f09d67fc640 1 -- 172.21.15.16:0/3660442086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09cc009860 con 0x5612c9bf3ba0 2024-09-13T16:33:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.823+0000 7f09d77fe640 1 --2- 172.21.15.16:0/3660442086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c9bf8d30 0x5612c9c04480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:33:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.824+0000 7f09d67fc640 1 --2- 172.21.15.16:0/3660442086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612c9bf3ba0 0x5612c9bfd440 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f09cc010fb0 tx=0x7f09cc01bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856ecdc845572839 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:33:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.824+0000 7f09baff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:33:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.824+0000 7f09baff5640 1 -- 172.21.15.16:0/3660442086 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc020070 con 0x5612c9bf3ba0 2024-09-13T16:33:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.824+0000 7f09baff5640 1 -- 172.21.15.16:0/3660442086 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f09cc016070 con 0x5612c9bf3ba0 2024-09-13T16:33:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.824+0000 7f09e0414740 1 -- 172.21.15.16:0/3660442086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612c9b7f680 con 0x5612c9bf3ba0 2024-09-13T16:33:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.824+0000 7f09baff5640 1 -- 172.21.15.16:0/3660442086 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09cc009cf0 con 0x5612c9bf3ba0 2024-09-13T16:33:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.824+0000 7f09e0414740 10 client.15735 fetch_fsmap learned FSMap version 29 2024-09-13T16:33:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.824+0000 7f09e0414740 1 -- 172.21.15.16:0/3660442086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5612c9b55330 con 0x5612c9bf3ba0 2024-09-13T16:33:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.824+0000 7f09baff5640 1 -- 172.21.15.16:0/3660442086 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5612c9b7f680 con 0x5612c9bf3ba0 2024-09-13T16:33:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.824+0000 7f09baff5640 1 -- 172.21.15.16:0/3660442086 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f09cc003cd0 con 0x5612c9bf3ba0 2024-09-13T16:33:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.824+0000 7f09e0414740 10 client.15735 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T16:33:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.825+0000 7f09e0414740 10 client.15735 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T16:33:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.825+0000 7f09e0414740 20 client.15735 populate_metadata read hostname 'smithi016' 2024-09-13T16:33:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.825+0000 7f09e0414740 1 --2- 172.21.15.16:0/3660442086 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x5612c9c08a50 0x5612c9c0ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.825+0000 7f09e0414740 4 client.15735 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T16:33:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.825+0000 7f09e0414740 1 -- 172.21.15.16:0/3660442086 --> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5612c9b4a5d0 con 0x5612c9c08a50 2024-09-13T16:33:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.825+0000 7f09d6ffd640 1 --2- 172.21.15.16:0/3660442086 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x5612c9c08a50 0x5612c9c0ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.825+0000 7f09d6ffd640 1 --2- 172.21.15.16:0/3660442086 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x5612c9c08a50 0x5612c9c0ae70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=912457ad8d618113 server_cookie=8ca49d906438d914 in_seq=0 out_seq=0 2024-09-13T16:33:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.825+0000 7f09baff5640 10 client.15735 ms_handle_connect on v2:172.21.15.107:6838/1914689806 2024-09-13T16:33:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.826+0000 7f09baff5640 1 -- 172.21.15.16:0/3660442086 <== mds.0 v2:172.21.15.107:6838/1914689806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5612c9b4a5d0 con 0x5612c9c08a50 2024-09-13T16:33:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.826+0000 7f09baff5640 10 client.15735 handle_command_reply: tid=0 2024-09-13T16:33:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.826+0000 7f09e0414740 1 client.15735 shutdown 2024-09-13T16:33:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.826+0000 7f09e0414740 1 -- 172.21.15.16:0/3660442086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612c9bf3ba0 msgr2=0x5612c9bfd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.826+0000 7f09e0414740 1 --2- 172.21.15.16:0/3660442086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612c9bf3ba0 0x5612c9bfd440 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f09cc010fb0 tx=0x7f09cc01bcb0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.826+0000 7f09e0414740 1 -- 172.21.15.16:0/3660442086 shutdown_connections 2024-09-13T16:33:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.826+0000 7f09e0414740 1 --2- 172.21.15.16:0/3660442086 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x5612c9c08a50 0x5612c9c0ae70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.826+0000 7f09e0414740 1 --2- 172.21.15.16:0/3660442086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c9bf8d30 0x5612c9c04480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.826+0000 7f09e0414740 1 --2- 172.21.15.16:0/3660442086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612c9bf3ba0 0x5612c9bfd440 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.826+0000 7f09baff5640 0 client.15735 ms_handle_reset on v2:172.21.15.107:6838/1914689806 2024-09-13T16:33:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.826+0000 7f09e0414740 1 --2- 172.21.15.16:0/3660442086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612c9bf31d0 0x5612c9bfcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.826+0000 7f09e0414740 1 -- 172.21.15.16:0/3660442086 >> 172.21.15.16:0/3660442086 conn(0x5612c9bee8c0 msgr2=0x5612c9bedb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:33:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.826+0000 7f09e0414740 1 -- 172.21.15.16:0/3660442086 shutdown_connections 2024-09-13T16:33:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.826+0000 7f09e0414740 1 -- 172.21.15.16:0/3660442086 wait complete. 2024-09-13T16:33:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.826+0000 7f09e0414740 20 client.15735 trim_cache size 0 max 16384 2024-09-13T16:33:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.839+0000 7f09e0414740 1 Processor -- start 2024-09-13T16:33:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.839+0000 7f09e0414740 1 -- start start 2024-09-13T16:33:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.839+0000 7f09e0414740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612c9bf31d0 0x5612c9bf4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.839+0000 7f09e0414740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612c9bf3ba0 0x5612c9c04440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.839+0000 7f09e0414740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c9bf8d30 0x5612c9bf4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.839+0000 7f09e0414740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c9b4a5d0 con 0x5612c9bf3ba0 2024-09-13T16:33:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.839+0000 7f09e0414740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c9b7f680 con 0x5612c9bf31d0 2024-09-13T16:33:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.839+0000 7f09e0414740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c9bfdc60 con 0x5612c9bf8d30 2024-09-13T16:33:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.839+0000 7f09d67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612c9bf3ba0 0x5612c9c04440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.839+0000 7f09d67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612c9bf3ba0 0x5612c9c04440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53074/0 (socket says 172.21.15.16:53074) 2024-09-13T16:33:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.839+0000 7f09d67fc640 1 -- 172.21.15.16:0/3020900944 learned_addr learned my addr 172.21.15.16:0/3020900944 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:33:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.839+0000 7f09d77fe640 1 --2- 172.21.15.16:0/3020900944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c9bf8d30 0x5612c9bf4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.839+0000 7f09d6ffd640 1 --2- 172.21.15.16:0/3020900944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612c9bf31d0 0x5612c9bf4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.839+0000 7f09d67fc640 1 -- 172.21.15.16:0/3020900944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c9bf8d30 msgr2=0x5612c9bf4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.839+0000 7f09d67fc640 1 --2- 172.21.15.16:0/3020900944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c9bf8d30 0x5612c9bf4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.839+0000 7f09d67fc640 1 -- 172.21.15.16:0/3020900944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612c9bf31d0 msgr2=0x5612c9bf4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.839+0000 7f09d67fc640 1 --2- 172.21.15.16:0/3020900944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612c9bf31d0 0x5612c9bf4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.839+0000 7f09d67fc640 1 -- 172.21.15.16:0/3020900944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612c9af7300 con 0x5612c9bf3ba0 2024-09-13T16:33:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.840+0000 7f09d6ffd640 1 --2- 172.21.15.16:0/3020900944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612c9bf31d0 0x5612c9bf4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:33:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.840+0000 7f09d67fc640 1 --2- 172.21.15.16:0/3020900944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612c9bf3ba0 0x5612c9c04440 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f09cc010df0 tx=0x7f09cc010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbcc743d0898322b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:33:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.840+0000 7f09bdffb640 1 -- 172.21.15.16:0/3020900944 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc020040 con 0x5612c9bf3ba0 2024-09-13T16:33:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.840+0000 7f09bdffb640 1 -- 172.21.15.16:0/3020900944 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f09cc012450 con 0x5612c9bf3ba0 2024-09-13T16:33:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.840+0000 7f09bdffb640 1 -- 172.21.15.16:0/3020900944 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc012ea0 con 0x5612c9bf3ba0 2024-09-13T16:33:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.840+0000 7f09e0414740 1 -- 172.21.15.16:0/3020900944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612c9bf3ba0 msgr2=0x5612c9c04440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.840+0000 7f09e0414740 1 --2- 172.21.15.16:0/3020900944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612c9bf3ba0 0x5612c9c04440 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f09cc010df0 tx=0x7f09cc010740 comp rx=0 tx=0).stop 2024-09-13T16:33:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.840+0000 7f09e0414740 1 -- 172.21.15.16:0/3020900944 shutdown_connections 2024-09-13T16:33:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.840+0000 7f09e0414740 1 --2- 172.21.15.16:0/3020900944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c9bf8d30 0x5612c9bf4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.840+0000 7f09e0414740 1 --2- 172.21.15.16:0/3020900944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612c9bf3ba0 0x5612c9c04440 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.840+0000 7f09e0414740 1 --2- 172.21.15.16:0/3020900944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612c9bf31d0 0x5612c9bf4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.840+0000 7f09e0414740 1 -- 172.21.15.16:0/3020900944 >> 172.21.15.16:0/3020900944 conn(0x5612c9bee8c0 msgr2=0x5612c9bf8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:33:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.840+0000 7f09e0414740 1 -- 172.21.15.16:0/3020900944 shutdown_connections 2024-09-13T16:33:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.840+0000 7f09e0414740 1 -- 172.21.15.16:0/3020900944 wait complete. 2024-09-13T16:33:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.841+0000 7f09e0414740 1 -- start start 2024-09-13T16:33:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.841+0000 7f09e0414740 1 Processor -- start 2024-09-13T16:33:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.841+0000 7f09e0414740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c9bf31d0 0x5612c9c03f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.841+0000 7f09e0414740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612c9bf3ba0 0x5612c9c044c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.841+0000 7f09e0414740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612c9bf8d30 0x5612c9c026b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.841+0000 7f09e0414740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c9c02db0 con 0x5612c9bf3ba0 2024-09-13T16:33:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.841+0000 7f09e0414740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c9c02f20 con 0x5612c9bf8d30 2024-09-13T16:33:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.841+0000 7f09e0414740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c9c030e0 con 0x5612c9bf31d0 2024-09-13T16:33:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.841+0000 7f09e0414740 10 client.0 _ll_register_callbacks cb 0x7f09cc009650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:33:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.841+0000 7f09d67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612c9bf3ba0 0x5612c9c044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.841+0000 7f09d67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612c9bf3ba0 0x5612c9c044c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53090/0 (socket says 172.21.15.16:53090) 2024-09-13T16:33:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.841+0000 7f09d67fc640 1 -- 172.21.15.16:0/1763433628 learned_addr learned my addr 172.21.15.16:0/1763433628 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:33:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.841+0000 7f09d77fe640 1 --2- 172.21.15.16:0/1763433628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612c9bf8d30 0x5612c9c026b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.841+0000 7f09d6ffd640 1 --2- 172.21.15.16:0/1763433628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c9bf31d0 0x5612c9c03f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.841+0000 7f09d67fc640 1 -- 172.21.15.16:0/1763433628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c9bf31d0 msgr2=0x5612c9c03f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.841+0000 7f09d67fc640 1 --2- 172.21.15.16:0/1763433628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c9bf31d0 0x5612c9c03f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.841+0000 7f09d67fc640 1 -- 172.21.15.16:0/1763433628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612c9bf8d30 msgr2=0x5612c9c026b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.841+0000 7f09d67fc640 1 --2- 172.21.15.16:0/1763433628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612c9bf8d30 0x5612c9c026b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.842+0000 7f09d67fc640 1 -- 172.21.15.16:0/1763433628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5612c9b55330 con 0x5612c9bf3ba0 2024-09-13T16:33:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.842+0000 7f09d77fe640 1 --2- 172.21.15.16:0/1763433628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612c9bf8d30 0x5612c9c026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:33:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.842+0000 7f09d67fc640 1 --2- 172.21.15.16:0/1763433628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612c9bf3ba0 0x5612c9c044c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f09cc01be20 tx=0x7f09cc01b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e6aae5ac23997c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:33:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.842+0000 7f09baff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:33:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.842+0000 7f09baff5640 1 -- 172.21.15.16:0/1763433628 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc020040 con 0x5612c9bf3ba0 2024-09-13T16:33:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.842+0000 7f09baff5640 1 -- 172.21.15.16:0/1763433628 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f09cc000e40 con 0x5612c9bf3ba0 2024-09-13T16:33:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.842+0000 7f09e0414740 1 -- 172.21.15.16:0/1763433628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f09cc01b910 con 0x5612c9bf3ba0 2024-09-13T16:33:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.842+0000 7f09baff5640 1 -- 172.21.15.16:0/1763433628 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09cc012af0 con 0x5612c9bf3ba0 2024-09-13T16:33:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.842+0000 7f09e0414740 10 client.15741 fetch_fsmap learned FSMap version 29 2024-09-13T16:33:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.842+0000 7f09e0414740 1 -- 172.21.15.16:0/1763433628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5612c9b1c8e0 con 0x5612c9bf3ba0 2024-09-13T16:33:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.842+0000 7f09baff5640 1 -- 172.21.15.16:0/1763433628 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f09cc01b910 con 0x5612c9bf3ba0 2024-09-13T16:33:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.842+0000 7f09baff5640 1 -- 172.21.15.16:0/1763433628 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f09cc033480 con 0x5612c9bf3ba0 2024-09-13T16:33:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.843+0000 7f09e0414740 10 client.15741 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T16:33:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.843+0000 7f09e0414740 10 client.15741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T16:33:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.843+0000 7f09e0414740 20 client.15741 populate_metadata read hostname 'smithi016' 2024-09-13T16:33:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.843+0000 7f09e0414740 1 --2- 172.21.15.16:0/1763433628 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x5612c9c60070 0x5612c9c62490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.843+0000 7f09e0414740 4 client.15741 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T16:33:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.843+0000 7f09e0414740 1 -- 172.21.15.16:0/1763433628 --> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5612c984a300 con 0x5612c9c60070 2024-09-13T16:33:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.843+0000 7f09d6ffd640 1 --2- 172.21.15.16:0/1763433628 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x5612c9c60070 0x5612c9c62490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.843+0000 7f09d6ffd640 1 --2- 172.21.15.16:0/1763433628 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x5612c9c60070 0x5612c9c62490 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6febc8bf81e3c0e7 server_cookie=4ccedfa2903e21db in_seq=0 out_seq=0 2024-09-13T16:33:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.843+0000 7f09baff5640 10 client.15741 ms_handle_connect on v2:172.21.15.107:6838/1914689806 2024-09-13T16:33:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.851+0000 7f09baff5640 1 -- 172.21.15.16:0/1763433628 <== mds.0 v2:172.21.15.107:6838/1914689806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5612c984a300 con 0x5612c9c60070 2024-09-13T16:33:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.851+0000 7f09baff5640 10 client.15741 handle_command_reply: tid=0 2024-09-13T16:33:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.851+0000 7f09e0414740 1 client.15741 shutdown 2024-09-13T16:33:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.851+0000 7f09e0414740 1 -- 172.21.15.16:0/1763433628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612c9bf3ba0 msgr2=0x5612c9c044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.851+0000 7f09e0414740 1 --2- 172.21.15.16:0/1763433628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612c9bf3ba0 0x5612c9c044c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f09cc01be20 tx=0x7f09cc01b820 comp rx=0 tx=0).stop 2024-09-13T16:33:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.851+0000 7f09e0414740 1 -- 172.21.15.16:0/1763433628 shutdown_connections 2024-09-13T16:33:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.851+0000 7f09e0414740 1 --2- 172.21.15.16:0/1763433628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612c9bf8d30 0x5612c9c026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.851+0000 7f09e0414740 1 --2- 172.21.15.16:0/1763433628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612c9bf3ba0 0x5612c9c044c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.851+0000 7f09e0414740 1 --2- 172.21.15.16:0/1763433628 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x5612c9c60070 0x5612c9c62490 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.851+0000 7f09e0414740 1 --2- 172.21.15.16:0/1763433628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c9bf31d0 0x5612c9c03f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.851+0000 7f09e0414740 1 -- 172.21.15.16:0/1763433628 >> 172.21.15.16:0/1763433628 conn(0x5612c9bee8c0 msgr2=0x5612c9bf80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:33:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.851+0000 7f09baff5640 0 client.15741 ms_handle_reset on v2:172.21.15.107:6838/1914689806 2024-09-13T16:33:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.851+0000 7f09e0414740 1 -- 172.21.15.16:0/1763433628 shutdown_connections 2024-09-13T16:33:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.851+0000 7f09e0414740 1 -- 172.21.15.16:0/1763433628 wait complete. 2024-09-13T16:33:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.851+0000 7f09e0414740 20 client.15741 trim_cache size 0 max 16384 2024-09-13T16:33:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-13T16:33:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-13T16:33:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-13T16:33:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-13T16:33:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.852+0000 7f09dd6f0640 1 -- 172.21.15.16:0/289056271 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f09c00a1090 msgr2=0x7f09c00a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.852+0000 7f09dd6f0640 1 --2- 172.21.15.16:0/289056271 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f09c00a1090 0x7f09c00a3550 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f09d0005fd0 tx=0x7f09d0009de0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.852+0000 7f09dd6f0640 1 -- 172.21.15.16:0/289056271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09d80ffd10 msgr2=0x7f09d8195ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.852+0000 7f09dd6f0640 1 --2- 172.21.15.16:0/289056271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09d80ffd10 0x7f09d8195ce0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f09c8005ab0 tx=0x7f09c80042f0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.852+0000 7f09dd6f0640 1 -- 172.21.15.16:0/289056271 shutdown_connections 2024-09-13T16:33:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.852+0000 7f09dd6f0640 1 --2- 172.21.15.16:0/289056271 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09d81018c0 0x7f09d819d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.852+0000 7f09dd6f0640 1 --2- 172.21.15.16:0/289056271 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f09c00a1090 0x7f09c00a3550 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.852+0000 7f09dd6f0640 1 --2- 172.21.15.16:0/289056271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09d8100f00 0x7f09d8196220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.852+0000 7f09dd6f0640 1 --2- 172.21.15.16:0/289056271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09d80ffd10 0x7f09d8195ce0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.852+0000 7f09dd6f0640 1 -- 172.21.15.16:0/289056271 >> 172.21.15.16:0/289056271 conn(0x7f09d80fb4b0 msgr2=0x7f09d8104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:33:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.852+0000 7f09dd6f0640 1 -- 172.21.15.16:0/289056271 shutdown_connections 2024-09-13T16:33:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:44.852+0000 7f09dd6f0640 1 -- 172.21.15.16:0/289056271 wait complete. 2024-09-13T16:33:46.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:45 smithi111 ceph-mon[34600]: pgmap v433: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 701 MiB/s wr, 186 op/s 2024-09-13T16:33:46.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:45 smithi107 ceph-mon[34139]: pgmap v433: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 701 MiB/s wr, 186 op/s 2024-09-13T16:33:46.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:45 smithi016 ceph-mon[31187]: pgmap v433: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 701 MiB/s wr, 186 op/s 2024-09-13T16:33:48.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:47 smithi111 ceph-mon[34600]: pgmap v434: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 645 MiB/s wr, 171 op/s 2024-09-13T16:33:48.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:47 smithi107 ceph-mon[34139]: pgmap v434: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 645 MiB/s wr, 171 op/s 2024-09-13T16:33:48.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:47 smithi016 ceph-mon[31187]: pgmap v434: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 645 MiB/s wr, 171 op/s 2024-09-13T16:33:50.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:49 smithi111 ceph-mon[34600]: pgmap v435: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 541 MiB/s wr, 149 op/s 2024-09-13T16:33:50.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:49 smithi107 ceph-mon[34139]: pgmap v435: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 541 MiB/s wr, 149 op/s 2024-09-13T16:33:50.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:49 smithi016 ceph-mon[31187]: pgmap v435: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 541 MiB/s wr, 149 op/s 2024-09-13T16:33:52.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:51 smithi111 ceph-mon[34600]: pgmap v436: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 382 MiB/s wr, 106 op/s 2024-09-13T16:33:52.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:51 smithi107 ceph-mon[34139]: pgmap v436: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 382 MiB/s wr, 106 op/s 2024-09-13T16:33:52.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:51 smithi016 ceph-mon[31187]: pgmap v436: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 382 MiB/s wr, 106 op/s 2024-09-13T16:33:54.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:53 smithi111 ceph-mon[34600]: pgmap v437: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 309 MiB/s wr, 90 op/s 2024-09-13T16:33:54.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:53 smithi107 ceph-mon[34139]: pgmap v437: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 309 MiB/s wr, 90 op/s 2024-09-13T16:33:54.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:53 smithi016 ceph-mon[31187]: pgmap v437: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 309 MiB/s wr, 90 op/s 2024-09-13T16:33:54.913 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-13T16:33:54.913 INFO:tasks.workunit.client.0.smithi016.stdout: Children see throughput for 1 readers = 8993912.00 kB/sec 2024-09-13T16:33:54.913 INFO:tasks.workunit.client.0.smithi016.stdout: Parent sees throughput for 1 readers = 8992239.29 kB/sec 2024-09-13T16:33:54.913 INFO:tasks.workunit.client.0.smithi016.stdout: Min throughput per process = 8993912.00 kB/sec 2024-09-13T16:33:54.913 INFO:tasks.workunit.client.0.smithi016.stdout: Max throughput per process = 8993912.00 kB/sec 2024-09-13T16:33:54.913 INFO:tasks.workunit.client.0.smithi016.stdout: Avg throughput per process = 8993912.00 kB/sec 2024-09-13T16:33:54.914 INFO:tasks.workunit.client.0.smithi016.stdout: Min xfer = 10485760.00 kB 2024-09-13T16:33:56.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:55 smithi107 ceph-mon[34139]: pgmap v438: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 200 MiB/s wr, 62 op/s 2024-09-13T16:33:56.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:55 smithi016 ceph-mon[31187]: pgmap v438: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 200 MiB/s wr, 62 op/s 2024-09-13T16:33:56.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:55 smithi111 ceph-mon[34600]: pgmap v438: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 200 MiB/s wr, 62 op/s 2024-09-13T16:33:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.951+0000 7fbba89f4640 1 -- 172.21.15.16:0/3941218338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbba40ffe60 msgr2=0x7fbba41002e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.951+0000 7fbba89f4640 1 --2- 172.21.15.16:0/3941218338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbba40ffe60 0x7fbba41002e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fbb94009900 tx=0x7fbb9402a410 comp rx=0 tx=0).stop 2024-09-13T16:33:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.951+0000 7fbba89f4640 1 -- 172.21.15.16:0/3941218338 shutdown_connections 2024-09-13T16:33:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.951+0000 7fbba89f4640 1 --2- 172.21.15.16:0/3941218338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbba4100820 0x7fbba41070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.951+0000 7fbba89f4640 1 --2- 172.21.15.16:0/3941218338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbba40ffe60 0x7fbba41002e0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.951+0000 7fbba89f4640 1 --2- 172.21.15.16:0/3941218338 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbba40fec70 0x7fbba40ff070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.951+0000 7fbba89f4640 1 -- 172.21.15.16:0/3941218338 >> 172.21.15.16:0/3941218338 conn(0x7fbba40fa3d0 msgr2=0x7fbba40fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:33:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.951+0000 7fbba89f4640 1 -- 172.21.15.16:0/3941218338 shutdown_connections 2024-09-13T16:33:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.951+0000 7fbba89f4640 1 -- 172.21.15.16:0/3941218338 wait complete. 2024-09-13T16:33:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.952+0000 7fbba89f4640 1 Processor -- start 2024-09-13T16:33:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.952+0000 7fbba89f4640 1 -- start start 2024-09-13T16:33:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.952+0000 7fbba89f4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbba40fec70 0x7fbba4068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.952+0000 7fbba89f4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbba40ffe60 0x7fbba419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.952+0000 7fbba89f4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbba4100820 0x7fbba419cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.952+0000 7fbba89f4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba419d780 con 0x7fbba4100820 2024-09-13T16:33:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.952+0000 7fbba89f4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba419d8f0 con 0x7fbba40fec70 2024-09-13T16:33:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.952+0000 7fbba89f4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba419dab0 con 0x7fbba40ffe60 2024-09-13T16:33:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.952+0000 7fbba2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbba4100820 0x7fbba419cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.952+0000 7fbba2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbba4100820 0x7fbba419cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54696/0 (socket says 172.21.15.16:54696) 2024-09-13T16:33:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.952+0000 7fbba2d9d640 1 -- 172.21.15.16:0/3991414915 learned_addr learned my addr 172.21.15.16:0/3991414915 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:33:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.952+0000 7fbba259c640 1 --2- 172.21.15.16:0/3991414915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbba40fec70 0x7fbba4068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.952+0000 7fbba1d9b640 1 --2- 172.21.15.16:0/3991414915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbba40ffe60 0x7fbba419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.953+0000 7fbba2d9d640 1 -- 172.21.15.16:0/3991414915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbba40ffe60 msgr2=0x7fbba419a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.953+0000 7fbba2d9d640 1 --2- 172.21.15.16:0/3991414915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbba40ffe60 0x7fbba419a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.953+0000 7fbba2d9d640 1 -- 172.21.15.16:0/3991414915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbba40fec70 msgr2=0x7fbba4068c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.953+0000 7fbba2d9d640 1 --2- 172.21.15.16:0/3991414915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbba40fec70 0x7fbba4068c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.953+0000 7fbba2d9d640 1 -- 172.21.15.16:0/3991414915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb94009560 con 0x7fbba4100820 2024-09-13T16:33:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.953+0000 7fbba259c640 1 --2- 172.21.15.16:0/3991414915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbba40fec70 0x7fbba4068c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:33:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.953+0000 7fbba2d9d640 1 --2- 172.21.15.16:0/3991414915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbba4100820 0x7fbba419cff0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c00eb10 tx=0x7fbb9c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:33:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.953+0000 7fbb9b7fe640 1 -- 172.21.15.16:0/3991414915 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c004b20 con 0x7fbba4100820 2024-09-13T16:33:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.953+0000 7fbb9b7fe640 1 -- 172.21.15.16:0/3991414915 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbb9c004c80 con 0x7fbba4100820 2024-09-13T16:33:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.953+0000 7fbb9b7fe640 1 -- 172.21.15.16:0/3991414915 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c005a10 con 0x7fbba4100820 2024-09-13T16:33:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.953+0000 7fbba89f4640 1 -- 172.21.15.16:0/3991414915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbba419dd90 con 0x7fbba4100820 2024-09-13T16:33:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.953+0000 7fbba89f4640 1 -- 172.21.15.16:0/3991414915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbba419e1c0 con 0x7fbba4100820 2024-09-13T16:33:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.956+0000 7fbb9b7fe640 1 -- 172.21.15.16:0/3991414915 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fbb9c005020 con 0x7fbba4100820 2024-09-13T16:33:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.956+0000 7fbb9b7fe640 1 --2- 172.21.15.16:0/3991414915 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fbb900a1110 0x7fbb900a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.957+0000 7fbb9b7fe640 1 -- 172.21.15.16:0/3991414915 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fbb9c014070 con 0x7fbba4100820 2024-09-13T16:33:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.957+0000 7fbba259c640 1 --2- 172.21.15.16:0/3991414915 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fbb900a1110 0x7fbb900a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.957+0000 7fbba259c640 1 --2- 172.21.15.16:0/3991414915 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fbb900a1110 0x7fbb900a35d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbba40ffcc0 tx=0x7fbb8c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:33:57.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.958+0000 7fbbab7f1740 1 Processor -- start 2024-09-13T16:33:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.958+0000 7fbbab7f1740 1 -- start start 2024-09-13T16:33:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.958+0000 7fbbab7f1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55846b4f41d0 0x55846b4f45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.958+0000 7fbbab7f1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55846b4f4ba0 0x55846b4f96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.958+0000 7fbbab7f1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55846b4f9d30 0x55846b4fc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.958+0000 7fbbab7f1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55846b45ec30 con 0x55846b4f41d0 2024-09-13T16:33:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.958+0000 7fbbab7f1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55846b45c3c0 con 0x55846b4f9d30 2024-09-13T16:33:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.958+0000 7fbbab7f1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55846b457cb0 con 0x55846b4f4ba0 2024-09-13T16:33:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.959+0000 7fbba259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55846b4f41d0 0x55846b4f45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.959+0000 7fbba259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55846b4f41d0 0x55846b4f45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54710/0 (socket says 172.21.15.16:54710) 2024-09-13T16:33:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.959+0000 7fbba259c640 1 -- 172.21.15.16:0/1874707996 learned_addr learned my addr 172.21.15.16:0/1874707996 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:33:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.959+0000 7fbba2d9d640 1 --2- 172.21.15.16:0/1874707996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55846b4f9d30 0x55846b4fc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.959+0000 7fbba1d9b640 1 --2- 172.21.15.16:0/1874707996 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55846b4f4ba0 0x55846b4f96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.959+0000 7fbba259c640 1 -- 172.21.15.16:0/1874707996 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55846b4f4ba0 msgr2=0x55846b4f96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.959+0000 7fbba259c640 1 --2- 172.21.15.16:0/1874707996 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55846b4f4ba0 0x55846b4f96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.959+0000 7fbba259c640 1 -- 172.21.15.16:0/1874707996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55846b4f9d30 msgr2=0x55846b4fc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:57.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.959+0000 7fbba259c640 1 --2- 172.21.15.16:0/1874707996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55846b4f9d30 0x55846b4fc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.959+0000 7fbba259c640 1 -- 172.21.15.16:0/1874707996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55846b3fe910 con 0x55846b4f41d0 2024-09-13T16:33:57.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.959+0000 7fbba1d9b640 1 --2- 172.21.15.16:0/1874707996 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55846b4f4ba0 0x55846b4f96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:33:57.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.959+0000 7fbba259c640 1 --2- 172.21.15.16:0/1874707996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55846b4f41d0 0x55846b4f45d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c004130 tx=0x7fbb8c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb7c5e01361172f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:33:57.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.959+0000 7fbb98ff9640 1 -- 172.21.15.16:0/1874707996 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c011c40 con 0x55846b4f41d0 2024-09-13T16:33:57.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.959+0000 7fbb98ff9640 1 -- 172.21.15.16:0/1874707996 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbb8c003db0 con 0x55846b4f41d0 2024-09-13T16:33:57.962 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.959+0000 7fbb98ff9640 1 -- 172.21.15.16:0/1874707996 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c020700 con 0x55846b4f41d0 2024-09-13T16:33:57.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.960+0000 7fbbab7f1740 1 -- 172.21.15.16:0/1874707996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55846b4f41d0 msgr2=0x55846b4f45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:57.966 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.960+0000 7fbbab7f1740 1 --2- 172.21.15.16:0/1874707996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55846b4f41d0 0x55846b4f45d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c004130 tx=0x7fbb8c014000 comp rx=0 tx=0).stop 2024-09-13T16:33:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.960+0000 7fbbab7f1740 1 -- 172.21.15.16:0/1874707996 shutdown_connections 2024-09-13T16:33:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.960+0000 7fbbab7f1740 1 --2- 172.21.15.16:0/1874707996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55846b4f9d30 0x55846b4fc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.960+0000 7fbbab7f1740 1 --2- 172.21.15.16:0/1874707996 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55846b4f4ba0 0x55846b4f96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.960+0000 7fbbab7f1740 1 --2- 172.21.15.16:0/1874707996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55846b4f41d0 0x55846b4f45d0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.960+0000 7fbbab7f1740 1 -- 172.21.15.16:0/1874707996 >> 172.21.15.16:0/1874707996 conn(0x55846b4ef8c0 msgr2=0x55846b4efcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:33:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.960+0000 7fbbab7f1740 1 -- 172.21.15.16:0/1874707996 shutdown_connections 2024-09-13T16:33:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.960+0000 7fbbab7f1740 1 -- 172.21.15.16:0/1874707996 wait complete. 2024-09-13T16:33:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.961+0000 7fbbab7f1740 1 -- start start 2024-09-13T16:33:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.961+0000 7fbbab7f1740 1 Processor -- start 2024-09-13T16:33:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.961+0000 7fbbab7f1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55846b4f41d0 0x55846b4fdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.961+0000 7fbbab7f1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55846b4f4ba0 0x55846b4fe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.961+0000 7fbbab7f1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55846b4f9d30 0x55846b505320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.961+0000 7fbbab7f1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55846b415e80 con 0x55846b4f9d30 2024-09-13T16:33:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.961+0000 7fbbab7f1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55846b415b20 con 0x55846b4f4ba0 2024-09-13T16:33:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.961+0000 7fbbab7f1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55846b3db260 con 0x55846b4f41d0 2024-09-13T16:33:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.961+0000 7fbbab7f1740 10 client.0 _ll_register_callbacks cb 0x55846b410df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:33:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.961+0000 7fbba2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55846b4f9d30 0x55846b505320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.961+0000 7fbba2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55846b4f9d30 0x55846b505320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54714/0 (socket says 172.21.15.16:54714) 2024-09-13T16:33:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.961+0000 7fbba2d9d640 1 -- 172.21.15.16:0/10963402 learned_addr learned my addr 172.21.15.16:0/10963402 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:33:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.961+0000 7fbba259c640 1 --2- 172.21.15.16:0/10963402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55846b4f41d0 0x55846b4fdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.962+0000 7fbba1d9b640 1 --2- 172.21.15.16:0/10963402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55846b4f4ba0 0x55846b4fe2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.962+0000 7fbba2d9d640 1 -- 172.21.15.16:0/10963402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55846b4f41d0 msgr2=0x55846b4fdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.962+0000 7fbba2d9d640 1 --2- 172.21.15.16:0/10963402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55846b4f41d0 0x55846b4fdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.962+0000 7fbba2d9d640 1 -- 172.21.15.16:0/10963402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55846b4f4ba0 msgr2=0x55846b4fe2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.962+0000 7fbba2d9d640 1 --2- 172.21.15.16:0/10963402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55846b4f4ba0 0x55846b4fe2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.962+0000 7fbba2d9d640 1 -- 172.21.15.16:0/10963402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb8c011da0 con 0x55846b4f9d30 2024-09-13T16:33:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.962+0000 7fbba2d9d640 1 --2- 172.21.15.16:0/10963402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55846b4f9d30 0x55846b505320 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0cadd0 tx=0x7fbb9c004df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc98fa3ecc73f38 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:33:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.962+0000 7fbb89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:33:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.962+0000 7fbb89ffb640 1 -- 172.21.15.16:0/10963402 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c0cd150 con 0x55846b4f9d30 2024-09-13T16:33:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.962+0000 7fbb89ffb640 1 -- 172.21.15.16:0/10963402 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fbb9c09a3b0 con 0x55846b4f9d30 2024-09-13T16:33:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.962+0000 7fbbab7f1740 1 -- 172.21.15.16:0/10963402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55846b480680 con 0x55846b4f9d30 2024-09-13T16:33:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.962+0000 7fbb89ffb640 1 -- 172.21.15.16:0/10963402 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb9c09e1e0 con 0x55846b4f9d30 2024-09-13T16:33:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.962+0000 7fbbab7f1740 10 client.15753 fetch_fsmap learned FSMap version 29 2024-09-13T16:33:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.962+0000 7fbbab7f1740 1 -- 172.21.15.16:0/10963402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55846b456330 con 0x55846b4f9d30 2024-09-13T16:33:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.962+0000 7fbb89ffb640 1 -- 172.21.15.16:0/10963402 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55846b480680 con 0x55846b4f9d30 2024-09-13T16:33:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.963+0000 7fbb89ffb640 1 -- 172.21.15.16:0/10963402 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbb9c01f060 con 0x55846b4f9d30 2024-09-13T16:33:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.963+0000 7fbbab7f1740 10 client.15753 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T16:33:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.963+0000 7fbbab7f1740 10 client.15753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T16:33:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.963+0000 7fbbab7f1740 20 client.15753 populate_metadata read hostname 'smithi016' 2024-09-13T16:33:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.963+0000 7fbbab7f1740 1 --2- 172.21.15.16:0/10963402 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55846b509a80 0x55846b50bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.963+0000 7fbbab7f1740 4 client.15753 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T16:33:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.963+0000 7fbbab7f1740 1 -- 172.21.15.16:0/10963402 --> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55846b44b5d0 con 0x55846b509a80 2024-09-13T16:33:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.963+0000 7fbba259c640 1 --2- 172.21.15.16:0/10963402 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55846b509a80 0x55846b50bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.963+0000 7fbb89ffb640 10 client.15753 ms_handle_connect on v2:172.21.15.107:6838/1914689806 2024-09-13T16:33:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.964+0000 7fbba259c640 1 --2- 172.21.15.16:0/10963402 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55846b509a80 0x55846b50bea0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23a5c0a11f7cb967 server_cookie=e23ef7a11eeeb1bf in_seq=0 out_seq=0 2024-09-13T16:33:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.964+0000 7fbb89ffb640 1 -- 172.21.15.16:0/10963402 <== mds.0 v2:172.21.15.107:6838/1914689806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55846b44b5d0 con 0x55846b509a80 2024-09-13T16:33:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.964+0000 7fbb89ffb640 10 client.15753 handle_command_reply: tid=0 2024-09-13T16:33:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.964+0000 7fbbab7f1740 1 client.15753 shutdown 2024-09-13T16:33:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.965+0000 7fbbab7f1740 1 -- 172.21.15.16:0/10963402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55846b4f9d30 msgr2=0x55846b505320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.965+0000 7fbbab7f1740 1 --2- 172.21.15.16:0/10963402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55846b4f9d30 0x55846b505320 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0cadd0 tx=0x7fbb9c004df0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.965+0000 7fbbab7f1740 1 -- 172.21.15.16:0/10963402 shutdown_connections 2024-09-13T16:33:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.965+0000 7fbbab7f1740 1 --2- 172.21.15.16:0/10963402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55846b4f9d30 0x55846b505320 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.965+0000 7fbbab7f1740 1 --2- 172.21.15.16:0/10963402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55846b4f4ba0 0x55846b4fe2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.965+0000 7fbbab7f1740 1 --2- 172.21.15.16:0/10963402 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55846b509a80 0x55846b50bea0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.965+0000 7fbbab7f1740 1 --2- 172.21.15.16:0/10963402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55846b4f41d0 0x55846b4fdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.965+0000 7fbb89ffb640 0 client.15753 ms_handle_reset on v2:172.21.15.107:6838/1914689806 2024-09-13T16:33:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.965+0000 7fbbab7f1740 1 -- 172.21.15.16:0/10963402 >> 172.21.15.16:0/10963402 conn(0x55846b4ef8c0 msgr2=0x55846b4eeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:33:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.965+0000 7fbbab7f1740 1 -- 172.21.15.16:0/10963402 shutdown_connections 2024-09-13T16:33:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.965+0000 7fbbab7f1740 1 -- 172.21.15.16:0/10963402 wait complete. 2024-09-13T16:33:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.965+0000 7fbbab7f1740 20 client.15753 trim_cache size 0 max 16384 2024-09-13T16:33:57.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.978+0000 7fbbab7f1740 1 Processor -- start 2024-09-13T16:33:57.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.978+0000 7fbbab7f1740 1 -- start start 2024-09-13T16:33:57.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.978+0000 7fbbab7f1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55846b4f41d0 0x55846b4fe260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:57.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.978+0000 7fbbab7f1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55846b4f4ba0 0x55846b5052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:57.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.978+0000 7fbbab7f1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55846b4f9d30 0x55846b4f1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:57.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.978+0000 7fbbab7f1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55846b4f4f80 con 0x55846b4f41d0 2024-09-13T16:33:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.978+0000 7fbbab7f1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55846b4f6030 con 0x55846b4f4ba0 2024-09-13T16:33:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.978+0000 7fbbab7f1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55846b44b5d0 con 0x55846b4f9d30 2024-09-13T16:33:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.978+0000 7fbba259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55846b4f41d0 0x55846b4fe260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.978+0000 7fbba259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55846b4f41d0 0x55846b4fe260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54722/0 (socket says 172.21.15.16:54722) 2024-09-13T16:33:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.978+0000 7fbba259c640 1 -- 172.21.15.16:0/3720177987 learned_addr learned my addr 172.21.15.16:0/3720177987 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:33:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.978+0000 7fbba2d9d640 1 --2- 172.21.15.16:0/3720177987 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55846b4f9d30 0x55846b4f1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.978+0000 7fbba1d9b640 1 --2- 172.21.15.16:0/3720177987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55846b4f4ba0 0x55846b5052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.978+0000 7fbba259c640 1 -- 172.21.15.16:0/3720177987 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55846b4f9d30 msgr2=0x55846b4f1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.978+0000 7fbba259c640 1 --2- 172.21.15.16:0/3720177987 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55846b4f9d30 0x55846b4f1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.979+0000 7fbba259c640 1 -- 172.21.15.16:0/3720177987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55846b4f4ba0 msgr2=0x55846b5052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.979+0000 7fbba259c640 1 --2- 172.21.15.16:0/3720177987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55846b4f4ba0 0x55846b5052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.979+0000 7fbba259c640 1 -- 172.21.15.16:0/3720177987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55846b3f8300 con 0x55846b4f41d0 2024-09-13T16:33:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.979+0000 7fbba259c640 1 --2- 172.21.15.16:0/3720177987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55846b4f41d0 0x55846b4fe260 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x55846b4f5420 tx=0x7fbb8c003f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43229d641e8cf65e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:33:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.979+0000 7fbb997fa640 1 -- 172.21.15.16:0/3720177987 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c031040 con 0x55846b4f41d0 2024-09-13T16:33:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.979+0000 7fbb997fa640 1 -- 172.21.15.16:0/3720177987 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbb8c0209c0 con 0x55846b4f41d0 2024-09-13T16:33:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.979+0000 7fbb997fa640 1 -- 172.21.15.16:0/3720177987 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c036730 con 0x55846b4f41d0 2024-09-13T16:33:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.980+0000 7fbbab7f1740 1 -- 172.21.15.16:0/3720177987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55846b4f41d0 msgr2=0x55846b4fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.980+0000 7fbbab7f1740 1 --2- 172.21.15.16:0/3720177987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55846b4f41d0 0x55846b4fe260 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x55846b4f5420 tx=0x7fbb8c003f20 comp rx=0 tx=0).stop 2024-09-13T16:33:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.980+0000 7fbbab7f1740 1 -- 172.21.15.16:0/3720177987 shutdown_connections 2024-09-13T16:33:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.980+0000 7fbbab7f1740 1 --2- 172.21.15.16:0/3720177987 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55846b4f9d30 0x55846b4f1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.980+0000 7fbbab7f1740 1 --2- 172.21.15.16:0/3720177987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55846b4f4ba0 0x55846b5052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.980+0000 7fbbab7f1740 1 --2- 172.21.15.16:0/3720177987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55846b4f41d0 0x55846b4fe260 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.980+0000 7fbbab7f1740 1 -- 172.21.15.16:0/3720177987 >> 172.21.15.16:0/3720177987 conn(0x55846b4ef8c0 msgr2=0x55846b4fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:33:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.980+0000 7fbbab7f1740 1 -- 172.21.15.16:0/3720177987 shutdown_connections 2024-09-13T16:33:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.980+0000 7fbbab7f1740 1 -- 172.21.15.16:0/3720177987 wait complete. 2024-09-13T16:33:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.980+0000 7fbbab7f1740 1 -- start start 2024-09-13T16:33:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.980+0000 7fbbab7f1740 1 Processor -- start 2024-09-13T16:33:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.980+0000 7fbbab7f1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55846b4f41d0 0x55846b503da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.981+0000 7fbbab7f1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55846b4f4ba0 0x55846b5042e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.981+0000 7fbbab7f1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55846b4f9d30 0x55846b55ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.981+0000 7fbbab7f1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55846b504a80 con 0x55846b4f41d0 2024-09-13T16:33:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.981+0000 7fbbab7f1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55846b504bf0 con 0x55846b4f4ba0 2024-09-13T16:33:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.981+0000 7fbbab7f1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55846b560470 con 0x55846b4f9d30 2024-09-13T16:33:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.981+0000 7fbbab7f1740 10 client.0 _ll_register_callbacks cb 0x7fbb9c099e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:33:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.981+0000 7fbba259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55846b4f41d0 0x55846b503da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.981+0000 7fbba259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55846b4f41d0 0x55846b503da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54730/0 (socket says 172.21.15.16:54730) 2024-09-13T16:33:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.981+0000 7fbba259c640 1 -- 172.21.15.16:0/859680616 learned_addr learned my addr 172.21.15.16:0/859680616 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:33:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.981+0000 7fbba1d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55846b4f4ba0 0x55846b5042e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.981+0000 7fbba2d9d640 1 --2- 172.21.15.16:0/859680616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55846b4f9d30 0x55846b55ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:57.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.981+0000 7fbba259c640 1 -- 172.21.15.16:0/859680616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55846b4f9d30 msgr2=0x55846b55ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:57.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.981+0000 7fbba259c640 1 --2- 172.21.15.16:0/859680616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55846b4f9d30 0x55846b55ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.981+0000 7fbba259c640 1 -- 172.21.15.16:0/859680616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55846b4f4ba0 msgr2=0x55846b5042e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:33:57.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.981+0000 7fbba259c640 1 --2- 172.21.15.16:0/859680616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55846b4f4ba0 0x55846b5042e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.981+0000 7fbba259c640 1 -- 172.21.15.16:0/859680616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55846b3fe910 con 0x55846b4f41d0 2024-09-13T16:33:57.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.981+0000 7fbba2d9d640 1 --2- 172.21.15.16:0/859680616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55846b4f9d30 0x55846b55ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:33:57.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.981+0000 7fbba1d9b640 1 --2- 172.21.15.16:0/859680616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55846b4f4ba0 0x55846b5042e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:33:57.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.981+0000 7fbba259c640 1 --2- 172.21.15.16:0/859680616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55846b4f41d0 0x55846b503da0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c003d30 tx=0x7fbb8c005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d42afe76d4a7dd85 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:33:57.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.981+0000 7fbb89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-13T16:33:57.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.982+0000 7fbb89ffb640 1 -- 172.21.15.16:0/859680616 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c031070 con 0x55846b4f41d0 2024-09-13T16:33:57.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.982+0000 7fbb89ffb640 1 -- 172.21.15.16:0/859680616 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fbb8c018070 con 0x55846b4f41d0 2024-09-13T16:33:57.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.982+0000 7fbbab7f1740 1 -- 172.21.15.16:0/859680616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb8c003db0 con 0x55846b4f41d0 2024-09-13T16:33:57.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.982+0000 7fbb89ffb640 1 -- 172.21.15.16:0/859680616 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb8c020630 con 0x55846b4f41d0 2024-09-13T16:33:57.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.982+0000 7fbbab7f1740 10 client.15759 fetch_fsmap learned FSMap version 29 2024-09-13T16:33:57.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.982+0000 7fbbab7f1740 1 -- 172.21.15.16:0/859680616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55846b41d8e0 con 0x55846b4f41d0 2024-09-13T16:33:57.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.982+0000 7fbb89ffb640 1 -- 172.21.15.16:0/859680616 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbb8c003db0 con 0x55846b4f41d0 2024-09-13T16:33:57.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.982+0000 7fbb89ffb640 1 -- 172.21.15.16:0/859680616 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbb8c01e040 con 0x55846b4f41d0 2024-09-13T16:33:57.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.982+0000 7fbbab7f1740 10 client.15759 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T16:33:57.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.982+0000 7fbbab7f1740 10 client.15759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T16:33:57.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.982+0000 7fbbab7f1740 20 client.15759 populate_metadata read hostname 'smithi016' 2024-09-13T16:33:57.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.982+0000 7fbbab7f1740 1 --2- 172.21.15.16:0/859680616 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55846b561dd0 0x55846b5641c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:33:57.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.982+0000 7fbbab7f1740 4 client.15759 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T16:33:57.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.982+0000 7fbbab7f1740 1 -- 172.21.15.16:0/859680616 --> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55846b14b300 con 0x55846b561dd0 2024-09-13T16:33:57.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.983+0000 7fbba1d9b640 1 --2- 172.21.15.16:0/859680616 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55846b561dd0 0x55846b5641c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:33:57.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.983+0000 7fbb89ffb640 10 client.15759 ms_handle_connect on v2:172.21.15.107:6838/1914689806 2024-09-13T16:33:57.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.983+0000 7fbba1d9b640 1 --2- 172.21.15.16:0/859680616 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55846b561dd0 0x55846b5641c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c581dcbe8b9b859 server_cookie=38bd2b6933e6467f in_seq=0 out_seq=0 2024-09-13T16:33:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.990+0000 7fbb89ffb640 1 -- 172.21.15.16:0/859680616 <== mds.0 v2:172.21.15.107:6838/1914689806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55846b14b300 con 0x55846b561dd0 2024-09-13T16:33:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.990+0000 7fbb89ffb640 10 client.15759 handle_command_reply: tid=0 2024-09-13T16:33:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.990+0000 7fbbab7f1740 1 client.15759 shutdown 2024-09-13T16:33:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.990+0000 7fbbab7f1740 1 -- 172.21.15.16:0/859680616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55846b4f41d0 msgr2=0x55846b503da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.990+0000 7fbbab7f1740 1 --2- 172.21.15.16:0/859680616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55846b4f41d0 0x55846b503da0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c003d30 tx=0x7fbb8c005970 comp rx=0 tx=0).stop 2024-09-13T16:33:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.990+0000 7fbbab7f1740 1 -- 172.21.15.16:0/859680616 shutdown_connections 2024-09-13T16:33:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.990+0000 7fbbab7f1740 1 --2- 172.21.15.16:0/859680616 >> [v2:172.21.15.107:6838/1914689806,v1:172.21.15.107:6839/1914689806] conn(0x55846b561dd0 0x55846b5641c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.990+0000 7fbbab7f1740 1 --2- 172.21.15.16:0/859680616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55846b4f9d30 0x55846b55ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.990+0000 7fbb89ffb640 0 client.15759 ms_handle_reset on v2:172.21.15.107:6838/1914689806 2024-09-13T16:33:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.990+0000 7fbbab7f1740 1 --2- 172.21.15.16:0/859680616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55846b4f4ba0 0x55846b5042e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.990+0000 7fbbab7f1740 1 --2- 172.21.15.16:0/859680616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55846b4f41d0 0x55846b503da0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.990+0000 7fbbab7f1740 1 -- 172.21.15.16:0/859680616 >> 172.21.15.16:0/859680616 conn(0x55846b4ef8c0 msgr2=0x55846b4ee2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:33:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.990+0000 7fbbab7f1740 1 -- 172.21.15.16:0/859680616 shutdown_connections 2024-09-13T16:33:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.990+0000 7fbbab7f1740 1 -- 172.21.15.16:0/859680616 wait complete. 2024-09-13T16:33:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.990+0000 7fbbab7f1740 20 client.15759 trim_cache size 0 max 16384 2024-09-13T16:33:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-13T16:33:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-13T16:33:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-13T16:33:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-13T16:33:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.991+0000 7fbba89f4640 1 -- 172.21.15.16:0/3991414915 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fbb900a1110 msgr2=0x7fbb900a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.991+0000 7fbba89f4640 1 --2- 172.21.15.16:0/3991414915 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fbb900a1110 0x7fbb900a35d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbba40ffcc0 tx=0x7fbb8c005f90 comp rx=0 tx=0).stop 2024-09-13T16:33:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.991+0000 7fbba89f4640 1 -- 172.21.15.16:0/3991414915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbba4100820 msgr2=0x7fbba419cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:33:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.991+0000 7fbba89f4640 1 --2- 172.21.15.16:0/3991414915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbba4100820 0x7fbba419cff0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c00eb10 tx=0x7fbb9c00c2d0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.991+0000 7fbba89f4640 1 -- 172.21.15.16:0/3991414915 shutdown_connections 2024-09-13T16:33:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.991+0000 7fbba89f4640 1 --2- 172.21.15.16:0/3991414915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbba4100820 0x7fbba419cff0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.991+0000 7fbba89f4640 1 --2- 172.21.15.16:0/3991414915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbba40ffe60 0x7fbba419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.991+0000 7fbba89f4640 1 --2- 172.21.15.16:0/3991414915 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fbb900a1110 0x7fbb900a35d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.991+0000 7fbba89f4640 1 --2- 172.21.15.16:0/3991414915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbba40fec70 0x7fbba4068c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:33:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.991+0000 7fbba89f4640 1 -- 172.21.15.16:0/3991414915 >> 172.21.15.16:0/3991414915 conn(0x7fbba40fa3d0 msgr2=0x7fbba41033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:33:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.991+0000 7fbba89f4640 1 -- 172.21.15.16:0/3991414915 shutdown_connections 2024-09-13T16:33:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-13T16:33:57.991+0000 7fbba89f4640 1 -- 172.21.15.16:0/3991414915 wait complete. 2024-09-13T16:33:58.162 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-13T16:33:58.162 INFO:tasks.workunit.client.0.smithi016.stdout: Children see throughput for 1 re-readers = 8872831.00 kB/sec 2024-09-13T16:33:58.162 INFO:tasks.workunit.client.0.smithi016.stdout: Parent sees throughput for 1 re-readers = 8871103.87 kB/sec 2024-09-13T16:33:58.162 INFO:tasks.workunit.client.0.smithi016.stdout: Min throughput per process = 8872831.00 kB/sec 2024-09-13T16:33:58.163 INFO:tasks.workunit.client.0.smithi016.stdout: Max throughput per process = 8872831.00 kB/sec 2024-09-13T16:33:58.163 INFO:tasks.workunit.client.0.smithi016.stdout: Avg throughput per process = 8872831.00 kB/sec 2024-09-13T16:33:58.163 INFO:tasks.workunit.client.0.smithi016.stdout: Min xfer = 10485760.00 kB 2024-09-13T16:33:58.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:57 smithi107 ceph-mon[34139]: pgmap v439: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 73 MiB/s wr, 30 op/s 2024-09-13T16:33:58.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:57 smithi016 ceph-mon[31187]: pgmap v439: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 73 MiB/s wr, 30 op/s 2024-09-13T16:33:58.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:57 smithi111 ceph-mon[34600]: pgmap v439: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 73 MiB/s wr, 30 op/s 2024-09-13T16:34:00.352 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:33:59 smithi107 ceph-mon[34139]: pgmap v440: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 14 MiB/s wr, 17 op/s 2024-09-13T16:34:00.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:33:59 smithi016 ceph-mon[31187]: pgmap v440: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 14 MiB/s wr, 17 op/s 2024-09-13T16:34:00.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:33:59 smithi111 ceph-mon[34600]: pgmap v440: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 14 MiB/s wr, 17 op/s 2024-09-13T16:34:02.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:01 smithi107 ceph-mon[34139]: pgmap v441: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-13T16:34:02.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:01 smithi016 ceph-mon[31187]: pgmap v441: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-13T16:34:02.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:01 smithi111 ceph-mon[34600]: pgmap v441: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-13T16:34:04.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:03 smithi107 ceph-mon[34139]: pgmap v442: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-09-13T16:34:04.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:03 smithi016 ceph-mon[31187]: pgmap v442: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-09-13T16:34:04.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:03 smithi111 ceph-mon[34600]: pgmap v442: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-09-13T16:34:04.989 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-13T16:34:04.989 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-13T16:34:04.990 INFO:tasks.workunit.client.0.smithi016.stdout: 2024-09-13T16:34:04.990 INFO:tasks.workunit.client.0.smithi016.stdout:iozone test complete. 2024-09-13T16:34:04.991 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-13T16:34:04.991 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-13T16:34:05.072 INFO:tasks.workunit:Stopping ['suites/iozone.sh'] on client.0... 2024-09-13T16:34:05.073 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-13T16:34:05.403 DEBUG:teuthology.parallel:result is None 2024-09-13T16:34:05.404 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-13T16:34:05.438 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-13T16:34:05.438 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-13T16:34:05.454 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi016.front.sepia.ceph.com 2024-09-13T16:34:05.489 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-13T16:34:05.500 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:34:05.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.630+0000 7f9ae26c9640 1 -- 172.21.15.16:0/3370506788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9adc0ffec0 msgr2=0x7f9adc100340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:05.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.630+0000 7f9ae26c9640 1 --2- 172.21.15.16:0/3370506788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9adc0ffec0 0x7f9adc100340 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4009900 tx=0x7f9ac402a410 comp rx=0 tx=0).stop 2024-09-13T16:34:05.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.630+0000 7f9ae26c9640 1 -- 172.21.15.16:0/3370506788 shutdown_connections 2024-09-13T16:34:05.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.630+0000 7f9ae26c9640 1 --2- 172.21.15.16:0/3370506788 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9adc100880 0x7f9adc107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:05.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.630+0000 7f9ae26c9640 1 --2- 172.21.15.16:0/3370506788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9adc0ffec0 0x7f9adc100340 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:05.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.630+0000 7f9ae26c9640 1 --2- 172.21.15.16:0/3370506788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9adc0fecd0 0x7f9adc0ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:05.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.630+0000 7f9ae26c9640 1 -- 172.21.15.16:0/3370506788 >> 172.21.15.16:0/3370506788 conn(0x7f9adc0fa430 msgr2=0x7f9adc0fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:05.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.630+0000 7f9ae26c9640 1 -- 172.21.15.16:0/3370506788 shutdown_connections 2024-09-13T16:34:05.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.630+0000 7f9ae26c9640 1 -- 172.21.15.16:0/3370506788 wait complete. 2024-09-13T16:34:05.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.631+0000 7f9ae26c9640 1 Processor -- start 2024-09-13T16:34:05.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.631+0000 7f9ae26c9640 1 -- start start 2024-09-13T16:34:05.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.631+0000 7f9ae26c9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9adc0fecd0 0x7f9adc195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:05.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.631+0000 7f9ae26c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9adc0ffec0 0x7f9adc196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:05.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.631+0000 7f9ae26c9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9adc100880 0x7f9adc19d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:05.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.631+0000 7f9ae26c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc19d840 con 0x7f9adc0ffec0 2024-09-13T16:34:05.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.631+0000 7f9ae26c9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc19d9b0 con 0x7f9adc100880 2024-09-13T16:34:05.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.631+0000 7f9ae26c9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc19db70 con 0x7f9adc0fecd0 2024-09-13T16:34:05.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.631+0000 7f9ad3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9adc0ffec0 0x7f9adc196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:05.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.631+0000 7f9ad3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9adc0ffec0 0x7f9adc196290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51606/0 (socket says 172.21.15.16:51606) 2024-09-13T16:34:05.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.631+0000 7f9ad3fff640 1 -- 172.21.15.16:0/420107958 learned_addr learned my addr 172.21.15.16:0/420107958 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:34:05.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.631+0000 7f9ae0c66640 1 --2- 172.21.15.16:0/420107958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9adc100880 0x7f9adc19d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:05.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.631+0000 7f9adbfff640 1 --2- 172.21.15.16:0/420107958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9adc0fecd0 0x7f9adc195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:05.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.631+0000 7f9ad3fff640 1 -- 172.21.15.16:0/420107958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9adc0fecd0 msgr2=0x7f9adc195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:05.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.631+0000 7f9ad3fff640 1 --2- 172.21.15.16:0/420107958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9adc0fecd0 0x7f9adc195d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:05.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.631+0000 7f9ad3fff640 1 -- 172.21.15.16:0/420107958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9adc100880 msgr2=0x7f9adc19d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:05.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.631+0000 7f9ad3fff640 1 --2- 172.21.15.16:0/420107958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9adc100880 0x7f9adc19d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:05.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.631+0000 7f9ad3fff640 1 -- 172.21.15.16:0/420107958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac4009560 con 0x7f9adc0ffec0 2024-09-13T16:34:05.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.632+0000 7f9ad3fff640 1 --2- 172.21.15.16:0/420107958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9adc0ffec0 0x7f9adc196290 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4006010 tx=0x7f9ac4005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:05.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.632+0000 7f9ad9ffb640 1 -- 172.21.15.16:0/420107958 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac4004cd0 con 0x7f9adc0ffec0 2024-09-13T16:34:05.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.632+0000 7f9ad9ffb640 1 -- 172.21.15.16:0/420107958 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9ac4003a80 con 0x7f9adc0ffec0 2024-09-13T16:34:05.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.632+0000 7f9ae26c9640 1 -- 172.21.15.16:0/420107958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9adc19ddf0 con 0x7f9adc0ffec0 2024-09-13T16:34:05.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.632+0000 7f9ad9ffb640 1 -- 172.21.15.16:0/420107958 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac403cad0 con 0x7f9adc0ffec0 2024-09-13T16:34:05.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.632+0000 7f9ae26c9640 1 -- 172.21.15.16:0/420107958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9adc19e260 con 0x7f9adc0ffec0 2024-09-13T16:34:05.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.633+0000 7f9ae26c9640 1 -- 172.21.15.16:0/420107958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9adc1061f0 con 0x7f9adc0ffec0 2024-09-13T16:34:05.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.634+0000 7f9ad9ffb640 1 -- 172.21.15.16:0/420107958 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f9ac4003bf0 con 0x7f9adc0ffec0 2024-09-13T16:34:05.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.635+0000 7f9ad9ffb640 1 --2- 172.21.15.16:0/420107958 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f9ac80a1110 0x7f9ac80a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:05.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.635+0000 7f9adbfff640 1 --2- 172.21.15.16:0/420107958 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f9ac80a1110 0x7f9ac80a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:05.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.635+0000 7f9ad9ffb640 1 -- 172.21.15.16:0/420107958 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f9ac40edb50 con 0x7f9adc0ffec0 2024-09-13T16:34:05.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.635+0000 7f9adbfff640 1 --2- 172.21.15.16:0/420107958 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f9ac80a1110 0x7f9ac80a35d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9acc005fd0 tx=0x7f9acc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:05.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.640+0000 7f9ad9ffb640 1 -- 172.21.15.16:0/420107958 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f9ac40b6fe0 con 0x7f9adc0ffec0 2024-09-13T16:34:05.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.814+0000 7f9ae26c9640 1 -- 172.21.15.16:0/420107958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9adc105c60 con 0x7f9adc0ffec0 2024-09-13T16:34:05.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.816+0000 7f9ad9ffb640 1 -- 172.21.15.16:0/420107958 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24593 (secure 0 0 0) 0x7f9ac4040090 con 0x7f9adc0ffec0 2024-09-13T16:34:05.819 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:34:05.819 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":93,"fsid":"0d241f6a-71ec-11ef-bceb-c7b262605968","created":"2024-09-13T16:21:17.971738+0000","modified":"2024-09-13T16:30:05.681521+0000","last_up_change":"2024-09-13T16:27:54.673413+0000","last_in_change":"2024-09-13T16:27:35.718390+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:24:22.981019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:28:37.746027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:28:38.602731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:28:41.633322+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e11ef8f-08d6-4e74-afd6-f49f4bd40032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6803","nonce":2079394186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6805","nonce":2079394186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6809","nonce":2079394186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6807","nonce":2079394186}]},"public_addr":"172.21.15.16:6803/2079394186","cluster_addr":"172.21.15.16:6805/2079394186","heartbeat_back_addr":"172.21.15.16:6809/2079394186","heartbeat_front_addr":"172.21.15.16:6807/2079394186","state":["exists","up"]},{"osd":1,"uuid":"f2cd3694-9c9e-4da7-976e-f80a8df120f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6801","nonce":4021360958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6803","nonce":4021360958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6807","nonce":4021360958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6805","nonce":4021360958}]},"public_addr":"172.21.15.107:6801/4021360958","cluster_addr":"172.21.15.107:6803/4021360958","heartbeat_back_addr":"172.21.15.107:6807/4021360958","heartbeat_front_addr":"172.21.15.107:6805/4021360958","state":["exists","up"]},{"osd":2,"uuid":"a5bbf2eb-6599-44d5-ad96-68bb89ee19f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6801","nonce":3227179825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6803","nonce":3227179825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6807","nonce":3227179825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6805","nonce":3227179825}]},"public_addr":"172.21.15.111:6801/3227179825","cluster_addr":"172.21.15.111:6803/3227179825","heartbeat_back_addr":"172.21.15.111:6807/3227179825","heartbeat_front_addr":"172.21.15.111:6805/3227179825","state":["exists","up"]},{"osd":3,"uuid":"d6295ffe-11bc-4964-a250-9fecca870b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6811","nonce":2390375937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6813","nonce":2390375937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6817","nonce":2390375937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6815","nonce":2390375937}]},"public_addr":"172.21.15.16:6811/2390375937","cluster_addr":"172.21.15.16:6813/2390375937","heartbeat_back_addr":"172.21.15.16:6817/2390375937","heartbeat_front_addr":"172.21.15.16:6815/2390375937","state":["exists","up"]},{"osd":4,"uuid":"0a8446e0-fc86-4646-beb6-a7b9d433093d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6809","nonce":1936582048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6811","nonce":1936582048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6815","nonce":1936582048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6813","nonce":1936582048}]},"public_addr":"172.21.15.107:6809/1936582048","cluster_addr":"172.21.15.107:6811/1936582048","heartbeat_back_addr":"172.21.15.107:6815/1936582048","heartbeat_front_addr":"172.21.15.107:6813/1936582048","state":["exists","up"]},{"osd":5,"uuid":"2c97a446-357e-4975-9e51-9c95fe84a663","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6809","nonce":483858448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6811","nonce":483858448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6815","nonce":483858448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6813","nonce":483858448}]},"public_addr":"172.21.15.111:6809/483858448","cluster_addr":"172.21.15.111:6811/483858448","heartbeat_back_addr":"172.21.15.111:6815/483858448","heartbeat_front_addr":"172.21.15.111:6813/483858448","state":["exists","up"]},{"osd":6,"uuid":"890239e6-e6fb-4f9c-b973-05cc66d199bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6819","nonce":1330214141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6821","nonce":1330214141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6825","nonce":1330214141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6823","nonce":1330214141}]},"public_addr":"172.21.15.16:6819/1330214141","cluster_addr":"172.21.15.16:6821/1330214141","heartbeat_back_addr":"172.21.15.16:6825/1330214141","heartbeat_front_addr":"172.21.15.16:6823/1330214141","state":["exists","up"]},{"osd":7,"uuid":"ee826a9c-f441-4903-9b1b-66b627f7b712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6817","nonce":1133876754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6819","nonce":1133876754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6823","nonce":1133876754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6821","nonce":1133876754}]},"public_addr":"172.21.15.107:6817/1133876754","cluster_addr":"172.21.15.107:6819/1133876754","heartbeat_back_addr":"172.21.15.107:6823/1133876754","heartbeat_front_addr":"172.21.15.107:6821/1133876754","state":["exists","up"]},{"osd":8,"uuid":"e19773f2-dec8-44e0-9d33-0252e07d6415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6817","nonce":837548190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6819","nonce":837548190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6823","nonce":837548190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6821","nonce":837548190}]},"public_addr":"172.21.15.111:6817/837548190","cluster_addr":"172.21.15.111:6819/837548190","heartbeat_back_addr":"172.21.15.111:6823/837548190","heartbeat_front_addr":"172.21.15.111:6821/837548190","state":["exists","up"]},{"osd":9,"uuid":"bae4d6db-8b90-4d02-8e83-7d8f2a6f8fdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6827","nonce":4175383686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6829","nonce":4175383686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6833","nonce":4175383686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6831","nonce":4175383686}]},"public_addr":"172.21.15.16:6827/4175383686","cluster_addr":"172.21.15.16:6829/4175383686","heartbeat_back_addr":"172.21.15.16:6833/4175383686","heartbeat_front_addr":"172.21.15.16:6831/4175383686","state":["exists","up"]},{"osd":10,"uuid":"fd0e7740-76c8-4c99-a349-7077b4905143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6825","nonce":2659550426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6827","nonce":2659550426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6831","nonce":2659550426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6829","nonce":2659550426}]},"public_addr":"172.21.15.107:6825/2659550426","cluster_addr":"172.21.15.107:6827/2659550426","heartbeat_back_addr":"172.21.15.107:6831/2659550426","heartbeat_front_addr":"172.21.15.107:6829/2659550426","state":["exists","up"]},{"osd":11,"uuid":"94a1751d-b00c-4e23-815d-3a8bc3dea146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6825","nonce":1633548683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6827","nonce":1633548683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6831","nonce":1633548683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6829","nonce":1633548683}]},"public_addr":"172.21.15.111:6825/1633548683","cluster_addr":"172.21.15.111:6827/1633548683","heartbeat_back_addr":"172.21.15.111:6831/1633548683","heartbeat_front_addr":"172.21.15.111:6829/1633548683","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:36.359852+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:58.014202+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:19.753672+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:43.404817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:06.001677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:29.361587+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:52.554847+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:15.957553+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:39.581093+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:04.911389+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:28.971280+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:53.473561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/2098378667":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/3536229016":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6801/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/2684687547":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6800/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/368410612":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/12158160":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6800/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6801/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/3754423379":"2024-09-14T16:21:40.126459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:34:05.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.818+0000 7f9ae26c9640 1 -- 172.21.15.16:0/420107958 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f9ac80a1110 msgr2=0x7f9ac80a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:05.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.818+0000 7f9ae26c9640 1 --2- 172.21.15.16:0/420107958 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f9ac80a1110 0x7f9ac80a35d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9acc005fd0 tx=0x7f9acc00b410 comp rx=0 tx=0).stop 2024-09-13T16:34:05.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.818+0000 7f9ae26c9640 1 -- 172.21.15.16:0/420107958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9adc0ffec0 msgr2=0x7f9adc196290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:05.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.818+0000 7f9ae26c9640 1 --2- 172.21.15.16:0/420107958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9adc0ffec0 0x7f9adc196290 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4006010 tx=0x7f9ac4005610 comp rx=0 tx=0).stop 2024-09-13T16:34:05.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.818+0000 7f9ae26c9640 1 -- 172.21.15.16:0/420107958 shutdown_connections 2024-09-13T16:34:05.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.818+0000 7f9ae26c9640 1 --2- 172.21.15.16:0/420107958 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f9ac80a1110 0x7f9ac80a35d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:05.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.818+0000 7f9ae26c9640 1 --2- 172.21.15.16:0/420107958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9adc100880 0x7f9adc19d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:05.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.818+0000 7f9ae26c9640 1 --2- 172.21.15.16:0/420107958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9adc0ffec0 0x7f9adc196290 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:05.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.818+0000 7f9ae26c9640 1 --2- 172.21.15.16:0/420107958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9adc0fecd0 0x7f9adc195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:05.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.818+0000 7f9ae26c9640 1 -- 172.21.15.16:0/420107958 >> 172.21.15.16:0/420107958 conn(0x7f9adc0fa430 msgr2=0x7f9adc1033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:05.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.818+0000 7f9ae26c9640 1 -- 172.21.15.16:0/420107958 shutdown_connections 2024-09-13T16:34:05.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.818+0000 7f9ae26c9640 1 -- 172.21.15.16:0/420107958 wait complete. 2024-09-13T16:34:05.832 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:24:22.981019+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T16:28:37.746027+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T16:28:38.602731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T16:28:41.633322+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T16:34:05.832 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:34:05.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.935+0000 7ff7d751f640 1 -- 172.21.15.16:0/908779268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7d01007e0 msgr2=0x7ff7d0102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:05.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.935+0000 7ff7d751f640 1 --2- 172.21.15.16:0/908779268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7d01007e0 0x7ff7d0102bd0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc009900 tx=0x7ff7cc02a410 comp rx=0 tx=0).stop 2024-09-13T16:34:05.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.935+0000 7ff7d751f640 1 -- 172.21.15.16:0/908779268 shutdown_connections 2024-09-13T16:34:05.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.935+0000 7ff7d751f640 1 --2- 172.21.15.16:0/908779268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7d0105a40 0x7ff7d0107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:05.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.935+0000 7ff7d751f640 1 --2- 172.21.15.16:0/908779268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7d0103110 0x7ff7d0105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:05.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.935+0000 7ff7d751f640 1 --2- 172.21.15.16:0/908779268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7d01007e0 0x7ff7d0102bd0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:05.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.935+0000 7ff7d751f640 1 -- 172.21.15.16:0/908779268 >> 172.21.15.16:0/908779268 conn(0x7ff7d00fa3e0 msgr2=0x7ff7d00fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:05.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.935+0000 7ff7d751f640 1 -- 172.21.15.16:0/908779268 shutdown_connections 2024-09-13T16:34:05.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.935+0000 7ff7d751f640 1 -- 172.21.15.16:0/908779268 wait complete. 2024-09-13T16:34:05.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.935+0000 7ff7d751f640 1 Processor -- start 2024-09-13T16:34:05.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.935+0000 7ff7d751f640 1 -- start start 2024-09-13T16:34:05.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.936+0000 7ff7d751f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7d01007e0 0x7ff7d0195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.936+0000 7ff7d751f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7d0103110 0x7ff7d0196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.936+0000 7ff7d751f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7d0105a40 0x7ff7d019d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.936+0000 7ff7d751f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d019d810 con 0x7ff7d0103110 2024-09-13T16:34:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.936+0000 7ff7d751f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d019d980 con 0x7ff7d0105a40 2024-09-13T16:34:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.936+0000 7ff7d751f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d019db40 con 0x7ff7d01007e0 2024-09-13T16:34:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.936+0000 7ff7d4aba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7d0103110 0x7ff7d0196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.936+0000 7ff7d4aba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7d0103110 0x7ff7d0196270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51630/0 (socket says 172.21.15.16:51630) 2024-09-13T16:34:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.936+0000 7ff7d4aba640 1 -- 172.21.15.16:0/995015273 learned_addr learned my addr 172.21.15.16:0/995015273 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:34:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.936+0000 7ff7d5abc640 1 --2- 172.21.15.16:0/995015273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7d0105a40 0x7ff7d019d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.936+0000 7ff7d52bb640 1 --2- 172.21.15.16:0/995015273 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7d01007e0 0x7ff7d0195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.936+0000 7ff7d4aba640 1 -- 172.21.15.16:0/995015273 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7d01007e0 msgr2=0x7ff7d0195d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.936+0000 7ff7d4aba640 1 --2- 172.21.15.16:0/995015273 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7d01007e0 0x7ff7d0195d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.936+0000 7ff7d4aba640 1 -- 172.21.15.16:0/995015273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7d0105a40 msgr2=0x7ff7d019d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.936+0000 7ff7d4aba640 1 --2- 172.21.15.16:0/995015273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7d0105a40 0x7ff7d019d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.936+0000 7ff7d4aba640 1 -- 172.21.15.16:0/995015273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7cc009560 con 0x7ff7d0103110 2024-09-13T16:34:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.936+0000 7ff7d52bb640 1 --2- 172.21.15.16:0/995015273 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7d01007e0 0x7ff7d0195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:34:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.936+0000 7ff7d4aba640 1 --2- 172.21.15.16:0/995015273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7d0103110 0x7ff7d0196270 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400e9c0 tx=0x7ff7c400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.936+0000 7ff7c27fc640 1 -- 172.21.15.16:0/995015273 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4009700 con 0x7ff7d0103110 2024-09-13T16:34:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.936+0000 7ff7c27fc640 1 -- 172.21.15.16:0/995015273 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff7c400cd80 con 0x7ff7d0103110 2024-09-13T16:34:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.936+0000 7ff7d751f640 1 -- 172.21.15.16:0/995015273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d019de20 con 0x7ff7d0103110 2024-09-13T16:34:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.936+0000 7ff7c27fc640 1 -- 172.21.15.16:0/995015273 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4010430 con 0x7ff7d0103110 2024-09-13T16:34:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.937+0000 7ff7d751f640 1 -- 172.21.15.16:0/995015273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d019e240 con 0x7ff7d0103110 2024-09-13T16:34:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.937+0000 7ff7d751f640 1 -- 172.21.15.16:0/995015273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7d01034f0 con 0x7ff7d0103110 2024-09-13T16:34:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.939+0000 7ff7c27fc640 1 -- 172.21.15.16:0/995015273 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff7c4004eb0 con 0x7ff7d0103110 2024-09-13T16:34:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.940+0000 7ff7c27fc640 1 --2- 172.21.15.16:0/995015273 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff7bc0a1090 0x7ff7bc0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.940+0000 7ff7c27fc640 1 -- 172.21.15.16:0/995015273 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7ff7c4014070 con 0x7ff7d0103110 2024-09-13T16:34:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.940+0000 7ff7d52bb640 1 --2- 172.21.15.16:0/995015273 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff7bc0a1090 0x7ff7bc0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:05.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.941+0000 7ff7d52bb640 1 --2- 172.21.15.16:0/995015273 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff7bc0a1090 0x7ff7bc0a3550 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc005ab0 tx=0x7ff7cc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:05.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:05.943+0000 7ff7c27fc640 1 -- 172.21.15.16:0/995015273 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff7c4096980 con 0x7ff7d0103110 2024-09-13T16:34:06.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:05 smithi016 ceph-mon[31187]: pgmap v443: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-13T16:34:06.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:05 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/420107958' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:34:06.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.114+0000 7ff7d751f640 1 -- 172.21.15.16:0/995015273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff7d010b540 con 0x7ff7d0103110 2024-09-13T16:34:06.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.114+0000 7ff7c27fc640 1 -- 172.21.15.16:0/995015273 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff7c40988b0 con 0x7ff7d0103110 2024-09-13T16:34:06.117 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-13T16:34:06.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.116+0000 7ff7d751f640 1 -- 172.21.15.16:0/995015273 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff7bc0a1090 msgr2=0x7ff7bc0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:06.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.116+0000 7ff7d751f640 1 --2- 172.21.15.16:0/995015273 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff7bc0a1090 0x7ff7bc0a3550 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc005ab0 tx=0x7ff7cc005a20 comp rx=0 tx=0).stop 2024-09-13T16:34:06.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.116+0000 7ff7d751f640 1 -- 172.21.15.16:0/995015273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7d0103110 msgr2=0x7ff7d0196270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:06.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.116+0000 7ff7d751f640 1 --2- 172.21.15.16:0/995015273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7d0103110 0x7ff7d0196270 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400e9c0 tx=0x7ff7c400ee80 comp rx=0 tx=0).stop 2024-09-13T16:34:06.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.116+0000 7ff7d751f640 1 -- 172.21.15.16:0/995015273 shutdown_connections 2024-09-13T16:34:06.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.116+0000 7ff7d751f640 1 --2- 172.21.15.16:0/995015273 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff7bc0a1090 0x7ff7bc0a3550 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:06.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.116+0000 7ff7d751f640 1 --2- 172.21.15.16:0/995015273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7d0105a40 0x7ff7d019d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:06.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.116+0000 7ff7d751f640 1 --2- 172.21.15.16:0/995015273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7d0103110 0x7ff7d0196270 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:06.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.116+0000 7ff7d751f640 1 --2- 172.21.15.16:0/995015273 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7d01007e0 0x7ff7d0195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:06.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.116+0000 7ff7d751f640 1 -- 172.21.15.16:0/995015273 >> 172.21.15.16:0/995015273 conn(0x7ff7d00fa3e0 msgr2=0x7ff7d01010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:06.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.117+0000 7ff7d751f640 1 -- 172.21.15.16:0/995015273 shutdown_connections 2024-09-13T16:34:06.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.117+0000 7ff7d751f640 1 -- 172.21.15.16:0/995015273 wait complete. 2024-09-13T16:34:06.131 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T16:34:06.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.234+0000 7fa2c3eb1640 1 -- 172.21.15.16:0/598002518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2bc0fec90 msgr2=0x7fa2bc0ff090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:06.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.234+0000 7fa2c3eb1640 1 --2- 172.21.15.16:0/598002518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2bc0fec90 0x7fa2bc0ff090 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8009900 tx=0x7fa2b802a410 comp rx=0 tx=0).stop 2024-09-13T16:34:06.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.234+0000 7fa2c3eb1640 1 -- 172.21.15.16:0/598002518 shutdown_connections 2024-09-13T16:34:06.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.234+0000 7fa2c3eb1640 1 --2- 172.21.15.16:0/598002518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2bc100840 0x7fa2bc1070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:06.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.234+0000 7fa2c3eb1640 1 --2- 172.21.15.16:0/598002518 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2bc0ffe80 0x7fa2bc100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.234+0000 7fa2c3eb1640 1 --2- 172.21.15.16:0/598002518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2bc0fec90 0x7fa2bc0ff090 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.234+0000 7fa2c3eb1640 1 -- 172.21.15.16:0/598002518 >> 172.21.15.16:0/598002518 conn(0x7fa2bc0fa3f0 msgr2=0x7fa2bc0fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.234+0000 7fa2c3eb1640 1 -- 172.21.15.16:0/598002518 shutdown_connections 2024-09-13T16:34:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.234+0000 7fa2c3eb1640 1 -- 172.21.15.16:0/598002518 wait complete. 2024-09-13T16:34:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.235+0000 7fa2c3eb1640 1 Processor -- start 2024-09-13T16:34:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.235+0000 7fa2c3eb1640 1 -- start start 2024-09-13T16:34:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.235+0000 7fa2c3eb1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2bc0fec90 0x7fa2bc197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.235+0000 7fa2c3eb1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2bc0ffe80 0x7fa2bc198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.235+0000 7fa2c3eb1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2bc100840 0x7fa2bc19f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.235+0000 7fa2c3eb1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc19f9e0 con 0x7fa2bc0fec90 2024-09-13T16:34:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.235+0000 7fa2c3eb1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc19fb50 con 0x7fa2bc100840 2024-09-13T16:34:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.235+0000 7fa2c3eb1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc19fd10 con 0x7fa2bc0ffe80 2024-09-13T16:34:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.235+0000 7fa2c1c4d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2bc0fec90 0x7fa2bc197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.235+0000 7fa2c1c4d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2bc0fec90 0x7fa2bc197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51650/0 (socket says 172.21.15.16:51650) 2024-09-13T16:34:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.235+0000 7fa2c1c4d640 1 -- 172.21.15.16:0/3288282456 learned_addr learned my addr 172.21.15.16:0/3288282456 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:34:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.235+0000 7fa2c244e640 1 --2- 172.21.15.16:0/3288282456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2bc100840 0x7fa2bc19f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.235+0000 7fa2c144c640 1 --2- 172.21.15.16:0/3288282456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2bc0ffe80 0x7fa2bc198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.235+0000 7fa2c1c4d640 1 -- 172.21.15.16:0/3288282456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2bc0ffe80 msgr2=0x7fa2bc198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.235+0000 7fa2c1c4d640 1 --2- 172.21.15.16:0/3288282456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2bc0ffe80 0x7fa2bc198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.235+0000 7fa2c1c4d640 1 -- 172.21.15.16:0/3288282456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2bc100840 msgr2=0x7fa2bc19f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.235+0000 7fa2c1c4d640 1 --2- 172.21.15.16:0/3288282456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2bc100840 0x7fa2bc19f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.235+0000 7fa2c1c4d640 1 -- 172.21.15.16:0/3288282456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b8009560 con 0x7fa2bc0fec90 2024-09-13T16:34:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.235+0000 7fa2c244e640 1 --2- 172.21.15.16:0/3288282456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2bc100840 0x7fa2bc19f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:34:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.235+0000 7fa2c1c4d640 1 --2- 172.21.15.16:0/3288282456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2bc0fec90 0x7fa2bc197ef0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8005ab0 tx=0x7fa2b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.236+0000 7fa2b2ffd640 1 -- 172.21.15.16:0/3288282456 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8031d70 con 0x7fa2bc0fec90 2024-09-13T16:34:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.236+0000 7fa2b2ffd640 1 -- 172.21.15.16:0/3288282456 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa2b80044e0 con 0x7fa2bc0fec90 2024-09-13T16:34:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.236+0000 7fa2c3eb1640 1 -- 172.21.15.16:0/3288282456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2bc19ff90 con 0x7fa2bc0fec90 2024-09-13T16:34:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.236+0000 7fa2b2ffd640 1 -- 172.21.15.16:0/3288282456 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8003b00 con 0x7fa2bc0fec90 2024-09-13T16:34:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.236+0000 7fa2c3eb1640 1 -- 172.21.15.16:0/3288282456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2bc1a03b0 con 0x7fa2bc0fec90 2024-09-13T16:34:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.237+0000 7fa2c3eb1640 1 -- 172.21.15.16:0/3288282456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2bc10a400 con 0x7fa2bc0fec90 2024-09-13T16:34:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.238+0000 7fa2b2ffd640 1 -- 172.21.15.16:0/3288282456 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa2b8004c50 con 0x7fa2bc0fec90 2024-09-13T16:34:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.238+0000 7fa2b2ffd640 1 --2- 172.21.15.16:0/3288282456 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa2a80a1110 0x7fa2a80a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.239+0000 7fa2c144c640 1 --2- 172.21.15.16:0/3288282456 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa2a80a1110 0x7fa2a80a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.239+0000 7fa2b2ffd640 1 -- 172.21.15.16:0/3288282456 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fa2b80edd40 con 0x7fa2bc0fec90 2024-09-13T16:34:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.239+0000 7fa2c144c640 1 --2- 172.21.15.16:0/3288282456 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa2a80a1110 0x7fa2a80a35d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac005fd0 tx=0x7fa2ac005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:06.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.243+0000 7fa2b2ffd640 1 -- 172.21.15.16:0/3288282456 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa2b80b7180 con 0x7fa2bc0fec90 2024-09-13T16:34:06.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:05 smithi107 ceph-mon[34139]: pgmap v443: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-13T16:34:06.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:05 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/420107958' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:34:06.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.413+0000 7fa2c3eb1640 1 -- 172.21.15.16:0/3288282456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa2bc1a05a0 con 0x7fa2bc0fec90 2024-09-13T16:34:06.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.413+0000 7fa2b2ffd640 1 -- 172.21.15.16:0/3288282456 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa2b80ba7a0 con 0x7fa2bc0fec90 2024-09-13T16:34:06.415 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-13T16:34:06.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.415+0000 7fa2c3eb1640 1 -- 172.21.15.16:0/3288282456 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa2a80a1110 msgr2=0x7fa2a80a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:06.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.415+0000 7fa2c3eb1640 1 --2- 172.21.15.16:0/3288282456 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa2a80a1110 0x7fa2a80a35d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac005fd0 tx=0x7fa2ac005850 comp rx=0 tx=0).stop 2024-09-13T16:34:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.415+0000 7fa2c3eb1640 1 -- 172.21.15.16:0/3288282456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2bc0fec90 msgr2=0x7fa2bc197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.415+0000 7fa2c3eb1640 1 --2- 172.21.15.16:0/3288282456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2bc0fec90 0x7fa2bc197ef0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8005ab0 tx=0x7fa2b80042f0 comp rx=0 tx=0).stop 2024-09-13T16:34:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.415+0000 7fa2c3eb1640 1 -- 172.21.15.16:0/3288282456 shutdown_connections 2024-09-13T16:34:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.415+0000 7fa2c3eb1640 1 --2- 172.21.15.16:0/3288282456 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fa2a80a1110 0x7fa2a80a35d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.415+0000 7fa2c3eb1640 1 --2- 172.21.15.16:0/3288282456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2bc100840 0x7fa2bc19f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.415+0000 7fa2c3eb1640 1 --2- 172.21.15.16:0/3288282456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2bc0ffe80 0x7fa2bc198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.415+0000 7fa2c3eb1640 1 --2- 172.21.15.16:0/3288282456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2bc0fec90 0x7fa2bc197ef0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.415+0000 7fa2c3eb1640 1 -- 172.21.15.16:0/3288282456 >> 172.21.15.16:0/3288282456 conn(0x7fa2bc0fa3f0 msgr2=0x7fa2bc068620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.415+0000 7fa2c3eb1640 1 -- 172.21.15.16:0/3288282456 shutdown_connections 2024-09-13T16:34:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.415+0000 7fa2c3eb1640 1 -- 172.21.15.16:0/3288282456 wait complete. 2024-09-13T16:34:06.428 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T16:34:06.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:05 smithi111 ceph-mon[34600]: pgmap v443: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-13T16:34:06.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:05 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/420107958' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:34:06.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.531+0000 7f3dfd8e9640 1 -- 172.21.15.16:0/2418262891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df8105a40 msgr2=0x7f3df8107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:06.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.531+0000 7f3dfd8e9640 1 --2- 172.21.15.16:0/2418262891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df8105a40 0x7f3df8107eb0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3df0009900 tx=0x7f3df002a410 comp rx=0 tx=0).stop 2024-09-13T16:34:06.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.532+0000 7f3dfd8e9640 1 -- 172.21.15.16:0/2418262891 shutdown_connections 2024-09-13T16:34:06.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.532+0000 7f3dfd8e9640 1 --2- 172.21.15.16:0/2418262891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df8105a40 0x7f3df8107eb0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:06.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.532+0000 7f3dfd8e9640 1 --2- 172.21.15.16:0/2418262891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3df8103110 0x7f3df8105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:06.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.532+0000 7f3dfd8e9640 1 --2- 172.21.15.16:0/2418262891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3df81007e0 0x7f3df8102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:06.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.532+0000 7f3dfd8e9640 1 -- 172.21.15.16:0/2418262891 >> 172.21.15.16:0/2418262891 conn(0x7f3df80fa3e0 msgr2=0x7f3df80fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:06.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.532+0000 7f3dfd8e9640 1 -- 172.21.15.16:0/2418262891 shutdown_connections 2024-09-13T16:34:06.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.532+0000 7f3dfd8e9640 1 -- 172.21.15.16:0/2418262891 wait complete. 2024-09-13T16:34:06.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.532+0000 7f3dfd8e9640 1 Processor -- start 2024-09-13T16:34:06.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.532+0000 7f3dfd8e9640 1 -- start start 2024-09-13T16:34:06.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.532+0000 7f3dfd8e9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3df81007e0 0x7f3df8195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:06.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.532+0000 7f3dfd8e9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3df8103110 0x7f3df81961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:06.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.532+0000 7f3dfd8e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df8105a40 0x7f3df819d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:06.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.532+0000 7f3dfd8e9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df819d790 con 0x7f3df8105a40 2024-09-13T16:34:06.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.532+0000 7f3dfd8e9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df819d900 con 0x7f3df8103110 2024-09-13T16:34:06.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.532+0000 7f3dfd8e9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df819dac0 con 0x7f3df81007e0 2024-09-13T16:34:06.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.533+0000 7f3df77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df8105a40 0x7f3df819d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:06.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.533+0000 7f3df77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df8105a40 0x7f3df819d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51664/0 (socket says 172.21.15.16:51664) 2024-09-13T16:34:06.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.533+0000 7f3df77fe640 1 -- 172.21.15.16:0/1325234211 learned_addr learned my addr 172.21.15.16:0/1325234211 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:34:06.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.533+0000 7f3df6ffd640 1 --2- 172.21.15.16:0/1325234211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3df81007e0 0x7f3df8195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:06.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.533+0000 7f3df67fc640 1 --2- 172.21.15.16:0/1325234211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3df8103110 0x7f3df81961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:06.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.533+0000 7f3df77fe640 1 -- 172.21.15.16:0/1325234211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3df81007e0 msgr2=0x7f3df8195c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:06.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.533+0000 7f3df77fe640 1 --2- 172.21.15.16:0/1325234211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3df81007e0 0x7f3df8195c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:06.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.533+0000 7f3df77fe640 1 -- 172.21.15.16:0/1325234211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3df8103110 msgr2=0x7f3df81961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:06.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.533+0000 7f3df77fe640 1 --2- 172.21.15.16:0/1325234211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3df8103110 0x7f3df81961c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:06.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.533+0000 7f3df77fe640 1 -- 172.21.15.16:0/1325234211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df0009560 con 0x7f3df8105a40 2024-09-13T16:34:06.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.533+0000 7f3df6ffd640 1 --2- 172.21.15.16:0/1325234211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3df81007e0 0x7f3df8195c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:34:06.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.533+0000 7f3df77fe640 1 --2- 172.21.15.16:0/1325234211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df8105a40 0x7f3df819d220 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3df002aa00 tx=0x7f3df00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:06.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.533+0000 7f3dfc8e7640 1 -- 172.21.15.16:0/1325234211 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df0031d70 con 0x7f3df8105a40 2024-09-13T16:34:06.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.533+0000 7f3dfc8e7640 1 -- 172.21.15.16:0/1325234211 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3df00044e0 con 0x7f3df8105a40 2024-09-13T16:34:06.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.533+0000 7f3dfd8e9640 1 -- 172.21.15.16:0/1325234211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df819dd40 con 0x7f3df8105a40 2024-09-13T16:34:06.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.533+0000 7f3dfc8e7640 1 -- 172.21.15.16:0/1325234211 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df0003ba0 con 0x7f3df8105a40 2024-09-13T16:34:06.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.533+0000 7f3dfd8e9640 1 -- 172.21.15.16:0/1325234211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df819e1e0 con 0x7f3df8105a40 2024-09-13T16:34:06.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.534+0000 7f3dfd8e9640 1 -- 172.21.15.16:0/1325234211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df80fbed0 con 0x7f3df8105a40 2024-09-13T16:34:06.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.535+0000 7f3dfc8e7640 1 -- 172.21.15.16:0/1325234211 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f3df0003d00 con 0x7f3df8105a40 2024-09-13T16:34:06.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.536+0000 7f3dfc8e7640 1 --2- 172.21.15.16:0/1325234211 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f3de00a1110 0x7f3de00a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:06.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.536+0000 7f3df6ffd640 1 --2- 172.21.15.16:0/1325234211 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f3de00a1110 0x7f3de00a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:06.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.536+0000 7f3dfc8e7640 1 -- 172.21.15.16:0/1325234211 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f3df00ee060 con 0x7f3df8105a40 2024-09-13T16:34:06.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.537+0000 7f3df6ffd640 1 --2- 172.21.15.16:0/1325234211 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f3de00a1110 0x7f3de00a35d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3df8196b50 tx=0x7f3de8009520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:06.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.540+0000 7f3dfc8e7640 1 -- 172.21.15.16:0/1325234211 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f3df00b75c0 con 0x7f3df8105a40 2024-09-13T16:34:06.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.712+0000 7f3dfd8e9640 1 -- 172.21.15.16:0/1325234211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3df810b7e0 con 0x7f3df8105a40 2024-09-13T16:34:06.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.712+0000 7f3dfc8e7640 1 -- 172.21.15.16:0/1325234211 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3df00400a0 con 0x7f3df8105a40 2024-09-13T16:34:06.714 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-09-13T16:34:06.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.714+0000 7f3dfd8e9640 1 -- 172.21.15.16:0/1325234211 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f3de00a1110 msgr2=0x7f3de00a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:06.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.714+0000 7f3dfd8e9640 1 --2- 172.21.15.16:0/1325234211 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f3de00a1110 0x7f3de00a35d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3df8196b50 tx=0x7f3de8009520 comp rx=0 tx=0).stop 2024-09-13T16:34:06.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.714+0000 7f3dfd8e9640 1 -- 172.21.15.16:0/1325234211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df8105a40 msgr2=0x7f3df819d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:06.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.714+0000 7f3dfd8e9640 1 --2- 172.21.15.16:0/1325234211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df8105a40 0x7f3df819d220 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3df002aa00 tx=0x7f3df00042f0 comp rx=0 tx=0).stop 2024-09-13T16:34:06.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.714+0000 7f3dfd8e9640 1 -- 172.21.15.16:0/1325234211 shutdown_connections 2024-09-13T16:34:06.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.714+0000 7f3dfd8e9640 1 --2- 172.21.15.16:0/1325234211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df8105a40 0x7f3df819d220 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:06.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.714+0000 7f3dfd8e9640 1 --2- 172.21.15.16:0/1325234211 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f3de00a1110 0x7f3de00a35d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:06.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.714+0000 7f3dfd8e9640 1 --2- 172.21.15.16:0/1325234211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3df8103110 0x7f3df81961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:06.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.714+0000 7f3dfd8e9640 1 --2- 172.21.15.16:0/1325234211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3df81007e0 0x7f3df8195c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:06.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.714+0000 7f3dfd8e9640 1 -- 172.21.15.16:0/1325234211 >> 172.21.15.16:0/1325234211 conn(0x7f3df80fa3e0 msgr2=0x7f3df8101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:06.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.714+0000 7f3dfd8e9640 1 -- 172.21.15.16:0/1325234211 shutdown_connections 2024-09-13T16:34:06.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.715+0000 7f3dfd8e9640 1 -- 172.21.15.16:0/1325234211 wait complete. 2024-09-13T16:34:06.727 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T16:34:06.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.830+0000 7fb41518a640 1 -- 172.21.15.16:0/1948706377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb410100810 msgr2=0x7fb410102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:06.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.830+0000 7fb41518a640 1 --2- 172.21.15.16:0/1948706377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb410100810 0x7fb410102c00 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8009900 tx=0x7fb3f802a410 comp rx=0 tx=0).stop 2024-09-13T16:34:06.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.830+0000 7fb41518a640 1 -- 172.21.15.16:0/1948706377 shutdown_connections 2024-09-13T16:34:06.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.830+0000 7fb41518a640 1 --2- 172.21.15.16:0/1948706377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb410105a70 0x7fb410107ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:06.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.830+0000 7fb41518a640 1 --2- 172.21.15.16:0/1948706377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb410103140 0x7fb410105530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:06.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.830+0000 7fb41518a640 1 --2- 172.21.15.16:0/1948706377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb410100810 0x7fb410102c00 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:06.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.830+0000 7fb41518a640 1 -- 172.21.15.16:0/1948706377 >> 172.21.15.16:0/1948706377 conn(0x7fb4100fa3f0 msgr2=0x7fb4100fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:06.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.830+0000 7fb41518a640 1 -- 172.21.15.16:0/1948706377 shutdown_connections 2024-09-13T16:34:06.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.830+0000 7fb41518a640 1 -- 172.21.15.16:0/1948706377 wait complete. 2024-09-13T16:34:06.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.831+0000 7fb41518a640 1 Processor -- start 2024-09-13T16:34:06.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.831+0000 7fb41518a640 1 -- start start 2024-09-13T16:34:06.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.831+0000 7fb41518a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb410100810 0x7fb410195c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:06.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.831+0000 7fb41518a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb410103140 0x7fb4101961d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:06.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.831+0000 7fb41518a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb410105a70 0x7fb41019d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:06.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.831+0000 7fb41518a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41019d7a0 con 0x7fb410103140 2024-09-13T16:34:06.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.831+0000 7fb41518a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41019d910 con 0x7fb410100810 2024-09-13T16:34:06.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.831+0000 7fb41518a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41019dad0 con 0x7fb410105a70 2024-09-13T16:34:06.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.831+0000 7fb40e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb410103140 0x7fb4101961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:06.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.831+0000 7fb40e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb410103140 0x7fb4101961d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51690/0 (socket says 172.21.15.16:51690) 2024-09-13T16:34:06.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.831+0000 7fb40e59c640 1 -- 172.21.15.16:0/2792904859 learned_addr learned my addr 172.21.15.16:0/2792904859 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:34:06.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.832+0000 7fb40f59e640 1 --2- 172.21.15.16:0/2792904859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb410105a70 0x7fb41019d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:06.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.832+0000 7fb40ed9d640 1 --2- 172.21.15.16:0/2792904859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb410100810 0x7fb410195c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:06.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.832+0000 7fb40e59c640 1 -- 172.21.15.16:0/2792904859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb410105a70 msgr2=0x7fb41019d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:06.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.832+0000 7fb40e59c640 1 --2- 172.21.15.16:0/2792904859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb410105a70 0x7fb41019d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:06.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.832+0000 7fb40e59c640 1 -- 172.21.15.16:0/2792904859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb410100810 msgr2=0x7fb410195c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:06.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.832+0000 7fb40e59c640 1 --2- 172.21.15.16:0/2792904859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb410100810 0x7fb410195c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:06.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.832+0000 7fb40e59c640 1 -- 172.21.15.16:0/2792904859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f8009560 con 0x7fb410103140 2024-09-13T16:34:06.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.832+0000 7fb40ed9d640 1 --2- 172.21.15.16:0/2792904859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb410100810 0x7fb410195c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:34:06.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.832+0000 7fb40e59c640 1 --2- 172.21.15.16:0/2792904859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb410103140 0x7fb4101961d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb40000e9c0 tx=0x7fb40000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:06.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.832+0000 7fb407fff640 1 -- 172.21.15.16:0/2792904859 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb400009700 con 0x7fb410103140 2024-09-13T16:34:06.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.832+0000 7fb41518a640 1 -- 172.21.15.16:0/2792904859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb41019ddb0 con 0x7fb410103140 2024-09-13T16:34:06.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.832+0000 7fb407fff640 1 -- 172.21.15.16:0/2792904859 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb40000cd80 con 0x7fb410103140 2024-09-13T16:34:06.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.832+0000 7fb407fff640 1 -- 172.21.15.16:0/2792904859 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb400010430 con 0x7fb410103140 2024-09-13T16:34:06.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.832+0000 7fb41518a640 1 -- 172.21.15.16:0/2792904859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb41019e280 con 0x7fb410103140 2024-09-13T16:34:06.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.833+0000 7fb41518a640 1 -- 172.21.15.16:0/2792904859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4100fbee0 con 0x7fb410103140 2024-09-13T16:34:06.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.834+0000 7fb407fff640 1 -- 172.21.15.16:0/2792904859 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb400004eb0 con 0x7fb410103140 2024-09-13T16:34:06.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.835+0000 7fb407fff640 1 --2- 172.21.15.16:0/2792904859 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb3fc0a1090 0x7fb3fc0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:06.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.835+0000 7fb407fff640 1 -- 172.21.15.16:0/2792904859 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb400014070 con 0x7fb410103140 2024-09-13T16:34:06.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.835+0000 7fb40ed9d640 1 --2- 172.21.15.16:0/2792904859 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb3fc0a1090 0x7fb3fc0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:06.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.835+0000 7fb40ed9d640 1 --2- 172.21.15.16:0/2792904859 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb3fc0a1090 0x7fb3fc0a3550 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8005ab0 tx=0x7fb3f8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:06.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:06.839+0000 7fb407fff640 1 -- 172.21.15.16:0/2792904859 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb400096600 con 0x7fb410103140 2024-09-13T16:34:07.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.012+0000 7fb41518a640 1 -- 172.21.15.16:0/2792904859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb41002c480 con 0x7fb410103140 2024-09-13T16:34:07.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.012+0000 7fb407fff640 1 -- 172.21.15.16:0/2792904859 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb400099c20 con 0x7fb410103140 2024-09-13T16:34:07.015 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-13T16:34:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.014+0000 7fb41518a640 1 -- 172.21.15.16:0/2792904859 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb3fc0a1090 msgr2=0x7fb3fc0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.014+0000 7fb41518a640 1 --2- 172.21.15.16:0/2792904859 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb3fc0a1090 0x7fb3fc0a3550 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8005ab0 tx=0x7fb3f8005a20 comp rx=0 tx=0).stop 2024-09-13T16:34:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.014+0000 7fb41518a640 1 -- 172.21.15.16:0/2792904859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb410103140 msgr2=0x7fb4101961d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.014+0000 7fb41518a640 1 --2- 172.21.15.16:0/2792904859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb410103140 0x7fb4101961d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb40000e9c0 tx=0x7fb40000ee80 comp rx=0 tx=0).stop 2024-09-13T16:34:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.014+0000 7fb41518a640 1 -- 172.21.15.16:0/2792904859 shutdown_connections 2024-09-13T16:34:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.014+0000 7fb41518a640 1 --2- 172.21.15.16:0/2792904859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb410105a70 0x7fb41019d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.014+0000 7fb41518a640 1 --2- 172.21.15.16:0/2792904859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb410103140 0x7fb4101961d0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.014+0000 7fb41518a640 1 --2- 172.21.15.16:0/2792904859 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb3fc0a1090 0x7fb3fc0a3550 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.014+0000 7fb41518a640 1 --2- 172.21.15.16:0/2792904859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb410100810 0x7fb410195c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.014+0000 7fb41518a640 1 -- 172.21.15.16:0/2792904859 >> 172.21.15.16:0/2792904859 conn(0x7fb4100fa3f0 msgr2=0x7fb410101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.015+0000 7fb41518a640 1 -- 172.21.15.16:0/2792904859 shutdown_connections 2024-09-13T16:34:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.015+0000 7fb41518a640 1 -- 172.21.15.16:0/2792904859 wait complete. 2024-09-13T16:34:07.026 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:06 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/995015273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:34:07.027 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:06 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3288282456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T16:34:07.027 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:06 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1325234211' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T16:34:07.028 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-13T16:34:07.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.129+0000 7ff605f1f640 1 -- 172.21.15.16:0/1407006488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6000ffec0 msgr2=0x7ff600100340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:07.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.129+0000 7ff605f1f640 1 --2- 172.21.15.16:0/1407006488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6000ffec0 0x7ff600100340 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4009900 tx=0x7ff5f402a410 comp rx=0 tx=0).stop 2024-09-13T16:34:07.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.129+0000 7ff605f1f640 1 -- 172.21.15.16:0/1407006488 shutdown_connections 2024-09-13T16:34:07.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.129+0000 7ff605f1f640 1 --2- 172.21.15.16:0/1407006488 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff600100880 0x7ff600107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:07.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.129+0000 7ff605f1f640 1 --2- 172.21.15.16:0/1407006488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6000ffec0 0x7ff600100340 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:07.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.129+0000 7ff605f1f640 1 --2- 172.21.15.16:0/1407006488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6000fecd0 0x7ff6000ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:07.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.129+0000 7ff605f1f640 1 -- 172.21.15.16:0/1407006488 >> 172.21.15.16:0/1407006488 conn(0x7ff6000fa430 msgr2=0x7ff6000fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:07.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.129+0000 7ff605f1f640 1 -- 172.21.15.16:0/1407006488 shutdown_connections 2024-09-13T16:34:07.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.129+0000 7ff605f1f640 1 -- 172.21.15.16:0/1407006488 wait complete. 2024-09-13T16:34:07.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.129+0000 7ff605f1f640 1 Processor -- start 2024-09-13T16:34:07.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.130+0000 7ff605f1f640 1 -- start start 2024-09-13T16:34:07.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.130+0000 7ff605f1f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6000fecd0 0x7ff600109590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:07.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.130+0000 7ff605f1f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6000ffec0 0x7ff600107bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:07.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.130+0000 7ff605f1f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff600100880 0x7ff600108110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:07.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.130+0000 7ff605f1f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff600108870 con 0x7ff6000ffec0 2024-09-13T16:34:07.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.130+0000 7ff605f1f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6001089e0 con 0x7ff6000fecd0 2024-09-13T16:34:07.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.130+0000 7ff605f1f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60019fd80 con 0x7ff600100880 2024-09-13T16:34:07.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.130+0000 7ff5feffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6000ffec0 0x7ff600107bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:07.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.130+0000 7ff5feffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6000ffec0 0x7ff600107bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51708/0 (socket says 172.21.15.16:51708) 2024-09-13T16:34:07.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.130+0000 7ff5feffd640 1 -- 172.21.15.16:0/2447463342 learned_addr learned my addr 172.21.15.16:0/2447463342 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:34:07.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.130+0000 7ff5fffff640 1 --2- 172.21.15.16:0/2447463342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff600100880 0x7ff600108110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:07.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.130+0000 7ff5ff7fe640 1 --2- 172.21.15.16:0/2447463342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6000fecd0 0x7ff600109590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:07.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.130+0000 7ff5feffd640 1 -- 172.21.15.16:0/2447463342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff600100880 msgr2=0x7ff600108110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:07.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.130+0000 7ff5feffd640 1 --2- 172.21.15.16:0/2447463342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff600100880 0x7ff600108110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:07.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.130+0000 7ff5feffd640 1 -- 172.21.15.16:0/2447463342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6000fecd0 msgr2=0x7ff600109590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:07.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.130+0000 7ff5feffd640 1 --2- 172.21.15.16:0/2447463342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6000fecd0 0x7ff600109590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:07.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.130+0000 7ff5feffd640 1 -- 172.21.15.16:0/2447463342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f4009560 con 0x7ff6000ffec0 2024-09-13T16:34:07.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.130+0000 7ff5ff7fe640 1 --2- 172.21.15.16:0/2447463342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6000fecd0 0x7ff600109590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:34:07.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.130+0000 7ff5feffd640 1 --2- 172.21.15.16:0/2447463342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6000ffec0 0x7ff600107bd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4005ab0 tx=0x7ff5f40052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:07.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.131+0000 7ff5fcff9640 1 -- 172.21.15.16:0/2447463342 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4031d70 con 0x7ff6000ffec0 2024-09-13T16:34:07.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.131+0000 7ff5fcff9640 1 -- 172.21.15.16:0/2447463342 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff5f40054a0 con 0x7ff6000ffec0 2024-09-13T16:34:07.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.131+0000 7ff605f1f640 1 -- 172.21.15.16:0/2447463342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff60019ffd0 con 0x7ff6000ffec0 2024-09-13T16:34:07.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.131+0000 7ff5fcff9640 1 -- 172.21.15.16:0/2447463342 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4003ac0 con 0x7ff6000ffec0 2024-09-13T16:34:07.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.131+0000 7ff605f1f640 1 -- 172.21.15.16:0/2447463342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6001a03f0 con 0x7ff6000ffec0 2024-09-13T16:34:07.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.132+0000 7ff605f1f640 1 -- 172.21.15.16:0/2447463342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff600106220 con 0x7ff6000ffec0 2024-09-13T16:34:07.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.133+0000 7ff5fcff9640 1 -- 172.21.15.16:0/2447463342 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff5f4003c40 con 0x7ff6000ffec0 2024-09-13T16:34:07.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.133+0000 7ff5fcff9640 1 --2- 172.21.15.16:0/2447463342 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff5e80a1090 0x7ff5e80a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:07.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.134+0000 7ff5fcff9640 1 -- 172.21.15.16:0/2447463342 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7ff5f40ee0b0 con 0x7ff6000ffec0 2024-09-13T16:34:07.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.134+0000 7ff5ff7fe640 1 --2- 172.21.15.16:0/2447463342 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff5e80a1090 0x7ff5e80a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:07.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.134+0000 7ff5ff7fe640 1 --2- 172.21.15.16:0/2447463342 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff5e80a1090 0x7ff5e80a3550 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff6000ffd20 tx=0x7ff5f0005e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:07.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.138+0000 7ff5fcff9640 1 -- 172.21.15.16:0/2447463342 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff5f40b74f0 con 0x7ff6000ffec0 2024-09-13T16:34:07.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:06 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/995015273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:34:07.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:06 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3288282456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T16:34:07.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:06 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1325234211' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T16:34:07.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.380+0000 7ff605f1f640 1 -- 172.21.15.16:0/2447463342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7ff600105c90 con 0x7ff6000ffec0 2024-09-13T16:34:07.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.395+0000 7ff5fcff9640 1 -- 172.21.15.16:0/2447463342 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) v1 ==== 81+0+733410 (secure 0 0 0) 0x7ff5f40bab10 con 0x7ff6000ffec0 2024-09-13T16:34:07.409 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:34:07.409 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-13T16:34:07.409 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 20, 2024-09-13T16:34:07.409 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.409 INFO:teuthology.orchestra.run.smithi016.stdout: "active_gid": 14150, 2024-09-13T16:34:07.409 INFO:teuthology.orchestra.run.smithi016.stdout: "active_name": "x", 2024-09-13T16:34:07.409 INFO:teuthology.orchestra.run.smithi016.stdout: "active_addrs": { 2024-09-13T16:34:07.409 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-13T16:34:07.409 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.410 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-13T16:34:07.410 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6800", 2024-09-13T16:34:07.410 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3739009683 2024-09-13T16:34:07.410 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.410 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.410 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-13T16:34:07.410 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6801", 2024-09-13T16:34:07.410 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3739009683 2024-09-13T16:34:07.410 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.410 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-13T16:34:07.410 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.410 INFO:teuthology.orchestra.run.smithi016.stdout: "active_addr": "172.21.15.16:6801/3739009683", 2024-09-13T16:34:07.410 INFO:teuthology.orchestra.run.smithi016.stdout: "active_change": "2024-09-13T16:22:04.485388+0000", 2024-09-13T16:34:07.411 INFO:teuthology.orchestra.run.smithi016.stdout: "active_mgr_features": 4540138322906710015, 2024-09-13T16:34:07.411 INFO:teuthology.orchestra.run.smithi016.stdout: "available": true, 2024-09-13T16:34:07.411 INFO:teuthology.orchestra.run.smithi016.stdout: "standbys": [ 2024-09-13T16:34:07.411 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.411 INFO:teuthology.orchestra.run.smithi016.stdout: "gid": 24109, 2024-09-13T16:34:07.411 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "z", 2024-09-13T16:34:07.411 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr_features": 4540138322906710015, 2024-09-13T16:34:07.411 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-09-13T16:34:07.411 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.411 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-09-13T16:34:07.411 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.411 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.411 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.412 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-13T16:34:07.412 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-13T16:34:07.412 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:07.412 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.412 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.412 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-13T16:34:07.412 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.412 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.412 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.412 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-09-13T16:34:07.412 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.412 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.412 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.413 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.413 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.413 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.413 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.413 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.413 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.413 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.413 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.413 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.413 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.413 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.413 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.414 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.414 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.414 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.414 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.414 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.414 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.414 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.414 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.414 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.414 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.414 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.414 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.414 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.414 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.415 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.415 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.415 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.415 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.415 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.415 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.415 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.415 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.415 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.415 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.415 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.415 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.415 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.416 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.416 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.416 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.416 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.416 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.416 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.416 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.416 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.416 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.416 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.416 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.416 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.416 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.416 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.417 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.417 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.417 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.417 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.417 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.417 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.417 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.417 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.417 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.417 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.417 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.417 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.418 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.418 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.418 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.418 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.418 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.418 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.418 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-09-13T16:34:07.418 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-09-13T16:34:07.418 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.418 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.418 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.418 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.418 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.419 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.419 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.419 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to", 2024-09-13T16:34:07.419 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.419 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.419 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.419 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.419 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-09-13T16:34:07.419 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-09-13T16:34:07.419 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.419 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.419 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.419 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-09-13T16:34:07.419 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.420 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.420 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.420 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-09-13T16:34:07.420 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.420 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.420 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.420 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.420 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-09-13T16:34:07.420 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-09-13T16:34:07.420 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.420 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.420 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.420 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.421 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.421 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.421 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.421 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-09-13T16:34:07.421 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.421 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.421 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.421 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.421 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-09-13T16:34:07.421 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-09-13T16:34:07.421 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.421 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.421 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.422 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.422 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.422 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.422 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.422 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-09-13T16:34:07.422 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.422 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.422 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.422 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.422 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-09-13T16:34:07.422 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-09-13T16:34:07.423 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.423 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.423 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.423 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-09-13T16:34:07.423 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.423 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.423 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.423 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-09-13T16:34:07.423 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.423 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.423 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.423 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.423 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-09-13T16:34:07.423 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-09-13T16:34:07.424 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.424 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.424 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.424 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.424 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.424 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.424 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.424 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-09-13T16:34:07.424 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.424 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.425 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.425 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.425 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-09-13T16:34:07.425 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-09-13T16:34:07.425 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.425 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.425 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.425 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.425 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.425 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.425 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.425 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-13T16:34:07.426 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.426 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.426 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.426 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.426 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-09-13T16:34:07.426 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-09-13T16:34:07.426 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.426 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.426 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.426 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.426 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.426 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.426 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.427 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-09-13T16:34:07.427 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.427 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.427 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.427 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.427 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.427 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.427 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.427 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-09-13T16:34:07.427 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.427 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.427 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.428 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-09-13T16:34:07.428 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-09-13T16:34:07.428 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.428 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.428 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.428 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.428 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.428 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.428 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.428 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-09-13T16:34:07.428 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.429 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.429 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.429 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.429 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-09-13T16:34:07.429 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-09-13T16:34:07.430 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.430 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.430 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.430 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-09-13T16:34:07.430 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.430 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.431 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.431 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-09-13T16:34:07.431 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-13T16:34:07.431 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.431 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.431 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.432 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-09-13T16:34:07.432 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-09-13T16:34:07.432 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-13T16:34:07.432 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.432 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.432 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-13T16:34:07.433 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-13T16:34:07.433 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-13T16:34:07.433 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.433 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-13T16:34:07.433 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-13T16:34:07.434 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.434 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.434 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.434 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-09-13T16:34:07.434 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-09-13T16:34:07.434 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-13T16:34:07.435 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.435 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.435 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-09-13T16:34:07.435 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-13T16:34:07.435 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-09-13T16:34:07.436 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.436 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-13T16:34:07.436 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.436 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.436 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.436 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.437 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-09-13T16:34:07.437 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-09-13T16:34:07.437 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.437 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.437 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.437 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-09-13T16:34:07.438 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.438 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.438 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.438 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-13T16:34:07.438 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-13T16:34:07.438 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.439 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.439 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.439 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-09-13T16:34:07.439 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-09-13T16:34:07.439 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-13T16:34:07.440 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.440 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.440 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-09-13T16:34:07.440 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-09-13T16:34:07.440 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-09-13T16:34:07.440 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.441 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-09-13T16:34:07.441 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-13T16:34:07.441 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.441 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.441 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.441 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-09-13T16:34:07.442 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-09-13T16:34:07.443 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.443 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.443 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.443 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-09-13T16:34:07.443 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.443 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.443 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.444 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-09-13T16:34:07.444 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-13T16:34:07.444 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.444 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.444 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.444 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-09-13T16:34:07.444 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-09-13T16:34:07.444 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-13T16:34:07.444 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.444 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.444 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-13T16:34:07.444 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-13T16:34:07.444 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-13T16:34:07.444 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.445 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-13T16:34:07.445 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-13T16:34:07.445 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.445 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.445 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.445 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.445 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.445 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.445 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.445 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.445 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.445 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.445 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.446 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.446 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.446 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.446 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.446 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.446 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.446 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.446 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.446 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.446 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.446 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.446 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.446 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.447 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.447 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.447 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.447 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.447 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.447 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.447 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.447 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.448 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.448 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.448 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.448 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.448 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.448 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.448 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.448 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.448 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.448 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.448 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.448 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.448 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.449 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.449 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.449 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.449 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.449 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.449 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.449 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.449 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.449 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.449 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.449 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.449 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.449 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.450 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.450 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.450 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.450 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.450 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.450 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.450 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.450 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.450 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.450 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.450 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.450 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.450 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.451 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.451 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.451 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-09-13T16:34:07.451 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-09-13T16:34:07.451 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-13T16:34:07.451 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.451 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.451 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-13T16:34:07.451 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.451 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.451 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.452 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-13T16:34:07.452 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.452 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.452 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.452 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.452 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-13T16:34:07.452 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-13T16:34:07.452 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.452 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.452 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.452 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-09-13T16:34:07.452 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.452 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.453 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.453 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-09-13T16:34:07.453 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-09-13T16:34:07.453 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap" 2024-09-13T16:34:07.453 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.453 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-09-13T16:34:07.453 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.453 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.453 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.453 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.453 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-09-13T16:34:07.453 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-09-13T16:34:07.453 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.454 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.454 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.454 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.454 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.454 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.454 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.454 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-13T16:34:07.454 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.454 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.454 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.454 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.454 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-13T16:34:07.454 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-13T16:34:07.455 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:07.455 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.455 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.455 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-13T16:34:07.455 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.455 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.455 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.455 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-13T16:34:07.455 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.455 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.455 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.455 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.455 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-09-13T16:34:07.456 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-09-13T16:34:07.456 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.456 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.456 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.456 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-13T16:34:07.456 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-13T16:34:07.456 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.456 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.456 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-13T16:34:07.456 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-13T16:34:07.456 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.456 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.457 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.457 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-09-13T16:34:07.457 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-09-13T16:34:07.457 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-13T16:34:07.457 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.457 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.457 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-13T16:34:07.457 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.457 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.457 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.457 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-13T16:34:07.457 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.458 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.458 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.458 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.458 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.458 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.458 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.458 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-09-13T16:34:07.458 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.458 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.459 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.459 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-09-13T16:34:07.459 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-09-13T16:34:07.459 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-13T16:34:07.459 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.459 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.459 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-13T16:34:07.459 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.459 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.459 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.459 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-13T16:34:07.459 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.460 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.460 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.460 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.460 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-09-13T16:34:07.460 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-09-13T16:34:07.460 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:07.460 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.460 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.460 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-09-13T16:34:07.460 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.460 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.461 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.461 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-13T16:34:07.461 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.461 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.461 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.461 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.461 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-09-13T16:34:07.461 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-09-13T16:34:07.461 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.461 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.461 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.461 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-09-13T16:34:07.461 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.461 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.462 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.462 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-13T16:34:07.462 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.462 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.462 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.462 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.462 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-09-13T16:34:07.462 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-09-13T16:34:07.462 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.462 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.462 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.462 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.463 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.463 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.463 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.463 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-13T16:34:07.463 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-13T16:34:07.463 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.463 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.463 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.463 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-09-13T16:34:07.463 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-09-13T16:34:07.463 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:07.463 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.464 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.464 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-13T16:34:07.464 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.464 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.464 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.464 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-13T16:34:07.464 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.464 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.464 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.464 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.464 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-09-13T16:34:07.464 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-09-13T16:34:07.464 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-13T16:34:07.465 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.465 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.465 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-09-13T16:34:07.465 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.465 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.465 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.465 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-13T16:34:07.465 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.465 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.465 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.465 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.465 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-09-13T16:34:07.465 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-09-13T16:34:07.466 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.466 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.466 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.466 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.466 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.466 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.466 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.466 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-13T16:34:07.466 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.466 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.466 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.466 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.466 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-09-13T16:34:07.467 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-09-13T16:34:07.467 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.467 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.467 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.467 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.467 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.467 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.467 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.467 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-13T16:34:07.467 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.467 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.467 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.467 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.468 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-09-13T16:34:07.468 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-09-13T16:34:07.468 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.468 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.468 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.468 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.468 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.468 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.468 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.468 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-13T16:34:07.468 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.468 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.469 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.469 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.469 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-09-13T16:34:07.469 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-09-13T16:34:07.469 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.469 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.469 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.469 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-13T16:34:07.469 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.469 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.469 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.469 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-13T16:34:07.470 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.470 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.470 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.470 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.470 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-09-13T16:34:07.470 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-09-13T16:34:07.470 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.470 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.470 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.470 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-13T16:34:07.470 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.470 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.470 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.470 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-09-13T16:34:07.471 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.471 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.471 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.471 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.471 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-09-13T16:34:07.471 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-09-13T16:34:07.471 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.471 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.471 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.471 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-13T16:34:07.471 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.472 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.472 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.472 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-09-13T16:34:07.472 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.472 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.472 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.472 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.472 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-09-13T16:34:07.472 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-09-13T16:34:07.472 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.473 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.473 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.473 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-09-13T16:34:07.473 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.473 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.473 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.473 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-13T16:34:07.473 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.473 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.473 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.473 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.473 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-09-13T16:34:07.473 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-09-13T16:34:07.474 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.474 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.474 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.474 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-13T16:34:07.474 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.474 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.474 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.474 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-09-13T16:34:07.474 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.474 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.474 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.474 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.474 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-09-13T16:34:07.474 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-09-13T16:34:07.475 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.475 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.475 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.475 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-13T16:34:07.475 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.475 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.475 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.475 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-09-13T16:34:07.475 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.475 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.475 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.475 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.475 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-09-13T16:34:07.476 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-09-13T16:34:07.476 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.476 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.476 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.476 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-13T16:34:07.476 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.476 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.476 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.476 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-09-13T16:34:07.476 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.476 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.476 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.476 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.477 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-09-13T16:34:07.477 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-09-13T16:34:07.477 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.477 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.477 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.477 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-13T16:34:07.477 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.477 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.477 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.477 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-09-13T16:34:07.477 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.477 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.477 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.477 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.478 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-09-13T16:34:07.478 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-09-13T16:34:07.478 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.478 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.478 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.478 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-13T16:34:07.478 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.478 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.478 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.478 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-09-13T16:34:07.478 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.478 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.478 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.479 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.479 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-09-13T16:34:07.479 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-09-13T16:34:07.479 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.479 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.479 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.479 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-13T16:34:07.479 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.479 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.479 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.479 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-09-13T16:34:07.479 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.479 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.479 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.480 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.480 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-09-13T16:34:07.480 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-09-13T16:34:07.480 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.480 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.480 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.480 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-13T16:34:07.480 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.480 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.480 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.480 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-13T16:34:07.480 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.481 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.481 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.481 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.481 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-09-13T16:34:07.481 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-09-13T16:34:07.481 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.481 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.481 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.481 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-09-13T16:34:07.481 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.481 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.481 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.481 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-09-13T16:34:07.481 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.482 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.482 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.482 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.482 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-09-13T16:34:07.482 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-09-13T16:34:07.482 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.482 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.482 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.482 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-13T16:34:07.482 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.482 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.482 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.482 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-13T16:34:07.482 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.482 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.483 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.483 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.483 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-09-13T16:34:07.483 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-09-13T16:34:07.483 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.483 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.483 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.483 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-13T16:34:07.483 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.483 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.483 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.483 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-09-13T16:34:07.483 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.484 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.484 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.484 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.484 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-09-13T16:34:07.484 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-09-13T16:34:07.484 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.484 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.484 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.484 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-13T16:34:07.484 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.484 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.484 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.484 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-09-13T16:34:07.485 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.485 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.485 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.485 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.485 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-09-13T16:34:07.485 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-09-13T16:34:07.485 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.485 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.485 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.485 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.485 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.485 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.486 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.486 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-09-13T16:34:07.486 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.486 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.486 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.486 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.486 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-09-13T16:34:07.486 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-09-13T16:34:07.486 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:07.486 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.486 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.486 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-13T16:34:07.486 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.487 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.487 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.487 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-13T16:34:07.487 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.487 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.487 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.487 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.487 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-09-13T16:34:07.487 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-09-13T16:34:07.487 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.487 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.487 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.487 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-09-13T16:34:07.487 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.488 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.488 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.488 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-13T16:34:07.488 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.488 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.488 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.488 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.488 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-09-13T16:34:07.488 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-09-13T16:34:07.489 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.489 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.489 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.489 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-09-13T16:34:07.489 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.489 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.489 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.489 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-13T16:34:07.489 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.489 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.489 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.489 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.490 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-09-13T16:34:07.490 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-09-13T16:34:07.490 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:07.490 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.490 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.490 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-09-13T16:34:07.490 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.490 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.490 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.490 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-09-13T16:34:07.490 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.490 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.490 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.491 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.491 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-09-13T16:34:07.491 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-09-13T16:34:07.491 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.491 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.491 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.491 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.491 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.491 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.491 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.491 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-13T16:34:07.491 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.491 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.492 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.492 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.492 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-09-13T16:34:07.492 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-09-13T16:34:07.492 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:07.492 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.492 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.492 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-13T16:34:07.492 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.492 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.492 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.492 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-09-13T16:34:07.492 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.493 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.493 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.493 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.493 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-09-13T16:34:07.493 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-09-13T16:34:07.493 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:07.493 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.493 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.493 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-13T16:34:07.493 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.493 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.493 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.493 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-09-13T16:34:07.493 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.494 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.494 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.494 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.494 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-09-13T16:34:07.494 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-09-13T16:34:07.494 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.494 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.494 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.494 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.494 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.494 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.494 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.494 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-13T16:34:07.495 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.495 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.495 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.495 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.495 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-09-13T16:34:07.495 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-09-13T16:34:07.495 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.495 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.495 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.495 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.495 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.495 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.495 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.495 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-13T16:34:07.496 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.496 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.496 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.496 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.496 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.496 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.496 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.496 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.496 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.496 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.496 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.496 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.496 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.496 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.497 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.497 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.497 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.497 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.497 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.497 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.497 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.497 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.497 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.497 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.497 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.497 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-09-13T16:34:07.497 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-09-13T16:34:07.498 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.498 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.498 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.498 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.498 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.498 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.498 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.498 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-13T16:34:07.498 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.498 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.498 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.498 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.498 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.498 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.499 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.499 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.499 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.499 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.499 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.499 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.499 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.499 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-13T16:34:07.499 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.499 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.499 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.499 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.499 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.500 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.500 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.500 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.500 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.500 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.500 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.500 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.500 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.500 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.500 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.500 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.500 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.500 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.500 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.501 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.501 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.501 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.501 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.501 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.501 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.501 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.501 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.501 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.501 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.501 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.501 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.502 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.502 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.502 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.502 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.502 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.502 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.502 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.502 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.502 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-13T16:34:07.502 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-13T16:34:07.502 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.502 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.502 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.502 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.503 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.503 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.503 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.503 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-13T16:34:07.503 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.503 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.503 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.503 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.503 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-13T16:34:07.503 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-13T16:34:07.503 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.503 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.503 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.503 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-09-13T16:34:07.504 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.504 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.504 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.504 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-13T16:34:07.504 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.504 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.504 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.504 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.504 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-09-13T16:34:07.505 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-09-13T16:34:07.505 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.505 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.505 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.505 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-13T16:34:07.505 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.505 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.505 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.505 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-09-13T16:34:07.505 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.505 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.505 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.505 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.506 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-09-13T16:34:07.506 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-09-13T16:34:07.506 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.506 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.506 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.506 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-13T16:34:07.506 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.506 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.506 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.506 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-13T16:34:07.506 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.506 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.506 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.506 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.506 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-09-13T16:34:07.507 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-09-13T16:34:07.507 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.507 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.507 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.507 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.507 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.507 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.507 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.507 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-09-13T16:34:07.507 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.507 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.507 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.507 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.507 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-13T16:34:07.508 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-13T16:34:07.508 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.508 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.508 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.508 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-09-13T16:34:07.508 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.508 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.508 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.508 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-09-13T16:34:07.508 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-09-13T16:34:07.508 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.508 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-09-13T16:34:07.508 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.508 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.509 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.509 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.509 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-09-13T16:34:07.509 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-09-13T16:34:07.509 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.509 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.509 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.509 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-09-13T16:34:07.509 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.509 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.509 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.509 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-13T16:34:07.509 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.509 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.510 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.510 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.510 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-09-13T16:34:07.510 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-09-13T16:34:07.510 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.510 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.510 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.510 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-13T16:34:07.510 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.510 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.510 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.510 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-13T16:34:07.510 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.510 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.511 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.511 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.511 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-09-13T16:34:07.511 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-09-13T16:34:07.511 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.511 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.511 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.511 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.511 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.511 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.511 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.511 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-13T16:34:07.512 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.512 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.512 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.512 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.512 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-09-13T16:34:07.512 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-09-13T16:34:07.512 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.512 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.512 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.512 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.512 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.512 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.512 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.512 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-13T16:34:07.513 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.513 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.513 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.513 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.513 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-09-13T16:34:07.513 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-09-13T16:34:07.513 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.513 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.513 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.513 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.513 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.513 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.513 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.514 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-13T16:34:07.514 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.514 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.514 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.514 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.514 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-09-13T16:34:07.514 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-09-13T16:34:07.514 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.514 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.514 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.514 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.514 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.514 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.515 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.515 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-13T16:34:07.515 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.515 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.515 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.515 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.515 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-09-13T16:34:07.515 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-09-13T16:34:07.515 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.515 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.515 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.515 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.515 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.516 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.516 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.516 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-13T16:34:07.516 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.516 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.516 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.516 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.516 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-09-13T16:34:07.516 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-09-13T16:34:07.516 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.516 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.516 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.516 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-09-13T16:34:07.517 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.517 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.517 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.517 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-09-13T16:34:07.517 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.517 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.517 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.517 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.517 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-09-13T16:34:07.517 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-09-13T16:34:07.517 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.517 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.518 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.518 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.518 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.518 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.518 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.518 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-13T16:34:07.518 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.518 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.518 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.518 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.518 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-09-13T16:34:07.518 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-09-13T16:34:07.518 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.518 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.519 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.519 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.519 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.519 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.519 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.519 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-13T16:34:07.519 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.519 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.519 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.519 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.519 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-09-13T16:34:07.519 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-09-13T16:34:07.519 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.520 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.520 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.520 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.520 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.520 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.520 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.520 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-13T16:34:07.520 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.520 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.520 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.520 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.520 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-09-13T16:34:07.520 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-09-13T16:34:07.521 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.521 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.521 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.521 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.521 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.521 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.521 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.521 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-09-13T16:34:07.521 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.521 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.521 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.521 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.521 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-09-13T16:34:07.521 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-09-13T16:34:07.522 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.522 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.522 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.522 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.522 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.522 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.522 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.522 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-13T16:34:07.522 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.522 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.522 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.522 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.522 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-09-13T16:34:07.522 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-09-13T16:34:07.522 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.523 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.523 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.523 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.523 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.523 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.523 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.523 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-13T16:34:07.523 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.523 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.523 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.523 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.523 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.523 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.523 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.523 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-09-13T16:34:07.524 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.524 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.524 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.524 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.524 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.524 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.524 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.524 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.524 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.524 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.524 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.524 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.524 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.525 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.525 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.525 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.525 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.525 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.525 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.525 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.525 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.525 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.525 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.525 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.525 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.525 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.525 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.528 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.528 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.528 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.528 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.528 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.528 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.529 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.529 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.529 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.529 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.529 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.529 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.529 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.529 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.529 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.529 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.529 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.529 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.529 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.530 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.530 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.530 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.530 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.530 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.530 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.530 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.530 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.530 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.530 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.530 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.530 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.531 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.531 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.531 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.531 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.531 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.531 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.531 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.531 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.531 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.531 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.531 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.531 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.531 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.532 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.532 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.532 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-09-13T16:34:07.532 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-09-13T16:34:07.532 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:07.532 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.532 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.532 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-09-13T16:34:07.532 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.532 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.532 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.532 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-13T16:34:07.532 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.532 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.533 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.533 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.533 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-09-13T16:34:07.533 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-09-13T16:34:07.533 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:07.533 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.533 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.533 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-09-13T16:34:07.533 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.533 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.533 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.533 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-13T16:34:07.534 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.534 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.534 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.534 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.534 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.534 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.534 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.534 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-09-13T16:34:07.534 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.534 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.534 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.534 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-13T16:34:07.534 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-13T16:34:07.534 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.535 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.535 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.535 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-13T16:34:07.535 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.535 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.535 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.535 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.535 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.535 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.535 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.535 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.536 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-09-13T16:34:07.536 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-13T16:34:07.536 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.536 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.536 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.536 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.536 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.536 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.536 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.536 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.536 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.536 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.537 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.537 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.537 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-13T16:34:07.537 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-13T16:34:07.537 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.537 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.537 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.537 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.537 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.537 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.537 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.537 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.537 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.538 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.538 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.538 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.538 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-09-13T16:34:07.538 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-09-13T16:34:07.538 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.538 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.538 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.538 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.538 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.538 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.538 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.538 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.539 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.539 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.539 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.539 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.539 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-13T16:34:07.539 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-13T16:34:07.539 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.539 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.539 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.539 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.539 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.539 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.539 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.539 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.540 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.540 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.540 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.540 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.540 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-09-13T16:34:07.540 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-13T16:34:07.540 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.540 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.540 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.540 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.540 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.540 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.540 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.541 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.541 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.541 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.541 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.541 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.541 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-13T16:34:07.541 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-13T16:34:07.541 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.541 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.541 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.541 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.541 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.541 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.542 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.542 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.542 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.542 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.542 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.542 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.542 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-13T16:34:07.542 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-13T16:34:07.542 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.542 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.542 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.542 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.542 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.542 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.543 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.543 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.543 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.543 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.543 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.543 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.543 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-13T16:34:07.543 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-13T16:34:07.543 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.543 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.543 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.543 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.544 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.544 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.544 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.544 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.544 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.544 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.544 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.544 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.544 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-13T16:34:07.544 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-13T16:34:07.544 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.545 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.545 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.545 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.545 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.545 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.545 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.545 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.545 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.545 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.545 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.545 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.545 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-13T16:34:07.546 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-13T16:34:07.546 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.546 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.546 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.546 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.546 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.546 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.546 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.546 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.547 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.547 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.547 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.547 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.547 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-13T16:34:07.547 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-13T16:34:07.547 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.547 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.547 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.547 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.547 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.547 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.548 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.548 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.548 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.548 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.548 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.548 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.548 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-13T16:34:07.548 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-13T16:34:07.548 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.548 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.548 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.548 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.548 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.549 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.549 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.549 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.549 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.549 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.549 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.549 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.549 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-13T16:34:07.549 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-13T16:34:07.549 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.549 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.549 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.549 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.550 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.550 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.550 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.550 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.550 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.550 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.550 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.550 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.550 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-09-13T16:34:07.550 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-13T16:34:07.550 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.550 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.550 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.551 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-13T16:34:07.551 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.551 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.551 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.551 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.551 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.551 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.551 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.551 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.551 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-13T16:34:07.551 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-13T16:34:07.551 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.551 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.551 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.552 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.552 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.552 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.552 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.552 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.552 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.552 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.552 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.552 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.552 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-09-13T16:34:07.552 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-09-13T16:34:07.552 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.553 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.553 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.553 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.553 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.553 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.553 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.553 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.553 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.553 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.553 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.553 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.553 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-09-13T16:34:07.553 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-13T16:34:07.553 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.554 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.554 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.554 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-13T16:34:07.554 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.554 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.554 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.554 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.554 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.554 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.554 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.554 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.554 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-13T16:34:07.554 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-13T16:34:07.555 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.555 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.555 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.555 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.555 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.555 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.555 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.555 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.555 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.555 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.555 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.555 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.555 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-13T16:34:07.556 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-13T16:34:07.556 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.556 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.556 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.556 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.556 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.556 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.556 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.556 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.556 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.556 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.556 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.556 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.557 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-13T16:34:07.557 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-13T16:34:07.557 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.557 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.557 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.557 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.557 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.557 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.557 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.557 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.557 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.557 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.557 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.558 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.558 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-13T16:34:07.558 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-13T16:34:07.558 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.558 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.558 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.558 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.558 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.558 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.558 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.558 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.558 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.558 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.559 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.559 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.559 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-09-13T16:34:07.559 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-13T16:34:07.559 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.559 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.559 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.559 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.559 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.559 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.559 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.559 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.559 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.560 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.560 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.560 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.560 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-13T16:34:07.560 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-13T16:34:07.560 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.560 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.560 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.560 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.560 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.560 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.560 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.560 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.561 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.561 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.561 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.561 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.561 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-13T16:34:07.561 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-13T16:34:07.561 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.561 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.561 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.561 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.562 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.562 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.562 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.562 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.562 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.562 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.562 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.562 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.562 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-13T16:34:07.562 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-13T16:34:07.562 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.562 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.562 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.562 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.563 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.563 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.563 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.563 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.563 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.563 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.563 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.563 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.563 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-13T16:34:07.563 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-13T16:34:07.563 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.563 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.563 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.563 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.563 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.564 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.564 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.564 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.564 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.564 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.564 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.564 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.564 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-13T16:34:07.564 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-13T16:34:07.564 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.564 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.564 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.564 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.564 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.565 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.565 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.565 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.565 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.565 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.565 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.565 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.565 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-13T16:34:07.565 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-13T16:34:07.565 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.565 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.565 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.565 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.566 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.566 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.566 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.566 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.566 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.566 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.566 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.566 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.566 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-13T16:34:07.566 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-13T16:34:07.566 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.566 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.566 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.566 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.567 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.567 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.567 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.567 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.567 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.567 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.567 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.567 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.567 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-13T16:34:07.567 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-13T16:34:07.567 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.567 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.567 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.567 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.568 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.568 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.568 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.568 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.568 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.568 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.568 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.568 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.568 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-09-13T16:34:07.568 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-13T16:34:07.568 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.568 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.568 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.569 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.569 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.569 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.569 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.569 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.569 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.569 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.569 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.569 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.570 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-13T16:34:07.570 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-13T16:34:07.570 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.570 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.570 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.570 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-13T16:34:07.570 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.570 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.570 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.570 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.570 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.570 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.570 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.570 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.571 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-13T16:34:07.571 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-13T16:34:07.571 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.571 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.571 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.571 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-13T16:34:07.571 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.571 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.571 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.571 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.571 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.571 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.571 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.571 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.572 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-13T16:34:07.572 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-13T16:34:07.572 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.572 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.572 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.572 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-09-13T16:34:07.572 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.572 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.572 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.572 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.572 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.572 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.572 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.572 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.573 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-13T16:34:07.573 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-13T16:34:07.573 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.573 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.573 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.573 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-09-13T16:34:07.573 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.573 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.573 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.573 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.573 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.573 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.573 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.574 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.574 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-09-13T16:34:07.574 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-13T16:34:07.574 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.574 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.574 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.574 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.574 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.574 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.574 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.574 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.574 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.574 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.574 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.575 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.575 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-13T16:34:07.575 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-13T16:34:07.575 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.575 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.575 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.575 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-13T16:34:07.575 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.575 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.575 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.575 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.575 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.575 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.575 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.575 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.576 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-09-13T16:34:07.576 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-13T16:34:07.576 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.576 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.576 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.576 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.576 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.576 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.576 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.576 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.576 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.576 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.576 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.576 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.577 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-09-13T16:34:07.577 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-13T16:34:07.577 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.577 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.577 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.577 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.577 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.577 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.577 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.577 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.577 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.577 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.577 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.577 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.578 INFO:teuthology.orchestra.run.smithi016.stdout: "UNSAFE_TLS_v1_2": { 2024-09-13T16:34:07.578 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-13T16:34:07.578 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.578 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.578 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.578 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.578 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.578 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.578 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.578 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.578 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.578 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.578 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.578 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.579 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-13T16:34:07.579 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-13T16:34:07.579 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.579 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.579 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.579 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-13T16:34:07.579 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.579 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.579 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.579 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.579 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.579 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.579 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.579 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.579 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-13T16:34:07.580 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-13T16:34:07.580 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.580 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.580 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.580 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-13T16:34:07.580 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.580 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.580 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.580 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.580 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.580 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.580 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.580 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.580 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-13T16:34:07.581 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-13T16:34:07.581 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.581 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.581 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.581 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-13T16:34:07.581 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.581 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.581 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.581 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.581 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.581 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.581 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.581 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.581 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-09-13T16:34:07.582 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-09-13T16:34:07.582 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.582 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.582 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.582 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.582 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.582 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.582 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.582 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.582 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.582 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.582 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.582 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.582 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-09-13T16:34:07.582 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-09-13T16:34:07.583 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.583 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.583 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.583 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.583 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.583 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.583 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.583 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.583 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.583 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.583 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.583 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.583 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-09-13T16:34:07.583 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-09-13T16:34:07.584 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.584 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.584 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.584 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.584 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.584 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.584 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.584 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-09-13T16:34:07.584 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.584 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.584 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.584 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.584 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-09-13T16:34:07.584 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-09-13T16:34:07.585 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.585 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.585 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.585 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-09-13T16:34:07.585 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.585 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.585 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.585 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.585 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.585 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.585 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.585 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.585 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-13T16:34:07.585 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-13T16:34:07.585 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.586 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.586 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.586 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.586 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.586 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.586 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.586 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.586 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.586 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.586 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.586 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.586 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.586 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.587 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.587 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.587 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.587 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.587 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.587 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.587 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.587 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.587 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.587 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.587 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.587 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.587 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.587 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.587 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.588 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.588 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.588 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.588 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.588 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.588 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.588 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.588 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.588 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.588 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.588 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.588 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.588 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.588 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.589 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.589 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.589 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.589 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.589 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.589 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.589 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.589 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.589 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.589 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.589 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.589 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.589 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.589 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.590 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.590 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.590 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.590 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.590 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.590 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.590 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.590 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.590 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.590 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.590 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.590 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.590 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.590 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.590 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.591 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.591 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.591 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.591 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.591 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.591 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.591 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.591 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.591 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.591 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.591 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-09-13T16:34:07.591 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-09-13T16:34:07.591 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.591 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.592 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.592 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.592 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.592 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.592 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.592 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-09-13T16:34:07.592 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.592 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.592 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.592 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.592 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-09-13T16:34:07.592 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-09-13T16:34:07.592 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.592 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.593 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.593 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.593 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.593 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.593 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.593 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.593 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.593 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.593 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.593 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.593 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-13T16:34:07.593 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-13T16:34:07.593 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.593 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.594 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.594 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-13T16:34:07.594 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.594 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.594 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.594 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.594 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.594 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.594 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.594 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.594 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-13T16:34:07.594 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-13T16:34:07.594 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.594 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.594 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.595 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-09-13T16:34:07.595 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.595 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.595 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.595 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.595 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.595 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.595 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.595 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.595 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-13T16:34:07.595 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-13T16:34:07.595 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.595 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.595 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.595 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.596 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.596 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.596 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.596 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.596 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.596 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.596 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.596 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.596 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-09-13T16:34:07.596 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-09-13T16:34:07.596 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.596 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.596 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.596 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-09-13T16:34:07.597 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.597 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.597 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.597 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.597 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.597 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.597 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.597 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.597 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-13T16:34:07.597 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-13T16:34:07.597 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.597 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.597 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.597 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-09-13T16:34:07.598 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.598 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.598 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.598 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.598 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-09-13T16:34:07.598 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.598 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.598 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.598 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.598 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.598 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.598 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-13T16:34:07.598 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-13T16:34:07.598 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.598 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.599 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.599 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-13T16:34:07.599 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-13T16:34:07.599 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-13T16:34:07.599 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.599 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.599 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.599 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.599 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.599 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.599 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-09-13T16:34:07.599 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-09-13T16:34:07.599 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.600 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.600 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.600 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.600 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.600 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.600 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.600 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.600 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.600 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.600 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.600 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.600 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.600 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.601 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.601 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-09-13T16:34:07.601 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.601 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.601 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.601 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-09-13T16:34:07.601 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-09-13T16:34:07.601 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.601 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.601 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.601 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.601 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.601 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.601 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.602 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-09-13T16:34:07.602 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.602 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.602 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.602 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.602 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.602 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.602 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.602 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.602 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.602 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.602 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.602 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.602 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.603 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.603 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.603 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.603 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.603 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.603 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.603 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.603 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.603 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.603 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.603 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.603 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.603 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.603 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.604 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.604 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.604 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.604 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.604 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.604 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.604 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.604 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.604 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.604 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.604 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.604 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.604 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.605 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.605 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.605 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.605 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.605 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.605 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.605 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.605 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.605 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.605 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.605 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.605 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.605 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.606 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.606 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.606 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.606 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.606 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.606 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.606 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.606 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.606 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.606 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.606 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.606 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.606 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.606 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.607 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.607 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.607 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.607 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.607 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.607 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.607 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.607 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-09-13T16:34:07.607 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-09-13T16:34:07.607 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:07.607 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.607 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.607 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-09-13T16:34:07.607 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.608 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.608 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.608 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-13T16:34:07.608 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.608 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.608 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.608 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.608 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-09-13T16:34:07.608 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-09-13T16:34:07.608 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.608 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.608 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.608 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-09-13T16:34:07.609 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.609 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.609 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.609 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-13T16:34:07.609 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.609 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.609 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.609 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.609 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-09-13T16:34:07.609 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-09-13T16:34:07.609 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:07.609 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.609 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.609 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-09-13T16:34:07.610 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.610 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.610 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.610 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-09-13T16:34:07.610 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.610 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.610 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.610 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.610 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-09-13T16:34:07.610 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-09-13T16:34:07.610 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:07.610 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.610 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.610 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-13T16:34:07.611 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.611 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.611 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.611 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-13T16:34:07.611 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.611 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.611 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.611 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.611 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-09-13T16:34:07.611 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-09-13T16:34:07.611 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.611 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.611 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.611 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.611 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.612 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.612 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.612 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-13T16:34:07.612 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.612 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.612 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.612 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.612 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-13T16:34:07.612 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-13T16:34:07.612 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:07.612 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.612 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.612 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-13T16:34:07.612 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.613 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.613 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.613 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-09-13T16:34:07.613 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.613 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.613 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.613 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.613 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-09-13T16:34:07.613 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-09-13T16:34:07.613 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:07.613 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.613 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.613 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-09-13T16:34:07.614 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.614 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.614 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.614 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-13T16:34:07.614 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.614 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.614 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.614 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.614 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.614 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.614 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.614 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-09-13T16:34:07.614 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.614 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.615 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.615 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.615 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.615 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.615 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.615 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.615 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.615 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.615 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.615 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.615 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.615 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.615 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.615 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.616 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.616 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.616 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.616 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.616 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.616 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.616 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.616 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.616 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.616 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.616 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.616 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.616 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.616 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.616 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.617 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.617 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.617 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.617 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.617 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.617 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.617 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.617 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.617 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.617 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.617 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.617 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.617 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.618 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.618 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.618 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.618 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.618 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.618 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.618 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.618 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.618 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.618 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.618 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.618 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.618 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.618 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.618 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.619 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.619 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.619 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.619 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.619 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.619 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.619 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.619 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.619 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.619 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.619 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.619 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.619 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.619 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.620 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-09-13T16:34:07.620 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-09-13T16:34:07.620 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.620 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.620 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.620 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-13T16:34:07.620 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.620 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.620 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.620 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.620 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.620 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.620 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.620 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.621 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-09-13T16:34:07.621 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-09-13T16:34:07.621 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.621 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.621 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.621 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-09-13T16:34:07.621 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.621 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.621 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.621 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.621 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.621 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.621 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.621 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.621 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-13T16:34:07.622 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-13T16:34:07.622 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.622 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.622 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.622 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-13T16:34:07.622 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.622 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.622 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.622 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.622 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.622 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.622 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.622 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.622 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.623 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.623 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.623 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-09-13T16:34:07.623 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-09-13T16:34:07.623 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-09-13T16:34:07.623 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.623 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-09-13T16:34:07.623 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-09-13T16:34:07.623 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.623 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.623 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.623 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-09-13T16:34:07.623 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.623 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.623 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.624 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-13T16:34:07.624 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.624 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.624 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.624 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.624 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-09-13T16:34:07.624 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-09-13T16:34:07.624 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.624 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.624 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.624 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-09-13T16:34:07.624 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.624 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.624 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.624 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-13T16:34:07.625 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.625 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.625 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.625 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.625 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-09-13T16:34:07.625 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-09-13T16:34:07.625 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.625 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.625 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.625 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.625 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.625 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.625 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.625 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-09-13T16:34:07.626 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.626 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.626 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.626 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.626 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-13T16:34:07.626 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-13T16:34:07.626 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:07.626 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.626 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.626 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-09-13T16:34:07.626 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-09-13T16:34:07.626 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.626 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.627 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-13T16:34:07.627 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.627 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.627 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.627 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.627 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.627 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.627 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.627 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.627 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.627 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.627 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.627 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.627 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.627 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.628 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.628 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.628 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.628 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.628 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.628 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.628 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.628 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.628 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.628 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.628 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.628 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.628 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.628 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.629 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.629 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.629 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.629 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.629 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.629 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.629 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.629 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.629 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.629 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.629 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.629 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.629 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.629 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.630 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.630 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.630 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.630 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.630 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.630 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.630 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.630 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.630 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.630 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.630 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.630 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.630 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.630 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.631 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.631 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.631 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.631 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.631 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.631 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.631 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.631 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.631 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.631 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.631 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.631 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.631 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.631 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.631 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.632 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.632 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.632 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.632 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-09-13T16:34:07.632 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-09-13T16:34:07.632 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.632 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.632 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.632 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.632 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.632 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.632 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.632 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-09-13T16:34:07.632 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.633 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.633 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.633 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.633 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-09-13T16:34:07.633 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-09-13T16:34:07.633 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.633 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.633 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.633 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-09-13T16:34:07.633 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.633 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.633 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.633 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-09-13T16:34:07.633 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.634 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.634 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.634 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.634 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-13T16:34:07.634 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-13T16:34:07.634 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.634 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.634 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.634 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-09-13T16:34:07.634 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.634 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.634 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.634 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-13T16:34:07.634 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.634 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.635 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.635 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.635 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-09-13T16:34:07.635 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-09-13T16:34:07.635 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.635 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.635 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.635 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-13T16:34:07.635 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-13T16:34:07.635 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-09-13T16:34:07.635 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.635 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-13T16:34:07.635 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.635 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.636 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.636 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.636 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-09-13T16:34:07.636 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-09-13T16:34:07.636 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.636 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.636 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.636 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.636 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.636 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.636 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.636 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-09-13T16:34:07.636 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.636 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.637 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.637 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.637 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-09-13T16:34:07.637 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-09-13T16:34:07.637 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.637 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.637 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.637 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-09-13T16:34:07.637 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.637 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.637 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.637 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-13T16:34:07.637 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.637 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.637 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.638 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.638 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.638 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.638 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.638 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-09-13T16:34:07.638 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.638 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.638 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.638 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.638 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.638 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.638 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.638 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.638 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.638 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.639 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.639 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.639 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.639 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.639 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.639 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.639 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.639 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.639 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.639 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.639 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.639 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.639 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.639 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.640 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.640 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.640 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.640 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.640 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.640 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.640 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.640 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.640 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.640 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.640 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.640 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.640 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.641 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.641 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.641 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.641 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.641 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.641 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.641 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.641 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.641 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.641 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.641 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.641 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.641 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.641 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.642 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.642 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.642 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.642 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.642 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.642 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.642 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.642 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.642 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.642 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.642 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.642 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.642 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.642 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.642 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.643 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.643 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.643 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.643 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.643 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.643 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.643 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.643 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.643 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.643 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.643 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-09-13T16:34:07.643 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.643 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.643 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.644 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.644 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.644 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.644 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.644 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.644 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.644 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.644 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.644 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.644 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.644 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.644 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.644 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.644 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.645 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.645 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.645 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.645 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.645 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.645 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.645 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.645 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.645 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.645 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.645 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.645 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.645 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.645 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.645 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.646 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.646 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.646 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.646 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.646 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.646 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.646 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.646 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.646 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.646 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.646 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.646 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.646 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.646 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.647 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.647 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.647 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.647 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.647 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.647 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.647 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.647 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.647 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.647 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.647 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.647 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.647 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.647 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.648 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.648 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.648 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.648 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.648 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.648 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.648 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.648 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.648 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.648 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.648 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.648 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.648 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.648 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.648 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.649 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.649 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-09-13T16:34:07.649 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.649 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.649 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.649 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-09-13T16:34:07.649 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-09-13T16:34:07.649 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.649 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.649 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.649 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-09-13T16:34:07.649 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.649 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.649 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.650 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-13T16:34:07.650 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.650 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.650 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.650 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.650 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-09-13T16:34:07.650 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-09-13T16:34:07.650 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.650 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.650 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.650 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-13T16:34:07.650 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-09-13T16:34:07.650 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.650 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.650 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-13T16:34:07.651 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.651 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.651 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.651 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.651 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.651 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.651 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.651 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.651 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.651 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.651 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.651 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.651 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.651 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.651 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.652 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.652 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.652 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.652 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.652 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.652 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.652 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.652 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.652 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.652 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.652 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.652 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.652 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.652 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.653 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.653 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.653 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.653 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.653 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.653 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.653 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.653 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.653 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.653 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.653 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.653 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.653 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.654 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.654 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.654 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.654 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.654 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.654 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.654 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.654 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.654 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.654 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.654 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.654 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.654 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.654 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.655 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.655 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.655 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.655 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.655 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.655 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.655 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.655 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.655 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.655 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.655 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.655 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.655 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.655 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.656 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.656 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.656 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.656 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.656 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.656 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.656 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.656 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-09-13T16:34:07.656 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.656 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.656 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.656 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-09-13T16:34:07.656 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-09-13T16:34:07.656 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.656 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.657 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.657 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-09-13T16:34:07.657 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.657 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.657 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.657 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-09-13T16:34:07.657 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-13T16:34:07.657 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.657 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.657 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.657 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.657 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.657 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.657 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.658 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.658 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.658 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.658 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.658 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.658 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.658 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.658 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.658 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.658 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.658 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.658 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.658 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.658 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.659 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.659 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.659 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.659 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.659 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.659 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.659 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.659 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.659 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.659 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.659 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.659 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.659 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.659 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.659 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.660 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.660 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.660 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.660 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.660 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.660 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.660 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.660 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.660 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.660 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.660 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.660 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.660 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.660 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.661 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.661 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.661 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.661 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.661 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.661 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.661 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.661 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.661 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.661 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.661 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.661 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.661 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.661 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.661 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.662 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.662 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.662 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.662 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.662 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.662 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.662 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.662 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.662 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-09-13T16:34:07.662 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-09-13T16:34:07.662 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.662 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.662 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.662 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.663 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.663 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.663 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.663 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-09-13T16:34:07.663 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-13T16:34:07.663 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.663 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.663 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.663 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-09-13T16:34:07.663 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-09-13T16:34:07.663 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.663 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.663 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.663 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-09-13T16:34:07.663 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.664 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.664 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.664 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-09-13T16:34:07.664 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.664 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.664 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.664 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.664 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-09-13T16:34:07.664 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-09-13T16:34:07.664 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.664 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.664 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.664 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-09-13T16:34:07.664 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.665 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.665 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.665 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-09-13T16:34:07.665 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.665 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.665 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.665 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.665 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-09-13T16:34:07.665 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-09-13T16:34:07.665 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.665 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.665 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.665 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.665 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.666 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.666 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.666 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-09-13T16:34:07.666 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.666 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.666 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.666 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.666 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-09-13T16:34:07.666 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-09-13T16:34:07.666 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.666 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.666 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.667 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-09-13T16:34:07.667 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.667 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.667 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.667 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-13T16:34:07.667 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-13T16:34:07.667 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.667 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.667 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.667 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.667 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.667 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.667 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-09-13T16:34:07.667 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.667 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.668 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.668 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.668 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.668 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.668 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.668 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.668 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.668 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.668 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.668 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.668 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.668 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.668 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.668 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.669 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.669 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.669 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.669 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.669 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.669 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.669 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.669 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.669 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.669 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.669 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.669 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.669 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.669 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.670 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.670 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.670 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.670 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.670 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.670 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.670 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.670 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.670 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.670 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.670 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.670 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.670 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.670 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.671 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.671 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.671 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.671 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.671 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.671 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.671 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.671 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.671 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.671 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.671 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.671 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.671 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.671 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.671 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.672 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.672 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.672 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.672 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.672 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.672 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.672 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.672 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.672 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.672 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.672 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.672 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.672 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.672 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.673 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.673 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.673 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.673 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-09-13T16:34:07.673 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.673 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.673 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.673 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.673 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.673 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.673 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.673 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.673 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.673 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.674 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.674 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.674 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.674 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.674 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.674 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.674 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.674 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.674 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.674 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.674 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.674 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.674 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.674 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.674 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.675 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.675 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.675 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.675 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.675 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.675 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.675 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.675 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.675 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.675 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.675 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.675 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.675 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.675 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.675 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.676 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.676 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.676 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.676 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.676 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.676 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.676 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.676 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.676 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.676 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.676 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.676 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.676 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.676 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.677 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.677 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.677 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.677 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.677 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.677 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.677 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.677 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.677 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.677 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.677 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.677 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.677 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.677 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.677 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.678 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.678 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.678 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.678 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.678 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.678 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.678 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.678 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-09-13T16:34:07.678 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.678 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.678 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.678 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.678 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.678 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.679 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.679 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.679 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.679 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.679 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.679 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.679 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.679 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.679 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.679 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.679 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.679 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.679 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.679 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.680 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.680 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.680 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.680 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.680 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.680 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.680 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.680 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.680 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.680 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.680 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.681 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.681 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.681 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.681 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.681 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.681 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.681 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.681 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.681 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.681 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.681 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.681 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.681 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.681 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.682 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.682 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.682 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.682 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.682 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.682 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.682 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.682 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.682 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.682 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.682 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.682 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.682 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.682 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.683 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.683 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.683 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.683 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.683 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.683 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.683 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.683 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.683 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.683 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.684 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.684 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.684 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.684 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.684 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-13T16:34:07.684 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.684 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.684 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.684 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-09-13T16:34:07.684 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-09-13T16:34:07.684 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.685 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.685 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.685 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.685 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.685 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.685 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.685 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-13T16:34:07.685 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.685 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.685 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.685 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.685 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.686 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.686 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.686 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.686 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.686 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.686 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.686 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.686 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.686 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.686 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.686 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.686 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.686 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.686 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.687 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.687 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.687 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.687 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.687 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.687 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.687 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.687 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.687 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.687 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.687 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.687 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.687 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.687 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.687 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.687 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.688 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.688 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.688 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.688 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.688 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.688 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.688 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.688 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.688 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.688 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.688 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.688 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.688 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.688 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.689 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.689 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.689 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.689 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.689 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.689 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.689 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.689 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.689 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.689 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.689 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.689 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.689 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.689 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.690 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.690 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.690 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.690 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.690 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.690 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.690 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.690 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.690 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.690 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.690 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-09-13T16:34:07.690 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-13T16:34:07.690 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.690 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.691 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.691 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.691 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.691 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.691 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.691 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-09-13T16:34:07.691 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-09-13T16:34:07.691 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-09-13T16:34:07.691 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.691 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-09-13T16:34:07.691 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.691 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.691 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.691 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.692 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.692 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.692 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.692 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-09-13T16:34:07.692 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.692 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.692 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.692 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.692 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.692 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.692 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.692 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.692 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.693 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.693 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.693 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.693 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.693 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.693 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.693 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.693 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.693 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.693 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.693 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.693 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.693 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.693 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.694 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.694 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.694 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.694 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.694 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.694 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.694 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.694 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.694 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.694 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.694 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.694 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.694 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.694 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.694 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.695 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.695 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.695 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.695 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.695 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.695 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.695 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.695 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.695 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.695 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.695 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.695 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.695 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.696 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.696 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.696 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.696 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.696 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.696 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.696 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.696 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.696 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.696 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.696 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.696 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.696 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.697 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.697 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.697 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.697 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.697 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.697 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.697 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.697 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.697 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.697 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.697 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.697 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.697 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-09-13T16:34:07.697 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.698 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.698 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.698 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.698 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.698 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.698 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.698 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.698 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.698 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.698 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.698 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.698 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.698 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.698 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.698 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.699 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.699 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.699 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.699 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.699 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.699 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.699 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.699 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.699 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.699 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.699 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.699 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.699 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.700 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.700 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.700 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.700 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.700 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.700 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.700 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.700 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.700 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.700 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.700 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.700 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.700 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.700 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.700 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.701 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.701 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.701 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.701 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.701 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.701 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.701 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.701 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.701 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.701 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.701 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.701 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.701 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.701 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.702 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.702 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.702 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.702 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.702 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.702 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.702 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.702 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.702 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.702 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.702 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.702 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.702 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.703 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.703 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.703 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.703 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.703 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.703 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-09-13T16:34:07.703 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.703 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.703 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.703 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.703 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.703 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.703 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.703 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.704 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.704 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.704 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.704 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.704 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.704 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.704 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.704 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.704 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.704 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.704 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.704 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.704 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.705 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.705 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.705 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.705 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.705 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.705 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.705 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.705 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.705 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.705 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.705 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.705 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.705 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.705 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.706 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.706 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.706 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.706 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.706 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.706 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.706 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.706 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.706 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.706 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.706 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.706 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.706 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.706 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.707 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.707 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.707 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.707 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.707 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.707 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.707 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.707 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.707 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.707 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.707 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.707 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.707 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.707 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.708 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.708 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.708 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.708 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.708 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.708 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.708 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.708 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.708 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.708 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.708 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-13T16:34:07.708 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-13T16:34:07.708 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:07.708 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.708 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.709 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-13T16:34:07.709 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.709 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.709 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.709 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.709 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.709 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.709 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.709 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.709 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-09-13T16:34:07.709 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-09-13T16:34:07.709 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-13T16:34:07.710 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.710 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.710 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-13T16:34:07.710 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-09-13T16:34:07.710 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.710 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.710 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-09-13T16:34:07.710 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-13T16:34:07.710 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.710 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.710 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.710 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.710 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.711 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.711 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-09-13T16:34:07.711 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.711 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.711 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.711 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-09-13T16:34:07.711 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-09-13T16:34:07.711 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.711 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.711 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.711 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.711 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.711 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.711 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.712 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-13T16:34:07.712 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.712 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.712 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.712 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.712 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-13T16:34:07.712 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-13T16:34:07.712 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.712 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.712 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.712 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.712 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.712 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.713 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.713 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.713 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.713 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.713 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.713 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.713 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.713 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.713 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.713 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.713 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.713 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.713 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.713 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.714 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.714 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.714 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.714 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.714 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.714 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.714 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.714 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.714 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.714 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.714 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.714 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.714 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.714 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.714 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.715 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.715 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.715 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.715 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.715 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.715 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.715 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.715 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.715 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.715 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.715 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.715 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.715 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.715 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.716 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.716 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.716 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.716 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.716 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.716 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.716 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.716 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.716 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.716 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.716 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.716 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.716 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.716 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.717 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.717 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.717 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.717 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.717 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.717 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.717 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.717 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.717 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.717 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.717 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.717 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.717 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.717 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.718 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.718 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.718 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.718 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.718 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.718 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-09-13T16:34:07.718 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-09-13T16:34:07.718 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.718 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.718 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.718 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-09-13T16:34:07.718 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.718 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.718 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.718 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-09-13T16:34:07.719 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.719 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.719 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.719 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.719 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-13T16:34:07.719 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-13T16:34:07.719 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:07.719 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.719 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.719 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-13T16:34:07.719 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.719 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.719 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.720 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-09-13T16:34:07.720 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.720 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.720 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.720 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.720 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.720 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.720 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.720 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-09-13T16:34:07.720 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.720 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.720 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.720 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-09-13T16:34:07.720 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-09-13T16:34:07.721 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.721 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.721 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.721 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.721 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.721 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.721 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.721 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.721 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.721 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.721 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.721 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.721 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-09-13T16:34:07.721 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-09-13T16:34:07.722 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.722 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.722 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.722 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.722 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.722 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.722 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.722 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-13T16:34:07.722 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-13T16:34:07.722 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.722 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.722 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.722 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.723 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.723 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.723 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.723 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.723 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.723 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.723 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.723 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.723 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.723 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.723 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.723 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.723 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.723 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.723 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.723 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.724 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.724 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.724 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.724 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.724 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.724 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.724 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.724 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.724 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.724 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.724 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.724 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.724 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.725 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.725 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.725 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.725 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.725 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.725 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.725 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.725 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.725 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.725 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.725 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.725 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.725 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.726 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.726 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.726 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.726 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.726 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.726 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.726 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.726 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.726 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.726 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.726 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.726 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.726 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.726 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.727 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.727 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.727 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.727 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.727 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.727 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.727 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.727 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.727 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.727 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.727 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.727 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.727 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.727 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-09-13T16:34:07.728 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-09-13T16:34:07.728 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.728 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.728 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.728 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.728 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.728 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.728 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.728 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.728 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.728 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.728 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.728 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.728 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-13T16:34:07.729 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-13T16:34:07.729 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.729 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.729 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.729 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-09-13T16:34:07.729 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.729 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.729 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.729 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.729 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.729 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.729 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.729 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.729 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-09-13T16:34:07.730 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-09-13T16:34:07.730 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-13T16:34:07.732 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.732 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.732 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-09-13T16:34:07.732 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.732 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.732 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.732 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.732 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.732 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.732 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.733 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.733 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-13T16:34:07.733 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-13T16:34:07.733 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.733 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.733 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.733 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-13T16:34:07.733 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.733 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.733 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.733 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-13T16:34:07.733 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.733 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.733 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.734 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.734 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-13T16:34:07.734 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-13T16:34:07.734 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.734 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.734 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.734 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-09-13T16:34:07.734 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.734 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.734 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.734 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-13T16:34:07.734 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.734 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.734 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.735 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.735 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-09-13T16:34:07.735 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-09-13T16:34:07.735 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.735 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.735 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.735 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-09-13T16:34:07.735 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.735 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.735 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.735 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.735 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.735 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.735 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.736 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.736 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-13T16:34:07.736 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-13T16:34:07.736 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.736 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.736 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.736 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-09-13T16:34:07.736 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.736 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.736 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.736 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-09-13T16:34:07.736 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-09-13T16:34:07.736 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.736 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.737 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.737 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.737 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.737 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.737 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-13T16:34:07.737 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-13T16:34:07.737 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.737 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.737 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.737 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-13T16:34:07.737 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-13T16:34:07.737 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-13T16:34:07.737 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.737 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.738 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.738 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.738 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.738 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.738 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.738 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.738 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.738 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-09-13T16:34:07.738 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.738 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.738 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.738 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.739 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.739 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.739 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.739 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.739 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.739 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.739 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.739 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.739 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.739 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.739 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.739 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.739 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.739 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.740 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.740 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.740 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.740 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.740 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.740 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.740 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.740 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.740 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.740 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.740 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.740 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.740 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.740 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.741 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.741 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.741 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.741 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.741 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.741 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.741 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.741 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.741 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.741 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.741 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.741 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.741 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.741 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.742 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.742 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.742 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.742 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.742 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.742 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.742 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.742 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.742 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.742 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.742 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.742 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.742 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.742 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.743 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.743 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.743 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.743 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.743 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.743 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.743 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.743 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.743 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.743 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.743 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.743 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.743 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.743 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-09-13T16:34:07.744 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-09-13T16:34:07.744 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.744 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.744 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.744 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-13T16:34:07.744 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.744 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.744 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.744 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.744 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.744 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.744 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.744 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.744 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-09-13T16:34:07.745 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-09-13T16:34:07.745 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.745 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.745 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.745 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.745 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.745 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.745 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.745 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.745 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.745 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.745 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.745 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.745 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-09-13T16:34:07.746 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-09-13T16:34:07.746 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.746 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.746 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.746 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.746 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.746 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.746 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.746 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.746 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.746 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.746 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.746 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.746 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.747 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.747 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.747 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-09-13T16:34:07.747 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.747 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.747 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.747 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-09-13T16:34:07.747 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-09-13T16:34:07.747 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.747 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.747 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.747 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.747 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.747 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.747 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.748 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.748 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.748 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.748 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.748 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.748 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-13T16:34:07.748 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-13T16:34:07.748 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.748 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.748 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.748 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.748 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.748 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.748 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.749 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.749 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.749 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.749 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.749 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.749 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.749 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.749 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.749 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.749 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.749 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.749 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.749 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.749 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.749 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.750 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.750 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.750 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.750 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.750 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.750 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.750 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.750 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.750 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.750 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.750 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.750 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.750 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.750 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.751 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.751 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.751 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.751 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.751 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.751 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.751 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.751 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.751 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.751 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.751 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.751 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.751 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.751 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.752 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.752 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.752 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.752 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.752 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.752 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.752 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.752 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.752 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.752 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.753 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.753 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.753 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.753 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.753 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.753 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.753 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.753 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.753 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.753 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.753 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.753 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.753 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.753 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.754 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.754 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.754 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.754 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.754 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.754 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.754 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.754 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.754 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-13T16:34:07.754 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-13T16:34:07.754 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.754 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.754 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.754 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.755 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.755 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.755 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.755 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.755 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.755 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.755 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.755 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.755 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-13T16:34:07.755 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-13T16:34:07.755 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.755 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.755 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.756 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.756 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.756 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.756 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.756 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.756 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.756 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.756 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.756 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.756 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.756 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.756 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.756 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-09-13T16:34:07.757 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.757 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.757 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.757 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.757 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.757 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.757 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.757 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.757 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.757 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.757 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.757 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.757 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.757 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.757 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.758 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.758 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.758 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.758 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.758 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.758 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.758 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.758 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.758 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.758 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.758 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.758 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.758 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.758 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.759 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.759 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.759 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.759 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.759 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.759 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.759 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.759 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.759 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.759 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.759 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.759 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.759 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.759 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.760 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.760 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.760 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.760 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.760 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.760 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.760 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.760 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.760 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.760 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.760 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.760 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.760 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.760 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.761 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.761 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.761 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.761 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.761 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.761 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.761 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.761 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.761 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.761 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.761 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.761 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.761 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.762 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.762 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.762 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.762 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.762 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.762 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.762 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-09-13T16:34:07.762 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.762 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.762 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.762 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.762 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.762 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.762 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.763 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.763 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.763 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.763 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.763 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.763 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.763 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.763 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.763 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.763 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.763 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.763 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.763 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.763 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.763 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.764 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.764 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.764 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.764 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.764 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.764 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.764 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.764 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.764 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.764 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.764 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.764 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.764 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.764 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.765 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.765 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.765 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.765 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.765 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.765 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.765 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.765 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.765 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.765 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.765 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.765 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.765 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.765 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.766 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.766 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.766 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.766 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.766 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.766 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.766 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.766 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.766 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.766 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.766 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.767 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.767 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.767 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.767 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.767 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.767 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.767 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.767 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.767 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.767 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.767 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.767 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.767 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-09-13T16:34:07.767 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-09-13T16:34:07.768 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.768 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.768 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.768 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-09-13T16:34:07.768 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.768 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.768 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.768 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-13T16:34:07.768 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.768 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.768 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.768 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.768 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.769 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.769 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.769 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-09-13T16:34:07.769 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.769 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.769 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.769 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.769 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.769 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.769 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.769 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.769 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.769 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.770 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.770 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.770 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.770 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.770 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.770 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.770 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.770 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.770 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.770 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.770 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.770 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.770 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.770 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.771 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.771 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.771 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.771 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.771 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.771 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.771 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.771 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.771 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.771 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.771 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.771 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.771 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.771 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.772 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.772 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.772 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.772 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.772 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.772 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.772 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.772 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.772 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.772 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.772 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.772 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.772 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.773 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.773 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.773 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.773 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.773 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.773 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.773 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.773 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.773 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.773 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.773 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.773 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.773 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.773 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.774 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.774 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.774 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.774 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.774 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.774 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.774 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.774 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.774 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-09-13T16:34:07.774 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-09-13T16:34:07.774 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.774 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.774 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.774 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-09-13T16:34:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-09-13T16:34:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-09-13T16:34:07.776 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.776 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.776 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.776 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.776 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.776 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.776 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.776 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.776 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-09-13T16:34:07.776 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-09-13T16:34:07.776 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.776 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.776 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.776 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.777 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.777 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.777 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.777 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.777 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.777 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.777 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.777 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.777 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-09-13T16:34:07.777 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-09-13T16:34:07.777 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.777 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.777 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.777 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.778 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.778 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.778 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.778 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.778 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.778 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.778 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.778 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.778 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-09-13T16:34:07.778 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-09-13T16:34:07.778 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-13T16:34:07.778 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.778 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.778 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.779 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.779 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.779 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.779 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.779 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.779 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.779 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.779 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.779 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-09-13T16:34:07.779 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-09-13T16:34:07.780 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.780 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.780 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.780 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.780 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.780 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.780 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.780 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.780 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.780 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.780 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.780 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.780 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-09-13T16:34:07.781 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-09-13T16:34:07.781 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.781 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.781 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.781 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.781 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.781 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.781 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.781 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.781 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.781 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.781 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.781 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.781 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-09-13T16:34:07.782 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-09-13T16:34:07.782 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.782 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.782 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.782 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.782 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.782 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.782 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.782 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.782 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.782 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.782 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.782 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.782 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-09-13T16:34:07.783 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-09-13T16:34:07.783 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.783 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.783 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.783 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.783 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-13T16:34:07.783 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-09-13T16:34:07.783 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.783 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.783 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.783 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.783 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.783 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.783 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-09-13T16:34:07.783 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-09-13T16:34:07.784 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.784 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.784 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.784 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.784 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.784 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.784 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.784 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.784 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.784 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.784 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.784 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.784 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-09-13T16:34:07.784 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-09-13T16:34:07.785 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.785 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.785 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.785 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.785 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.785 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.785 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.785 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.785 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.785 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.785 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.785 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.785 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-09-13T16:34:07.785 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-09-13T16:34:07.786 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.786 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.786 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.786 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.786 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.786 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.786 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.786 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.786 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.786 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.786 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.786 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.786 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.786 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.787 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.787 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-09-13T16:34:07.787 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.787 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.787 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.787 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-09-13T16:34:07.787 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-09-13T16:34:07.787 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.787 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.787 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.787 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.787 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.787 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.787 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.788 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-09-13T16:34:07.788 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.788 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.788 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.788 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.788 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-09-13T16:34:07.788 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-09-13T16:34:07.788 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.788 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.788 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.788 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.788 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.788 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.789 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.789 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-09-13T16:34:07.789 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.789 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.789 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.789 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.789 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.789 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.789 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.789 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.789 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.789 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.789 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.789 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.790 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.790 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.790 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.790 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.790 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.790 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.790 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.790 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.790 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.790 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.790 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.790 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.790 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.790 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.790 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.791 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.791 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.791 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.791 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.791 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.791 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.791 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.791 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.791 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.791 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.791 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.791 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.791 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.791 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.792 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.792 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.792 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.792 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.792 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.792 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.792 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.792 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.792 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.792 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.792 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.792 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.792 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.792 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.793 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.793 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.793 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.793 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.793 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.793 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.793 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.793 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.793 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.793 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.793 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.793 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.793 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.793 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.794 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.794 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.794 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.794 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.794 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.794 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.794 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.794 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.794 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-09-13T16:34:07.794 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.794 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.794 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.794 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.794 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.795 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.795 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.795 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.795 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.795 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.795 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.795 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.795 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.795 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.795 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.795 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.795 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.796 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.796 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.796 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.796 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.796 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.796 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.796 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.796 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.796 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.796 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.796 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.796 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.796 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.796 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.797 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.797 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.797 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.797 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.797 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.797 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.797 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.797 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.797 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.797 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.797 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.797 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.797 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.797 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.798 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.798 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.798 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.798 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.798 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.798 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.798 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.798 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.798 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.798 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.798 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.798 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.798 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.798 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.799 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.799 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.799 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.799 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.799 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.799 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.799 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.799 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.799 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.799 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.799 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.799 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.799 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.799 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.800 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.800 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.800 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.800 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-09-13T16:34:07.800 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.800 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.800 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.800 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.800 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.800 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.800 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.800 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.800 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.800 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.801 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.801 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.801 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.801 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.801 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.801 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.801 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.801 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.801 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.801 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.801 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.801 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.801 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.802 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.802 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.802 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.802 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.802 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.802 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.802 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.802 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.802 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.802 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.802 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.802 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.802 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.802 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.803 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.803 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.803 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.803 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.803 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.803 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.803 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.803 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.803 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.803 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.803 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.803 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.803 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.803 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.804 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.804 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.804 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.804 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.804 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.804 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.804 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.804 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.804 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.804 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.804 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.804 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.804 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.804 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.805 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.805 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.805 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.805 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.805 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.805 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.805 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.805 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.805 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.805 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.805 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.805 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-09-13T16:34:07.805 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.805 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.806 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.806 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-09-13T16:34:07.806 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-09-13T16:34:07.806 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.806 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.806 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.806 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-13T16:34:07.806 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.806 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.806 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.806 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.806 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.806 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.806 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.807 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.807 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-13T16:34:07.807 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-13T16:34:07.807 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:07.807 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.807 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.807 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-09-13T16:34:07.807 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.807 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.807 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.807 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.807 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.807 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.807 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.808 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.808 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.808 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.808 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.808 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.808 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.808 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.808 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.808 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.808 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.808 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.808 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.808 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.808 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.808 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.809 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.809 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.809 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.809 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.809 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.809 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.809 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.809 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.809 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.809 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.809 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.809 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.810 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.811 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.812 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.813 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.813 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.813 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.813 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.813 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.813 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.813 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-09-13T16:34:07.813 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.813 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.813 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.813 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-09-13T16:34:07.813 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-09-13T16:34:07.813 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.813 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.814 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.814 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.814 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.814 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.814 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.814 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-13T16:34:07.814 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.814 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.814 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.814 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.814 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-09-13T16:34:07.814 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-09-13T16:34:07.814 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.815 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.815 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.815 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.815 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.815 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.815 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.815 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-13T16:34:07.815 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.815 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.815 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.815 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.815 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-09-13T16:34:07.815 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-09-13T16:34:07.815 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.816 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.816 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.816 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.816 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.816 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.816 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.816 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-13T16:34:07.816 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.816 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.816 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.816 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.816 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-09-13T16:34:07.816 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-09-13T16:34:07.817 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.817 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.817 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.817 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.817 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.817 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.817 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.817 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-13T16:34:07.817 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.817 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.817 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.817 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.817 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-09-13T16:34:07.817 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-09-13T16:34:07.817 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.818 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.818 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.818 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.818 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.818 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.818 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.818 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-13T16:34:07.818 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.818 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.818 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.818 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.818 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-09-13T16:34:07.818 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-09-13T16:34:07.818 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-09-13T16:34:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-09-13T16:34:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.819 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.820 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.820 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.820 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.820 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.820 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.820 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.820 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.820 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.820 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.820 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.820 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-09-13T16:34:07.820 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-09-13T16:34:07.820 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.820 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.821 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.821 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-13T16:34:07.821 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.821 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.821 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.821 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.821 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.821 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.821 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.821 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.821 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-13T16:34:07.821 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-13T16:34:07.821 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.821 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.822 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.822 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.822 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.822 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.822 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.822 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.822 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.822 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.822 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.822 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.822 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-13T16:34:07.822 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-13T16:34:07.822 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.822 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.823 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.823 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-09-13T16:34:07.823 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-09-13T16:34:07.823 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.823 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.823 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.823 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.823 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.823 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.823 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.823 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-09-13T16:34:07.823 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-09-13T16:34:07.823 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.823 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.823 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.824 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-09-13T16:34:07.824 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.824 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.824 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.824 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.824 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.824 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.824 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.824 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.824 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-09-13T16:34:07.824 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-09-13T16:34:07.824 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.824 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.824 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.825 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.825 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.825 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.825 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.825 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.825 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.825 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.825 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.825 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.825 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-09-13T16:34:07.825 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-09-13T16:34:07.825 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.825 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.826 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.826 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.826 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.826 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.826 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.826 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.826 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.826 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.826 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.826 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.826 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.826 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.826 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.826 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.826 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.826 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.827 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.827 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.827 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.827 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.827 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.827 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.827 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.827 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.827 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.827 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.827 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.827 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.827 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.827 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.827 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.828 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.828 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.828 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.828 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.828 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.828 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.828 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.828 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.828 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.828 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.828 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.828 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.828 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.828 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.829 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.829 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.829 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.829 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.829 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.829 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.829 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.829 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.829 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.829 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.829 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.829 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.829 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.830 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.830 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.830 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.830 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.830 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.830 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.830 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.830 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.830 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.830 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.830 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.830 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.830 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.830 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.831 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.831 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.831 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.831 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.831 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.831 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.831 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.831 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.831 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-09-13T16:34:07.831 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-09-13T16:34:07.831 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.831 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.831 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.832 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.832 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.832 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.832 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.832 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.832 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.832 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.832 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.832 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.832 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-09-13T16:34:07.832 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-09-13T16:34:07.832 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.832 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.832 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.832 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.832 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.833 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.833 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.833 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.833 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.833 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.833 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.833 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.833 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-09-13T16:34:07.833 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-09-13T16:34:07.833 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.833 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.833 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.833 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-13T16:34:07.833 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.834 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.834 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.834 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.834 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.834 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.834 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.834 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.834 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.834 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.834 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.834 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-09-13T16:34:07.834 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.834 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.834 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.835 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.835 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.835 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.835 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.835 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.835 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.835 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.835 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.835 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.835 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.835 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.835 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.835 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.835 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.836 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.836 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.836 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.836 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.836 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.836 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.836 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.836 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.836 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.836 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.836 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.836 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.836 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.837 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.837 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.837 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.837 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.837 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.837 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.837 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.837 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.837 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.837 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.837 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.837 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.837 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.837 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.837 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.838 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.838 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.838 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.838 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.838 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.838 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.838 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.838 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.838 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.838 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.838 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.838 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.838 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.839 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.839 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.839 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.839 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.839 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.839 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.839 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.839 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.839 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.839 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.839 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.839 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.839 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.839 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.840 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.840 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.840 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.840 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.840 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-09-13T16:34:07.840 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.840 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.840 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.840 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.840 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.840 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.840 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.840 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.840 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.841 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.841 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.841 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.841 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.841 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.841 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.841 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.841 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.841 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.841 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.841 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.841 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.841 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.841 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.842 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.842 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.842 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.842 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.842 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.842 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.842 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.842 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.842 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.842 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.842 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.842 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.842 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.842 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.843 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.843 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.843 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.843 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.843 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.843 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.843 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.843 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.843 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.843 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.843 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.843 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.843 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.844 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.844 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.844 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.844 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.844 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.844 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.844 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.844 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.844 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.844 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.844 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.844 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.844 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.844 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.845 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.845 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.845 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.845 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.845 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.845 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.845 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.845 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.845 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.845 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-09-13T16:34:07.845 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-09-13T16:34:07.845 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.845 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.845 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.845 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-09-13T16:34:07.846 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.846 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.846 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.846 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-13T16:34:07.846 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.846 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.846 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.846 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.846 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-09-13T16:34:07.846 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-09-13T16:34:07.846 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.846 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.846 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.846 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-13T16:34:07.846 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.847 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.847 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.847 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-13T16:34:07.847 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.847 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.847 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.847 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.847 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-09-13T16:34:07.847 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-09-13T16:34:07.847 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.847 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.847 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.847 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.847 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.848 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.848 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.848 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-13T16:34:07.848 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.848 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.848 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.848 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.848 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.848 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.848 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.848 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-09-13T16:34:07.848 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.848 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.848 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.848 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-09-13T16:34:07.849 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-09-13T16:34:07.849 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-13T16:34:07.849 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.849 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.849 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-09-13T16:34:07.849 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.849 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.849 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.849 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.849 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.849 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.849 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.849 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.850 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-09-13T16:34:07.850 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-09-13T16:34:07.850 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.850 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.850 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.850 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.850 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.850 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.850 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.850 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.850 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.850 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.850 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.851 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.851 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-13T16:34:07.851 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-13T16:34:07.851 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:07.851 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.851 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.851 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-13T16:34:07.851 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.851 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.851 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.851 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.851 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.851 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.851 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.852 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.852 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.852 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.852 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.852 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.852 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.852 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.852 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.852 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.852 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.852 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.852 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.852 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.852 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.853 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.853 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.853 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.853 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.853 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.853 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.853 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.853 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.853 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.853 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.853 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.853 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.853 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.854 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.854 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.854 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.854 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.854 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.854 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.854 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.854 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.854 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.854 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.854 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.854 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.854 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.854 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.855 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.855 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.855 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.855 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.855 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.855 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.855 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.855 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.855 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.855 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.855 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.855 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.855 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.855 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.856 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.856 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.856 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.856 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.856 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.856 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.856 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.856 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.856 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.856 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.856 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.856 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.856 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.857 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.857 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.857 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.857 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-09-13T16:34:07.857 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-09-13T16:34:07.857 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.857 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.857 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.857 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.857 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.857 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.857 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.857 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.858 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.858 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.858 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.858 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.858 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-09-13T16:34:07.858 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-09-13T16:34:07.858 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.858 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.858 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.858 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-09-13T16:34:07.858 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.858 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.858 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.858 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.859 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.859 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.859 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.859 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.859 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-09-13T16:34:07.859 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-09-13T16:34:07.859 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.859 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.859 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.859 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-13T16:34:07.859 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.859 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.859 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.859 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.860 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.860 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.860 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.860 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.860 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.860 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.860 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-13T16:34:07.860 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.860 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.860 INFO:teuthology.orchestra.run.smithi016.stdout: "gid": 24110, 2024-09-13T16:34:07.860 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "y", 2024-09-13T16:34:07.860 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr_features": 4540138322906710015, 2024-09-13T16:34:07.860 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-09-13T16:34:07.860 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.861 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-09-13T16:34:07.861 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.861 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.861 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.861 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-13T16:34:07.861 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-13T16:34:07.861 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:07.861 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.861 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.861 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-13T16:34:07.861 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.861 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.861 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.861 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-09-13T16:34:07.862 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.862 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.862 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.862 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.862 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.862 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.862 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.862 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.862 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.862 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.862 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.862 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.862 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.862 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.862 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.863 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.863 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.863 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.863 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.863 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.863 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.863 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.863 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.863 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.863 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.863 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.863 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.863 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.863 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.864 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.864 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.864 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.864 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.864 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.864 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.864 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.864 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.864 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.864 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.864 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.864 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.864 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.865 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.865 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.865 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.865 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.865 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.865 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.865 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.865 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.865 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.865 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.865 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.865 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.865 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.866 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.866 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.866 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.866 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.866 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.866 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.866 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.866 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.866 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.866 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.866 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.866 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.866 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.866 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.867 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.867 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.867 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.867 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.867 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.867 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-09-13T16:34:07.867 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-09-13T16:34:07.867 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.867 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.867 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.867 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.867 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.867 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.867 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.867 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to", 2024-09-13T16:34:07.868 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.868 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.868 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.868 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.868 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-09-13T16:34:07.868 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-09-13T16:34:07.868 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.868 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.868 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.868 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-09-13T16:34:07.868 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.868 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.868 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.868 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-09-13T16:34:07.869 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.869 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.869 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.869 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.869 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-09-13T16:34:07.869 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-09-13T16:34:07.869 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.869 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.869 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.869 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.869 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.869 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.869 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.870 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-09-13T16:34:07.870 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.870 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.870 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.870 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.870 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-09-13T16:34:07.870 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-09-13T16:34:07.870 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.870 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.870 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.870 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.870 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.870 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.870 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.871 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-09-13T16:34:07.871 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.871 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.871 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.871 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.871 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-09-13T16:34:07.871 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-09-13T16:34:07.871 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.871 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.871 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.871 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-09-13T16:34:07.871 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.871 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.871 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.872 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-09-13T16:34:07.872 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.872 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.872 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.872 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.872 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-09-13T16:34:07.872 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-09-13T16:34:07.872 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.872 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.872 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.872 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.872 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.872 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.872 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.873 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-09-13T16:34:07.873 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.873 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.873 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.873 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.873 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-09-13T16:34:07.873 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-09-13T16:34:07.873 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.873 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.873 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.873 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.873 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.873 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.873 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.874 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-13T16:34:07.874 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.874 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.874 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.874 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.874 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-09-13T16:34:07.874 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-09-13T16:34:07.874 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.874 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.874 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.874 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.874 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.874 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.874 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.874 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-09-13T16:34:07.875 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.875 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.875 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.875 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.875 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.875 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.875 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.875 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-09-13T16:34:07.875 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.875 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.875 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.875 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-09-13T16:34:07.875 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-09-13T16:34:07.875 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.876 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.876 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.876 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.876 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.876 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.876 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.876 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-09-13T16:34:07.876 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.876 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.876 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.876 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.876 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-09-13T16:34:07.876 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-09-13T16:34:07.876 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.877 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.877 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.877 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-09-13T16:34:07.877 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.877 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.877 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.877 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-09-13T16:34:07.877 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-13T16:34:07.877 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.877 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.877 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.877 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-09-13T16:34:07.877 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-09-13T16:34:07.877 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-13T16:34:07.878 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.878 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.878 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-13T16:34:07.878 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-13T16:34:07.878 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-13T16:34:07.878 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.878 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-13T16:34:07.878 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-13T16:34:07.878 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.878 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.878 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.878 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-09-13T16:34:07.878 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-09-13T16:34:07.879 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-13T16:34:07.879 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.879 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.879 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-09-13T16:34:07.879 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-13T16:34:07.879 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-09-13T16:34:07.879 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.879 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-13T16:34:07.879 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.879 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.879 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.879 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.879 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-09-13T16:34:07.879 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-09-13T16:34:07.880 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.880 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.880 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.880 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-09-13T16:34:07.880 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.880 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.880 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.880 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-13T16:34:07.880 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-13T16:34:07.880 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.880 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.880 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.880 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-09-13T16:34:07.880 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-09-13T16:34:07.881 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-13T16:34:07.881 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.881 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.881 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-09-13T16:34:07.881 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-09-13T16:34:07.881 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-09-13T16:34:07.881 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.881 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-09-13T16:34:07.881 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-13T16:34:07.881 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.881 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.881 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.881 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-09-13T16:34:07.881 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-09-13T16:34:07.882 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.882 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.882 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.882 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-09-13T16:34:07.882 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.882 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.882 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.882 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-09-13T16:34:07.882 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-13T16:34:07.882 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.882 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.882 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.882 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-09-13T16:34:07.882 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-09-13T16:34:07.882 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-13T16:34:07.883 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.883 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.883 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-13T16:34:07.883 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-13T16:34:07.883 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-13T16:34:07.883 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.883 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-13T16:34:07.883 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-13T16:34:07.883 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.883 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.883 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.883 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.883 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.884 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.884 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.884 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.884 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.884 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.884 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.884 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.884 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.884 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.884 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.884 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.884 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.884 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.884 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.885 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.885 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.885 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.885 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.885 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.885 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.885 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.885 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.885 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.885 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.885 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.885 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.885 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.886 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.886 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.886 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.886 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.886 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.886 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.886 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.886 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.886 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.886 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.886 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.886 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.886 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.887 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.887 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.887 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.887 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.887 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.887 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.887 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.887 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.887 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.887 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.887 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.887 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.887 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.887 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.887 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.887 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.888 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.888 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.888 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.888 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.888 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.888 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.888 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.888 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.888 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.888 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.888 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.888 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.888 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-09-13T16:34:07.888 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-09-13T16:34:07.889 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-13T16:34:07.889 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.889 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.889 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-13T16:34:07.889 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.889 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.889 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.889 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-13T16:34:07.889 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.889 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.889 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.889 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.889 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-13T16:34:07.889 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-13T16:34:07.890 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.890 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.890 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.890 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-09-13T16:34:07.890 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.890 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.890 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.890 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-09-13T16:34:07.890 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-09-13T16:34:07.890 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap" 2024-09-13T16:34:07.890 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.890 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-09-13T16:34:07.891 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.891 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.891 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.891 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.891 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-09-13T16:34:07.891 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-09-13T16:34:07.891 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.891 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.891 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.891 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.891 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.891 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.891 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.891 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-13T16:34:07.892 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.892 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.892 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.892 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.892 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-13T16:34:07.892 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-13T16:34:07.892 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:07.892 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.892 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.892 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-13T16:34:07.892 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.892 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.892 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.892 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-13T16:34:07.893 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.893 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.893 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.893 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.893 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-09-13T16:34:07.893 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-09-13T16:34:07.893 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.893 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.893 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.893 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-13T16:34:07.893 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-13T16:34:07.893 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.893 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.894 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-13T16:34:07.894 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-13T16:34:07.894 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.894 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.894 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.894 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-09-13T16:34:07.894 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-09-13T16:34:07.894 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-13T16:34:07.894 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.894 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.894 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-13T16:34:07.894 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.894 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.894 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.895 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-13T16:34:07.895 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.895 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.895 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.895 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.895 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.895 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.895 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.895 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-09-13T16:34:07.895 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.895 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.895 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.895 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-09-13T16:34:07.895 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-09-13T16:34:07.896 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-13T16:34:07.896 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.896 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.896 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-13T16:34:07.896 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.896 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.896 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.896 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-13T16:34:07.896 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.896 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.896 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.896 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.896 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-09-13T16:34:07.897 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-09-13T16:34:07.897 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:07.897 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.897 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.897 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-09-13T16:34:07.897 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.897 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.897 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.897 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-13T16:34:07.897 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.897 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.897 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.897 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.897 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-09-13T16:34:07.898 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-09-13T16:34:07.898 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.898 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.898 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.898 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-09-13T16:34:07.898 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.898 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.898 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.898 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-13T16:34:07.898 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.898 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.898 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.898 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.898 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-09-13T16:34:07.899 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-09-13T16:34:07.899 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.899 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.899 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.899 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.899 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.899 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.899 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.899 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-13T16:34:07.899 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-13T16:34:07.899 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.899 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.899 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.899 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-09-13T16:34:07.899 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-09-13T16:34:07.900 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:07.900 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.900 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.900 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-13T16:34:07.900 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.900 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.900 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.900 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-13T16:34:07.900 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.900 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.900 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.900 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.900 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-09-13T16:34:07.900 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-09-13T16:34:07.901 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-13T16:34:07.901 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.901 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.901 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-09-13T16:34:07.901 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.901 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.901 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.901 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-13T16:34:07.901 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.901 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.901 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.901 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.901 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-09-13T16:34:07.901 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-09-13T16:34:07.902 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.902 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.902 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.902 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.902 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.902 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.902 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.902 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-13T16:34:07.902 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.902 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.902 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.902 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.902 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-09-13T16:34:07.902 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-09-13T16:34:07.903 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.903 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.903 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.903 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.903 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.903 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.903 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.903 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-13T16:34:07.903 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.903 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.903 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.903 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.903 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-09-13T16:34:07.904 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-09-13T16:34:07.904 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.904 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.904 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.904 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.904 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.904 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.904 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.904 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-13T16:34:07.904 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.904 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.904 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.904 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.904 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-09-13T16:34:07.905 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-09-13T16:34:07.905 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.905 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.905 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.905 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-13T16:34:07.905 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.905 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.905 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.905 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-13T16:34:07.905 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.905 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.905 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.905 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.906 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-09-13T16:34:07.906 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-09-13T16:34:07.906 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.906 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.906 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.906 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-13T16:34:07.906 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.906 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.906 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.906 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-09-13T16:34:07.906 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.906 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.906 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.906 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.906 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-09-13T16:34:07.907 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-09-13T16:34:07.907 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.907 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.907 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.907 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-13T16:34:07.907 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.907 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.907 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.907 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-09-13T16:34:07.907 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.907 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.907 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.907 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.907 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-09-13T16:34:07.908 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-09-13T16:34:07.908 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.908 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.908 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.908 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-09-13T16:34:07.908 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.908 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.908 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.908 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-13T16:34:07.908 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.908 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.908 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.909 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.909 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-09-13T16:34:07.909 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-09-13T16:34:07.909 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.909 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.909 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.909 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-13T16:34:07.909 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.909 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.909 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.909 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-09-13T16:34:07.909 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.909 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.909 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.910 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.910 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-09-13T16:34:07.910 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-09-13T16:34:07.910 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.910 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.910 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.910 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-13T16:34:07.910 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.910 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.910 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.910 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-09-13T16:34:07.910 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.910 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.910 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.911 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.911 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-09-13T16:34:07.911 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-09-13T16:34:07.911 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.911 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.911 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.911 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-13T16:34:07.911 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.911 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.911 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.911 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-09-13T16:34:07.911 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.911 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.911 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.912 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.912 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-09-13T16:34:07.912 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-09-13T16:34:07.912 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.912 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.912 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.912 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-13T16:34:07.912 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.912 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.912 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.912 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-09-13T16:34:07.912 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.912 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.912 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.913 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.913 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-09-13T16:34:07.913 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-09-13T16:34:07.913 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.913 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.913 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.913 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-13T16:34:07.913 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.913 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.913 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.913 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-09-13T16:34:07.913 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.913 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.914 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.914 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.914 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-09-13T16:34:07.914 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-09-13T16:34:07.914 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.914 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.914 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.914 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-13T16:34:07.914 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.914 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.914 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.914 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-09-13T16:34:07.914 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.914 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.915 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.915 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.915 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-09-13T16:34:07.915 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-09-13T16:34:07.915 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.915 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.915 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.915 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-13T16:34:07.915 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.915 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.915 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.915 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-13T16:34:07.915 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.916 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.916 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.916 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.916 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-09-13T16:34:07.916 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-09-13T16:34:07.916 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.916 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.916 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.916 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-09-13T16:34:07.916 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.916 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.916 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.916 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-09-13T16:34:07.916 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.917 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.917 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.917 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.917 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-09-13T16:34:07.917 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-09-13T16:34:07.917 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.917 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.917 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.917 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-13T16:34:07.917 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.917 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.917 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.917 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-13T16:34:07.917 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.918 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.918 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.918 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.918 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-09-13T16:34:07.918 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-09-13T16:34:07.918 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.918 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.918 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.918 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-13T16:34:07.918 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.918 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.918 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.918 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-09-13T16:34:07.918 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.919 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.919 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.919 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.919 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-09-13T16:34:07.919 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-09-13T16:34:07.919 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.919 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.919 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.919 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-13T16:34:07.919 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.919 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.919 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.919 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-09-13T16:34:07.919 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.920 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.920 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.920 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.920 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-09-13T16:34:07.920 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-09-13T16:34:07.920 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.920 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.920 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.920 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.920 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.920 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.920 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.920 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-09-13T16:34:07.921 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.921 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.921 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.921 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.921 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-09-13T16:34:07.921 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-09-13T16:34:07.921 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:07.921 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.921 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.921 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-13T16:34:07.921 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.921 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.921 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.921 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-13T16:34:07.922 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.922 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.922 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.922 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.922 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-09-13T16:34:07.922 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-09-13T16:34:07.922 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.922 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.922 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.922 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-09-13T16:34:07.922 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.922 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.922 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.922 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-13T16:34:07.923 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.923 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.923 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.923 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.923 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-09-13T16:34:07.923 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-09-13T16:34:07.923 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.923 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.923 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.923 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-09-13T16:34:07.923 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.923 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.923 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.923 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-13T16:34:07.924 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.924 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.924 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.924 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.924 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-09-13T16:34:07.924 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-09-13T16:34:07.924 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:07.924 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.924 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.924 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-09-13T16:34:07.924 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.924 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.924 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.924 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-09-13T16:34:07.925 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.925 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.925 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.925 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.925 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-09-13T16:34:07.925 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-09-13T16:34:07.925 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.925 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.925 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.925 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.925 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.925 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.925 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.925 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-13T16:34:07.926 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.926 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.926 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.926 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.926 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-09-13T16:34:07.926 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-09-13T16:34:07.926 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:07.926 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.926 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.926 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-13T16:34:07.926 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.926 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.926 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.926 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-09-13T16:34:07.926 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.927 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.927 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.927 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.927 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-09-13T16:34:07.927 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-09-13T16:34:07.927 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:07.927 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.927 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.927 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-13T16:34:07.927 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.927 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.927 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.927 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-09-13T16:34:07.927 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.928 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.928 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.928 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.928 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-09-13T16:34:07.928 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-09-13T16:34:07.928 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.928 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.928 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.928 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.928 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.928 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.928 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.928 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-13T16:34:07.928 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.929 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.929 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.929 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.929 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-09-13T16:34:07.929 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-09-13T16:34:07.929 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.929 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.929 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.929 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.929 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.929 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.929 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.929 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-13T16:34:07.929 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.929 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.930 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.930 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.930 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.930 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.930 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.930 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.930 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.930 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.930 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.930 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.930 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.930 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.930 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.930 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.931 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.931 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.931 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.931 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.931 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.931 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.931 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.931 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.931 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.931 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-09-13T16:34:07.931 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-09-13T16:34:07.931 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.931 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.931 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.931 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.932 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.932 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.932 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.932 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-13T16:34:07.932 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.932 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.932 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.932 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.932 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.932 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.932 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.932 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.932 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.932 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.933 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.933 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.933 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.933 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-13T16:34:07.933 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.933 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.933 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.933 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.933 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.933 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.933 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.933 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.933 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.933 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.934 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.934 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.934 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.934 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.934 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.934 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.934 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.934 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.934 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.934 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.934 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.934 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.934 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.934 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.934 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.935 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.935 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.935 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.935 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.935 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.935 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.935 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.935 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.935 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.935 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.935 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.935 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.935 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.935 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.936 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-13T16:34:07.936 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-13T16:34:07.936 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.936 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.936 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.936 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.936 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.936 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.936 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.936 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-13T16:34:07.936 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.936 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.936 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.936 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.936 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-13T16:34:07.937 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-13T16:34:07.937 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.937 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.937 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.937 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-09-13T16:34:07.937 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.937 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.940 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.940 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-13T16:34:07.940 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.940 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.940 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.940 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.940 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-09-13T16:34:07.940 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-09-13T16:34:07.940 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.941 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.941 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.941 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-13T16:34:07.941 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.941 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.941 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.941 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-09-13T16:34:07.941 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.942 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.942 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.942 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.942 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-09-13T16:34:07.942 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-09-13T16:34:07.942 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.942 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.942 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.942 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-13T16:34:07.942 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.942 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.942 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.942 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-13T16:34:07.942 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.943 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.943 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.943 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.943 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-09-13T16:34:07.943 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-09-13T16:34:07.943 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.943 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.943 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.943 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.943 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.943 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.943 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.943 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-09-13T16:34:07.944 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.944 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.944 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.944 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.944 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-13T16:34:07.944 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-13T16:34:07.944 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.944 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.944 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.944 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-09-13T16:34:07.944 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.944 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.944 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.944 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-09-13T16:34:07.945 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-09-13T16:34:07.945 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.945 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-09-13T16:34:07.945 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.945 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.945 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.945 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.945 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-09-13T16:34:07.945 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-09-13T16:34:07.945 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.945 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.945 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.945 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-09-13T16:34:07.945 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.946 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.946 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.946 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-13T16:34:07.946 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.946 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.946 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.946 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.946 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-09-13T16:34:07.946 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-09-13T16:34:07.946 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.946 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.946 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.946 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-13T16:34:07.947 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.947 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.947 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.947 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-13T16:34:07.947 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.947 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.947 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.947 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.947 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-09-13T16:34:07.947 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-09-13T16:34:07.947 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.947 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.947 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.947 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.948 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.948 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.948 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.948 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-13T16:34:07.948 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.948 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.948 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.948 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.948 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-09-13T16:34:07.948 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-09-13T16:34:07.948 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.948 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.948 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.948 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.949 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.949 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.949 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.949 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-13T16:34:07.949 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.949 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.949 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.949 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.949 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-09-13T16:34:07.949 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-09-13T16:34:07.949 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.949 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.949 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.950 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.950 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.950 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.950 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.950 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-13T16:34:07.950 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.950 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.950 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.950 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.950 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-09-13T16:34:07.950 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-09-13T16:34:07.950 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.950 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.950 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.950 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.951 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.951 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.951 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.951 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-13T16:34:07.951 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.951 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.951 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.951 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.951 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-09-13T16:34:07.951 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-09-13T16:34:07.951 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.951 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.951 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.951 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.952 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.952 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.952 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.952 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-13T16:34:07.952 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.952 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.952 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.952 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.952 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-09-13T16:34:07.952 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-09-13T16:34:07.952 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.952 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.952 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.952 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-09-13T16:34:07.952 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.953 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.953 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.953 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-09-13T16:34:07.953 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.953 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.953 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.953 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.953 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-09-13T16:34:07.953 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-09-13T16:34:07.953 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.953 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.953 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.953 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.953 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.954 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.954 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.954 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-13T16:34:07.954 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.954 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.954 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.954 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.954 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-09-13T16:34:07.954 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-09-13T16:34:07.954 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.954 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.954 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.954 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.954 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.955 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.955 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.955 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-13T16:34:07.955 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.955 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.955 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.955 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.955 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-09-13T16:34:07.955 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-09-13T16:34:07.955 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.955 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.955 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.955 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.955 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.956 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.956 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.956 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-13T16:34:07.956 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.956 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.956 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.956 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.956 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-09-13T16:34:07.956 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-09-13T16:34:07.956 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.956 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.956 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.957 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.957 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.957 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.957 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.957 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-09-13T16:34:07.957 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.957 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.957 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.957 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.957 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-09-13T16:34:07.957 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-09-13T16:34:07.957 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.958 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.958 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.958 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.958 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.958 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.958 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.958 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-13T16:34:07.958 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.958 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.958 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.958 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.958 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-09-13T16:34:07.958 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-09-13T16:34:07.958 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.959 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.959 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.959 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.959 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.959 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.959 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.959 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-13T16:34:07.959 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.959 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.959 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.959 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.959 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.959 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.959 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.960 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-09-13T16:34:07.960 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.960 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.960 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.960 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:07.960 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:07.960 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.960 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.960 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.960 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.960 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.960 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.960 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.960 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.961 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.961 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.961 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.961 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.961 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.961 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.961 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.961 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.961 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.961 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.961 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.961 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:07.961 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:07.961 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.962 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.962 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.962 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.962 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.962 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.962 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.962 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.962 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.962 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.962 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.962 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.962 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:07.962 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:07.963 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.963 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.963 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.963 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:07.963 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.963 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.963 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:07.963 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:07.963 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:07.963 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:07.963 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:07.963 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:07.963 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:07.963 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:07.963 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.964 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.964 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.964 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.964 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.964 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:07.964 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:07.964 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.964 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.964 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.964 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.964 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.964 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.964 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.964 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.965 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.965 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.965 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.965 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.965 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-09-13T16:34:07.965 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-09-13T16:34:07.965 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:07.965 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.965 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.965 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-09-13T16:34:07.965 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.965 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.965 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.965 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-13T16:34:07.966 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.966 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.966 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.966 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.966 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-09-13T16:34:07.966 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-09-13T16:34:07.966 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:07.966 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.966 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:07.966 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-09-13T16:34:07.966 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.966 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.966 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.966 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-13T16:34:07.967 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.967 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.967 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.967 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.967 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:07.967 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.967 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:07.967 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-09-13T16:34:07.967 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:07.967 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:07.967 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:07.967 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-13T16:34:07.967 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-13T16:34:07.967 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:07.967 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.968 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.968 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-13T16:34:07.968 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.968 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.968 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.968 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.968 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.968 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.968 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.968 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.968 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-09-13T16:34:07.968 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-13T16:34:07.968 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.969 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.969 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.969 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.969 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.969 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.969 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.969 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.969 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.969 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.969 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.969 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.969 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-13T16:34:07.969 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-13T16:34:07.969 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.970 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.970 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.970 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.970 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.970 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.970 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.970 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.970 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.970 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.970 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.970 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.970 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-09-13T16:34:07.970 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-09-13T16:34:07.970 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.971 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.971 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.971 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.971 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.971 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.971 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.971 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.971 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.971 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.971 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.971 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.971 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-13T16:34:07.971 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-13T16:34:07.971 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.972 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.972 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.972 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.972 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.972 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.972 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.972 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.972 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.972 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.972 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.972 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.972 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-09-13T16:34:07.972 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-13T16:34:07.973 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.973 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.973 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.973 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.973 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.973 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.973 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.973 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.973 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.973 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.973 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.973 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.973 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-13T16:34:07.973 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-13T16:34:07.974 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.974 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.974 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.974 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.974 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.974 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.974 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.974 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.974 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.974 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.974 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.974 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.974 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-13T16:34:07.974 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-13T16:34:07.975 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.975 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.975 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.975 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.975 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.975 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.975 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.975 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.975 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.975 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.975 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.975 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.975 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-13T16:34:07.975 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-13T16:34:07.976 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.976 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.976 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.976 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.976 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.976 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.976 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.976 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.976 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.976 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.976 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.976 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.976 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-13T16:34:07.976 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-13T16:34:07.977 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.977 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.977 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.977 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.977 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.977 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.977 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.977 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.977 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.977 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.977 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.977 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.977 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-13T16:34:07.977 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-13T16:34:07.978 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.978 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.978 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.978 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.978 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.978 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.978 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.978 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.978 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.978 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.978 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.978 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.978 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-13T16:34:07.978 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-13T16:34:07.979 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.979 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.979 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.979 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.979 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.979 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.979 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.979 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.979 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.979 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.979 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.979 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.979 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-13T16:34:07.979 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-13T16:34:07.979 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.980 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.980 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.980 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.980 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.980 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.980 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.980 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.980 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.980 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.980 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.980 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.980 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-13T16:34:07.980 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-13T16:34:07.980 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.981 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.981 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.981 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.981 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.981 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.981 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.981 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.981 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.981 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.981 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.981 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.981 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-09-13T16:34:07.981 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-13T16:34:07.982 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.982 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.982 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.982 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-13T16:34:07.982 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.982 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.982 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.982 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.982 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.982 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.982 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.982 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.982 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-13T16:34:07.982 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-13T16:34:07.982 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.983 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.983 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.983 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.983 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.983 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.983 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.983 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.983 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.983 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.983 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.983 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.983 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-09-13T16:34:07.983 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-09-13T16:34:07.983 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.983 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.984 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.984 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.984 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.984 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.984 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.984 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.984 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.984 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.984 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.984 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.984 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-09-13T16:34:07.984 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-13T16:34:07.984 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.984 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.985 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.985 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-13T16:34:07.985 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.985 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.985 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.985 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.985 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.985 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.985 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.985 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.985 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-13T16:34:07.985 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-13T16:34:07.985 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.985 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.986 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.986 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.986 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.986 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.986 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.986 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.986 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.986 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.986 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.986 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.986 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-13T16:34:07.986 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-13T16:34:07.986 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.986 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.987 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.987 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.987 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.987 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.987 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.987 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.987 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.987 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.987 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.987 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.987 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-13T16:34:07.987 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-13T16:34:07.987 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.987 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.988 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.988 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.988 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.988 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.988 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.988 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.988 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.988 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.988 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.988 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.988 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-13T16:34:07.988 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-13T16:34:07.988 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.988 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.989 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.989 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.989 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.989 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.989 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.989 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.989 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.989 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.989 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.989 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.989 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-09-13T16:34:07.989 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-13T16:34:07.989 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.989 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.989 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.990 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:07.990 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.990 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.990 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.990 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.990 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.990 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.990 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.990 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.990 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-13T16:34:07.990 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-13T16:34:07.990 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.990 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.990 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.991 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.991 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.991 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.991 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.991 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.991 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.991 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.991 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.991 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.991 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-13T16:34:07.991 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-13T16:34:07.991 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.991 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.992 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.992 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.992 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.992 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.992 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.992 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.992 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.992 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.992 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.992 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.992 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-13T16:34:07.992 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-13T16:34:07.992 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.992 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.993 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.993 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.993 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.993 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.993 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.993 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.993 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.993 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.993 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.993 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.993 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-13T16:34:07.993 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-13T16:34:07.993 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.993 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.993 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.994 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.994 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.994 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.994 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.994 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.994 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.994 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.994 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.994 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.994 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-13T16:34:07.994 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-13T16:34:07.994 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.994 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.994 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.995 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.995 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.995 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.995 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.995 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.995 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.995 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.995 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.995 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.995 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-13T16:34:07.995 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-13T16:34:07.995 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.995 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.995 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.996 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.996 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.996 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.996 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.996 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.996 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.996 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.996 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.996 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.996 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-13T16:34:07.996 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-13T16:34:07.996 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.996 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.997 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.997 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.997 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.997 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.997 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.997 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.997 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.997 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.997 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.997 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.997 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-13T16:34:07.997 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-13T16:34:07.997 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.997 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.997 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.998 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:07.998 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.998 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.998 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.998 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.998 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.998 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.998 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.998 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.998 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-09-13T16:34:07.998 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-13T16:34:07.998 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:07.998 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.999 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:07.999 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:07.999 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:07.999 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:07.999 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:07.999 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:07.999 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:07.999 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:07.999 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:07.999 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:07.999 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-13T16:34:07.999 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-13T16:34:07.999 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:07.999 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:07.999 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.000 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-13T16:34:08.000 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.000 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.000 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.000 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.000 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.000 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.000 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.000 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.000 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-13T16:34:08.000 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-13T16:34:08.000 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.000 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.001 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.001 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-13T16:34:08.001 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.001 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.001 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.001 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.001 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.001 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.001 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.001 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.001 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-13T16:34:08.001 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-13T16:34:08.001 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.001 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.002 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.002 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-09-13T16:34:08.002 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.002 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.002 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.002 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.002 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.002 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.002 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.002 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.002 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-13T16:34:08.002 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-13T16:34:08.002 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.002 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.002 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.003 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-09-13T16:34:08.003 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.003 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.003 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.003 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.003 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.003 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.003 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.003 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.003 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-09-13T16:34:08.003 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-13T16:34:08.003 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.003 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.003 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.003 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.004 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.004 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.004 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.004 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.004 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.004 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.004 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.004 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.004 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-13T16:34:08.004 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-13T16:34:08.004 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.004 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.004 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.004 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-13T16:34:08.005 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.005 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.005 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.005 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.005 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.005 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.005 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.005 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.005 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-09-13T16:34:08.005 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-13T16:34:08.005 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.005 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.005 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.005 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.005 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.006 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.006 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.006 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.006 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.006 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.006 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.006 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.006 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-09-13T16:34:08.006 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-13T16:34:08.006 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.006 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.006 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.006 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.006 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.006 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.007 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.007 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.007 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.007 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.007 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.007 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.007 INFO:teuthology.orchestra.run.smithi016.stdout: "UNSAFE_TLS_v1_2": { 2024-09-13T16:34:08.007 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-13T16:34:08.007 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.007 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.007 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.007 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.007 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.007 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.008 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.008 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.008 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.008 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.008 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.008 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.008 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-13T16:34:08.008 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-13T16:34:08.008 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.008 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.008 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.008 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-13T16:34:08.008 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.008 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.009 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.009 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.009 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.009 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.009 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.009 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.009 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-13T16:34:08.009 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-13T16:34:08.009 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.009 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.009 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.009 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-13T16:34:08.009 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.009 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.010 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.010 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.010 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.010 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.010 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.010 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.010 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-13T16:34:08.010 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-13T16:34:08.010 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.010 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.010 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.010 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-13T16:34:08.010 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.010 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.011 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.011 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.011 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.011 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.011 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.011 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.011 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-09-13T16:34:08.011 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-09-13T16:34:08.011 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.011 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.011 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.011 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.011 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.011 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.011 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.012 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.012 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.012 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.012 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.012 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.012 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-09-13T16:34:08.012 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-09-13T16:34:08.012 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.012 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.012 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.012 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.012 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.012 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.012 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.013 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.013 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.013 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.013 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.013 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.013 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-09-13T16:34:08.013 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-09-13T16:34:08.013 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.013 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.013 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.013 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.013 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.013 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.014 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.014 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-09-13T16:34:08.014 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.014 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.014 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.014 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.014 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-09-13T16:34:08.014 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-09-13T16:34:08.014 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.014 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.014 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.014 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-09-13T16:34:08.014 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.014 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.014 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.015 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.015 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.015 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.015 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.015 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.015 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-13T16:34:08.015 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-13T16:34:08.015 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.015 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.015 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.015 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.015 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.015 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.015 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.016 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.016 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.016 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.016 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.016 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.016 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.016 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.016 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.016 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.016 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.016 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.016 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.016 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.016 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.017 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.017 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.017 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.017 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.017 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.017 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.017 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.017 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.017 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.017 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.017 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.017 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.017 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.017 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.017 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.018 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.018 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.018 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.018 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.018 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.018 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.018 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.018 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.018 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.018 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.018 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.018 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.018 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.018 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.018 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.019 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.019 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.019 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.019 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.019 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.019 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.019 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.019 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.019 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.019 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.019 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.019 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.019 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.019 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.020 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.020 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.020 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.020 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.020 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.020 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.020 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.020 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.020 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.020 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.020 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.020 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.020 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.020 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.021 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.021 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.021 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.021 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-09-13T16:34:08.021 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-09-13T16:34:08.021 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.021 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.021 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.021 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.021 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.021 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.021 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.021 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-09-13T16:34:08.021 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.022 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.022 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.022 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.022 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-09-13T16:34:08.022 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-09-13T16:34:08.022 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.022 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.022 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.022 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.022 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.022 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.022 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.022 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.023 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.023 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.023 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.023 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.023 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-13T16:34:08.023 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-13T16:34:08.023 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.023 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.023 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.023 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-13T16:34:08.023 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.023 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.023 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.023 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.023 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.024 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.024 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.024 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.024 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-13T16:34:08.024 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-13T16:34:08.024 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.024 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.024 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.024 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-09-13T16:34:08.024 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.024 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.024 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.024 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.024 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.025 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.025 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.025 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.025 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-13T16:34:08.025 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-13T16:34:08.025 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.025 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.025 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.025 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.025 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.025 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.025 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.025 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.025 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.026 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.026 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.026 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.026 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-09-13T16:34:08.026 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-09-13T16:34:08.026 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.026 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.026 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.026 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-09-13T16:34:08.026 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.026 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.026 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.026 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.026 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.026 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.027 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.027 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.027 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-13T16:34:08.027 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-13T16:34:08.027 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.027 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.027 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.027 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-09-13T16:34:08.027 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.027 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.027 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.027 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.027 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-09-13T16:34:08.027 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.028 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.028 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.028 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.028 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.028 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.028 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-13T16:34:08.028 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-13T16:34:08.028 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.028 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.028 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.028 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-13T16:34:08.028 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-13T16:34:08.028 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-13T16:34:08.028 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.029 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.029 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.029 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.029 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.029 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.029 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-09-13T16:34:08.029 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-09-13T16:34:08.029 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.029 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.029 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.029 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.029 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.029 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.029 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.029 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.030 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.030 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.030 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.030 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.030 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.030 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.030 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.030 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-09-13T16:34:08.030 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.030 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.030 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.030 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-09-13T16:34:08.030 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-09-13T16:34:08.030 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.030 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.031 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.031 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.031 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.031 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.031 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.031 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-09-13T16:34:08.031 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.031 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.031 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.031 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.031 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.031 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.031 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.031 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.031 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.032 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.032 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.032 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.032 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.032 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.032 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.032 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.032 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.032 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.032 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.032 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.032 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.032 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.032 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.033 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.033 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.033 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.033 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.033 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.033 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.033 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.033 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.033 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.033 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.033 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.033 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.033 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.033 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.033 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.034 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.034 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.034 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.034 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.034 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.034 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.034 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.034 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.034 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.034 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.034 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.034 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.034 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.035 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.035 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.035 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.035 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.035 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.035 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.035 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.035 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.035 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.035 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.035 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.035 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.035 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.035 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.035 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.036 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.036 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.036 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.036 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.036 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.036 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.036 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.036 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.036 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-09-13T16:34:08.036 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-09-13T16:34:08.036 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:08.036 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.036 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.036 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-09-13T16:34:08.037 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.037 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.037 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.037 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-13T16:34:08.037 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.037 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.037 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.037 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.037 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-09-13T16:34:08.037 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-09-13T16:34:08.037 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.037 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.037 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.037 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-09-13T16:34:08.038 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.038 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.038 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.038 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-13T16:34:08.038 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.038 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.038 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.038 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.038 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-09-13T16:34:08.038 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-09-13T16:34:08.038 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:08.038 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.038 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.038 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-09-13T16:34:08.039 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.039 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.039 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.039 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-09-13T16:34:08.039 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.039 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.039 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.039 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.039 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-09-13T16:34:08.039 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-09-13T16:34:08.039 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:08.039 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.039 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.039 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-13T16:34:08.040 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.040 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.040 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.040 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-13T16:34:08.040 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.040 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.040 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.040 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.040 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-09-13T16:34:08.040 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-09-13T16:34:08.040 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.040 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.040 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.040 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.041 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.041 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.041 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.041 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-13T16:34:08.041 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.041 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.041 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.041 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.041 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-13T16:34:08.041 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-13T16:34:08.041 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:08.041 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.041 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.041 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-13T16:34:08.042 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.042 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.042 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.042 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-09-13T16:34:08.042 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.042 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.042 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.042 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.042 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-09-13T16:34:08.042 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-09-13T16:34:08.042 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:08.042 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.042 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.042 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-09-13T16:34:08.042 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.043 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.043 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.043 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-13T16:34:08.043 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.043 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.043 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.043 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.043 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.043 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.043 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.043 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-09-13T16:34:08.043 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.043 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.043 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.044 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.044 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.044 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.044 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.044 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.044 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.044 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.044 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.044 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.044 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.044 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.044 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.044 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.044 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.044 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.045 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.045 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.045 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.045 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.045 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.045 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.045 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.045 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.045 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.045 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.045 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.045 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.045 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.045 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.046 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.046 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.046 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.046 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.046 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.046 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.046 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.046 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.046 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.046 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.046 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.046 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.046 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.046 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.046 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.047 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.047 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.047 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.047 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.047 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.047 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.047 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.047 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.047 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.047 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.047 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.047 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.047 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.047 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.048 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.048 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.048 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.048 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.048 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.048 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.048 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.048 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.048 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.048 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.048 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.048 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.048 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-09-13T16:34:08.048 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-09-13T16:34:08.049 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.049 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.049 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.049 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-13T16:34:08.049 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.049 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.049 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.049 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.049 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.049 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.049 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.049 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.049 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-09-13T16:34:08.049 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-09-13T16:34:08.050 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.050 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.050 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.050 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-09-13T16:34:08.050 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.050 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.050 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.050 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.050 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.050 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.050 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.050 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.050 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-13T16:34:08.050 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-13T16:34:08.051 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.051 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.051 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.051 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-13T16:34:08.051 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.051 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.051 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.051 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.051 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.051 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.051 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.051 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.051 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.051 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.052 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.052 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-09-13T16:34:08.052 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-09-13T16:34:08.052 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-09-13T16:34:08.052 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.052 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-09-13T16:34:08.052 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-09-13T16:34:08.052 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.052 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.052 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.052 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-09-13T16:34:08.052 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.052 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.052 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.053 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-13T16:34:08.053 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.053 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.053 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.053 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.053 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-09-13T16:34:08.053 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-09-13T16:34:08.053 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.053 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.053 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.053 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-09-13T16:34:08.053 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.053 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.053 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.053 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-13T16:34:08.054 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.054 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.054 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.054 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.054 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-09-13T16:34:08.054 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-09-13T16:34:08.054 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.054 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.054 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.054 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.054 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.054 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.054 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.054 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-09-13T16:34:08.055 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.055 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.055 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.055 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.055 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-13T16:34:08.055 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-13T16:34:08.055 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:08.055 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.055 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.055 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-09-13T16:34:08.055 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-09-13T16:34:08.055 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.055 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.055 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-13T16:34:08.055 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.056 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.056 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.056 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.056 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.056 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.056 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.056 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.056 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.056 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.056 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.056 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.056 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.056 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.056 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.057 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.057 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.057 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.057 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.057 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.057 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.057 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.057 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.057 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.057 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.057 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.057 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.057 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.057 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.057 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.058 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.058 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.058 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.058 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.058 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.058 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.058 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.058 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.058 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.058 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.058 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.058 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.058 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.058 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.058 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.059 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.059 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.059 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.059 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.059 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.059 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.059 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.059 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.059 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.059 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.059 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.059 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.059 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.059 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.059 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.060 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.060 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.060 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.060 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.060 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.060 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.060 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.060 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.060 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.060 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.060 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.060 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.060 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.061 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.061 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-09-13T16:34:08.061 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-09-13T16:34:08.061 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.061 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.061 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.061 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.061 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.061 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.061 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.061 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-09-13T16:34:08.061 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.061 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.061 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.062 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.062 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-09-13T16:34:08.062 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-09-13T16:34:08.062 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.062 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.062 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.062 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-09-13T16:34:08.062 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.062 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.062 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.062 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-09-13T16:34:08.062 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.062 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.062 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.062 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.063 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-13T16:34:08.063 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-13T16:34:08.063 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.063 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.063 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.063 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-09-13T16:34:08.063 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.063 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.063 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.063 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-13T16:34:08.063 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.063 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.063 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.063 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.064 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-09-13T16:34:08.064 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-09-13T16:34:08.064 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.064 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.064 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.064 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-13T16:34:08.064 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-13T16:34:08.064 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-09-13T16:34:08.064 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.064 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-13T16:34:08.064 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.064 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.064 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.064 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.065 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-09-13T16:34:08.065 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-09-13T16:34:08.065 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.065 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.065 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.065 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.065 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.065 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.065 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.065 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-09-13T16:34:08.065 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.065 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.065 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.065 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.065 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-09-13T16:34:08.066 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-09-13T16:34:08.066 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.066 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.066 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.066 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-09-13T16:34:08.066 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.066 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.066 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.066 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-13T16:34:08.066 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.066 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.066 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.066 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.066 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.067 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.067 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.067 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-09-13T16:34:08.067 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.067 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.067 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.067 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.067 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.067 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.067 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.067 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.067 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.067 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.067 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.067 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.068 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.068 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.068 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.068 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.068 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.068 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.068 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.068 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.068 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.068 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.068 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.068 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.068 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.068 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.069 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.069 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.069 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.069 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.069 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.069 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.069 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.069 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.069 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.069 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.069 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.069 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.069 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.069 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.069 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.070 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.070 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.070 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.070 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.070 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.070 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.070 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.070 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.070 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.070 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.070 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.070 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.070 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.070 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.070 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.071 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.071 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.071 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.071 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.071 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.071 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.071 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.071 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.071 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.071 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.071 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.071 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.071 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.071 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.072 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.072 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.072 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.072 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.072 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.072 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.072 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-09-13T16:34:08.072 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.072 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.072 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.072 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.072 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.072 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.072 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.073 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.073 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.073 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.073 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.073 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.073 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.073 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.073 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.073 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.073 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.073 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.073 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.074 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.074 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.074 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.074 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.074 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.074 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.074 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.074 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.074 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.074 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.074 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.074 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.074 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.074 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.075 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.075 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.075 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.075 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.075 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.075 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.075 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.075 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.075 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.075 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.075 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.075 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.075 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.075 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.075 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.076 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.076 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.076 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.076 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.076 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.076 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.076 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.076 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.076 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.076 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.076 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.076 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.076 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.076 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.077 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.077 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.077 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.077 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.077 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.077 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.077 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.077 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.077 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.077 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.077 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.077 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.077 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.078 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.078 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-09-13T16:34:08.078 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.078 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.078 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.078 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-09-13T16:34:08.078 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-09-13T16:34:08.078 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.078 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.078 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.078 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-09-13T16:34:08.078 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.078 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.078 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.078 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-13T16:34:08.079 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.079 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.079 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.079 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.079 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-09-13T16:34:08.079 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-09-13T16:34:08.079 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.079 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.079 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.079 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-13T16:34:08.079 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-09-13T16:34:08.079 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.079 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.079 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-13T16:34:08.080 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.080 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.080 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.080 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.080 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.080 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.080 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.080 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.080 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.080 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.080 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.080 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.080 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.080 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.081 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.081 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.081 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.081 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.081 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.081 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.081 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.081 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.081 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.081 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.081 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.081 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.081 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.081 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.082 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.082 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.082 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.082 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.082 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.082 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.082 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.082 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.082 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.082 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.082 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.082 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.082 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.082 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.082 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.083 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.083 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.083 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.083 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.083 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.083 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.083 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.083 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.083 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.083 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.083 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.083 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.083 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.083 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.084 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.084 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.084 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.084 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.084 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.084 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.084 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.084 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.084 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.084 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.084 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.084 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.084 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.084 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.085 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.085 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.085 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.085 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.085 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.085 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.085 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-09-13T16:34:08.085 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.085 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.085 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.085 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-09-13T16:34:08.085 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-09-13T16:34:08.085 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.085 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.086 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.086 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-09-13T16:34:08.086 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.086 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.086 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.086 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-09-13T16:34:08.086 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-13T16:34:08.086 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.086 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.086 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.086 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.086 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.086 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.086 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.086 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.087 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.087 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.087 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.087 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.087 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.087 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.087 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.087 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.087 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.087 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.087 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.088 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.088 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.088 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.088 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.088 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.088 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.088 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.088 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.088 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.088 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.088 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.088 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.088 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.088 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.089 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.089 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.089 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.089 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.089 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.089 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.089 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.089 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.089 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.089 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.089 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.089 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.089 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.089 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.090 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.090 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.090 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.090 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.090 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.090 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.090 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.090 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.090 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.090 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.090 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.090 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.090 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.090 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.091 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.091 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.091 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.091 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.091 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.091 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.091 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.091 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.091 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.091 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.091 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.091 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.091 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-09-13T16:34:08.091 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-09-13T16:34:08.092 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.092 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.092 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.092 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.092 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.092 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.092 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.092 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-09-13T16:34:08.092 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-13T16:34:08.092 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.092 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.092 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.092 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-09-13T16:34:08.092 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-09-13T16:34:08.093 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.093 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.093 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.093 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-09-13T16:34:08.093 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.093 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.093 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.093 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-09-13T16:34:08.093 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.093 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.093 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.093 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.093 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-09-13T16:34:08.093 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-09-13T16:34:08.094 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.094 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.094 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.094 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-09-13T16:34:08.094 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.094 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.094 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.094 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-09-13T16:34:08.094 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.094 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.094 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.094 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.094 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-09-13T16:34:08.094 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-09-13T16:34:08.094 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.095 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.095 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.095 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.095 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.095 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.095 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.095 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-09-13T16:34:08.095 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.095 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.095 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.095 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.095 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-09-13T16:34:08.095 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-09-13T16:34:08.095 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.096 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.096 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.096 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-09-13T16:34:08.096 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.096 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.096 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.096 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-13T16:34:08.096 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-13T16:34:08.096 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.096 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.096 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.096 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.096 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.096 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.097 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-09-13T16:34:08.097 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.097 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.097 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.097 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.097 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.097 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.097 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.097 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.097 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.097 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.097 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.097 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.097 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.097 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.098 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.098 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.098 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.098 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.098 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.098 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.098 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.098 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.098 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.098 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.098 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.098 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.098 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.098 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.098 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.099 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.099 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.099 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.099 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.099 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.099 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.099 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.099 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.099 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.099 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.099 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.099 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.099 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.099 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.100 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.100 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.100 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.100 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.100 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.100 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.100 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.100 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.100 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.100 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.101 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.101 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.101 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.101 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.101 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.101 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.101 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.101 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.101 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.101 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.101 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.101 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.101 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.101 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.102 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.102 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.102 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.102 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.102 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.102 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.102 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.102 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.102 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.102 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-09-13T16:34:08.102 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.102 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.102 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.102 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.103 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.103 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.103 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.103 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.103 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.103 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.103 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.103 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.103 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.103 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.103 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.103 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.103 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.103 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.103 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.104 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.104 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.104 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.104 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.104 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.104 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.104 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.104 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.104 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.104 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.104 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.104 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.104 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.104 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.105 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.105 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.105 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.105 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.105 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.105 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.105 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.105 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.105 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.105 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.105 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.105 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.105 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.105 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.106 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.106 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.106 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.106 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.106 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.106 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.106 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.106 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.106 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.106 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.106 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.106 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.106 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.106 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.107 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.107 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.107 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.107 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.107 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.107 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.107 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.107 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.107 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.107 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.107 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.107 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.107 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.107 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.107 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.108 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-09-13T16:34:08.108 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.108 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.108 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.108 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.108 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.108 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.108 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.108 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.108 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.108 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.108 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.108 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.108 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.109 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.109 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.109 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.109 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.109 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.109 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.109 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.109 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.109 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.109 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.109 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.109 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.109 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.109 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.109 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.110 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.110 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.110 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.110 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.110 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.110 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.110 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.110 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.110 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.110 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.110 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.110 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.110 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.110 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.111 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.111 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.111 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.111 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.111 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.111 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.111 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.111 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.111 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.111 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.111 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.111 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.111 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.111 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.111 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.112 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.112 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.112 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.112 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.112 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.112 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.112 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.112 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.112 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.112 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.112 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.112 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.112 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.112 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.113 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.113 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.113 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.113 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.113 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.113 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-13T16:34:08.113 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.113 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.113 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.113 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-09-13T16:34:08.113 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-09-13T16:34:08.113 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.113 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.113 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.113 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.114 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.114 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.114 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.114 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-13T16:34:08.114 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.114 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.114 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.114 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.114 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.114 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.114 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.114 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.114 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.115 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.115 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.115 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.115 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.115 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.115 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.115 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.115 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.115 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.115 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.115 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.115 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.115 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.115 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.115 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.116 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.116 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.116 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.116 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.116 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.116 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.116 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.116 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.116 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.116 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.116 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.116 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.116 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.116 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.117 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.117 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.117 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.117 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.117 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.117 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.117 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.117 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.117 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.117 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.117 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.117 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.117 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.117 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.117 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.118 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.118 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.118 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.118 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.118 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.118 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.118 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.118 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.118 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.118 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.118 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.118 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.118 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.118 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.119 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.119 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.119 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.119 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.119 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.119 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.119 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.119 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-09-13T16:34:08.119 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-13T16:34:08.119 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.119 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.119 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.119 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.119 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.119 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.120 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.120 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-09-13T16:34:08.120 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-09-13T16:34:08.120 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-09-13T16:34:08.120 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.120 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-09-13T16:34:08.120 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.120 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.120 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.120 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.120 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.120 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.120 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.120 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-09-13T16:34:08.121 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.121 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.121 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.121 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.121 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.121 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.121 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.121 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.121 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.121 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.121 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.121 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.121 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.121 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.122 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.122 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.122 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.122 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.122 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.122 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.122 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.122 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.122 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.122 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.122 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.122 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.122 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.122 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.122 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.123 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.123 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.123 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.123 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.123 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.123 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.123 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.123 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.123 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.123 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.123 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.123 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.123 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.123 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.123 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.124 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.124 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.124 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.124 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.124 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.124 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.124 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.124 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.124 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.124 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.124 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.124 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.124 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.124 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.125 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.125 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.125 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.125 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.125 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.125 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.125 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.125 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.125 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.125 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.125 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.125 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.125 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.125 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.125 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.126 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.126 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.126 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.126 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-09-13T16:34:08.126 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.126 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.126 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.126 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.126 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.126 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.126 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.126 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.126 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.126 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.126 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.127 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.127 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.127 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.127 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.127 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.127 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.127 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.127 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.127 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.127 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.127 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.127 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.127 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.127 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.128 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.128 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.128 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.128 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.128 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.128 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.128 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.128 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.128 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.128 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.128 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.128 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.128 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.128 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.129 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.129 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.129 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.129 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.129 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.129 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.129 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.129 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.129 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.129 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.129 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.129 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.129 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.129 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.129 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.130 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.130 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.130 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.130 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.130 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.130 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.130 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.130 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.130 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.130 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.130 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.130 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.130 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.130 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.131 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.131 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.131 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.131 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.131 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.131 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.131 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.131 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.131 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-09-13T16:34:08.131 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.131 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.131 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.131 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.131 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.132 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.132 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.132 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.132 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.132 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.132 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.132 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.132 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.132 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.132 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.132 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.132 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.132 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.132 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.132 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.133 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.133 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.133 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.133 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.133 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.133 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.133 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.133 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.133 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.133 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.133 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.133 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.133 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.134 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.134 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.134 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.134 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.134 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.134 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.134 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.134 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.134 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.134 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.134 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.134 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.134 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.134 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.135 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.135 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.135 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.135 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.135 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.135 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.135 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.135 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.135 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.135 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.135 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.135 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.135 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.135 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.136 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.136 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.136 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.136 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.136 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.136 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.136 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.136 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.136 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.136 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.136 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.136 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.136 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-13T16:34:08.137 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-13T16:34:08.137 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:08.137 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.137 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.137 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-13T16:34:08.137 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.137 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.137 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.137 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.137 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.137 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.137 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.137 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.137 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-09-13T16:34:08.137 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-09-13T16:34:08.138 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-13T16:34:08.138 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.138 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.138 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-13T16:34:08.138 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-09-13T16:34:08.138 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.138 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.138 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-09-13T16:34:08.138 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-13T16:34:08.138 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.138 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.138 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.138 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.138 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.139 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.139 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-09-13T16:34:08.139 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.139 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.139 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.139 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-09-13T16:34:08.139 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-09-13T16:34:08.139 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.139 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.139 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.139 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.139 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.139 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.139 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.140 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-13T16:34:08.140 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.140 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.140 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.140 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.140 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-13T16:34:08.140 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-13T16:34:08.140 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.140 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.140 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.140 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.140 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.140 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.140 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.140 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.141 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.141 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.141 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.141 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.144 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.144 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.144 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.144 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.144 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.144 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.144 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.144 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.144 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.145 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.145 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.145 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.145 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.145 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.145 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.145 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.145 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.145 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.145 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.145 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.145 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.145 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.146 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.146 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.146 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.146 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.146 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.146 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.146 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.146 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.146 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.146 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.146 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.146 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.146 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.146 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.147 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.147 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.147 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.147 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.147 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.147 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.147 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.147 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.147 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.147 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.147 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.147 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.147 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.147 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.147 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.148 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.148 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.148 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.148 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.148 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.148 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.148 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.148 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.148 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.148 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.148 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.148 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.148 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.148 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.149 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.149 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.149 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.149 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.149 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.149 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-09-13T16:34:08.149 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-09-13T16:34:08.149 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.149 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.149 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.149 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-09-13T16:34:08.149 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.149 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.149 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.150 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-09-13T16:34:08.150 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.150 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.150 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.150 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.150 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-13T16:34:08.150 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-13T16:34:08.150 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:08.150 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.150 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.150 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-13T16:34:08.150 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.150 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.150 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.150 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-09-13T16:34:08.151 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.151 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.151 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.151 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.151 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.151 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.151 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.151 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-09-13T16:34:08.151 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.151 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.151 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.151 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-09-13T16:34:08.151 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-09-13T16:34:08.151 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.152 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.152 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.152 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.152 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.152 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.152 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.152 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.152 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.152 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.152 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.152 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.152 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-09-13T16:34:08.152 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-09-13T16:34:08.152 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.153 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.153 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.153 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.153 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.153 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.153 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.153 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-13T16:34:08.153 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-13T16:34:08.153 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.153 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.153 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.153 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.153 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.153 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.154 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.154 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.154 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.154 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.154 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.154 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.154 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.154 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.154 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.154 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.154 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.154 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.154 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.154 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.154 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.155 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.155 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.155 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.155 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.155 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.155 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.155 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.155 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.155 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.155 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.155 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.155 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.155 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.155 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.155 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.156 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.156 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.156 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.156 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.156 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.156 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.156 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.156 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.156 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.156 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.156 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.156 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.156 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.156 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.156 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.157 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.157 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.157 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.157 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.157 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.157 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.157 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.157 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.157 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.157 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.157 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.157 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.157 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.157 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.158 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.158 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.158 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.158 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.158 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.158 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.158 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.158 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.158 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-09-13T16:34:08.158 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-09-13T16:34:08.158 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.158 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.158 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.158 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.159 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.159 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.159 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.159 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.159 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.159 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.159 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.159 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.159 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-13T16:34:08.159 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-13T16:34:08.159 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.159 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.159 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.159 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-09-13T16:34:08.159 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.160 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.160 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.160 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.160 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.160 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.160 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.160 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.160 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-09-13T16:34:08.160 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-09-13T16:34:08.160 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-13T16:34:08.160 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.160 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.160 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-09-13T16:34:08.160 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.161 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.161 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.161 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.161 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.161 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.161 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.161 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.161 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-13T16:34:08.161 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-13T16:34:08.161 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.161 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.161 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.161 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-13T16:34:08.161 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.162 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.162 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.162 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-13T16:34:08.162 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.162 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.162 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.162 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.162 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-13T16:34:08.162 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-13T16:34:08.162 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.162 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.162 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.162 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-09-13T16:34:08.162 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.163 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.163 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.163 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-13T16:34:08.163 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.163 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.163 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.163 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.163 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-09-13T16:34:08.163 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-09-13T16:34:08.163 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.163 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.163 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.163 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-09-13T16:34:08.163 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.163 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.164 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.164 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.164 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.164 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.164 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.164 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.164 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-13T16:34:08.164 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-13T16:34:08.164 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.164 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.164 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.164 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-09-13T16:34:08.164 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.164 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.165 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.165 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-09-13T16:34:08.165 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-09-13T16:34:08.165 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.165 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.165 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.165 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.165 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.165 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.165 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-13T16:34:08.165 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-13T16:34:08.165 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.165 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.165 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.165 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-13T16:34:08.166 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-13T16:34:08.166 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-13T16:34:08.166 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.166 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.166 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.166 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.166 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.166 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.166 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.166 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.166 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.166 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-09-13T16:34:08.166 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.166 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.167 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.167 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.167 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.167 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.167 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.167 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.167 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.167 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.167 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.167 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.167 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.167 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.167 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.167 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.167 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.168 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.168 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.168 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.168 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.168 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.168 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.168 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.168 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.168 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.168 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.168 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.168 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.168 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.168 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.169 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.169 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.169 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.169 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.169 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.169 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.169 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.169 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.169 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.169 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.169 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.169 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.169 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.169 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.169 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.170 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.170 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.170 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.170 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.170 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.170 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.170 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.170 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.170 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.170 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.170 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.170 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.170 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.170 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.171 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.171 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.171 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.171 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.171 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.171 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.171 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.171 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.171 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.171 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.171 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.171 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.171 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.171 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-09-13T16:34:08.171 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-09-13T16:34:08.172 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.172 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.172 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.172 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-13T16:34:08.172 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.172 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.172 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.172 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.172 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.172 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.172 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.172 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.172 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-09-13T16:34:08.172 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-09-13T16:34:08.173 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.173 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.173 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.173 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.173 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.173 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.173 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.173 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.173 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.173 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.173 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.173 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.173 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-09-13T16:34:08.173 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-09-13T16:34:08.174 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.174 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.174 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.174 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.174 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.174 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.174 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.174 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.174 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.174 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.174 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.174 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.174 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.174 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.174 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.175 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-09-13T16:34:08.175 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.175 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.175 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.175 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-09-13T16:34:08.175 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-09-13T16:34:08.175 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.175 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.175 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.175 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.175 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.175 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.175 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.175 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.176 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.176 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.176 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.176 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.176 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-13T16:34:08.176 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-13T16:34:08.176 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.176 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.176 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.176 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.176 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.176 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.176 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.177 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.177 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.177 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.177 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.177 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.177 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.177 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.177 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.177 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.177 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.177 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.177 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.177 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.177 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.178 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.178 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.178 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.178 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.178 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.178 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.178 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.178 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.178 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.178 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.178 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.178 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.178 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.178 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.179 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.179 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.179 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.179 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.179 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.179 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.179 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.179 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.179 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.179 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.179 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.179 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.179 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.179 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.180 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.180 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.180 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.180 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.180 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.180 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.180 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.180 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.180 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.180 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.180 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.180 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.180 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.180 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.181 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.181 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.181 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.181 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.181 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.181 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.181 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.181 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.181 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.181 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.181 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.181 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.181 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.181 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.181 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.182 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.182 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.182 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.182 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.182 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-13T16:34:08.182 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-13T16:34:08.182 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.182 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.182 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.182 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.182 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.182 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.182 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.182 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.183 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.183 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.183 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.183 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.183 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-13T16:34:08.183 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-13T16:34:08.183 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.183 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.183 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.183 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.183 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.183 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.183 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.184 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.184 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.184 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.184 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.184 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.184 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.184 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.184 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.184 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-09-13T16:34:08.184 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.184 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.184 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.184 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.184 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.185 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.185 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.185 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.185 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.185 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.185 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.185 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.185 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.185 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.185 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.185 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.185 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.185 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.185 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.186 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.186 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.186 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.186 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.186 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.186 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.186 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.186 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.186 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.186 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.186 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.186 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.186 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.186 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.187 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.187 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.187 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.187 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.187 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.187 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.187 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.187 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.187 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.187 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.187 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.187 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.187 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.187 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.188 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.188 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.188 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.188 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.188 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.188 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.188 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.188 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.188 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.188 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.188 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.188 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.188 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.188 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.189 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.189 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.189 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.189 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.189 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.189 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.189 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.189 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.189 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.189 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.189 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.189 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.189 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.189 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.189 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.190 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-09-13T16:34:08.190 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.190 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.190 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.190 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.190 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.190 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.190 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.190 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.190 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.190 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.190 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.190 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.190 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.191 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.191 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.191 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.191 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.191 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.191 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.191 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.191 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.191 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.191 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.191 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.191 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.191 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.191 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.192 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.192 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.192 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.192 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.192 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.192 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.192 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.192 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.192 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.192 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.192 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.192 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.192 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.192 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.193 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.193 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.193 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.193 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.193 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.193 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.193 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.193 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.193 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.193 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.193 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.193 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.193 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.193 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.194 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.194 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.194 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.194 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.194 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.194 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.194 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.194 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.194 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.194 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.194 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.194 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.194 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.194 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.195 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.195 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.195 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.195 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.195 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-09-13T16:34:08.195 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-09-13T16:34:08.195 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.195 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.195 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.195 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-09-13T16:34:08.195 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.195 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.195 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.195 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-13T16:34:08.196 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.196 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.196 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.196 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.196 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.196 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.196 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.196 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-09-13T16:34:08.196 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.196 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.196 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.196 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.196 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.196 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.197 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.197 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.197 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.197 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.197 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.197 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.197 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.197 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.197 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.197 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.197 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.197 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.197 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.198 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.198 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.198 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.198 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.198 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.198 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.198 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.198 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.198 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.198 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.198 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.198 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.198 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.198 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.199 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.199 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.199 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.199 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.199 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.199 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.199 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.199 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.199 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.199 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.199 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.199 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.199 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.200 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.200 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.200 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.200 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.200 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.200 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.200 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.200 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.200 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.200 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.200 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.200 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.200 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.201 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.201 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.201 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.201 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.201 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.201 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.201 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.201 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.201 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.201 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.201 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.201 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.201 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.201 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.202 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-09-13T16:34:08.202 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-09-13T16:34:08.202 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.202 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.202 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.202 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.202 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.202 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.202 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.202 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.202 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.202 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.202 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.202 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.203 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-09-13T16:34:08.203 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-09-13T16:34:08.203 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.203 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.203 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.203 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-09-13T16:34:08.203 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.203 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.203 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.203 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.203 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.203 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.203 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.203 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.203 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-09-13T16:34:08.204 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-09-13T16:34:08.204 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.204 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.204 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.204 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.204 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.204 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.204 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.204 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.204 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.204 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.204 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.204 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.204 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-09-13T16:34:08.205 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-09-13T16:34:08.205 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.205 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.205 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.205 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.205 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.205 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.205 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.205 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.205 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.205 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.205 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.205 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.205 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-09-13T16:34:08.206 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-09-13T16:34:08.206 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-13T16:34:08.206 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.206 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.206 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.206 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.206 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.206 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.206 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.206 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.206 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.206 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.206 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.206 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-09-13T16:34:08.207 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-09-13T16:34:08.207 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.207 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.207 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.207 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.207 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.207 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.207 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.207 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.207 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.207 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.207 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.207 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.207 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-09-13T16:34:08.208 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-09-13T16:34:08.208 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.208 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.208 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.208 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.208 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.208 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.208 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.208 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.208 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.208 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.208 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.208 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.208 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-09-13T16:34:08.209 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-09-13T16:34:08.209 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.209 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.209 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.209 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.209 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.209 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.209 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.209 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.209 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.209 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.209 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.209 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.209 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-09-13T16:34:08.210 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-09-13T16:34:08.210 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.210 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.210 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.210 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.210 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-13T16:34:08.210 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-09-13T16:34:08.210 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.210 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.210 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.210 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.210 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.210 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.210 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-09-13T16:34:08.210 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-09-13T16:34:08.211 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.211 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.211 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.211 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.211 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.211 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.211 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.211 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.211 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.211 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.211 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.211 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.211 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-09-13T16:34:08.211 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-09-13T16:34:08.212 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.212 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.212 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.212 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.212 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.212 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.212 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.212 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.212 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.212 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.212 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.212 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.212 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-09-13T16:34:08.212 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-09-13T16:34:08.213 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.213 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.213 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.213 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.213 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.213 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.213 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.213 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.213 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.213 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.213 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.213 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.213 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.213 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.213 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.214 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-09-13T16:34:08.214 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.214 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.214 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.214 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-09-13T16:34:08.214 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-09-13T16:34:08.214 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.214 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.214 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.214 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.214 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.214 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.214 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.215 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-09-13T16:34:08.215 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.215 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.215 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.215 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.215 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-09-13T16:34:08.215 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-09-13T16:34:08.215 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.215 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.215 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.215 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.215 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.215 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.215 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.216 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-09-13T16:34:08.216 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.216 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.216 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.216 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.216 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.216 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.216 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.216 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.216 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.216 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.216 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.216 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.217 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.217 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.217 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.217 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.217 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.217 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.217 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.217 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.217 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.217 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.217 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.217 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.217 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.217 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.218 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.218 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.218 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.218 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.218 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.218 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.218 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.218 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.218 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.218 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.218 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.218 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.218 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.218 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.218 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.219 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.219 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.219 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.219 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.219 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.219 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.219 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.219 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.219 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.219 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.219 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.219 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.219 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.219 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.220 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.220 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.220 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.220 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.220 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.220 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.220 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.220 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.220 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.220 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.220 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.220 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.220 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.220 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.221 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.221 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.221 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.221 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.221 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.221 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.221 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.221 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.221 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-09-13T16:34:08.221 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.221 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.221 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.221 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.221 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.222 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.222 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.222 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.222 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.222 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.222 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.222 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.222 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.222 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.222 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.222 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.222 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.222 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.222 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.223 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.223 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.223 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.223 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.223 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.223 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.223 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.223 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.223 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.223 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.223 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.223 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.223 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.223 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.223 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.224 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.224 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.224 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.224 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.224 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.224 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.224 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.224 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.224 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.224 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.224 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.224 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.224 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.224 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.224 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.225 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.225 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.225 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.225 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.225 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.225 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.225 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.225 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.225 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.225 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.225 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.225 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.225 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.225 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.226 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.226 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.226 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.226 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.226 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.226 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.226 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.226 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.226 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.226 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.226 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.226 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.226 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.227 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-09-13T16:34:08.227 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.227 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.227 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.227 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.227 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.227 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.227 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.227 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.227 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.227 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.227 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.227 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.227 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.228 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.228 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.228 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.228 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.228 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.228 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.228 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.228 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.228 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.228 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.228 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.228 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.228 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.228 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.229 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.229 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.229 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.229 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.229 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.229 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.229 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.229 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.229 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.229 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.229 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.229 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.229 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.229 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.230 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.230 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.230 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.230 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.230 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.230 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.230 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.230 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.230 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.230 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.230 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.230 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.230 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.230 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.231 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.231 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.231 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.231 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.231 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.231 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.231 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.231 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.231 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.231 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.231 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.231 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.231 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.231 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.231 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.232 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.232 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.232 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.232 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.232 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.232 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.232 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-09-13T16:34:08.232 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.232 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.232 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.232 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-09-13T16:34:08.232 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-09-13T16:34:08.232 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.232 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.233 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.233 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-13T16:34:08.233 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.233 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.233 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.233 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.233 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.233 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.233 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.233 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.233 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-13T16:34:08.233 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-13T16:34:08.233 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:08.233 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.234 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.234 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-09-13T16:34:08.234 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.234 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.234 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.234 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.234 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.234 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.234 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.234 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.234 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.234 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.234 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.234 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.235 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.235 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.235 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.235 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.235 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.235 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.235 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.235 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.235 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.235 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.235 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.235 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.235 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.235 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.236 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.236 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.236 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.236 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.236 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.236 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.236 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.236 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.236 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.236 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.236 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.236 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.236 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.236 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.237 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.237 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.237 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.237 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.237 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.237 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.237 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.237 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.237 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.237 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.237 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.237 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.237 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.237 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.238 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.238 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.238 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.238 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.238 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.238 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.238 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.238 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.238 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.238 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.238 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.238 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.238 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.238 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.239 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.239 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.239 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.239 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.239 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.239 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.239 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.239 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.239 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.239 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.239 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.239 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.239 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.239 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-09-13T16:34:08.239 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.240 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.240 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.240 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-09-13T16:34:08.240 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-09-13T16:34:08.240 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.240 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.240 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.240 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.240 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.240 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.240 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.240 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-13T16:34:08.240 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.241 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.241 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.241 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.241 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-09-13T16:34:08.241 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-09-13T16:34:08.241 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.241 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.241 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.241 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.241 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.241 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.241 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.241 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-13T16:34:08.241 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.241 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.242 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.242 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.242 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-09-13T16:34:08.242 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-09-13T16:34:08.242 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.242 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.242 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.242 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.242 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.242 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.242 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.242 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-13T16:34:08.242 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.242 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.243 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.243 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.243 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-09-13T16:34:08.243 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-09-13T16:34:08.243 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.243 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.243 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.243 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.243 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.243 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.243 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.243 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-13T16:34:08.243 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.243 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.244 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.244 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.244 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-09-13T16:34:08.244 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-09-13T16:34:08.244 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.244 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.244 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.244 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.244 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.244 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.244 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.244 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-13T16:34:08.244 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.244 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.244 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.245 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.245 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-09-13T16:34:08.245 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-09-13T16:34:08.245 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.245 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.245 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.245 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.245 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.245 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.245 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.245 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.245 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.245 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.245 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.246 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.246 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-09-13T16:34:08.246 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-09-13T16:34:08.246 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.246 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.246 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.246 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.246 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.246 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.246 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.246 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.246 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.246 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.246 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.246 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.247 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-09-13T16:34:08.247 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-09-13T16:34:08.247 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.247 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.247 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.247 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-13T16:34:08.247 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.247 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.247 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.247 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.247 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.247 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.247 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.247 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.248 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-13T16:34:08.248 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-13T16:34:08.248 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.248 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.248 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.248 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.248 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.248 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.248 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.248 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.248 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.248 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.248 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.248 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.249 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-13T16:34:08.249 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-13T16:34:08.249 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.249 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.249 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.249 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-09-13T16:34:08.249 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-09-13T16:34:08.249 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.249 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.249 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.249 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.249 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.249 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.249 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.249 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-09-13T16:34:08.250 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-09-13T16:34:08.250 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.250 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.250 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.250 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-09-13T16:34:08.250 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.250 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.250 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.250 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.250 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.250 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.250 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.250 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.250 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-09-13T16:34:08.251 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-09-13T16:34:08.251 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.251 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.251 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.251 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.251 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.251 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.251 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.251 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.251 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.251 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.251 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.251 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.251 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-09-13T16:34:08.251 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-09-13T16:34:08.252 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.252 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.252 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.252 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.252 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.252 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.252 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.252 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.252 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.252 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.252 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.252 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.252 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.252 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.253 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.253 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.253 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.253 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.253 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.253 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.253 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.253 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.253 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.253 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.253 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.253 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.253 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.254 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.254 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.254 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.254 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.254 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.254 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.254 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.254 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.254 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.254 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.254 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.254 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.254 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.254 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.255 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.255 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.255 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.255 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.255 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.255 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.255 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.255 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.255 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.255 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.255 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.255 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.255 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.255 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.256 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.256 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.256 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.256 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.256 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.256 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.256 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.256 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.256 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.256 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.256 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.256 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.256 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.256 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.257 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.257 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.257 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.257 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.257 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.257 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.257 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.257 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.257 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.257 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.257 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.257 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.257 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.257 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-09-13T16:34:08.258 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-09-13T16:34:08.258 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.258 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.258 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.258 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.258 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.258 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.258 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.258 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.258 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.258 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.258 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.258 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.258 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-09-13T16:34:08.259 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-09-13T16:34:08.259 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.259 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.259 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.259 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.259 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.259 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.259 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.259 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.259 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.259 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.259 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.259 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.259 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-09-13T16:34:08.259 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-09-13T16:34:08.260 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.260 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.260 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.260 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-13T16:34:08.260 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.260 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.260 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.260 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.260 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.260 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.260 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.260 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.260 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.261 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.261 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.261 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-09-13T16:34:08.261 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.261 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.261 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.261 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.261 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.261 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.261 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.261 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.261 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.261 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.261 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.262 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.262 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.262 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.262 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.262 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.262 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.262 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.262 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.262 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.262 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.262 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.262 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.262 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.262 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.262 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.263 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.263 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.263 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.263 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.263 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.263 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.263 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.263 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.263 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.263 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.263 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.263 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.263 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.263 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.264 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.264 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.264 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.264 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.264 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.264 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.264 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.264 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.264 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.264 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.264 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.264 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.264 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.264 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.265 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.265 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.265 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.265 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.265 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.265 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.265 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.265 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.265 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.265 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.265 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.265 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.265 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.265 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.265 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.266 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.266 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.266 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.266 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.266 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.266 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.266 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.266 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-09-13T16:34:08.266 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.266 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.266 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.266 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.266 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.267 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.267 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.267 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.267 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.267 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.267 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.267 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.267 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.267 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.267 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.267 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.267 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.267 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.267 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.268 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.268 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.268 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.268 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.268 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.268 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.268 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.268 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.268 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.268 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.268 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.268 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.268 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.268 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.269 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.269 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.269 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.269 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.269 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.269 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.269 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.269 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.269 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.269 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.269 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.269 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.269 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.269 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.270 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.270 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.270 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.270 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.270 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.270 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.270 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.270 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.270 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.270 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.270 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.270 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.270 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.270 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.271 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.271 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.271 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.271 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.271 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.271 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.271 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.271 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.271 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.271 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.271 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.271 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.271 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-09-13T16:34:08.271 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-09-13T16:34:08.272 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.272 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.272 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.272 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-09-13T16:34:08.272 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.272 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.272 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.272 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-13T16:34:08.272 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.272 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.272 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.272 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.272 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-09-13T16:34:08.273 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-09-13T16:34:08.273 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.273 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.273 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.273 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-13T16:34:08.273 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.273 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.273 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.273 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-13T16:34:08.273 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.273 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.273 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.273 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.273 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-09-13T16:34:08.274 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-09-13T16:34:08.274 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.274 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.274 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.274 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.274 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.274 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.274 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.274 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-13T16:34:08.274 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.274 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.274 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.274 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.274 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.275 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.275 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.275 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-09-13T16:34:08.275 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.275 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.275 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.275 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-09-13T16:34:08.275 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-09-13T16:34:08.275 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-13T16:34:08.275 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.275 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.275 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-09-13T16:34:08.275 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.275 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.276 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.276 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.276 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.276 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.276 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.276 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.276 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-09-13T16:34:08.276 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-09-13T16:34:08.276 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.276 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.276 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.276 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.276 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.276 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.277 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.277 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.277 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.277 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.277 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.277 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.277 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-13T16:34:08.277 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-13T16:34:08.277 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:08.277 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.277 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.277 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-13T16:34:08.277 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.277 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.277 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.278 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.278 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.278 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.278 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.278 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.278 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.278 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.278 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.278 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.278 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.278 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.278 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.278 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.278 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.279 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.279 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.279 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.279 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.279 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.279 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.279 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.279 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.279 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.279 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.279 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.279 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.279 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.279 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.280 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.280 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.280 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.280 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.280 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.280 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.280 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.280 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.280 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.280 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.280 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.280 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.280 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.280 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.281 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.281 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.281 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.281 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.281 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.281 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.281 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.281 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.281 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.281 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.281 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.281 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.281 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.282 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.282 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.282 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.282 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.282 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.282 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.282 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.282 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.282 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.282 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.282 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.282 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.282 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.282 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.282 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.283 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.283 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.283 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.283 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.283 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.283 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-09-13T16:34:08.283 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-09-13T16:34:08.283 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.283 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.283 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.283 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.283 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.283 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.283 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.284 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.284 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.284 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.284 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.284 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.284 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-09-13T16:34:08.284 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-09-13T16:34:08.284 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.284 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.284 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.284 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-09-13T16:34:08.284 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.284 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.284 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.285 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.285 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.285 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.285 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.285 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.285 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-09-13T16:34:08.285 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-09-13T16:34:08.285 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.285 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.285 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.285 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-13T16:34:08.285 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.285 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.285 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.286 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.286 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.286 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.286 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.286 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.286 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.286 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.286 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-13T16:34:08.286 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.286 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.286 INFO:teuthology.orchestra.run.smithi016.stdout: "modules": [ 2024-09-13T16:34:08.286 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-09-13T16:34:08.286 INFO:teuthology.orchestra.run.smithi016.stdout: "dashboard", 2024-09-13T16:34:08.286 INFO:teuthology.orchestra.run.smithi016.stdout: "iostat", 2024-09-13T16:34:08.287 INFO:teuthology.orchestra.run.smithi016.stdout: "nfs", 2024-09-13T16:34:08.287 INFO:teuthology.orchestra.run.smithi016.stdout: "restful" 2024-09-13T16:34:08.287 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.287 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-09-13T16:34:08.287 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.287 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-09-13T16:34:08.287 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.287 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.287 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.287 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-13T16:34:08.287 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-13T16:34:08.287 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:08.287 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.287 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.288 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-13T16:34:08.288 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.288 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.288 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.288 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-09-13T16:34:08.288 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.288 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.288 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.288 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.288 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.288 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.288 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.288 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.288 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.289 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.289 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.289 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.289 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.289 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.289 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.289 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.289 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.289 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.289 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.289 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.289 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.289 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.289 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.290 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.290 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.290 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.290 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.290 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.290 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.290 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.290 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.290 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.290 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.290 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.290 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.290 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.290 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.291 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.291 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.291 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.291 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.291 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.291 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.291 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.291 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.291 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.291 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.291 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.291 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.291 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.291 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.291 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.292 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.292 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.292 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.292 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.292 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.292 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.292 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.292 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.292 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.292 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.292 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.292 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.292 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.292 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.293 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.293 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.293 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.293 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.293 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.293 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.293 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.293 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.293 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-09-13T16:34:08.293 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-09-13T16:34:08.293 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.293 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.293 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.293 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.294 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.294 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.294 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.294 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to", 2024-09-13T16:34:08.294 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.294 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.294 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.294 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.294 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-09-13T16:34:08.294 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-09-13T16:34:08.294 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.294 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.294 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.295 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-09-13T16:34:08.295 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.295 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.295 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.295 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-09-13T16:34:08.295 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.295 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.295 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.295 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.295 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-09-13T16:34:08.295 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-09-13T16:34:08.295 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.295 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.295 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.296 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.296 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.296 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.296 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.296 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-09-13T16:34:08.296 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.296 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.296 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.296 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.296 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-09-13T16:34:08.296 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-09-13T16:34:08.296 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.296 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.296 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.296 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.297 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.297 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.297 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.297 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-09-13T16:34:08.297 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.297 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.297 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.297 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.297 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-09-13T16:34:08.297 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-09-13T16:34:08.297 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.297 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.297 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.298 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-09-13T16:34:08.298 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.298 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.298 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.298 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-09-13T16:34:08.298 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.298 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.298 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.298 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.298 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-09-13T16:34:08.298 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-09-13T16:34:08.298 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.298 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.298 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.298 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.299 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.299 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.299 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.299 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-09-13T16:34:08.299 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.299 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.299 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.299 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.299 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-09-13T16:34:08.299 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-09-13T16:34:08.299 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.299 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.299 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.299 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.300 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.300 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.300 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.300 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-13T16:34:08.300 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.300 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.300 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.300 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.300 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-09-13T16:34:08.300 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-09-13T16:34:08.300 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.300 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.300 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.300 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.301 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.301 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.301 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.301 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-09-13T16:34:08.301 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.301 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.301 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.301 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.301 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.301 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.301 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.301 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-09-13T16:34:08.301 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.301 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.302 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.302 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-09-13T16:34:08.302 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-09-13T16:34:08.302 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.302 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.302 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.302 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.302 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.302 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.302 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.302 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-09-13T16:34:08.302 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.302 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.302 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.302 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.303 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-09-13T16:34:08.303 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-09-13T16:34:08.303 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.303 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.303 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.303 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-09-13T16:34:08.303 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.303 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.303 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.303 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-09-13T16:34:08.303 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-13T16:34:08.303 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.303 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.303 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.304 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-09-13T16:34:08.304 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-09-13T16:34:08.304 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-13T16:34:08.304 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.304 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.304 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-13T16:34:08.304 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-13T16:34:08.304 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-13T16:34:08.304 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.304 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-13T16:34:08.304 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-13T16:34:08.304 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.304 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.304 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.305 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-09-13T16:34:08.305 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-09-13T16:34:08.305 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-13T16:34:08.305 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.305 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.305 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-09-13T16:34:08.305 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-13T16:34:08.305 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-09-13T16:34:08.305 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.305 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-13T16:34:08.305 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.305 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.305 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.305 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.305 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-09-13T16:34:08.306 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-09-13T16:34:08.306 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.306 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.306 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.306 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-09-13T16:34:08.306 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.306 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.306 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.306 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-13T16:34:08.306 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-13T16:34:08.306 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.306 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.306 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.306 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-09-13T16:34:08.306 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-09-13T16:34:08.307 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-13T16:34:08.307 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.307 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.307 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-09-13T16:34:08.307 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-09-13T16:34:08.307 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-09-13T16:34:08.307 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.307 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-09-13T16:34:08.307 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-13T16:34:08.307 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.307 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.307 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.307 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-09-13T16:34:08.307 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-09-13T16:34:08.308 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.308 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.308 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.308 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-09-13T16:34:08.308 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.308 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.308 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.308 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-09-13T16:34:08.308 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-13T16:34:08.308 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.308 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.308 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.308 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-09-13T16:34:08.308 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-09-13T16:34:08.309 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-13T16:34:08.309 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.309 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.309 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-13T16:34:08.309 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-13T16:34:08.309 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-13T16:34:08.309 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.309 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-13T16:34:08.309 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-13T16:34:08.309 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.309 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.309 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.309 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.309 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.309 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.310 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.310 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.310 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.310 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.310 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.310 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.310 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.310 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.310 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.310 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.310 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.310 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.310 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.310 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.311 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.311 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.311 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.311 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.311 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.311 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.311 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.311 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.311 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.311 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.311 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.311 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.311 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.311 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.311 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.312 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.312 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.312 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.312 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.312 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.312 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.312 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.312 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.312 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.312 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.312 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.312 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.312 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.312 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.313 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.313 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.313 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.313 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.313 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.313 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.313 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.313 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.313 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.313 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.313 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.313 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.313 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.313 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.314 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.314 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.314 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.314 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.314 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.314 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.314 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.314 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.314 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.314 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.314 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-09-13T16:34:08.314 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-09-13T16:34:08.314 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-13T16:34:08.314 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.315 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.315 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-13T16:34:08.315 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.315 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.315 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.315 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-13T16:34:08.315 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.315 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.315 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.315 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.315 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-13T16:34:08.315 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-13T16:34:08.315 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.315 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.315 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.316 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-09-13T16:34:08.316 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.316 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.316 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.316 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-09-13T16:34:08.316 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-09-13T16:34:08.316 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap" 2024-09-13T16:34:08.316 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.316 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-09-13T16:34:08.316 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.316 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.316 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.316 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.316 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-09-13T16:34:08.317 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-09-13T16:34:08.317 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.317 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.317 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.317 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.317 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.317 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.317 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.317 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-13T16:34:08.317 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.317 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.317 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.317 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.317 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-13T16:34:08.318 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-13T16:34:08.318 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:08.318 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.318 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.318 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-13T16:34:08.318 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.318 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.318 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.318 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-13T16:34:08.318 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.318 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.318 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.318 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.318 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-09-13T16:34:08.319 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-09-13T16:34:08.319 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.319 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.319 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.319 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-13T16:34:08.319 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-13T16:34:08.319 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.319 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.319 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-13T16:34:08.319 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-13T16:34:08.319 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.319 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.319 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.319 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-09-13T16:34:08.320 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-09-13T16:34:08.320 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-13T16:34:08.320 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.320 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.320 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-13T16:34:08.320 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.320 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.320 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.320 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-13T16:34:08.320 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.320 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.320 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.320 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.320 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.321 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.321 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.321 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-09-13T16:34:08.321 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.321 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.321 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.321 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-09-13T16:34:08.321 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-09-13T16:34:08.321 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-13T16:34:08.321 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.321 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.321 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-13T16:34:08.321 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.321 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.321 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.322 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-13T16:34:08.322 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.322 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.322 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.322 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.322 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-09-13T16:34:08.322 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-09-13T16:34:08.322 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:08.322 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.322 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.322 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-09-13T16:34:08.322 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.322 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.322 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.322 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-13T16:34:08.323 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.323 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.323 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.323 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.323 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-09-13T16:34:08.323 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-09-13T16:34:08.323 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.323 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.323 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.323 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-09-13T16:34:08.323 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.323 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.323 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.323 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-13T16:34:08.324 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.324 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.324 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.324 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.324 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-09-13T16:34:08.324 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-09-13T16:34:08.324 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.324 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.324 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.324 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.324 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.324 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.324 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.324 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-13T16:34:08.325 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-13T16:34:08.325 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.325 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.325 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.325 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-09-13T16:34:08.325 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-09-13T16:34:08.325 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:08.325 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.325 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.325 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-13T16:34:08.325 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.325 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.325 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.325 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-13T16:34:08.325 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.326 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.326 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.326 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.326 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-09-13T16:34:08.326 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-09-13T16:34:08.326 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-13T16:34:08.326 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.326 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.326 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-09-13T16:34:08.326 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.326 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.326 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.326 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-13T16:34:08.327 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.327 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.327 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.327 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.327 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-09-13T16:34:08.327 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-09-13T16:34:08.327 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.327 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.327 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.327 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.327 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.327 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.327 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.327 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-13T16:34:08.328 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.328 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.328 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.328 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.328 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-09-13T16:34:08.328 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-09-13T16:34:08.328 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.328 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.328 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.328 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.328 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.328 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.328 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.328 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-13T16:34:08.329 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.329 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.329 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.329 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.329 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-09-13T16:34:08.329 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-09-13T16:34:08.329 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.329 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.329 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.329 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.329 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.329 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.329 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.329 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-13T16:34:08.329 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.330 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.330 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.330 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.330 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-09-13T16:34:08.330 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-09-13T16:34:08.330 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.330 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.330 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.330 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-13T16:34:08.330 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.330 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.330 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.330 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-13T16:34:08.330 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.331 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.331 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.331 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.331 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-09-13T16:34:08.331 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-09-13T16:34:08.331 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.331 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.331 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.331 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-13T16:34:08.331 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.331 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.331 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.331 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-09-13T16:34:08.331 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.332 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.332 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.332 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.332 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-09-13T16:34:08.332 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-09-13T16:34:08.332 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.332 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.332 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.332 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-13T16:34:08.332 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.332 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.332 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.332 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-09-13T16:34:08.332 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.332 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.333 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.333 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.333 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-09-13T16:34:08.333 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-09-13T16:34:08.333 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.333 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.333 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.333 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-09-13T16:34:08.333 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.333 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.333 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.333 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-13T16:34:08.333 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.333 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.334 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.334 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.334 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-09-13T16:34:08.334 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-09-13T16:34:08.334 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.334 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.334 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.334 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-13T16:34:08.334 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.334 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.334 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.334 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-09-13T16:34:08.334 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.334 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.334 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.335 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.335 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-09-13T16:34:08.335 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-09-13T16:34:08.335 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.335 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.335 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.335 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-13T16:34:08.335 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.335 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.335 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.335 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-09-13T16:34:08.335 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.335 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.335 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.336 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.336 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-09-13T16:34:08.336 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-09-13T16:34:08.336 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.336 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.336 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.336 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-13T16:34:08.336 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.336 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.336 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.336 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-09-13T16:34:08.336 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.336 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.336 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.337 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.337 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-09-13T16:34:08.337 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-09-13T16:34:08.337 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.337 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.337 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.337 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-13T16:34:08.337 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.337 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.337 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.337 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-09-13T16:34:08.337 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.337 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.337 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.337 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.338 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-09-13T16:34:08.338 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-09-13T16:34:08.338 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.338 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.338 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.338 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-13T16:34:08.338 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.338 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.338 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.338 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-09-13T16:34:08.338 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.338 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.338 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.338 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.339 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-09-13T16:34:08.339 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-09-13T16:34:08.339 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.339 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.339 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.339 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-13T16:34:08.339 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.339 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.339 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.339 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-09-13T16:34:08.339 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.339 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.339 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.339 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.340 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-09-13T16:34:08.340 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-09-13T16:34:08.340 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.340 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.340 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.340 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-13T16:34:08.340 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.340 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.340 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.340 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-13T16:34:08.340 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.340 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.340 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.340 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.340 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-09-13T16:34:08.341 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-09-13T16:34:08.341 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.341 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.341 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.341 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-09-13T16:34:08.341 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.341 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.341 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.341 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-09-13T16:34:08.341 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.341 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.341 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.341 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.341 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-09-13T16:34:08.342 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-09-13T16:34:08.342 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.342 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.342 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.342 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-13T16:34:08.342 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.342 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.342 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.342 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-13T16:34:08.342 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.342 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.342 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.342 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.342 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-09-13T16:34:08.343 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-09-13T16:34:08.343 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.343 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.343 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.343 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-13T16:34:08.343 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.343 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.343 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.343 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-09-13T16:34:08.343 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.343 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.343 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.343 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.343 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-09-13T16:34:08.343 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-09-13T16:34:08.344 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.344 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.344 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.344 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-13T16:34:08.344 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.344 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.344 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.344 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-09-13T16:34:08.344 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.344 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.344 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.344 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.344 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-09-13T16:34:08.344 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-09-13T16:34:08.345 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.345 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.345 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.345 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.345 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.345 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.345 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.345 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-09-13T16:34:08.345 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.345 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.345 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.345 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.345 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-09-13T16:34:08.345 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-09-13T16:34:08.346 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:08.346 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.346 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.346 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-13T16:34:08.346 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.346 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.346 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.346 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-13T16:34:08.346 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.346 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.346 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.346 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.346 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-09-13T16:34:08.346 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-09-13T16:34:08.347 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.347 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.347 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.347 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-09-13T16:34:08.347 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.347 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.347 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.347 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-13T16:34:08.347 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.347 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.347 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.347 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.347 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-09-13T16:34:08.347 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-09-13T16:34:08.347 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.348 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.348 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.348 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-09-13T16:34:08.348 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.348 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.348 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.348 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-13T16:34:08.348 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.348 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.348 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.348 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.348 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-09-13T16:34:08.348 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-09-13T16:34:08.348 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:08.349 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.349 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.349 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-09-13T16:34:08.349 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.349 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.349 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.349 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-09-13T16:34:08.349 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.349 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.349 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.349 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.352 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-09-13T16:34:08.352 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-09-13T16:34:08.352 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.352 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.352 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.352 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.352 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.352 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.352 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.352 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-13T16:34:08.353 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.353 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.353 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.353 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.353 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-09-13T16:34:08.353 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-09-13T16:34:08.353 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:08.353 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.353 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.353 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-13T16:34:08.353 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.353 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.353 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.353 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-09-13T16:34:08.354 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.354 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.354 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.354 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.354 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-09-13T16:34:08.354 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-09-13T16:34:08.354 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:08.354 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.354 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.354 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-13T16:34:08.354 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.354 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.354 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.354 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-09-13T16:34:08.354 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.355 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.355 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.355 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.355 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-09-13T16:34:08.355 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-09-13T16:34:08.355 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.355 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.355 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.355 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.355 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.355 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.355 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.355 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-13T16:34:08.355 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.355 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.356 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.356 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.356 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-09-13T16:34:08.356 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-09-13T16:34:08.356 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.356 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.356 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.356 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.356 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.356 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.356 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.356 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-13T16:34:08.356 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.356 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.357 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.357 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.357 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.357 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.357 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.357 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.357 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.357 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.357 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.357 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.357 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.357 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.357 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.357 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.357 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.358 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.358 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.358 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.358 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.358 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.358 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.358 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.358 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.358 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-09-13T16:34:08.358 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-09-13T16:34:08.358 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.358 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.358 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.359 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.359 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.359 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.359 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.359 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-13T16:34:08.359 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.359 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.359 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.359 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.359 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.359 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.359 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.359 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.359 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.360 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.360 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.360 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.360 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.360 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-13T16:34:08.360 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.360 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.360 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.360 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.360 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.360 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.360 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.360 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.360 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.361 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.361 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.361 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.361 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.361 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.361 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.361 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.361 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.361 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.361 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.361 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.361 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.361 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.361 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.362 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.362 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.362 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.362 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.362 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.362 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.362 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.362 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.362 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.362 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.362 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.362 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.362 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.362 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.363 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.363 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.363 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-13T16:34:08.363 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-13T16:34:08.363 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.363 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.363 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.363 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.363 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.363 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.363 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.363 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-13T16:34:08.363 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.363 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.363 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.364 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.364 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-13T16:34:08.364 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-13T16:34:08.364 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.364 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.364 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.364 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-09-13T16:34:08.364 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.364 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.364 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.364 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-13T16:34:08.364 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.364 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.365 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.365 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.365 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-09-13T16:34:08.365 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-09-13T16:34:08.365 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.365 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.365 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.365 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-13T16:34:08.365 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.365 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.365 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.365 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-09-13T16:34:08.365 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.365 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.365 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.366 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.366 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-09-13T16:34:08.366 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-09-13T16:34:08.366 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.366 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.366 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.366 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-13T16:34:08.366 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.366 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.366 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.366 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-13T16:34:08.366 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.366 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.366 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.367 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.367 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-09-13T16:34:08.367 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-09-13T16:34:08.367 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.367 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.367 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.367 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.367 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.367 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.367 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.367 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-09-13T16:34:08.367 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.367 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.367 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.368 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.368 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-13T16:34:08.368 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-13T16:34:08.368 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.368 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.368 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.368 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-09-13T16:34:08.368 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.368 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.368 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.368 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-09-13T16:34:08.368 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-09-13T16:34:08.368 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.368 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-09-13T16:34:08.368 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.369 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.369 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.369 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.369 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-09-13T16:34:08.369 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-09-13T16:34:08.369 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.369 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.369 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.369 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-09-13T16:34:08.369 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.369 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.369 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.369 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-13T16:34:08.369 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.370 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.370 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.370 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.370 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-09-13T16:34:08.370 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-09-13T16:34:08.370 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.370 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.370 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.370 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-13T16:34:08.370 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.370 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.370 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.370 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-13T16:34:08.370 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.370 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.371 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.371 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.371 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-09-13T16:34:08.371 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-09-13T16:34:08.371 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.371 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.371 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.371 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.371 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.371 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.371 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.371 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-13T16:34:08.371 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.371 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.372 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.372 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.372 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-09-13T16:34:08.372 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-09-13T16:34:08.372 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.372 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.372 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.372 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.372 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.372 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.372 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.372 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-13T16:34:08.372 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.373 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.373 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.373 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.373 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-09-13T16:34:08.373 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-09-13T16:34:08.373 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.373 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.373 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.373 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.373 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.373 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.373 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.373 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-13T16:34:08.373 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.374 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.374 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.374 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.374 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-09-13T16:34:08.374 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-09-13T16:34:08.374 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.374 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.374 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.374 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.374 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.374 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.374 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.374 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-13T16:34:08.374 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.374 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.375 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.375 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.375 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-09-13T16:34:08.375 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-09-13T16:34:08.375 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.375 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.375 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.375 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.375 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.375 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.375 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.375 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-13T16:34:08.375 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.375 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.376 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.376 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.376 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-09-13T16:34:08.376 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-09-13T16:34:08.376 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.376 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.376 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.376 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-09-13T16:34:08.376 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.376 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.376 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.376 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-09-13T16:34:08.376 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.376 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.377 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.377 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.377 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-09-13T16:34:08.377 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-09-13T16:34:08.377 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.377 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.377 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.377 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.377 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.377 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.377 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.377 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-13T16:34:08.377 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.377 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.378 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.378 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.378 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-09-13T16:34:08.378 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-09-13T16:34:08.378 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.378 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.378 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.378 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.378 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.378 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.378 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.378 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-13T16:34:08.378 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.378 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.378 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.379 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.379 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-09-13T16:34:08.379 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-09-13T16:34:08.379 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.379 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.379 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.379 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.379 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.379 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.379 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.379 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-13T16:34:08.379 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.379 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.379 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.380 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.380 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-09-13T16:34:08.380 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-09-13T16:34:08.380 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.380 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.380 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.380 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.380 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.380 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.380 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.380 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-09-13T16:34:08.380 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.380 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.380 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.381 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.381 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-09-13T16:34:08.381 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-09-13T16:34:08.381 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.381 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.381 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.381 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.381 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.381 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.381 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.381 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-13T16:34:08.381 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.381 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.381 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.382 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.382 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-09-13T16:34:08.382 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-09-13T16:34:08.382 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.382 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.382 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.382 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.382 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.382 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.382 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.382 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-13T16:34:08.382 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.382 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.382 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.383 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.383 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.383 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.383 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.383 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-09-13T16:34:08.383 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.383 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.383 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.383 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.383 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.383 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.383 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.383 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.384 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.384 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.384 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.384 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.384 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.384 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.384 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.384 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.384 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.384 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.384 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.384 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.384 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.384 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.384 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.385 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.385 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.385 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.385 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.385 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.385 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.385 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.385 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.385 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.385 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.385 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.385 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.385 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.385 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.386 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.386 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.386 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.386 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.386 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.386 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.386 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.386 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.386 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.386 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.386 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.386 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.386 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.386 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.387 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.388 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.388 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.388 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.388 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.388 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.388 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.388 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.388 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-09-13T16:34:08.388 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-09-13T16:34:08.388 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:08.388 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.388 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.388 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-09-13T16:34:08.388 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.389 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.389 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.389 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-13T16:34:08.389 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.389 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.389 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.389 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.389 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-09-13T16:34:08.389 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-09-13T16:34:08.389 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:08.389 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.389 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.389 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-09-13T16:34:08.389 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.390 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.390 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.390 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-13T16:34:08.390 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.390 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.390 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.390 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.390 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.390 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.390 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.390 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-09-13T16:34:08.390 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.390 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.390 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.391 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-13T16:34:08.391 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-13T16:34:08.391 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.391 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.391 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.391 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-13T16:34:08.391 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.391 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.391 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.391 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.391 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.391 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.391 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.391 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.392 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-09-13T16:34:08.392 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-13T16:34:08.392 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.392 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.392 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.392 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.392 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.392 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.392 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.392 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.392 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.392 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.392 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.392 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.393 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-13T16:34:08.393 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-13T16:34:08.393 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.393 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.393 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.393 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.393 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.393 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.393 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.393 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.393 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.393 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.393 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.393 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.393 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-09-13T16:34:08.394 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-09-13T16:34:08.394 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.394 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.394 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.394 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.394 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.394 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.394 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.394 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.394 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.394 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.394 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.394 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.394 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-13T16:34:08.395 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-13T16:34:08.395 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.395 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.395 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.395 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.395 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.395 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.395 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.395 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.395 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.395 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.395 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.395 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.395 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-09-13T16:34:08.396 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-13T16:34:08.396 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.396 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.396 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.396 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.396 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.396 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.396 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.396 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.396 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.396 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.396 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.396 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.396 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-13T16:34:08.397 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-13T16:34:08.397 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.397 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.397 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.397 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.397 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.397 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.397 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.397 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.397 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.397 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.397 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.397 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.397 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-13T16:34:08.398 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-13T16:34:08.398 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.398 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.398 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.398 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.398 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.398 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.398 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.398 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.398 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.398 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.398 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.398 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.398 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-13T16:34:08.399 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-13T16:34:08.399 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.399 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.399 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.399 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.399 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.399 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.399 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.399 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.399 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.399 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.399 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.399 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.399 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-13T16:34:08.399 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-13T16:34:08.400 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.400 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.400 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.400 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.400 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.400 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.400 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.400 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.400 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.400 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.400 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.400 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.400 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-13T16:34:08.400 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-13T16:34:08.400 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.401 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.401 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.401 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.401 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.401 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.401 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.401 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.401 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.401 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.401 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.401 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.401 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-13T16:34:08.401 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-13T16:34:08.401 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.402 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.402 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.402 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.402 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.402 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.402 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.402 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.402 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.402 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.402 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.402 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.402 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-13T16:34:08.402 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-13T16:34:08.402 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.403 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.403 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.403 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.403 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.403 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.403 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.403 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.403 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.403 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.403 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.403 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.403 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-13T16:34:08.403 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-13T16:34:08.403 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.403 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.404 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.404 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.404 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.404 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.404 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.404 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.404 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.404 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.404 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.404 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.404 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-09-13T16:34:08.404 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-13T16:34:08.404 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.404 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.405 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.405 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-13T16:34:08.405 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.405 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.405 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.405 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.405 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.405 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.405 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.405 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.405 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-13T16:34:08.405 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-13T16:34:08.405 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.405 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.406 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.406 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.406 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.406 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.406 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.406 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.406 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.406 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.406 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.406 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.406 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-09-13T16:34:08.406 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-09-13T16:34:08.406 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.406 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.406 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.407 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.407 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.407 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.407 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.407 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.407 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.407 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.407 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.407 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.407 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-09-13T16:34:08.407 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-13T16:34:08.407 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.407 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.407 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.408 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-13T16:34:08.408 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.408 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.408 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.408 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.408 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.408 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.408 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.408 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.408 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-13T16:34:08.408 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-13T16:34:08.408 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.408 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.408 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.409 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.409 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.409 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.409 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.409 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.409 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.409 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.409 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.409 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.409 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-13T16:34:08.409 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-13T16:34:08.409 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.409 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.409 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.410 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.410 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.410 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.410 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.410 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.410 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.410 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.410 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.410 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.410 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-13T16:34:08.410 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-13T16:34:08.410 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.410 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.410 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.411 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.411 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.411 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.411 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.411 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.411 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.411 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.411 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.411 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.411 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-13T16:34:08.411 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-13T16:34:08.411 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.411 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.411 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.412 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.412 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.412 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.412 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.412 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.412 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.412 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.412 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.412 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.412 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-09-13T16:34:08.412 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-13T16:34:08.412 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.412 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.412 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.413 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.413 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.413 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.413 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.413 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.413 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.413 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.413 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.413 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.413 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-13T16:34:08.413 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-13T16:34:08.413 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.413 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.413 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.413 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.414 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.414 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.414 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.414 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.414 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.414 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.414 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.414 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.414 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-13T16:34:08.414 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-13T16:34:08.414 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.414 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.414 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.414 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.415 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.415 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.415 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.415 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.415 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.415 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.415 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.415 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.415 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-13T16:34:08.415 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-13T16:34:08.415 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.415 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.415 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.415 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.416 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.416 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.416 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.416 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.416 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.416 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.416 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.416 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.416 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-13T16:34:08.416 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-13T16:34:08.416 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.416 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.416 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.416 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.416 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.417 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.417 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.417 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.417 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.417 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.417 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.417 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.417 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-13T16:34:08.417 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-13T16:34:08.417 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.417 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.417 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.417 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.417 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.417 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.418 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.418 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.418 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.418 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.418 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.418 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.418 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-13T16:34:08.418 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-13T16:34:08.418 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.418 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.418 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.418 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.418 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.418 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.419 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.419 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.419 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.419 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.419 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.419 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.419 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-13T16:34:08.419 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-13T16:34:08.419 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.419 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.419 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.419 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.419 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.419 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.420 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.420 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.420 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.420 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.420 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.420 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.420 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-13T16:34:08.420 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-13T16:34:08.420 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.420 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.420 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.420 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.420 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.420 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.421 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.421 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.421 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.421 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.421 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.421 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.421 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-09-13T16:34:08.421 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-13T16:34:08.421 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.421 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.421 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.421 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.421 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.421 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.422 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.422 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.422 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.422 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.422 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.422 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.422 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-13T16:34:08.422 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-13T16:34:08.422 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.422 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.422 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.422 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-13T16:34:08.422 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.422 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.422 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.423 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.423 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.423 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.423 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.423 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.423 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-13T16:34:08.423 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-13T16:34:08.423 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.423 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.423 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.423 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-13T16:34:08.423 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.423 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.423 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.424 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.424 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.424 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.424 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.424 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.424 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-13T16:34:08.424 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-13T16:34:08.424 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.424 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.424 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.424 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-09-13T16:34:08.424 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.424 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.424 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.425 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.425 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.425 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.425 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.425 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.425 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-13T16:34:08.425 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-13T16:34:08.425 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.425 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.425 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.425 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-09-13T16:34:08.425 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.425 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.425 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.425 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.426 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.426 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.426 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.426 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.426 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-09-13T16:34:08.426 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-13T16:34:08.426 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.426 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.426 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.426 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.426 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.426 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.426 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.426 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.427 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.427 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.427 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.427 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.427 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-13T16:34:08.427 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-13T16:34:08.427 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.427 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.427 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.427 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-13T16:34:08.427 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.427 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.427 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.427 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.428 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.428 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.428 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.428 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.428 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-09-13T16:34:08.428 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-13T16:34:08.428 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.428 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.428 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.428 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.428 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.428 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.428 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.428 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.429 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.429 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.429 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.429 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.429 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-09-13T16:34:08.429 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-13T16:34:08.429 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.429 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.429 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.429 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.429 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.429 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.429 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.429 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.430 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.430 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.430 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.430 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.430 INFO:teuthology.orchestra.run.smithi016.stdout: "UNSAFE_TLS_v1_2": { 2024-09-13T16:34:08.430 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-13T16:34:08.430 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.430 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.430 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.430 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.430 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.430 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.430 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.430 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.430 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.431 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.431 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.431 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.431 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-13T16:34:08.431 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-13T16:34:08.431 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.431 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.431 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.431 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-13T16:34:08.431 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.431 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.431 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.431 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.431 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.432 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.432 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.432 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.432 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-13T16:34:08.432 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-13T16:34:08.432 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.432 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.432 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.432 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-13T16:34:08.432 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.432 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.432 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.432 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.432 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.433 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.433 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.433 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.433 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-13T16:34:08.433 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-13T16:34:08.433 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.433 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.433 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.433 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-13T16:34:08.433 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.433 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.433 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.433 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.433 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.433 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.434 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.434 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.434 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-09-13T16:34:08.434 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-09-13T16:34:08.434 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.434 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.434 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.434 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.434 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.434 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.434 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.434 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.434 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.434 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.435 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.435 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.435 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-09-13T16:34:08.435 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-09-13T16:34:08.435 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.435 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.435 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.435 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.435 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.435 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.435 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.435 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.435 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.435 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.436 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.436 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.436 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-09-13T16:34:08.436 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-09-13T16:34:08.436 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.436 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.436 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.436 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.436 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.436 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.436 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.436 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-09-13T16:34:08.436 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.436 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.437 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.437 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.437 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-09-13T16:34:08.437 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-09-13T16:34:08.437 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.437 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.437 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.437 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-09-13T16:34:08.437 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.437 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.437 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.437 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.437 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.437 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.438 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.438 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.438 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-13T16:34:08.438 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-13T16:34:08.438 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.438 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.438 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.438 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.438 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.438 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.438 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.438 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.438 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.438 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.438 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.439 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.439 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.439 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.439 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.439 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.439 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.439 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.439 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.439 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.439 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.439 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.439 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.439 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.439 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.440 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.440 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.440 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.440 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.440 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.440 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.440 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.440 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.440 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.440 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.440 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.440 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.440 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.440 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.441 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.441 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.441 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.441 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.441 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.441 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.441 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.441 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.441 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.441 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.441 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.441 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.441 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.441 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.442 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.442 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.442 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.442 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.442 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.442 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.442 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.442 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.442 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.442 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.442 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.442 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.442 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.442 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.443 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.443 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.443 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.443 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.443 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.443 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.443 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.443 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.443 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.443 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.443 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.443 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.443 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.443 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.443 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.444 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-09-13T16:34:08.444 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-09-13T16:34:08.444 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.444 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.444 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.444 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.444 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.444 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.444 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.444 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-09-13T16:34:08.444 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.444 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.444 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.444 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.445 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-09-13T16:34:08.445 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-09-13T16:34:08.445 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.445 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.445 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.445 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.445 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.445 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.445 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.445 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.445 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.445 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.445 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.445 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.445 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-13T16:34:08.446 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-13T16:34:08.446 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.446 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.446 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.446 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-13T16:34:08.446 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.446 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.446 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.446 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.446 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.446 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.446 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.446 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.446 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-13T16:34:08.447 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-13T16:34:08.447 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.447 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.447 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.447 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-09-13T16:34:08.447 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.447 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.447 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.447 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.447 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.447 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.447 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.447 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.447 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-13T16:34:08.447 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-13T16:34:08.448 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.448 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.448 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.448 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.448 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.448 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.448 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.448 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.448 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.448 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.448 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.448 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.448 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-09-13T16:34:08.448 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-09-13T16:34:08.449 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.449 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.449 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.449 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-09-13T16:34:08.449 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.449 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.449 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.449 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.449 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.449 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.449 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.449 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.449 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-13T16:34:08.449 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-13T16:34:08.449 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.450 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.450 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.450 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-09-13T16:34:08.450 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.450 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.450 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.450 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.450 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-09-13T16:34:08.450 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.450 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.450 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.450 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.450 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.450 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.451 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-13T16:34:08.451 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-13T16:34:08.451 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.451 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.451 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.451 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-13T16:34:08.451 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-13T16:34:08.451 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-13T16:34:08.451 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.451 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.451 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.451 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.451 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.451 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.452 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-09-13T16:34:08.452 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-09-13T16:34:08.452 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.452 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.452 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.452 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.452 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.452 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.452 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.452 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.452 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.452 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.452 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.452 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.453 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.453 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.453 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.453 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-09-13T16:34:08.453 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.453 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.453 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.453 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-09-13T16:34:08.453 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-09-13T16:34:08.453 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.453 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.453 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.453 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.453 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.454 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.454 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.454 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-09-13T16:34:08.454 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.454 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.454 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.454 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.454 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.454 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.454 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.454 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.454 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.454 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.454 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.454 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.455 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.455 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.455 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.455 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.455 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.455 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.455 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.455 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.455 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.455 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.455 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.455 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.455 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.455 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.456 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.456 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.456 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.456 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.456 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.456 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.456 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.456 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.456 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.456 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.456 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.456 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.456 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.456 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.457 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.457 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.457 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.457 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.457 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.457 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.457 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.457 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.457 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.457 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.457 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.457 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.457 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.457 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.457 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.458 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.458 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.458 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.458 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.458 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.458 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.458 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.458 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.458 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.458 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.458 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.458 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.458 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.458 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.459 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.459 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.459 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.459 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.459 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.459 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-09-13T16:34:08.459 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-09-13T16:34:08.459 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:08.459 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.459 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.459 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-09-13T16:34:08.459 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.459 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.459 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.459 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-13T16:34:08.460 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.460 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.460 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.460 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.460 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-09-13T16:34:08.460 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-09-13T16:34:08.460 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.460 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.460 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.460 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-09-13T16:34:08.460 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.460 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.460 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.460 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-13T16:34:08.461 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.461 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.461 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.461 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.461 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-09-13T16:34:08.461 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-09-13T16:34:08.461 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:08.461 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.461 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.461 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-09-13T16:34:08.461 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.461 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.461 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.461 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-09-13T16:34:08.462 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.462 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.462 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.462 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.462 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-09-13T16:34:08.462 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-09-13T16:34:08.462 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:08.462 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.462 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.462 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-13T16:34:08.462 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.462 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.462 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.462 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-13T16:34:08.463 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.463 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.463 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.463 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.463 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-09-13T16:34:08.463 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-09-13T16:34:08.463 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.463 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.463 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.463 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.463 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.463 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.463 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.463 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-13T16:34:08.463 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.464 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.464 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.464 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.464 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-13T16:34:08.464 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-13T16:34:08.464 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:08.464 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.464 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.464 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-13T16:34:08.464 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.464 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.464 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.464 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-09-13T16:34:08.464 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.465 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.465 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.465 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.465 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-09-13T16:34:08.465 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-09-13T16:34:08.465 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:08.465 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.465 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.465 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-09-13T16:34:08.465 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.465 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.465 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.465 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-13T16:34:08.465 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.465 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.466 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.466 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.466 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.466 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.466 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.466 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-09-13T16:34:08.466 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.466 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.466 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.466 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.466 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.466 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.466 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.466 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.467 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.467 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.467 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.467 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.467 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.467 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.467 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.467 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.467 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.467 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.467 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.467 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.467 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.467 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.468 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.468 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.468 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.468 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.468 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.468 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.468 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.468 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.468 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.468 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.468 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.468 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.468 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.468 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.469 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.469 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.469 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.469 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.469 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.469 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.469 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.469 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.469 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.469 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.469 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.469 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.469 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.469 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.469 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.470 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.470 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.470 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.470 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.470 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.470 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.470 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.470 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.470 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.470 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.470 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.470 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.470 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.470 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.471 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.471 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.471 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.471 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.471 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.471 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.471 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.471 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.471 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-09-13T16:34:08.471 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-09-13T16:34:08.471 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.471 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.471 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.471 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-13T16:34:08.472 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.472 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.472 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.472 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.472 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.472 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.472 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.472 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.472 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-09-13T16:34:08.472 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-09-13T16:34:08.472 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.472 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.472 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.472 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-09-13T16:34:08.473 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.473 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.473 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.473 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.473 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.473 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.473 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.473 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.473 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-13T16:34:08.473 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-13T16:34:08.473 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.473 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.473 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.473 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-13T16:34:08.473 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.474 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.474 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.474 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.474 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.474 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.474 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.474 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.474 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.474 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.474 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.474 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-09-13T16:34:08.474 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-09-13T16:34:08.474 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-09-13T16:34:08.474 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.475 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-09-13T16:34:08.475 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-09-13T16:34:08.475 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.475 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.475 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.475 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-09-13T16:34:08.475 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.475 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.475 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.475 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-13T16:34:08.475 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.475 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.475 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.475 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.476 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-09-13T16:34:08.476 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-09-13T16:34:08.476 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.476 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.476 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.476 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-09-13T16:34:08.476 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.476 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.476 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.476 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-13T16:34:08.476 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.476 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.476 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.476 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.477 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-09-13T16:34:08.477 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-09-13T16:34:08.477 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.477 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.477 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.477 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.477 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.477 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.477 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.477 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-09-13T16:34:08.477 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.477 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.477 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.477 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.478 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-13T16:34:08.478 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-13T16:34:08.478 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:08.478 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.478 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.478 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-09-13T16:34:08.478 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-09-13T16:34:08.478 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.478 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.478 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-13T16:34:08.478 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.478 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.478 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.478 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.479 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.479 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.479 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.479 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.479 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.479 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.479 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.479 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.479 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.479 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.479 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.479 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.479 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.479 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.479 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.480 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.480 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.480 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.480 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.480 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.480 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.480 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.480 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.480 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.480 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.480 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.480 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.480 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.480 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.481 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.481 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.481 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.481 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.481 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.481 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.481 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.481 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.481 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.481 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.481 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.481 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.481 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.481 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.482 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.482 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.482 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.482 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.482 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.482 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.482 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.482 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.482 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.482 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.482 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.482 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.482 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.482 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.482 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.483 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.483 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.483 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.483 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.483 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.483 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.483 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.483 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.483 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.483 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.483 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.483 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.483 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-09-13T16:34:08.484 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-09-13T16:34:08.484 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.484 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.484 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.484 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.484 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.484 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.484 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.484 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-09-13T16:34:08.484 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.484 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.484 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.484 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.485 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-09-13T16:34:08.485 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-09-13T16:34:08.485 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.485 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.485 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.485 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-09-13T16:34:08.485 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.485 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.485 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.485 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-09-13T16:34:08.485 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.485 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.485 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.485 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.486 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-13T16:34:08.486 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-13T16:34:08.486 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.486 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.486 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.486 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-09-13T16:34:08.486 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.486 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.486 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.486 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-13T16:34:08.486 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.486 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.486 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.486 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.487 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-09-13T16:34:08.487 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-09-13T16:34:08.487 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.487 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.487 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.487 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-13T16:34:08.487 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-13T16:34:08.487 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-09-13T16:34:08.487 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.487 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-13T16:34:08.487 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.487 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.487 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.487 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.488 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-09-13T16:34:08.488 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-09-13T16:34:08.488 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.488 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.488 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.488 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.488 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.488 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.488 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.488 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-09-13T16:34:08.488 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.488 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.488 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.488 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.489 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-09-13T16:34:08.489 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-09-13T16:34:08.489 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.489 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.489 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.489 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-09-13T16:34:08.489 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.489 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.489 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.489 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-13T16:34:08.489 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.489 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.489 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.489 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.489 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.490 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.490 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.490 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-09-13T16:34:08.490 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.490 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.490 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.490 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.490 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.490 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.490 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.490 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.490 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.490 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.490 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.491 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.491 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.491 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.491 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.491 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.491 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.491 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.491 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.491 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.491 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.491 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.491 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.491 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.491 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.492 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.492 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.492 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.492 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.492 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.492 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.492 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.492 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.492 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.492 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.492 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.492 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.492 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.492 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.493 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.493 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.493 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.493 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.493 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.493 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.493 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.493 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.493 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.493 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.493 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.493 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.493 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.493 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.494 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.494 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.494 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.494 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.494 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.494 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.494 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.494 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.494 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.494 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.494 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.494 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.494 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.494 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.495 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.495 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.495 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.495 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.495 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.495 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.495 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.495 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.495 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.495 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-09-13T16:34:08.495 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.495 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.495 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.495 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.495 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.496 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.496 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.496 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.496 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.496 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.496 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.496 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.496 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.496 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.496 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.496 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.496 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.496 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.496 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.497 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.497 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.497 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.497 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.497 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.497 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.497 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.497 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.497 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.497 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.497 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.497 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.497 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.497 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.498 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.498 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.498 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.498 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.498 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.498 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.498 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.498 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.498 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.498 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.498 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.498 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.498 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.498 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.498 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.499 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.499 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.499 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.499 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.499 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.499 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.499 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.499 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.499 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.499 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.499 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.499 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.499 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.499 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.500 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.500 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.500 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.500 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.500 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.500 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.500 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.500 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.500 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.500 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.500 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.500 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.500 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.500 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.501 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-09-13T16:34:08.501 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.501 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.501 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.501 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-09-13T16:34:08.501 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-09-13T16:34:08.501 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.501 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.501 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.501 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-09-13T16:34:08.501 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.501 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.501 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.501 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-13T16:34:08.502 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.502 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.502 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.502 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.502 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-09-13T16:34:08.502 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-09-13T16:34:08.502 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.502 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.502 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.502 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-13T16:34:08.502 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-09-13T16:34:08.502 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.502 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.502 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-13T16:34:08.503 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.503 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.503 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.503 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.503 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.503 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.503 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.503 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.503 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.503 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.503 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.503 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.503 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.503 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.503 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.504 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.504 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.504 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.504 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.504 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.504 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.504 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.504 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.504 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.504 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.504 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.504 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.504 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.504 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.505 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.505 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.505 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.505 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.505 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.505 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.505 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.505 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.505 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.505 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.505 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.505 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.505 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.506 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.506 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.506 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.506 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.506 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.506 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.506 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.506 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.506 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.506 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.506 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.506 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.506 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.506 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.506 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.507 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.507 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.507 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.507 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.507 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.507 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.507 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.507 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.507 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.507 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.507 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.507 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.507 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.507 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.508 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.508 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.508 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.508 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.508 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.508 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.508 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-09-13T16:34:08.508 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.508 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.508 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.508 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-09-13T16:34:08.508 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-09-13T16:34:08.508 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.508 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.509 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.509 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-09-13T16:34:08.509 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.509 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.509 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.509 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-09-13T16:34:08.509 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-13T16:34:08.509 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.509 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.509 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.509 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.509 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.509 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.509 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.510 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.510 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.510 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.510 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.510 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.510 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.510 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.510 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.510 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.510 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.510 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.510 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.510 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.510 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.511 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.511 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.511 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.511 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.511 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.511 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.511 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.511 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.511 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.511 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.511 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.511 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.511 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.511 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.512 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.512 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.512 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.512 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.512 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.512 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.512 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.512 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.512 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.512 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.512 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.512 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.512 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.512 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.513 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.513 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.513 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.513 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.513 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.513 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.513 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.513 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.513 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.513 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.513 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.513 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.513 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.513 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.514 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.514 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.514 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.514 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.514 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.514 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.514 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.514 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.514 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.514 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.514 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-09-13T16:34:08.514 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-09-13T16:34:08.514 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.514 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.514 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.515 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.515 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.515 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.515 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.515 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-09-13T16:34:08.515 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-13T16:34:08.515 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.515 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.515 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.515 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-09-13T16:34:08.515 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-09-13T16:34:08.515 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.515 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.515 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.515 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-09-13T16:34:08.516 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.516 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.516 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.516 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-09-13T16:34:08.516 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.516 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.516 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.516 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.516 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-09-13T16:34:08.516 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-09-13T16:34:08.516 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.516 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.516 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.516 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-09-13T16:34:08.517 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.517 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.517 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.517 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-09-13T16:34:08.517 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.517 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.517 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.517 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.517 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-09-13T16:34:08.517 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-09-13T16:34:08.517 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.517 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.517 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.517 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.518 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.518 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.518 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.518 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-09-13T16:34:08.518 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.518 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.518 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.518 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.518 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-09-13T16:34:08.518 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-09-13T16:34:08.518 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.518 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.518 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.518 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-09-13T16:34:08.519 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.519 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.519 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.519 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-13T16:34:08.519 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-13T16:34:08.519 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.519 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.519 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.519 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.519 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.519 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.519 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-09-13T16:34:08.519 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.519 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.519 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.520 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.520 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.520 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.520 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.520 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.520 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.520 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.520 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.520 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.520 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.520 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.520 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.520 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.521 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.521 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.521 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.521 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.521 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.521 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.521 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.521 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.521 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.521 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.521 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.521 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.521 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.521 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.522 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.522 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.522 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.522 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.522 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.522 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.522 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.522 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.522 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.522 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.522 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.522 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.522 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.522 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.523 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.523 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.523 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.523 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.523 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.523 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.523 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.523 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.523 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.523 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.523 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.523 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.523 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.523 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.524 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.524 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.524 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.524 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.524 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.524 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.524 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.524 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.524 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.524 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.524 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.524 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.524 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.524 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.525 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.525 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.525 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.525 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.525 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-09-13T16:34:08.525 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.525 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.525 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.525 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.525 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.525 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.525 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.525 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.525 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.526 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.526 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.526 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.526 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.526 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.526 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.526 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.526 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.526 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.526 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.526 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.526 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.526 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.526 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.527 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.527 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.527 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.527 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.527 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.527 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.527 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.527 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.527 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.527 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.527 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.527 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.527 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.527 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.528 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.528 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.528 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.528 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.528 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.528 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.528 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.528 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.528 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.528 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.528 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.528 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.528 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.528 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.529 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.529 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.529 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.529 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.529 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.529 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.529 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.529 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.529 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.529 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.529 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.529 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.529 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.529 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.530 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.530 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.530 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.530 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.530 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.530 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.530 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.530 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.530 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.530 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.530 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.530 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-09-13T16:34:08.530 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.530 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.530 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.531 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.531 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.531 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.531 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.531 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.531 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.531 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.531 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.531 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.531 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.531 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.531 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.531 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.531 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.532 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.532 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.532 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.532 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.532 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.532 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.532 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.532 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.532 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.532 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.532 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.532 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.532 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.532 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.533 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.533 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.533 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.533 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.533 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.533 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.533 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.533 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.533 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.533 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.533 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.533 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.533 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.533 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.534 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.534 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.534 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.534 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.534 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.534 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.534 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.534 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.534 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.534 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.534 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.534 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.534 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.535 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.535 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.535 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.535 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.535 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.535 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.535 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.535 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.535 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.535 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.535 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.535 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.535 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.535 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.535 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.536 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.536 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.536 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.536 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-13T16:34:08.536 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.536 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.536 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.536 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-09-13T16:34:08.536 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-09-13T16:34:08.536 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.536 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.536 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.536 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.536 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.537 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.537 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.537 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-13T16:34:08.537 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.537 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.537 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.537 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.537 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.537 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.537 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.537 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.537 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.537 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.537 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.537 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.538 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.538 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.538 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.538 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.538 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.538 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.538 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.538 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.538 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.538 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.538 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.538 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.538 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.538 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.539 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.539 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.539 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.539 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.539 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.539 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.539 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.539 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.539 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.539 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.539 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.539 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.539 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.539 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.539 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.540 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.540 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.540 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.540 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.540 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.540 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.540 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.540 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.540 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.540 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.540 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.540 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.540 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.541 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.541 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.541 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.541 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.541 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.541 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.541 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.541 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.541 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.541 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.541 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.541 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.541 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.541 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.542 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.542 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.542 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.542 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.542 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.542 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.542 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-09-13T16:34:08.542 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-13T16:34:08.542 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.542 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.542 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.542 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.542 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.542 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.542 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.543 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-09-13T16:34:08.543 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-09-13T16:34:08.543 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-09-13T16:34:08.543 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.543 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-09-13T16:34:08.543 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.543 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.543 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.543 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.543 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.543 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.543 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.543 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-09-13T16:34:08.543 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.544 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.544 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.544 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.544 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.544 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.544 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.544 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.544 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.544 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.544 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.544 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.544 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.544 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.544 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.545 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.545 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.545 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.545 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.545 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.545 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.545 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.545 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.545 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.545 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.545 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.545 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.545 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.545 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.546 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.546 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.546 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.546 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.546 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.546 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.546 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.546 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.546 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.546 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.546 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.546 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.546 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.546 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.546 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.547 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.547 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.547 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.547 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.547 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.547 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.547 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.547 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.547 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.547 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.547 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.547 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.547 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.548 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.548 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.548 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.548 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.548 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.548 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.548 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.548 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.548 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.548 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.548 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.548 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.548 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.548 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.549 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.549 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.549 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.549 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.549 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.549 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-09-13T16:34:08.549 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.549 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.549 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.549 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.549 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.549 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.549 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.549 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.549 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.550 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.550 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.550 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.550 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.550 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.550 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.550 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.550 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.550 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.550 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.550 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.550 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.550 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.550 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.551 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.551 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.551 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.551 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.551 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.551 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.551 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.551 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.551 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.551 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.551 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.551 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.551 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.551 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.552 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.552 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.552 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.552 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.552 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.552 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.552 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.552 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.552 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.552 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.552 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.552 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.552 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.552 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.553 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.553 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.553 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.553 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.553 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.553 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.553 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.553 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.553 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.553 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.553 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.553 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.553 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.553 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.553 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.554 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.554 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.554 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.554 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.554 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.554 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.554 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.554 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.554 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.554 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.554 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-09-13T16:34:08.554 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.554 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.554 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.555 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.555 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.555 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.555 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.555 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.555 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.555 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.555 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.555 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.555 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.555 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.555 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.555 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.555 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.555 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.556 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.556 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.556 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.556 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.556 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.556 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.556 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.556 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.556 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.556 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.556 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.556 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.556 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.557 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.557 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.557 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.557 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.557 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.557 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.557 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.557 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.557 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.557 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.557 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.557 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.557 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.560 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.560 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.560 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.560 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.560 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.560 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.560 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.560 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.560 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.560 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.561 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.561 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.561 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.561 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.561 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.561 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.561 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.561 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.561 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.561 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.561 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.561 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.561 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.561 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.562 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.562 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.562 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.562 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.562 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.562 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-13T16:34:08.562 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-13T16:34:08.562 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:08.562 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.562 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.562 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-13T16:34:08.562 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.562 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.562 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.562 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.563 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.563 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.563 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.563 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.563 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-09-13T16:34:08.563 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-09-13T16:34:08.563 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-13T16:34:08.563 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.563 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.563 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-13T16:34:08.563 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-09-13T16:34:08.563 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.563 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.563 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-09-13T16:34:08.564 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-13T16:34:08.564 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.564 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.564 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.564 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.564 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.564 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.564 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-09-13T16:34:08.564 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.564 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.564 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.564 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-09-13T16:34:08.564 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-09-13T16:34:08.564 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.564 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.565 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.565 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.565 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.565 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.565 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.565 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-13T16:34:08.565 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.565 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.565 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.565 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.565 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-13T16:34:08.565 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-13T16:34:08.565 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.565 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.565 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.566 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.566 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.566 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.566 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.566 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.566 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.566 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.566 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.566 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.566 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.566 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.566 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.566 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.566 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.567 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.567 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.567 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.567 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.567 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.567 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.567 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.567 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.567 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.567 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.567 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.567 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.567 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.567 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.567 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.568 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.568 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.568 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.568 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.568 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.568 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.568 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.568 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.568 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.568 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.568 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.568 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.568 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.568 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.569 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.569 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.569 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.569 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.569 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.569 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.569 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.569 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.569 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.569 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.569 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.569 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.569 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.569 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.569 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.570 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.570 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.570 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.570 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.570 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.570 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.570 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.570 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.570 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.570 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.570 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.570 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.570 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.570 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.571 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.571 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.571 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.571 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.571 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.571 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.571 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.571 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-09-13T16:34:08.571 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-09-13T16:34:08.571 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.571 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.571 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.571 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-09-13T16:34:08.571 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.572 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.572 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.572 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-09-13T16:34:08.572 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.572 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.572 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.572 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.572 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-13T16:34:08.572 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-13T16:34:08.572 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:08.572 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.572 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.572 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-13T16:34:08.572 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.573 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.573 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.573 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-09-13T16:34:08.573 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.573 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.573 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.573 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.573 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.573 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.573 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.573 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-09-13T16:34:08.573 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.573 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.573 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.574 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-09-13T16:34:08.574 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-09-13T16:34:08.574 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.574 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.574 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.574 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.574 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.574 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.574 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.574 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.574 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.574 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.574 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.574 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.575 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-09-13T16:34:08.575 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-09-13T16:34:08.575 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.575 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.575 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.575 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.575 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.575 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.575 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.575 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-13T16:34:08.575 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-13T16:34:08.575 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.575 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.575 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.576 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.576 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.576 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.576 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.576 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.576 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.576 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.576 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.576 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.576 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.576 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.576 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.576 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.577 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.577 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.577 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.577 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.577 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.577 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.577 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.577 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.577 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.577 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.577 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.577 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.577 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.577 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.578 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.578 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.578 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.578 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.578 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.578 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.578 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.578 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.578 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.578 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.578 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.578 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.578 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.578 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.579 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.579 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.579 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.579 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.579 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.579 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.579 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.579 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.579 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.579 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.579 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.579 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.579 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.579 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.579 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.580 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.580 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.580 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.580 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.580 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.580 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.580 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.580 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.580 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.580 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.580 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.580 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.580 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.580 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.581 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-09-13T16:34:08.581 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-09-13T16:34:08.581 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.581 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.581 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.581 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.581 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.581 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.581 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.581 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.581 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.581 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.581 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.581 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.581 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-13T16:34:08.582 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-13T16:34:08.582 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.582 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.582 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.582 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-09-13T16:34:08.582 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.582 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.582 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.582 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.582 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.582 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.582 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.582 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.582 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-09-13T16:34:08.583 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-09-13T16:34:08.583 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-13T16:34:08.583 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.583 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.583 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-09-13T16:34:08.583 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.583 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.583 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.583 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.583 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.583 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.583 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.583 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.583 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-13T16:34:08.583 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-13T16:34:08.584 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.584 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.584 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.584 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-13T16:34:08.584 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.584 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.584 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.584 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-13T16:34:08.584 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.584 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.584 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.584 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.584 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-13T16:34:08.584 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-13T16:34:08.585 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.585 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.585 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.585 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-09-13T16:34:08.585 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.585 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.585 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.585 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-13T16:34:08.585 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.585 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.585 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.585 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.585 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-09-13T16:34:08.585 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-09-13T16:34:08.586 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.586 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.586 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.586 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-09-13T16:34:08.586 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.586 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.586 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.586 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.586 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.586 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.586 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.586 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.586 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-13T16:34:08.586 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-13T16:34:08.586 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.587 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.587 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.587 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-09-13T16:34:08.587 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.587 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.587 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.587 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-09-13T16:34:08.587 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-09-13T16:34:08.587 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.587 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.587 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.587 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.587 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.587 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.588 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-13T16:34:08.588 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-13T16:34:08.588 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.588 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.588 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.588 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-13T16:34:08.588 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-13T16:34:08.588 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-13T16:34:08.588 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.588 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.588 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.588 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.588 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.588 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.589 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.589 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.589 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.589 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-09-13T16:34:08.589 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.589 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.589 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.589 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.589 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.589 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.589 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.589 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.589 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.589 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.589 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.590 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.590 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.590 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.590 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.590 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.590 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.590 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.590 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.590 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.590 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.590 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.590 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.590 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.590 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.591 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.591 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.591 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.591 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.591 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.591 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.591 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.591 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.591 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.591 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.591 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.591 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.591 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.591 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.592 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.592 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.592 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.592 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.592 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.592 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.592 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.592 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.592 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.592 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.592 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.592 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.592 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.592 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.593 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.593 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.593 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.593 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.593 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.593 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.593 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.593 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.593 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.593 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.593 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.593 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.593 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.593 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.594 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.594 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.594 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.594 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.594 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.594 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.594 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-09-13T16:34:08.594 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-09-13T16:34:08.594 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.594 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.594 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.594 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-13T16:34:08.594 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.594 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.595 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.595 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.595 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.595 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.595 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.595 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.595 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-09-13T16:34:08.595 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-09-13T16:34:08.595 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.595 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.595 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.595 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.595 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.595 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.595 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.596 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.596 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.596 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.596 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.596 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.596 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-09-13T16:34:08.596 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-09-13T16:34:08.596 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.596 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.596 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.596 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.596 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.596 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.596 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.596 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.597 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.597 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.597 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.597 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.597 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.597 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.597 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.597 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-09-13T16:34:08.597 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.597 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.597 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.597 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-09-13T16:34:08.597 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-09-13T16:34:08.597 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.598 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.598 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.598 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.598 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.598 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.598 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.598 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.598 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.598 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.598 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.598 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.598 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-13T16:34:08.598 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-13T16:34:08.598 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.598 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.599 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.599 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.599 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.599 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.599 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.599 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.599 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.599 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.599 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.599 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.599 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.599 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.599 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.599 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.600 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.600 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.600 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.600 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.600 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.600 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.600 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.600 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.600 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.600 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.600 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.600 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.600 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.600 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.601 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.601 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.601 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.601 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.601 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.601 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.601 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.601 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.601 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.601 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.601 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.601 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.601 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.601 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.601 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.602 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.602 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.602 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.602 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.602 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.602 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.602 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.602 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.602 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.602 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.602 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.602 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.602 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.603 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.603 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.603 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.603 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.603 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.603 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.603 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.603 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.603 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.603 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.603 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.603 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.603 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.603 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.604 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.604 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.604 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.604 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.604 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.604 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.604 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.604 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.604 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.604 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.604 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-13T16:34:08.604 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-13T16:34:08.604 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.604 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.605 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.605 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.605 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.605 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.605 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.605 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.605 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.605 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.605 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.605 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.605 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-13T16:34:08.605 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-13T16:34:08.605 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.605 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.605 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.606 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.606 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.606 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.606 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.606 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.606 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.606 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.606 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.606 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.606 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.606 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.606 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.606 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-09-13T16:34:08.606 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.610 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.610 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.610 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.610 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.610 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.610 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.610 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.610 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.610 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.610 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.610 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.610 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.610 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.610 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.611 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.611 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.611 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.611 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.611 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.611 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.611 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.611 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.611 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.611 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.611 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.611 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.611 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.611 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.612 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.612 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.612 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.612 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.612 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.612 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-09-13T16:34:08.612 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.612 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.612 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.612 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.612 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.612 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.612 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.612 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.612 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.613 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.613 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.613 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.613 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.613 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.613 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.613 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.613 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.613 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.613 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.613 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.613 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.613 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.613 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.614 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.614 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.614 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.614 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.614 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.614 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.614 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.614 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.614 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.614 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.614 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.614 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.614 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.614 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.614 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.615 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.615 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.615 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.615 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.615 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.615 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.615 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.615 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.615 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.615 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.615 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.615 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.615 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.615 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.615 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.616 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.616 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.616 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.616 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.616 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.616 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.616 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.616 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.616 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.616 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.616 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.616 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.616 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.616 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.617 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.617 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.617 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.617 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.617 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.617 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.617 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-09-13T16:34:08.617 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-09-13T16:34:08.617 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.617 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.617 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.617 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-09-13T16:34:08.617 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.617 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.617 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.618 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-13T16:34:08.618 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.618 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.618 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.618 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.618 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.618 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.618 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.618 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-09-13T16:34:08.618 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.618 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.618 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.618 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.618 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.618 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.619 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.619 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.619 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.619 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.619 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.619 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.619 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.619 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.619 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.619 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.619 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.619 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.619 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.620 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.620 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.620 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.620 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.620 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.620 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.620 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.620 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.620 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.620 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.620 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.620 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.620 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.620 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.621 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.621 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.621 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.621 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.621 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.621 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.621 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.621 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.621 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.621 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.621 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.621 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.621 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.621 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.622 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.622 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.622 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.622 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.622 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.622 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.622 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.622 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.622 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.622 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.622 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.622 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.622 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.622 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.622 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.623 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.623 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.623 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.623 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.623 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.623 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.623 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.623 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.623 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.623 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.623 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.623 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-09-13T16:34:08.623 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-09-13T16:34:08.623 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.624 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.624 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.624 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.624 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.624 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.624 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.624 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.624 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.624 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.624 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.624 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.624 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-09-13T16:34:08.624 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-09-13T16:34:08.624 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.625 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.625 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.625 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-09-13T16:34:08.625 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.625 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.625 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.625 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.625 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.625 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.625 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.625 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.625 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-09-13T16:34:08.625 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-09-13T16:34:08.625 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.626 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.626 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.626 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.626 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.626 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.626 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.626 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.626 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.626 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.626 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.626 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.626 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-09-13T16:34:08.626 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-09-13T16:34:08.626 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.627 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.627 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.627 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.627 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.627 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.627 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.627 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.627 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.627 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.627 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.627 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.627 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-09-13T16:34:08.627 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-09-13T16:34:08.627 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-13T16:34:08.628 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.628 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.628 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.628 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.628 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.628 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.628 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.628 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.628 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.628 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.628 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.628 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-09-13T16:34:08.628 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-09-13T16:34:08.628 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.629 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.629 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.629 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.629 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.629 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.629 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.629 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.629 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.629 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.629 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.629 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.629 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-09-13T16:34:08.629 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-09-13T16:34:08.629 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.630 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.630 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.630 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.630 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.630 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.630 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.630 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.630 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.630 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.630 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.630 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.630 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-09-13T16:34:08.630 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-09-13T16:34:08.630 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.630 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.631 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.631 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.631 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.631 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.631 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.631 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.631 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.631 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.631 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.631 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.631 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-09-13T16:34:08.631 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-09-13T16:34:08.631 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.631 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.631 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.632 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.632 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-13T16:34:08.632 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-09-13T16:34:08.632 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.632 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.632 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.632 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.632 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.632 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.632 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-09-13T16:34:08.632 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-09-13T16:34:08.632 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.632 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.632 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.633 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.633 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.633 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.633 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.633 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.633 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.633 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.633 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.633 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.633 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-09-13T16:34:08.633 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-09-13T16:34:08.633 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.633 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.633 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.633 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.634 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.634 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.634 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.634 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.634 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.634 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.634 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.634 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.634 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-09-13T16:34:08.634 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-09-13T16:34:08.634 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.634 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.634 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.634 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.635 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.635 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.635 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.635 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.635 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.635 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.635 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.635 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.635 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.635 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.635 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.635 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-09-13T16:34:08.635 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.635 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.635 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.636 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-09-13T16:34:08.636 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-09-13T16:34:08.636 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.636 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.636 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.636 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.636 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.636 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.636 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.636 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-09-13T16:34:08.636 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.636 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.636 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.637 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.637 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-09-13T16:34:08.637 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-09-13T16:34:08.637 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.637 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.637 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.637 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.637 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.637 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.637 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.637 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-09-13T16:34:08.637 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.637 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.637 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.638 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.638 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.638 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.638 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.638 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.638 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.638 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.638 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.638 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.638 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.638 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.638 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.638 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.638 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.639 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.639 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.639 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.639 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.639 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.639 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.639 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.639 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.639 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.639 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.639 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.639 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.639 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.639 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.640 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.640 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.640 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.640 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.640 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.640 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.640 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.640 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.640 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.640 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.640 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.640 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.640 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.640 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.641 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.641 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.641 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.641 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.641 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.641 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.641 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.641 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.641 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.641 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.641 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.641 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.641 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.641 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.642 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.642 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.642 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.642 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.642 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.642 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.642 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.642 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.642 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.642 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.642 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.642 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.642 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.643 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.643 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.643 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.643 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.643 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.643 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-09-13T16:34:08.643 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.643 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.643 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.643 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.643 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.643 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.643 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.643 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.643 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.644 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.644 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.644 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.644 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.644 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.644 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.644 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.644 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.644 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.644 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.644 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.644 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.644 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.644 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.645 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.645 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.645 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.645 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.645 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.645 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.645 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.645 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.645 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.645 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.645 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.645 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.645 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.645 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.646 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.646 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.646 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.646 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.646 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.646 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.646 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.646 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.646 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.646 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.646 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.646 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.646 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.646 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.647 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.647 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.647 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.647 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.647 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.647 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.647 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.647 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.647 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.647 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.647 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.647 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.647 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.647 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.648 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.648 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.648 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.648 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.648 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.648 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.648 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.648 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.648 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.648 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.648 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.648 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-09-13T16:34:08.648 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.648 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.648 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.649 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.649 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.649 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.649 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.649 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.649 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.649 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.649 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.649 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.649 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.649 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.649 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.649 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.649 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.650 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.650 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.650 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.650 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.650 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.650 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.650 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.650 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.650 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.650 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.650 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.650 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.650 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.650 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.651 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.651 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.651 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.651 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.651 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.651 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.651 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.651 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.651 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.651 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.651 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.651 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.651 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.651 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.652 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.652 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.652 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.652 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.652 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.652 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.652 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.652 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.652 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.652 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.652 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.652 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.652 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.652 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.653 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.653 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.653 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.653 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.653 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.653 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.653 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.653 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.653 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.653 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.653 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.653 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.653 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.653 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.653 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.654 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.654 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.654 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-09-13T16:34:08.654 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.654 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.654 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.654 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-09-13T16:34:08.654 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-09-13T16:34:08.654 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.654 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.654 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.654 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-13T16:34:08.654 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.654 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.655 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.655 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.655 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.655 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.655 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.655 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.655 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-13T16:34:08.655 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-13T16:34:08.655 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:08.655 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.655 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.655 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-09-13T16:34:08.655 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.655 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.656 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.656 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.656 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.656 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.656 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.656 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.656 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.656 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.656 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.656 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.656 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.656 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.656 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.656 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.657 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.657 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.657 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.657 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.657 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.657 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.657 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.657 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.657 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.657 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.657 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.657 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.657 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.657 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.658 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.658 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.658 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.658 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.658 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.658 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.658 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.658 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.658 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.658 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.658 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.658 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.658 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.658 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.659 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.659 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.659 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.659 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.659 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.659 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.659 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.659 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.659 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.659 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.659 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.659 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.659 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.659 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.660 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.660 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.660 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.660 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.660 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.660 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.660 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.660 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.660 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.660 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.660 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.660 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.660 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.660 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.661 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.661 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.661 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.661 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.661 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.661 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.661 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.661 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.661 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.661 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-09-13T16:34:08.661 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.661 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.661 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.661 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-09-13T16:34:08.661 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-09-13T16:34:08.662 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.662 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.662 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.662 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.662 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.662 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.662 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.662 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-13T16:34:08.662 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.662 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.662 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.662 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.662 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-09-13T16:34:08.662 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-09-13T16:34:08.663 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.663 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.663 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.663 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.663 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.663 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.663 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.663 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-13T16:34:08.663 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.663 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.663 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.663 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.663 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-09-13T16:34:08.663 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-09-13T16:34:08.664 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.664 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.664 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.664 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.664 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.664 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.664 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.664 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-13T16:34:08.664 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.664 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.664 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.664 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.664 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-09-13T16:34:08.664 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-09-13T16:34:08.665 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.665 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.665 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.665 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.665 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.665 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.665 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.665 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-13T16:34:08.665 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.665 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.665 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.665 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.665 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-09-13T16:34:08.665 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-09-13T16:34:08.665 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.666 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.666 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.666 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.666 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.666 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.666 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.666 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-13T16:34:08.666 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.666 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.666 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.666 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.666 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-09-13T16:34:08.666 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-09-13T16:34:08.666 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.667 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.667 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.667 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.667 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.667 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.667 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.667 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.667 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.667 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.667 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.667 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.667 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-09-13T16:34:08.667 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-09-13T16:34:08.667 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.667 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.668 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.668 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.668 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.668 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.668 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.668 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.668 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.668 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.668 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.668 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.668 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-09-13T16:34:08.668 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-09-13T16:34:08.668 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.668 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.669 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.669 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-13T16:34:08.669 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.669 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.669 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.669 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.669 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.669 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.669 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.669 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.669 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-13T16:34:08.669 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-13T16:34:08.669 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.669 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.669 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.670 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.670 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.670 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.670 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.670 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.670 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.670 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.670 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.670 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.670 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-13T16:34:08.670 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-13T16:34:08.670 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.670 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.671 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.671 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-09-13T16:34:08.671 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-09-13T16:34:08.671 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.671 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.671 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.671 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.671 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.671 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.671 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.671 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-09-13T16:34:08.671 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-09-13T16:34:08.671 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.671 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.672 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.672 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-09-13T16:34:08.672 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.672 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.672 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.672 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.672 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.672 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.672 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.672 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.672 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-09-13T16:34:08.672 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-09-13T16:34:08.672 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.672 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.673 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.673 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.673 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.673 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.673 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.673 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.673 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.673 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.673 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.673 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.673 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-09-13T16:34:08.673 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-09-13T16:34:08.673 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.673 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.674 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.674 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.674 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.674 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.674 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.674 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.674 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.674 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.674 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.674 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.674 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.674 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.674 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.674 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.675 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.675 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.675 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.675 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.675 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.675 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.675 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.675 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.675 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.675 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.675 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.675 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.675 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.675 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.675 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.676 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.676 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.676 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.676 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.676 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.676 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.676 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.676 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.676 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.676 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.676 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.676 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.676 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.677 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.677 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.677 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.677 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.677 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.677 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.677 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.677 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.677 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.677 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.677 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.677 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.677 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.677 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.678 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.678 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.678 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.678 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.678 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.678 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.678 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.678 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.678 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.678 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.678 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.678 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.678 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.678 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.679 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.679 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.679 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.679 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.679 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.679 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.679 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.679 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.679 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.679 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-09-13T16:34:08.679 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-09-13T16:34:08.679 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.679 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.680 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.680 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.680 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.680 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.680 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.680 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.680 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.680 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.680 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.680 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-09-13T16:34:08.680 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-09-13T16:34:08.680 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.680 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-09-13T16:34:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-09-13T16:34:08.681 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-13T16:34:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-09-13T16:34:08.683 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.683 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.683 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.683 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.683 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.683 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.683 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.683 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.683 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.683 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.683 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.683 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.683 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.683 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.684 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.685 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.686 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.687 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.687 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.687 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.687 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.687 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.687 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.687 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.687 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.687 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.687 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.687 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.687 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.687 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.687 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-09-13T16:34:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.688 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.689 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.689 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.689 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.689 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.689 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.689 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.689 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.689 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.689 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.689 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.689 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.689 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.689 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.689 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.690 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.690 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.690 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.690 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.690 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.690 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.690 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.690 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.690 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.690 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.690 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.690 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.690 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.691 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.691 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.691 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.691 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.691 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.691 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.691 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.691 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.691 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.691 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.691 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.691 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.691 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.691 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.691 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.692 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.692 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.692 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.692 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.692 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.692 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.692 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.692 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.692 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.692 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.692 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.692 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.692 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.692 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.693 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.693 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.693 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.693 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.693 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.693 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.693 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.693 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.693 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-09-13T16:34:08.693 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-09-13T16:34:08.693 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.693 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.693 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.694 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-09-13T16:34:08.694 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.694 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.694 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.694 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-13T16:34:08.694 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.694 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.694 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.694 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.694 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-09-13T16:34:08.694 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-09-13T16:34:08.694 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.694 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.694 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.695 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-13T16:34:08.695 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.695 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.695 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.695 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-13T16:34:08.695 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.695 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.695 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.695 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.695 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-09-13T16:34:08.695 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-09-13T16:34:08.695 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.695 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.695 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.696 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-13T16:34:08.696 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.696 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.696 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.696 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-13T16:34:08.696 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.696 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.696 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.696 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.696 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.696 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.696 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.696 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-09-13T16:34:08.696 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-13T16:34:08.697 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-13T16:34:08.697 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-13T16:34:08.697 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-09-13T16:34:08.697 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-09-13T16:34:08.697 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-13T16:34:08.697 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.697 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.697 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-09-13T16:34:08.697 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.697 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.697 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.697 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.697 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.697 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.697 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.698 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.698 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-09-13T16:34:08.698 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-09-13T16:34:08.698 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.698 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.698 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.698 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.698 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.698 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.698 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.698 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.698 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.698 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.698 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.699 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.699 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-13T16:34:08.699 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-13T16:34:08.699 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-13T16:34:08.699 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.699 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.699 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-13T16:34:08.699 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.699 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.699 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.699 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.699 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.699 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.699 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.700 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.700 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-13T16:34:08.700 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-13T16:34:08.700 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.700 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.700 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.700 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.700 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.700 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.700 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.700 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.700 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.700 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.700 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.700 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.701 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.701 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.701 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.701 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.701 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.701 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.701 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.701 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-13T16:34:08.701 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-13T16:34:08.701 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.701 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.701 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.701 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.701 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.702 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.702 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.702 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.702 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.702 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.702 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.702 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.702 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-13T16:34:08.702 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-13T16:34:08.702 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.702 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.702 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.702 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-13T16:34:08.702 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.703 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.703 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-13T16:34:08.703 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-13T16:34:08.703 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-13T16:34:08.703 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-13T16:34:08.703 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-13T16:34:08.703 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-13T16:34:08.703 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-13T16:34:08.703 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.703 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.703 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.703 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.703 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.703 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.704 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-13T16:34:08.704 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-13T16:34:08.704 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-13T16:34:08.704 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.704 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-13T16:34:08.704 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-13T16:34:08.704 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.704 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.704 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.704 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.704 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.704 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.704 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.704 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.705 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-09-13T16:34:08.705 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-09-13T16:34:08.705 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.705 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.705 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.705 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-13T16:34:08.705 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.705 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.705 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.705 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.705 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.705 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.705 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.705 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.705 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-09-13T16:34:08.706 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-09-13T16:34:08.706 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-13T16:34:08.706 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.706 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.706 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-09-13T16:34:08.706 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.706 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.706 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.706 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.706 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.706 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.706 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.706 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.706 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-09-13T16:34:08.707 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-09-13T16:34:08.707 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-13T16:34:08.707 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-13T16:34:08.707 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-13T16:34:08.707 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-13T16:34:08.707 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-13T16:34:08.707 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-13T16:34:08.707 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-13T16:34:08.707 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-13T16:34:08.707 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-13T16:34:08.707 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-13T16:34:08.707 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-13T16:34:08.707 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.707 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.707 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.708 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.708 INFO:teuthology.orchestra.run.smithi016.stdout: "services": { 2024-09-13T16:34:08.708 INFO:teuthology.orchestra.run.smithi016.stdout: "dashboard": "https://172.21.15.16:8443/" 2024-09-13T16:34:08.708 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.708 INFO:teuthology.orchestra.run.smithi016.stdout: "always_on_modules": { 2024-09-13T16:34:08.708 INFO:teuthology.orchestra.run.smithi016.stdout: "octopus": [ 2024-09-13T16:34:08.708 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-13T16:34:08.708 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-13T16:34:08.708 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-13T16:34:08.708 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-13T16:34:08.708 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-13T16:34:08.708 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-13T16:34:08.708 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-13T16:34:08.708 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-13T16:34:08.709 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-13T16:34:08.709 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-13T16:34:08.709 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.709 INFO:teuthology.orchestra.run.smithi016.stdout: "pacific": [ 2024-09-13T16:34:08.709 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-13T16:34:08.709 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-13T16:34:08.709 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-13T16:34:08.709 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-13T16:34:08.709 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-13T16:34:08.709 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-13T16:34:08.709 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-13T16:34:08.709 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-13T16:34:08.709 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-13T16:34:08.709 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-13T16:34:08.710 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.710 INFO:teuthology.orchestra.run.smithi016.stdout: "quincy": [ 2024-09-13T16:34:08.710 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-13T16:34:08.710 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-13T16:34:08.710 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-13T16:34:08.710 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-13T16:34:08.710 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-13T16:34:08.710 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-13T16:34:08.710 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-13T16:34:08.710 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-13T16:34:08.710 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-13T16:34:08.710 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-13T16:34:08.710 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-13T16:34:08.710 INFO:teuthology.orchestra.run.smithi016.stdout: "reef": [ 2024-09-13T16:34:08.711 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-13T16:34:08.711 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-13T16:34:08.711 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-13T16:34:08.711 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-13T16:34:08.711 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-13T16:34:08.711 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-13T16:34:08.711 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-13T16:34:08.711 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-13T16:34:08.711 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-13T16:34:08.711 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-13T16:34:08.711 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-13T16:34:08.711 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.711 INFO:teuthology.orchestra.run.smithi016.stdout: "last_failure_osd_epoch": 3, 2024-09-13T16:34:08.711 INFO:teuthology.orchestra.run.smithi016.stdout: "active_clients": [ 2024-09-13T16:34:08.711 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.712 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-09-13T16:34:08.712 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-13T16:34:08.712 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.712 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-13T16:34:08.712 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-13T16:34:08.712 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3937301827 2024-09-13T16:34:08.712 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.712 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-13T16:34:08.712 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.712 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.712 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "libcephsqlite", 2024-09-13T16:34:08.712 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-13T16:34:08.712 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.712 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-13T16:34:08.713 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-13T16:34:08.713 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2872798669 2024-09-13T16:34:08.713 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.713 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-13T16:34:08.713 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.713 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.713 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-09-13T16:34:08.713 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-13T16:34:08.713 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.713 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-13T16:34:08.713 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-13T16:34:08.713 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3459043361 2024-09-13T16:34:08.713 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.713 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-13T16:34:08.713 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:08.714 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.714 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-09-13T16:34:08.714 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-13T16:34:08.714 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-13T16:34:08.714 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-13T16:34:08.714 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-13T16:34:08.714 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2780553042 2024-09-13T16:34:08.714 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.714 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-13T16:34:08.714 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:08.714 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-13T16:34:08.714 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-13T16:34:08.716 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:07 smithi107 ceph-mon[34139]: pgmap v444: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-13T16:34:08.716 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:07 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/2792904859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T16:34:08.716 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:07 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/2447463342' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-13T16:34:08.716 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:06 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/995015273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:34:08.716 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:06 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3288282456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T16:34:08.716 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:06 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1325234211' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T16:34:08.717 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:07 smithi111 ceph-mon[34600]: pgmap v444: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-13T16:34:08.717 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:07 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/2792904859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T16:34:08.717 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:07 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/2447463342' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-13T16:34:08.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.403+0000 7ff605f1f640 1 -- 172.21.15.16:0/2447463342 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff5e80a1090 msgr2=0x7ff5e80a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:08.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.403+0000 7ff605f1f640 1 --2- 172.21.15.16:0/2447463342 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff5e80a1090 0x7ff5e80a3550 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff6000ffd20 tx=0x7ff5f0005e70 comp rx=0 tx=0).stop 2024-09-13T16:34:08.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.403+0000 7ff605f1f640 1 -- 172.21.15.16:0/2447463342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6000ffec0 msgr2=0x7ff600107bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:08.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.403+0000 7ff605f1f640 1 --2- 172.21.15.16:0/2447463342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6000ffec0 0x7ff600107bd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4005ab0 tx=0x7ff5f40052b0 comp rx=0 tx=0).stop 2024-09-13T16:34:08.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.403+0000 7ff605f1f640 1 -- 172.21.15.16:0/2447463342 shutdown_connections 2024-09-13T16:34:08.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.403+0000 7ff605f1f640 1 --2- 172.21.15.16:0/2447463342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff600100880 0x7ff600108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:08.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.403+0000 7ff605f1f640 1 --2- 172.21.15.16:0/2447463342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6000ffec0 0x7ff600107bd0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:08.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.403+0000 7ff605f1f640 1 --2- 172.21.15.16:0/2447463342 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7ff5e80a1090 0x7ff5e80a3550 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:08.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.403+0000 7ff605f1f640 1 --2- 172.21.15.16:0/2447463342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6000fecd0 0x7ff600109590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:08.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.403+0000 7ff605f1f640 1 -- 172.21.15.16:0/2447463342 >> 172.21.15.16:0/2447463342 conn(0x7ff6000fa430 msgr2=0x7ff6001033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:08.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.404+0000 7ff605f1f640 1 -- 172.21.15.16:0/2447463342 shutdown_connections 2024-09-13T16:34:08.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:07.404+0000 7ff605f1f640 1 -- 172.21.15.16:0/2447463342 wait complete. 2024-09-13T16:34:08.724 DEBUG:tasks.check_counter:Getting stats from a 2024-09-13T16:34:08.725 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-13T16:34:08.725 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:07 smithi016 ceph-mon[31187]: pgmap v444: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-13T16:34:08.725 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:07 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2792904859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T16:34:08.725 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:07 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2447463342' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-13T16:34:08.928 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:34:10.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:09 smithi107 ceph-mon[34139]: pgmap v445: 129 pgs: 129 active+clean; 7.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.4 KiB/s wr, 70 op/s 2024-09-13T16:34:10.352 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:09 smithi016 ceph-mon[31187]: pgmap v445: 129 pgs: 129 active+clean; 7.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.4 KiB/s wr, 70 op/s 2024-09-13T16:34:10.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:09 smithi111 ceph-mon[34600]: pgmap v445: 129 pgs: 129 active+clean; 7.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.4 KiB/s wr, 70 op/s 2024-09-13T16:34:10.953 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-13T16:34:10.954 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T16:34:10.954 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 415, 2024-09-13T16:34:10.954 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 407, 2024-09-13T16:34:10.954 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 836784, 2024-09-13T16:34:10.954 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 148752, 2024-09-13T16:34:10.954 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-09-13T16:34:10.954 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-09-13T16:34:10.954 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.053450063, 2024-09-13T16:34:10.954 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.018432605, 2024-09-13T16:34:10.954 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.025851933, 2024-09-13T16:34:10.954 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.009991716, 2024-09-13T16:34:10.954 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:10.955 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 407, 2024-09-13T16:34:10.955 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.023357788, 2024-09-13T16:34:10.955 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000057390 2024-09-13T16:34:10.955 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.955 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:10.955 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.955 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.955 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.955 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.955 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 734640, 2024-09-13T16:34:10.955 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 54208 2024-09-13T16:34:10.955 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.955 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T16:34:10.955 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 493, 2024-09-13T16:34:10.956 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 563, 2024-09-13T16:34:10.956 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 127154, 2024-09-13T16:34:10.956 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 347277, 2024-09-13T16:34:10.956 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-09-13T16:34:10.956 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 4, 2024-09-13T16:34:10.956 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.043708360, 2024-09-13T16:34:10.956 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.015013699, 2024-09-13T16:34:10.956 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.011443684, 2024-09-13T16:34:10.956 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.013594566, 2024-09-13T16:34:10.956 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:10.956 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 561, 2024-09-13T16:34:10.956 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.022473549, 2024-09-13T16:34:10.956 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000040059 2024-09-13T16:34:10.956 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.957 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:10.957 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.957 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.957 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.957 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.957 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-13T16:34:10.957 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 194208 2024-09-13T16:34:10.957 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.957 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T16:34:10.957 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 4, 2024-09-13T16:34:10.957 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 6, 2024-09-13T16:34:10.957 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1629, 2024-09-13T16:34:10.958 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 1256, 2024-09-13T16:34:10.958 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-09-13T16:34:10.958 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 4, 2024-09-13T16:34:10.958 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.001630779, 2024-09-13T16:34:10.958 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000071813, 2024-09-13T16:34:10.958 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.001363243, 2024-09-13T16:34:10.958 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000092465, 2024-09-13T16:34:10.958 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:10.958 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 4, 2024-09-13T16:34:10.958 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.003157649, 2024-09-13T16:34:10.958 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000789412 2024-09-13T16:34:10.958 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.958 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:10.958 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.959 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.959 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.959 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.959 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-13T16:34:10.959 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-09-13T16:34:10.959 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.959 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-09-13T16:34:10.959 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 1, 2024-09-13T16:34:10.959 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-09-13T16:34:10.959 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.959 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-MDSRank": { 2024-09-13T16:34:10.959 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-09-13T16:34:10.959 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-09-13T16:34:10.960 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1396, 2024-09-13T16:34:10.960 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.039344575, 2024-09-13T16:34:10.960 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000028183 2024-09-13T16:34:10.960 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:10.960 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.960 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-PurgeQueue": { 2024-09-13T16:34:10.960 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-09-13T16:34:10.960 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-09-13T16:34:10.960 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.960 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.960 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.960 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:10.960 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.961 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-09-13T16:34:10.961 INFO:teuthology.orchestra.run.smithi016.stdout: "request": 0, 2024-09-13T16:34:10.961 INFO:teuthology.orchestra.run.smithi016.stdout: "reply": 0, 2024-09-13T16:34:10.961 INFO:teuthology.orchestra.run.smithi016.stdout: "reply_latency": { 2024-09-13T16:34:10.961 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.961 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.961 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.961 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.961 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_reply": 0, 2024-09-13T16:34:10.961 INFO:teuthology.orchestra.run.smithi016.stdout: "forward": 0, 2024-09-13T16:34:10.961 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_complete": 0, 2024-09-13T16:34:10.961 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_keys": 0, 2024-09-13T16:34:10.961 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_commit": 0, 2024-09-13T16:34:10.961 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_split": 0, 2024-09-13T16:34:10.962 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_merge": 0, 2024-09-13T16:34:10.962 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes": 0, 2024-09-13T16:34:10.962 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_top": 0, 2024-09-13T16:34:10.962 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_bottom": 0, 2024-09-13T16:34:10.962 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pin_tail": 0, 2024-09-13T16:34:10.962 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pinned": 0, 2024-09-13T16:34:10.962 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_expired": 0, 2024-09-13T16:34:10.962 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_with_caps": 0, 2024-09-13T16:34:10.962 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": 0, 2024-09-13T16:34:10.962 INFO:teuthology.orchestra.run.smithi016.stdout: "subtrees": 1, 2024-09-13T16:34:10.962 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse": 0, 2024-09-13T16:34:10.962 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_hit": 0, 2024-09-13T16:34:10.962 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_forward": 0, 2024-09-13T16:34:10.963 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_discover": 0, 2024-09-13T16:34:10.963 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_dir_fetch": 0, 2024-09-13T16:34:10.963 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_remote_ino": 0, 2024-09-13T16:34:10.963 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_lock": 0, 2024-09-13T16:34:10.963 INFO:teuthology.orchestra.run.smithi016.stdout: "load_cent": 0, 2024-09-13T16:34:10.963 INFO:teuthology.orchestra.run.smithi016.stdout: "q": 0, 2024-09-13T16:34:10.963 INFO:teuthology.orchestra.run.smithi016.stdout: "exported": 0, 2024-09-13T16:34:10.963 INFO:teuthology.orchestra.run.smithi016.stdout: "exported_inodes": 0, 2024-09-13T16:34:10.963 INFO:teuthology.orchestra.run.smithi016.stdout: "imported": 0, 2024-09-13T16:34:10.963 INFO:teuthology.orchestra.run.smithi016.stdout: "imported_inodes": 0, 2024-09-13T16:34:10.963 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_dir_fetch": 0, 2024-09-13T16:34:10.963 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_backtrace_fetch": 0, 2024-09-13T16:34:10.963 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_peer_discover": 0, 2024-09-13T16:34:10.963 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rfiles": 0, 2024-09-13T16:34:10.964 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rbytes": 0, 2024-09-13T16:34:10.964 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rsnaps": 0, 2024-09-13T16:34:10.964 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_fetch": 0, 2024-09-13T16:34:10.964 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_set_tag": 0, 2024-09-13T16:34:10.964 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_repaired": 0, 2024-09-13T16:34:10.964 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_inotable_repaired": 0, 2024-09-13T16:34:10.964 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_inodes": 0, 2024-09-13T16:34:10.964 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_base_inodes": 0, 2024-09-13T16:34:10.964 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dirfrag_rstats": 0, 2024-09-13T16:34:10.964 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_file_inodes": 0, 2024-09-13T16:34:10.964 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_inode_file_caps": 0, 2024-09-13T16:34:10.964 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_revoke": 0, 2024-09-13T16:34:10.964 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_grant": 0, 2024-09-13T16:34:10.964 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_trunc": 0, 2024-09-13T16:34:10.964 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-13T16:34:10.965 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-13T16:34:10.965 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps": 0, 2024-09-13T16:34:10.965 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps_dirty": 0, 2024-09-13T16:34:10.965 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_cap_release": 0, 2024-09-13T16:34:10.965 INFO:teuthology.orchestra.run.smithi016.stdout: "process_request_cap_release": 0 2024-09-13T16:34:10.965 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.965 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_cache": { 2024-09-13T16:34:10.965 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update": 0, 2024-09-13T16:34:10.965 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update_receipt": 0, 2024-09-13T16:34:10.965 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_try_discover": 0, 2024-09-13T16:34:10.965 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_send_discover": 0, 2024-09-13T16:34:10.965 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_handle_discover": 0, 2024-09-13T16:34:10.965 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays": 0, 2024-09-13T16:34:10.966 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_delayed": 0, 2024-09-13T16:34:10.966 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_enqueuing": 0, 2024-09-13T16:34:10.966 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_created": 0, 2024-09-13T16:34:10.966 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_enqueued": 0, 2024-09-13T16:34:10.966 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_reintegrated": 0, 2024-09-13T16:34:10.966 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_migrated": 0, 2024-09-13T16:34:10.966 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_processing": 0, 2024-09-13T16:34:10.966 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_enqueued": 0, 2024-09-13T16:34:10.966 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_prioritized": 0, 2024-09-13T16:34:10.966 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_started": 0, 2024-09-13T16:34:10.966 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_completed": 0, 2024-09-13T16:34:10.966 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_enqueue_scrub": 0, 2024-09-13T16:34:10.966 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_exportdir": 0, 2024-09-13T16:34:10.966 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_flush": 0, 2024-09-13T16:34:10.967 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragmentdir": 0, 2024-09-13T16:34:10.967 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragstats": 0, 2024-09-13T16:34:10.967 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_inodestats": 0 2024-09-13T16:34:10.967 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.967 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_log": { 2024-09-13T16:34:10.967 INFO:teuthology.orchestra.run.smithi016.stdout: "evadd": 0, 2024-09-13T16:34:10.967 INFO:teuthology.orchestra.run.smithi016.stdout: "evex": 0, 2024-09-13T16:34:10.967 INFO:teuthology.orchestra.run.smithi016.stdout: "evtrm": 0, 2024-09-13T16:34:10.967 INFO:teuthology.orchestra.run.smithi016.stdout: "ev": 2, 2024-09-13T16:34:10.967 INFO:teuthology.orchestra.run.smithi016.stdout: "evexg": 0, 2024-09-13T16:34:10.967 INFO:teuthology.orchestra.run.smithi016.stdout: "evexd": 0, 2024-09-13T16:34:10.967 INFO:teuthology.orchestra.run.smithi016.stdout: "segadd": 0, 2024-09-13T16:34:10.967 INFO:teuthology.orchestra.run.smithi016.stdout: "segex": 0, 2024-09-13T16:34:10.967 INFO:teuthology.orchestra.run.smithi016.stdout: "segtrm": 0, 2024-09-13T16:34:10.968 INFO:teuthology.orchestra.run.smithi016.stdout: "seg": 1, 2024-09-13T16:34:10.968 INFO:teuthology.orchestra.run.smithi016.stdout: "segexg": 0, 2024-09-13T16:34:10.968 INFO:teuthology.orchestra.run.smithi016.stdout: "segexd": 0, 2024-09-13T16:34:10.968 INFO:teuthology.orchestra.run.smithi016.stdout: "expos": 4194304, 2024-09-13T16:34:10.968 INFO:teuthology.orchestra.run.smithi016.stdout: "wrpos": 4195338, 2024-09-13T16:34:10.968 INFO:teuthology.orchestra.run.smithi016.stdout: "rdpos": 4194844, 2024-09-13T16:34:10.968 INFO:teuthology.orchestra.run.smithi016.stdout: "jlat": { 2024-09-13T16:34:10.968 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.968 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.968 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.968 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.968 INFO:teuthology.orchestra.run.smithi016.stdout: "replayed": 2 2024-09-13T16:34:10.968 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.968 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_mem": { 2024-09-13T16:34:10.969 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 3, 2024-09-13T16:34:10.969 INFO:teuthology.orchestra.run.smithi016.stdout: "ino+": 3, 2024-09-13T16:34:10.969 INFO:teuthology.orchestra.run.smithi016.stdout: "ino-": 0, 2024-09-13T16:34:10.969 INFO:teuthology.orchestra.run.smithi016.stdout: "dir": 1, 2024-09-13T16:34:10.969 INFO:teuthology.orchestra.run.smithi016.stdout: "dir+": 1, 2024-09-13T16:34:10.969 INFO:teuthology.orchestra.run.smithi016.stdout: "dir-": 0, 2024-09-13T16:34:10.969 INFO:teuthology.orchestra.run.smithi016.stdout: "dn": 0, 2024-09-13T16:34:10.969 INFO:teuthology.orchestra.run.smithi016.stdout: "dn+": 0, 2024-09-13T16:34:10.969 INFO:teuthology.orchestra.run.smithi016.stdout: "dn-": 0, 2024-09-13T16:34:10.969 INFO:teuthology.orchestra.run.smithi016.stdout: "cap": 0, 2024-09-13T16:34:10.969 INFO:teuthology.orchestra.run.smithi016.stdout: "cap+": 0, 2024-09-13T16:34:10.969 INFO:teuthology.orchestra.run.smithi016.stdout: "cap-": 0, 2024-09-13T16:34:10.969 INFO:teuthology.orchestra.run.smithi016.stdout: "rss": 47348, 2024-09-13T16:34:10.969 INFO:teuthology.orchestra.run.smithi016.stdout: "heap": 207124 2024-09-13T16:34:10.970 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.970 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_server": { 2024-09-13T16:34:10.970 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_client_request": 0, 2024-09-13T16:34:10.970 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_server_request": 0, 2024-09-13T16:34:10.970 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_request": 0, 2024-09-13T16:34:10.970 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_session": 0, 2024-09-13T16:34:10.970 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_peer_request": 0, 2024-09-13T16:34:10.970 INFO:teuthology.orchestra.run.smithi016.stdout: "req_create_latency": { 2024-09-13T16:34:10.970 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.970 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.970 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.970 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.970 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getattr_latency": { 2024-09-13T16:34:10.970 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.971 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.971 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.971 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.971 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getfilelock_latency": { 2024-09-13T16:34:10.971 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.971 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.971 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.971 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.971 INFO:teuthology.orchestra.run.smithi016.stdout: "req_link_latency": { 2024-09-13T16:34:10.971 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.971 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.971 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.971 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.971 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookup_latency": { 2024-09-13T16:34:10.971 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.972 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.972 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.972 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.972 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookuphash_latency": { 2024-09-13T16:34:10.972 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.972 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.972 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.972 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.972 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupino_latency": { 2024-09-13T16:34:10.972 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.972 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.972 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.972 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.973 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupname_latency": { 2024-09-13T16:34:10.973 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.973 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.973 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.973 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.973 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupparent_latency": { 2024-09-13T16:34:10.973 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.973 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.973 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.973 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.973 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupsnap_latency": { 2024-09-13T16:34:10.973 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.973 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.973 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.974 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.974 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lssnap_latency": { 2024-09-13T16:34:10.974 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.974 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.974 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.974 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.974 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mkdir_latency": { 2024-09-13T16:34:10.974 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.974 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.974 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.974 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.974 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mknod_latency": { 2024-09-13T16:34:10.974 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.975 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.975 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.975 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.975 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mksnap_latency": { 2024-09-13T16:34:10.975 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.976 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.976 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.976 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.976 INFO:teuthology.orchestra.run.smithi016.stdout: "req_open_latency": { 2024-09-13T16:34:10.976 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.976 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.976 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.976 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.976 INFO:teuthology.orchestra.run.smithi016.stdout: "req_readdir_latency": { 2024-09-13T16:34:10.976 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.976 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.976 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.976 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.977 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rename_latency": { 2024-09-13T16:34:10.977 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.977 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.977 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.977 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.977 INFO:teuthology.orchestra.run.smithi016.stdout: "req_renamesnap_latency": { 2024-09-13T16:34:10.977 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.977 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.977 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.977 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.977 INFO:teuthology.orchestra.run.smithi016.stdout: "req_snapdiff_latency": { 2024-09-13T16:34:10.977 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.977 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.977 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.978 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.978 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmdir_latency": { 2024-09-13T16:34:10.978 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.978 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.978 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.978 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.978 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmsnap_latency": { 2024-09-13T16:34:10.978 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.978 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.978 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.978 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.978 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmxattr_latency": { 2024-09-13T16:34:10.978 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.979 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.979 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.979 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.979 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setattr_latency": { 2024-09-13T16:34:10.979 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.979 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.979 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.979 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.979 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setdirlayout_latency": { 2024-09-13T16:34:10.979 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.979 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.979 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.979 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.979 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setfilelock_latency": { 2024-09-13T16:34:10.980 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.980 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.980 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.980 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.980 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setlayout_latency": { 2024-09-13T16:34:10.980 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.980 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.980 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.980 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.980 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setxattr_latency": { 2024-09-13T16:34:10.980 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.980 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.980 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.981 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.981 INFO:teuthology.orchestra.run.smithi016.stdout: "req_symlink_latency": { 2024-09-13T16:34:10.981 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.981 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.981 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.981 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.981 INFO:teuthology.orchestra.run.smithi016.stdout: "req_unlink_latency": { 2024-09-13T16:34:10.981 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.981 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.981 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.981 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.981 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_revoke_eviction": 0, 2024-09-13T16:34:10.981 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_acquisition_throttle": 0, 2024-09-13T16:34:10.981 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getvxattr_latency": { 2024-09-13T16:34:10.981 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.982 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.982 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.982 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:10.982 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.982 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_sessions": { 2024-09-13T16:34:10.982 INFO:teuthology.orchestra.run.smithi016.stdout: "session_count": 1, 2024-09-13T16:34:10.982 INFO:teuthology.orchestra.run.smithi016.stdout: "session_add": 1, 2024-09-13T16:34:10.982 INFO:teuthology.orchestra.run.smithi016.stdout: "session_remove": 0, 2024-09-13T16:34:10.982 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_open": 1, 2024-09-13T16:34:10.982 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_stale": 0, 2024-09-13T16:34:10.982 INFO:teuthology.orchestra.run.smithi016.stdout: "total_load": 0, 2024-09-13T16:34:10.982 INFO:teuthology.orchestra.run.smithi016.stdout: "average_load": 0, 2024-09-13T16:34:10.982 INFO:teuthology.orchestra.run.smithi016.stdout: "avg_session_uptime": 1216, 2024-09-13T16:34:10.982 INFO:teuthology.orchestra.run.smithi016.stdout: "mdthresh_evicted": 0 2024-09-13T16:34:10.983 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.983 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-09-13T16:34:10.983 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-09-13T16:34:10.983 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-09-13T16:34:10.983 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T16:34:10.983 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-09-13T16:34:10.983 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T16:34:10.983 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-09-13T16:34:10.983 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T16:34:10.983 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T16:34:10.983 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T16:34:10.983 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T16:34:10.983 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T16:34:10.984 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-09-13T16:34:10.984 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T16:34:10.984 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T16:34:10.984 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-09-13T16:34:10.984 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-09-13T16:34:10.984 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-09-13T16:34:10.984 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-09-13T16:34:10.984 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T16:34:10.984 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T16:34:10.984 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T16:34:10.984 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T16:34:10.984 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T16:34:10.984 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-09-13T16:34:10.985 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-09-13T16:34:10.985 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-09-13T16:34:10.985 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T16:34:10.985 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T16:34:10.985 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-09-13T16:34:10.985 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-09-13T16:34:10.985 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-09-13T16:34:10.985 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-09-13T16:34:10.985 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T16:34:10.985 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-09-13T16:34:10.985 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T16:34:10.985 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-09-13T16:34:10.985 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 197336, 2024-09-13T16:34:10.985 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 53, 2024-09-13T16:34:10.985 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-09-13T16:34:10.986 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-09-13T16:34:10.986 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-09-13T16:34:10.986 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-09-13T16:34:10.986 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-09-13T16:34:10.986 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-09-13T16:34:10.986 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-09-13T16:34:10.986 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-09-13T16:34:10.986 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 8856, 2024-09-13T16:34:10.986 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 166, 2024-09-13T16:34:10.986 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T16:34:10.986 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-09-13T16:34:10.986 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-09-13T16:34:10.986 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-09-13T16:34:10.986 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 8368, 2024-09-13T16:34:10.986 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 13, 2024-09-13T16:34:10.987 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-09-13T16:34:10.987 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-09-13T16:34:10.987 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-09-13T16:34:10.987 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-09-13T16:34:10.987 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.987 INFO:teuthology.orchestra.run.smithi016.stdout: "objecter": { 2024-09-13T16:34:10.987 INFO:teuthology.orchestra.run.smithi016.stdout: "op_active": 0, 2024-09-13T16:34:10.987 INFO:teuthology.orchestra.run.smithi016.stdout: "op_laggy": 0, 2024-09-13T16:34:10.987 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send": 800, 2024-09-13T16:34:10.987 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send_bytes": 21, 2024-09-13T16:34:10.987 INFO:teuthology.orchestra.run.smithi016.stdout: "op_resend": 3, 2024-09-13T16:34:10.987 INFO:teuthology.orchestra.run.smithi016.stdout: "op_reply": 800, 2024-09-13T16:34:10.987 INFO:teuthology.orchestra.run.smithi016.stdout: "op_latency": { 2024-09-13T16:34:10.987 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 800, 2024-09-13T16:34:10.987 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.509992208, 2024-09-13T16:34:10.988 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000637490 2024-09-13T16:34:10.988 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.988 INFO:teuthology.orchestra.run.smithi016.stdout: "op_inflight": 0, 2024-09-13T16:34:10.988 INFO:teuthology.orchestra.run.smithi016.stdout: "oplen_avg": { 2024-09-13T16:34:10.988 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 800, 2024-09-13T16:34:10.988 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 801 2024-09-13T16:34:10.988 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.988 INFO:teuthology.orchestra.run.smithi016.stdout: "op": 800, 2024-09-13T16:34:10.988 INFO:teuthology.orchestra.run.smithi016.stdout: "op_r": 800, 2024-09-13T16:34:10.988 INFO:teuthology.orchestra.run.smithi016.stdout: "op_w": 0, 2024-09-13T16:34:10.988 INFO:teuthology.orchestra.run.smithi016.stdout: "op_rmw": 0, 2024-09-13T16:34:10.988 INFO:teuthology.orchestra.run.smithi016.stdout: "op_pg": 0, 2024-09-13T16:34:10.988 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_stat": 489, 2024-09-13T16:34:10.988 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_create": 0, 2024-09-13T16:34:10.989 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_read": 309, 2024-09-13T16:34:10.989 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_write": 0, 2024-09-13T16:34:10.989 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writefull": 0, 2024-09-13T16:34:10.989 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writesame": 0, 2024-09-13T16:34:10.989 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_append": 0, 2024-09-13T16:34:10.989 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_zero": 0, 2024-09-13T16:34:10.989 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_truncate": 0, 2024-09-13T16:34:10.989 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_delete": 0, 2024-09-13T16:34:10.989 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_mapext": 0, 2024-09-13T16:34:10.989 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_sparse_read": 0, 2024-09-13T16:34:10.989 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_clonerange": 0, 2024-09-13T16:34:10.989 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_getxattr": 1, 2024-09-13T16:34:10.989 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_setxattr": 0, 2024-09-13T16:34:10.989 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_cmpxattr": 0, 2024-09-13T16:34:10.989 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_rmxattr": 0, 2024-09-13T16:34:10.990 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_resetxattrs": 0, 2024-09-13T16:34:10.990 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_call": 0, 2024-09-13T16:34:10.990 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_watch": 0, 2024-09-13T16:34:10.990 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_notify": 0, 2024-09-13T16:34:10.990 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_src_cmpxattr": 0, 2024-09-13T16:34:10.990 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls": 0, 2024-09-13T16:34:10.990 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls_filter": 0, 2024-09-13T16:34:10.990 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_other": 0, 2024-09-13T16:34:10.990 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_active": 0, 2024-09-13T16:34:10.990 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_send": 0, 2024-09-13T16:34:10.990 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_resend": 0, 2024-09-13T16:34:10.990 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_ping": 0, 2024-09-13T16:34:10.990 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_active": 0, 2024-09-13T16:34:10.990 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_send": 0, 2024-09-13T16:34:10.990 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_resend": 0, 2024-09-13T16:34:10.990 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_active": 0, 2024-09-13T16:34:10.991 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_send": 0, 2024-09-13T16:34:10.991 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_resend": 0, 2024-09-13T16:34:10.991 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_active": 0, 2024-09-13T16:34:10.991 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_send": 0, 2024-09-13T16:34:10.991 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_resend": 0, 2024-09-13T16:34:10.991 INFO:teuthology.orchestra.run.smithi016.stdout: "command_active": 0, 2024-09-13T16:34:10.991 INFO:teuthology.orchestra.run.smithi016.stdout: "command_send": 0, 2024-09-13T16:34:10.991 INFO:teuthology.orchestra.run.smithi016.stdout: "command_resend": 0, 2024-09-13T16:34:10.991 INFO:teuthology.orchestra.run.smithi016.stdout: "map_epoch": 93, 2024-09-13T16:34:10.991 INFO:teuthology.orchestra.run.smithi016.stdout: "map_full": 0, 2024-09-13T16:34:10.991 INFO:teuthology.orchestra.run.smithi016.stdout: "map_inc": 2, 2024-09-13T16:34:10.991 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_sessions": 6, 2024-09-13T16:34:10.991 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_open": 6, 2024-09-13T16:34:10.991 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_close": 0, 2024-09-13T16:34:10.991 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_laggy": 0, 2024-09-13T16:34:10.991 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_wr": 0, 2024-09-13T16:34:10.992 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_rd": 2, 2024-09-13T16:34:10.992 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_del": 0 2024-09-13T16:34:10.992 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.992 INFO:teuthology.orchestra.run.smithi016.stdout: "oft": { 2024-09-13T16:34:10.992 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_objs": 0, 2024-09-13T16:34:10.992 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_kv_pairs": 0, 2024-09-13T16:34:10.992 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_updates": 0, 2024-09-13T16:34:10.992 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_removes": 0 2024-09-13T16:34:10.992 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.992 INFO:teuthology.orchestra.run.smithi016.stdout: "osd-slow-ops": { 2024-09-13T16:34:10.992 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_ops_count": 0 2024-09-13T16:34:10.992 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.992 INFO:teuthology.orchestra.run.smithi016.stdout: "purge_queue": { 2024-09-13T16:34:10.992 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops": 0, 2024-09-13T16:34:10.992 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops_high_water": 0, 2024-09-13T16:34:10.993 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing": 0, 2024-09-13T16:34:10.993 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_high_water": 0, 2024-09-13T16:34:10.993 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed_ops": 0, 2024-09-13T16:34:10.993 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed": 0, 2024-09-13T16:34:10.993 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_item_in_journal": 0 2024-09-13T16:34:10.993 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.993 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T16:34:10.993 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-13T16:34:10.993 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-13T16:34:10.993 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-13T16:34:10.993 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 908, 2024-09-13T16:34:10.993 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 875388, 2024-09-13T16:34:10.993 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:10.993 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 908, 2024-09-13T16:34:10.994 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-13T16:34:10.994 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-13T16:34:10.994 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 908, 2024-09-13T16:34:10.994 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 875388, 2024-09-13T16:34:10.994 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-13T16:34:10.994 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.994 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.994 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.994 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:10.994 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.994 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_bytes": { 2024-09-13T16:34:10.994 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-13T16:34:10.994 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-13T16:34:10.994 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-13T16:34:10.994 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-09-13T16:34:10.994 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-09-13T16:34:10.995 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:10.995 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:10.995 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 800, 2024-09-13T16:34:10.995 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 1039, 2024-09-13T16:34:10.995 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 800, 2024-09-13T16:34:10.995 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 1039, 2024-09-13T16:34:10.995 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-13T16:34:10.995 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.995 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.995 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.995 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:10.995 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.995 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_ops": { 2024-09-13T16:34:10.995 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-13T16:34:10.996 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 1024, 2024-09-13T16:34:10.996 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-13T16:34:10.996 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-09-13T16:34:10.996 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-09-13T16:34:10.996 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:10.996 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:10.996 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 800, 2024-09-13T16:34:10.996 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 800, 2024-09-13T16:34:10.996 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 800, 2024-09-13T16:34:10.996 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 800, 2024-09-13T16:34:10.996 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-13T16:34:10.996 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.996 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.997 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.997 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:10.997 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.997 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle": { 2024-09-13T16:34:10.997 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-13T16:34:10.997 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-09-13T16:34:10.997 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-13T16:34:10.997 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-09-13T16:34:10.997 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-09-13T16:34:10.997 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:10.997 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:10.997 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-13T16:34:10.997 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-13T16:34:10.997 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-09-13T16:34:10.998 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-09-13T16:34:10.998 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-13T16:34:10.998 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.998 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.998 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.998 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:10.998 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:10.998 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle-0x55cf305dc780": { 2024-09-13T16:34:10.998 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-13T16:34:10.998 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-09-13T16:34:10.998 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-13T16:34:10.998 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-09-13T16:34:10.998 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-09-13T16:34:10.998 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:10.999 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:10.999 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-13T16:34:10.999 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-13T16:34:10.999 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-09-13T16:34:10.999 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-09-13T16:34:10.999 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-13T16:34:10.999 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:10.999 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:10.999 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:10.999 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:10.999 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:10.999 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-13T16:34:11.770 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-13T16:34:11.770 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-13T16:34:11.770 DEBUG:tasks.check_counter:Getting stats from d 2024-09-13T16:34:11.770 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-13T16:34:11.985 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:34:12.050 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:11 smithi016 ceph-mon[31187]: pgmap v446: 129 pgs: 129 active+clean; 4.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.4 KiB/s wr, 121 op/s 2024-09-13T16:34:12.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:11 smithi107 ceph-mon[34139]: pgmap v446: 129 pgs: 129 active+clean; 4.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.4 KiB/s wr, 121 op/s 2024-09-13T16:34:12.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:11 smithi111 ceph-mon[34600]: pgmap v446: 129 pgs: 129 active+clean; 4.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.4 KiB/s wr, 121 op/s 2024-09-13T16:34:13.721 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-13T16:34:13.722 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T16:34:13.722 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 0, 2024-09-13T16:34:13.722 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 2, 2024-09-13T16:34:13.722 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 0, 2024-09-13T16:34:13.722 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 0, 2024-09-13T16:34:13.722 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-09-13T16:34:13.722 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-09-13T16:34:13.722 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.001380746, 2024-09-13T16:34:13.722 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000000000, 2024-09-13T16:34:13.722 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.000322126, 2024-09-13T16:34:13.722 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-13T16:34:13.722 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:13.723 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:13.723 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:13.723 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:13.723 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:13.723 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:13.723 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:13.723 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:13.723 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:13.723 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:13.723 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-13T16:34:13.723 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-09-13T16:34:13.723 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:13.723 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T16:34:13.724 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 104, 2024-09-13T16:34:13.724 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 99, 2024-09-13T16:34:13.724 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 710432, 2024-09-13T16:34:13.724 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 55536, 2024-09-13T16:34:13.724 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-09-13T16:34:13.724 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-09-13T16:34:13.724 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.018365523, 2024-09-13T16:34:13.724 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.007024478, 2024-09-13T16:34:13.724 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.017023623, 2024-09-13T16:34:13.724 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-13T16:34:13.724 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:13.724 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 98, 2024-09-13T16:34:13.725 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.004998084, 2024-09-13T16:34:13.725 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000051000 2024-09-13T16:34:13.725 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:13.725 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:13.725 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:13.725 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:13.725 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:13.725 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:13.725 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 710432, 2024-09-13T16:34:13.725 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 55536 2024-09-13T16:34:13.725 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:13.725 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T16:34:13.725 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 5, 2024-09-13T16:34:13.725 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 77, 2024-09-13T16:34:13.726 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 3120, 2024-09-13T16:34:13.726 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 89728, 2024-09-13T16:34:13.726 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 3, 2024-09-13T16:34:13.726 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-09-13T16:34:13.726 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.009119022, 2024-09-13T16:34:13.726 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.005054585, 2024-09-13T16:34:13.726 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.003340571, 2024-09-13T16:34:13.726 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-13T16:34:13.726 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:13.726 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 76, 2024-09-13T16:34:13.726 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.007835485, 2024-09-13T16:34:13.726 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000103098 2024-09-13T16:34:13.726 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:13.727 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:13.727 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:13.727 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:13.727 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:13.727 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:13.727 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 3120, 2024-09-13T16:34:13.727 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 89728 2024-09-13T16:34:13.727 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:13.727 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-09-13T16:34:13.727 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-09-13T16:34:13.727 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-09-13T16:34:13.727 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:13.727 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-09-13T16:34:13.727 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-09-13T16:34:13.728 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-09-13T16:34:13.728 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T16:34:13.728 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-09-13T16:34:13.728 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T16:34:13.728 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-09-13T16:34:13.728 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T16:34:13.728 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T16:34:13.728 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T16:34:13.728 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T16:34:13.728 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T16:34:13.728 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-09-13T16:34:13.728 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T16:34:13.728 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T16:34:13.728 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-09-13T16:34:13.729 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-09-13T16:34:13.729 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-09-13T16:34:13.729 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-09-13T16:34:13.729 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T16:34:13.729 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T16:34:13.729 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T16:34:13.729 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T16:34:13.729 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T16:34:13.729 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-09-13T16:34:13.729 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-09-13T16:34:13.729 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-09-13T16:34:13.729 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T16:34:13.729 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T16:34:13.729 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-09-13T16:34:13.730 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-09-13T16:34:13.730 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-09-13T16:34:13.730 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-09-13T16:34:13.730 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T16:34:13.730 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-09-13T16:34:13.730 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T16:34:13.730 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-09-13T16:34:13.730 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 148880, 2024-09-13T16:34:13.730 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 29, 2024-09-13T16:34:13.730 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-09-13T16:34:13.730 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-09-13T16:34:13.730 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-09-13T16:34:13.730 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-09-13T16:34:13.730 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-09-13T16:34:13.730 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-09-13T16:34:13.731 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-09-13T16:34:13.731 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-09-13T16:34:13.731 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-09-13T16:34:13.731 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-09-13T16:34:13.731 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T16:34:13.731 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-09-13T16:34:13.731 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-09-13T16:34:13.731 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-09-13T16:34:13.731 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 928, 2024-09-13T16:34:13.731 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-09-13T16:34:13.731 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-09-13T16:34:13.731 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-09-13T16:34:13.731 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-09-13T16:34:13.731 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-09-13T16:34:13.732 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:13.732 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T16:34:13.732 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-13T16:34:13.732 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-13T16:34:13.732 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-13T16:34:13.732 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 105, 2024-09-13T16:34:13.732 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 695743, 2024-09-13T16:34:13.732 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:13.732 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 105, 2024-09-13T16:34:13.732 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-13T16:34:13.732 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-13T16:34:13.732 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 105, 2024-09-13T16:34:13.732 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 695743, 2024-09-13T16:34:13.732 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-13T16:34:13.732 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:13.733 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:13.733 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:13.733 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:13.733 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:13.733 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-13T16:34:14.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:13 smithi016 ceph-mon[31187]: pgmap v447: 129 pgs: 129 active+clean; 225 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.0 KiB/s wr, 223 op/s 2024-09-13T16:34:14.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:13 smithi107 ceph-mon[34139]: pgmap v447: 129 pgs: 129 active+clean; 225 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.0 KiB/s wr, 223 op/s 2024-09-13T16:34:14.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:13 smithi111 ceph-mon[34600]: pgmap v447: 129 pgs: 129 active+clean; 225 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.0 KiB/s wr, 223 op/s 2024-09-13T16:34:14.586 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-13T16:34:14.586 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-13T16:34:14.586 DEBUG:tasks.check_counter:Getting stats from g 2024-09-13T16:34:14.587 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-13T16:34:14.801 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:34:16.100 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:15 smithi016 ceph-mon[31187]: pgmap v448: 129 pgs: 129 active+clean; 1014 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 224 op/s 2024-09-13T16:34:16.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:15 smithi107 ceph-mon[34139]: pgmap v448: 129 pgs: 129 active+clean; 1014 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 224 op/s 2024-09-13T16:34:16.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:15 smithi111 ceph-mon[34600]: pgmap v448: 129 pgs: 129 active+clean; 1014 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 224 op/s 2024-09-13T16:34:16.659 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-13T16:34:16.659 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T16:34:16.659 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 311, 2024-09-13T16:34:16.659 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 313, 2024-09-13T16:34:16.659 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 104441, 2024-09-13T16:34:16.659 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 96722, 2024-09-13T16:34:16.659 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 6, 2024-09-13T16:34:16.659 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-09-13T16:34:16.659 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.037239525, 2024-09-13T16:34:16.659 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.014618503, 2024-09-13T16:34:16.659 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.008788866, 2024-09-13T16:34:16.660 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.009248055, 2024-09-13T16:34:16.660 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:16.660 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 311, 2024-09-13T16:34:16.660 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.016747401, 2024-09-13T16:34:16.660 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000053850 2024-09-13T16:34:16.660 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.660 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:16.660 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.660 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.660 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.660 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.660 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-13T16:34:16.661 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-09-13T16:34:16.661 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.661 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T16:34:16.661 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 543, 2024-09-13T16:34:16.661 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 613, 2024-09-13T16:34:16.661 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 142592, 2024-09-13T16:34:16.661 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 363967, 2024-09-13T16:34:16.661 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-09-13T16:34:16.661 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 4, 2024-09-13T16:34:16.661 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.042594399, 2024-09-13T16:34:16.661 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.013381599, 2024-09-13T16:34:16.661 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.012697116, 2024-09-13T16:34:16.661 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.013443066, 2024-09-13T16:34:16.661 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:16.662 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 612, 2024-09-13T16:34:16.662 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.022595271, 2024-09-13T16:34:16.662 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000036920 2024-09-13T16:34:16.662 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.662 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:16.662 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.662 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.662 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.662 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.662 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 3120, 2024-09-13T16:34:16.662 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 196592 2024-09-13T16:34:16.662 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.662 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T16:34:16.662 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 112, 2024-09-13T16:34:16.663 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 106, 2024-09-13T16:34:16.663 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 733572, 2024-09-13T16:34:16.663 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 55888, 2024-09-13T16:34:16.663 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-09-13T16:34:16.663 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 3, 2024-09-13T16:34:16.663 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.019515685, 2024-09-13T16:34:16.663 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.006715286, 2024-09-13T16:34:16.663 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.019144551, 2024-09-13T16:34:16.663 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000081805, 2024-09-13T16:34:16.663 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:16.663 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 105, 2024-09-13T16:34:16.663 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.007125948, 2024-09-13T16:34:16.663 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000067866 2024-09-13T16:34:16.663 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.664 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:16.664 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.664 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.664 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.664 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.664 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 732224, 2024-09-13T16:34:16.664 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 54944 2024-09-13T16:34:16.664 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.664 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-09-13T16:34:16.664 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 1, 2024-09-13T16:34:16.664 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-09-13T16:34:16.664 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.664 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-MDSRank": { 2024-09-13T16:34:16.665 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-09-13T16:34:16.665 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-09-13T16:34:16.665 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1467, 2024-09-13T16:34:16.665 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.036407139, 2024-09-13T16:34:16.665 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000024817 2024-09-13T16:34:16.665 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:16.665 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.665 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-PurgeQueue": { 2024-09-13T16:34:16.665 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-09-13T16:34:16.665 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-09-13T16:34:16.665 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.665 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.665 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.665 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:16.666 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.666 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-09-13T16:34:16.666 INFO:teuthology.orchestra.run.smithi016.stdout: "request": 0, 2024-09-13T16:34:16.666 INFO:teuthology.orchestra.run.smithi016.stdout: "reply": 0, 2024-09-13T16:34:16.666 INFO:teuthology.orchestra.run.smithi016.stdout: "reply_latency": { 2024-09-13T16:34:16.666 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.666 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.666 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.666 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.666 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_reply": 0, 2024-09-13T16:34:16.666 INFO:teuthology.orchestra.run.smithi016.stdout: "forward": 0, 2024-09-13T16:34:16.666 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_complete": 0, 2024-09-13T16:34:16.666 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_keys": 0, 2024-09-13T16:34:16.667 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_commit": 0, 2024-09-13T16:34:16.667 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_split": 0, 2024-09-13T16:34:16.667 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_merge": 0, 2024-09-13T16:34:16.667 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes": 0, 2024-09-13T16:34:16.667 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_top": 0, 2024-09-13T16:34:16.667 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_bottom": 0, 2024-09-13T16:34:16.667 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pin_tail": 0, 2024-09-13T16:34:16.667 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pinned": 0, 2024-09-13T16:34:16.667 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_expired": 0, 2024-09-13T16:34:16.667 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_with_caps": 0, 2024-09-13T16:34:16.667 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": 0, 2024-09-13T16:34:16.667 INFO:teuthology.orchestra.run.smithi016.stdout: "subtrees": 1, 2024-09-13T16:34:16.667 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse": 0, 2024-09-13T16:34:16.667 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_hit": 0, 2024-09-13T16:34:16.667 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_forward": 0, 2024-09-13T16:34:16.668 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_discover": 0, 2024-09-13T16:34:16.668 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_dir_fetch": 0, 2024-09-13T16:34:16.668 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_remote_ino": 0, 2024-09-13T16:34:16.668 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_lock": 0, 2024-09-13T16:34:16.668 INFO:teuthology.orchestra.run.smithi016.stdout: "load_cent": 0, 2024-09-13T16:34:16.668 INFO:teuthology.orchestra.run.smithi016.stdout: "q": 0, 2024-09-13T16:34:16.668 INFO:teuthology.orchestra.run.smithi016.stdout: "exported": 0, 2024-09-13T16:34:16.668 INFO:teuthology.orchestra.run.smithi016.stdout: "exported_inodes": 0, 2024-09-13T16:34:16.668 INFO:teuthology.orchestra.run.smithi016.stdout: "imported": 0, 2024-09-13T16:34:16.668 INFO:teuthology.orchestra.run.smithi016.stdout: "imported_inodes": 0, 2024-09-13T16:34:16.668 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_dir_fetch": 0, 2024-09-13T16:34:16.668 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_backtrace_fetch": 0, 2024-09-13T16:34:16.668 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_peer_discover": 0, 2024-09-13T16:34:16.669 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rfiles": 0, 2024-09-13T16:34:16.669 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rbytes": 0, 2024-09-13T16:34:16.669 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rsnaps": 0, 2024-09-13T16:34:16.669 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_fetch": 0, 2024-09-13T16:34:16.669 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_set_tag": 0, 2024-09-13T16:34:16.669 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_repaired": 0, 2024-09-13T16:34:16.669 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_inotable_repaired": 0, 2024-09-13T16:34:16.669 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_inodes": 0, 2024-09-13T16:34:16.669 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_base_inodes": 0, 2024-09-13T16:34:16.669 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dirfrag_rstats": 0, 2024-09-13T16:34:16.669 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_file_inodes": 0, 2024-09-13T16:34:16.669 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_inode_file_caps": 0, 2024-09-13T16:34:16.669 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_revoke": 0, 2024-09-13T16:34:16.669 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_grant": 0, 2024-09-13T16:34:16.670 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_trunc": 0, 2024-09-13T16:34:16.670 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-13T16:34:16.670 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-13T16:34:16.670 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps": 0, 2024-09-13T16:34:16.670 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps_dirty": 0, 2024-09-13T16:34:16.670 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_cap_release": 0, 2024-09-13T16:34:16.670 INFO:teuthology.orchestra.run.smithi016.stdout: "process_request_cap_release": 0 2024-09-13T16:34:16.670 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.670 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_cache": { 2024-09-13T16:34:16.670 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update": 0, 2024-09-13T16:34:16.670 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update_receipt": 0, 2024-09-13T16:34:16.670 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_try_discover": 0, 2024-09-13T16:34:16.670 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_send_discover": 0, 2024-09-13T16:34:16.670 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_handle_discover": 0, 2024-09-13T16:34:16.671 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays": 0, 2024-09-13T16:34:16.671 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_delayed": 0, 2024-09-13T16:34:16.671 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_enqueuing": 0, 2024-09-13T16:34:16.671 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_created": 0, 2024-09-13T16:34:16.671 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_enqueued": 0, 2024-09-13T16:34:16.671 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_reintegrated": 0, 2024-09-13T16:34:16.671 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_migrated": 0, 2024-09-13T16:34:16.671 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_processing": 0, 2024-09-13T16:34:16.671 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_enqueued": 0, 2024-09-13T16:34:16.671 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_prioritized": 0, 2024-09-13T16:34:16.671 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_started": 0, 2024-09-13T16:34:16.671 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_completed": 0, 2024-09-13T16:34:16.671 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_enqueue_scrub": 0, 2024-09-13T16:34:16.671 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_exportdir": 0, 2024-09-13T16:34:16.672 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_flush": 0, 2024-09-13T16:34:16.672 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragmentdir": 0, 2024-09-13T16:34:16.672 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragstats": 0, 2024-09-13T16:34:16.672 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_inodestats": 0 2024-09-13T16:34:16.672 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.672 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_log": { 2024-09-13T16:34:16.672 INFO:teuthology.orchestra.run.smithi016.stdout: "evadd": 0, 2024-09-13T16:34:16.672 INFO:teuthology.orchestra.run.smithi016.stdout: "evex": 0, 2024-09-13T16:34:16.672 INFO:teuthology.orchestra.run.smithi016.stdout: "evtrm": 0, 2024-09-13T16:34:16.672 INFO:teuthology.orchestra.run.smithi016.stdout: "ev": 6, 2024-09-13T16:34:16.672 INFO:teuthology.orchestra.run.smithi016.stdout: "evexg": 0, 2024-09-13T16:34:16.672 INFO:teuthology.orchestra.run.smithi016.stdout: "evexd": 0, 2024-09-13T16:34:16.672 INFO:teuthology.orchestra.run.smithi016.stdout: "segadd": 0, 2024-09-13T16:34:16.673 INFO:teuthology.orchestra.run.smithi016.stdout: "segex": 0, 2024-09-13T16:34:16.673 INFO:teuthology.orchestra.run.smithi016.stdout: "segtrm": 0, 2024-09-13T16:34:16.675 INFO:teuthology.orchestra.run.smithi016.stdout: "seg": 1, 2024-09-13T16:34:16.675 INFO:teuthology.orchestra.run.smithi016.stdout: "segexg": 0, 2024-09-13T16:34:16.676 INFO:teuthology.orchestra.run.smithi016.stdout: "segexd": 0, 2024-09-13T16:34:16.676 INFO:teuthology.orchestra.run.smithi016.stdout: "expos": 4194304, 2024-09-13T16:34:16.676 INFO:teuthology.orchestra.run.smithi016.stdout: "wrpos": 4196404, 2024-09-13T16:34:16.676 INFO:teuthology.orchestra.run.smithi016.stdout: "rdpos": 4196263, 2024-09-13T16:34:16.676 INFO:teuthology.orchestra.run.smithi016.stdout: "jlat": { 2024-09-13T16:34:16.676 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.676 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.676 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.676 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.676 INFO:teuthology.orchestra.run.smithi016.stdout: "replayed": 6 2024-09-13T16:34:16.676 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.676 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_mem": { 2024-09-13T16:34:16.676 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 3, 2024-09-13T16:34:16.676 INFO:teuthology.orchestra.run.smithi016.stdout: "ino+": 3, 2024-09-13T16:34:16.677 INFO:teuthology.orchestra.run.smithi016.stdout: "ino-": 0, 2024-09-13T16:34:16.677 INFO:teuthology.orchestra.run.smithi016.stdout: "dir": 1, 2024-09-13T16:34:16.677 INFO:teuthology.orchestra.run.smithi016.stdout: "dir+": 1, 2024-09-13T16:34:16.677 INFO:teuthology.orchestra.run.smithi016.stdout: "dir-": 0, 2024-09-13T16:34:16.677 INFO:teuthology.orchestra.run.smithi016.stdout: "dn": 0, 2024-09-13T16:34:16.677 INFO:teuthology.orchestra.run.smithi016.stdout: "dn+": 0, 2024-09-13T16:34:16.677 INFO:teuthology.orchestra.run.smithi016.stdout: "dn-": 0, 2024-09-13T16:34:16.677 INFO:teuthology.orchestra.run.smithi016.stdout: "cap": 0, 2024-09-13T16:34:16.677 INFO:teuthology.orchestra.run.smithi016.stdout: "cap+": 0, 2024-09-13T16:34:16.677 INFO:teuthology.orchestra.run.smithi016.stdout: "cap-": 0, 2024-09-13T16:34:16.677 INFO:teuthology.orchestra.run.smithi016.stdout: "rss": 47384, 2024-09-13T16:34:16.677 INFO:teuthology.orchestra.run.smithi016.stdout: "heap": 207124 2024-09-13T16:34:16.677 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.677 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_server": { 2024-09-13T16:34:16.678 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_client_request": 0, 2024-09-13T16:34:16.678 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_server_request": 0, 2024-09-13T16:34:16.678 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_request": 0, 2024-09-13T16:34:16.678 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_session": 0, 2024-09-13T16:34:16.678 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_peer_request": 0, 2024-09-13T16:34:16.678 INFO:teuthology.orchestra.run.smithi016.stdout: "req_create_latency": { 2024-09-13T16:34:16.678 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.678 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.678 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.678 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.678 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getattr_latency": { 2024-09-13T16:34:16.678 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.678 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.678 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.679 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getfilelock_latency": { 2024-09-13T16:34:16.679 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.679 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.679 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.679 INFO:teuthology.orchestra.run.smithi016.stdout: "req_link_latency": { 2024-09-13T16:34:16.679 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.679 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.679 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.679 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookup_latency": { 2024-09-13T16:34:16.679 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.679 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.679 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.680 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookuphash_latency": { 2024-09-13T16:34:16.680 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.680 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.680 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.680 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupino_latency": { 2024-09-13T16:34:16.680 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.680 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.680 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.680 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupname_latency": { 2024-09-13T16:34:16.680 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.681 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.681 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.681 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.681 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupparent_latency": { 2024-09-13T16:34:16.681 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.681 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.681 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.681 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.681 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupsnap_latency": { 2024-09-13T16:34:16.681 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.681 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.681 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.681 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.682 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lssnap_latency": { 2024-09-13T16:34:16.682 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.682 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.682 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.682 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mkdir_latency": { 2024-09-13T16:34:16.682 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.682 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.682 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.682 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mknod_latency": { 2024-09-13T16:34:16.683 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.683 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.683 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.683 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.683 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mksnap_latency": { 2024-09-13T16:34:16.683 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.683 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.683 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.683 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.684 INFO:teuthology.orchestra.run.smithi016.stdout: "req_open_latency": { 2024-09-13T16:34:16.684 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.684 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.684 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.684 INFO:teuthology.orchestra.run.smithi016.stdout: "req_readdir_latency": { 2024-09-13T16:34:16.684 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.684 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.684 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.684 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rename_latency": { 2024-09-13T16:34:16.684 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.684 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.685 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.685 INFO:teuthology.orchestra.run.smithi016.stdout: "req_renamesnap_latency": { 2024-09-13T16:34:16.685 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.685 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.685 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.685 INFO:teuthology.orchestra.run.smithi016.stdout: "req_snapdiff_latency": { 2024-09-13T16:34:16.685 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.685 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.685 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.685 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmdir_latency": { 2024-09-13T16:34:16.686 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.686 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.686 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.686 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmsnap_latency": { 2024-09-13T16:34:16.686 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.686 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.686 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.686 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmxattr_latency": { 2024-09-13T16:34:16.686 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.686 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.686 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.686 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setattr_latency": { 2024-09-13T16:34:16.687 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.687 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.687 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.687 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.687 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setdirlayout_latency": { 2024-09-13T16:34:16.687 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.687 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.687 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.687 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.687 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setfilelock_latency": { 2024-09-13T16:34:16.687 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.687 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.687 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.687 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.688 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setlayout_latency": { 2024-09-13T16:34:16.688 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.688 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.688 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.688 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.688 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setxattr_latency": { 2024-09-13T16:34:16.688 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.688 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.688 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.688 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.688 INFO:teuthology.orchestra.run.smithi016.stdout: "req_symlink_latency": { 2024-09-13T16:34:16.688 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.688 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.689 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.689 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.689 INFO:teuthology.orchestra.run.smithi016.stdout: "req_unlink_latency": { 2024-09-13T16:34:16.689 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.689 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.689 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.689 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.689 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_revoke_eviction": 0, 2024-09-13T16:34:16.689 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_acquisition_throttle": 0, 2024-09-13T16:34:16.689 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getvxattr_latency": { 2024-09-13T16:34:16.689 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.689 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.689 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.690 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:16.690 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.690 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_sessions": { 2024-09-13T16:34:16.690 INFO:teuthology.orchestra.run.smithi016.stdout: "session_count": 1, 2024-09-13T16:34:16.690 INFO:teuthology.orchestra.run.smithi016.stdout: "session_add": 3, 2024-09-13T16:34:16.690 INFO:teuthology.orchestra.run.smithi016.stdout: "session_remove": 2, 2024-09-13T16:34:16.690 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_open": 2, 2024-09-13T16:34:16.690 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_stale": 0, 2024-09-13T16:34:16.690 INFO:teuthology.orchestra.run.smithi016.stdout: "total_load": 0, 2024-09-13T16:34:16.690 INFO:teuthology.orchestra.run.smithi016.stdout: "average_load": 0, 2024-09-13T16:34:16.690 INFO:teuthology.orchestra.run.smithi016.stdout: "avg_session_uptime": 2261, 2024-09-13T16:34:16.690 INFO:teuthology.orchestra.run.smithi016.stdout: "mdthresh_evicted": 0 2024-09-13T16:34:16.690 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.690 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-09-13T16:34:16.691 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-09-13T16:34:16.691 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-09-13T16:34:16.691 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T16:34:16.691 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-09-13T16:34:16.691 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T16:34:16.691 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-09-13T16:34:16.691 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T16:34:16.691 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T16:34:16.691 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T16:34:16.691 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T16:34:16.691 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T16:34:16.691 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-09-13T16:34:16.691 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T16:34:16.691 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T16:34:16.692 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-09-13T16:34:16.692 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-09-13T16:34:16.692 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-09-13T16:34:16.692 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-09-13T16:34:16.692 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T16:34:16.692 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T16:34:16.692 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T16:34:16.692 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T16:34:16.692 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T16:34:16.692 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-09-13T16:34:16.692 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-09-13T16:34:16.692 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-09-13T16:34:16.692 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T16:34:16.692 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T16:34:16.693 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-09-13T16:34:16.693 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-09-13T16:34:16.693 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-09-13T16:34:16.693 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-09-13T16:34:16.693 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T16:34:16.693 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-09-13T16:34:16.693 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T16:34:16.693 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-09-13T16:34:16.693 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 197319, 2024-09-13T16:34:16.693 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 52, 2024-09-13T16:34:16.693 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-09-13T16:34:16.693 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-09-13T16:34:16.693 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-09-13T16:34:16.693 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-09-13T16:34:16.694 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-09-13T16:34:16.694 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-09-13T16:34:16.694 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-09-13T16:34:16.694 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-09-13T16:34:16.694 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 8856, 2024-09-13T16:34:16.694 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 166, 2024-09-13T16:34:16.694 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T16:34:16.694 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-09-13T16:34:16.694 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-09-13T16:34:16.694 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-09-13T16:34:16.694 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 8368, 2024-09-13T16:34:16.694 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 13, 2024-09-13T16:34:16.694 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-09-13T16:34:16.694 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-09-13T16:34:16.694 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-09-13T16:34:16.695 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-09-13T16:34:16.695 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.695 INFO:teuthology.orchestra.run.smithi016.stdout: "objecter": { 2024-09-13T16:34:16.695 INFO:teuthology.orchestra.run.smithi016.stdout: "op_active": 0, 2024-09-13T16:34:16.695 INFO:teuthology.orchestra.run.smithi016.stdout: "op_laggy": 0, 2024-09-13T16:34:16.695 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send": 852, 2024-09-13T16:34:16.695 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send_bytes": 21, 2024-09-13T16:34:16.695 INFO:teuthology.orchestra.run.smithi016.stdout: "op_resend": 3, 2024-09-13T16:34:16.695 INFO:teuthology.orchestra.run.smithi016.stdout: "op_reply": 852, 2024-09-13T16:34:16.695 INFO:teuthology.orchestra.run.smithi016.stdout: "op_latency": { 2024-09-13T16:34:16.695 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 852, 2024-09-13T16:34:16.695 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.462992944, 2024-09-13T16:34:16.695 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000543418 2024-09-13T16:34:16.695 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.696 INFO:teuthology.orchestra.run.smithi016.stdout: "op_inflight": 0, 2024-09-13T16:34:16.696 INFO:teuthology.orchestra.run.smithi016.stdout: "oplen_avg": { 2024-09-13T16:34:16.696 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 852, 2024-09-13T16:34:16.696 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 853 2024-09-13T16:34:16.696 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.696 INFO:teuthology.orchestra.run.smithi016.stdout: "op": 852, 2024-09-13T16:34:16.696 INFO:teuthology.orchestra.run.smithi016.stdout: "op_r": 852, 2024-09-13T16:34:16.696 INFO:teuthology.orchestra.run.smithi016.stdout: "op_w": 0, 2024-09-13T16:34:16.696 INFO:teuthology.orchestra.run.smithi016.stdout: "op_rmw": 0, 2024-09-13T16:34:16.696 INFO:teuthology.orchestra.run.smithi016.stdout: "op_pg": 0, 2024-09-13T16:34:16.696 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_stat": 532, 2024-09-13T16:34:16.696 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_create": 0, 2024-09-13T16:34:16.696 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_read": 318, 2024-09-13T16:34:16.696 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_write": 0, 2024-09-13T16:34:16.697 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writefull": 0, 2024-09-13T16:34:16.697 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writesame": 0, 2024-09-13T16:34:16.697 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_append": 0, 2024-09-13T16:34:16.697 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_zero": 0, 2024-09-13T16:34:16.697 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_truncate": 0, 2024-09-13T16:34:16.697 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_delete": 0, 2024-09-13T16:34:16.697 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_mapext": 0, 2024-09-13T16:34:16.697 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_sparse_read": 0, 2024-09-13T16:34:16.697 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_clonerange": 0, 2024-09-13T16:34:16.697 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_getxattr": 1, 2024-09-13T16:34:16.697 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_setxattr": 0, 2024-09-13T16:34:16.697 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_cmpxattr": 0, 2024-09-13T16:34:16.697 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_rmxattr": 0, 2024-09-13T16:34:16.697 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_resetxattrs": 0, 2024-09-13T16:34:16.697 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_call": 0, 2024-09-13T16:34:16.698 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_watch": 0, 2024-09-13T16:34:16.698 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_notify": 0, 2024-09-13T16:34:16.698 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_src_cmpxattr": 0, 2024-09-13T16:34:16.698 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls": 0, 2024-09-13T16:34:16.698 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls_filter": 0, 2024-09-13T16:34:16.698 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_other": 0, 2024-09-13T16:34:16.698 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_active": 0, 2024-09-13T16:34:16.698 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_send": 0, 2024-09-13T16:34:16.698 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_resend": 0, 2024-09-13T16:34:16.698 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_ping": 0, 2024-09-13T16:34:16.698 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_active": 0, 2024-09-13T16:34:16.698 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_send": 0, 2024-09-13T16:34:16.698 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_resend": 0, 2024-09-13T16:34:16.698 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_active": 0, 2024-09-13T16:34:16.698 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_send": 0, 2024-09-13T16:34:16.699 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_resend": 0, 2024-09-13T16:34:16.699 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_active": 0, 2024-09-13T16:34:16.699 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_send": 0, 2024-09-13T16:34:16.699 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_resend": 0, 2024-09-13T16:34:16.699 INFO:teuthology.orchestra.run.smithi016.stdout: "command_active": 0, 2024-09-13T16:34:16.699 INFO:teuthology.orchestra.run.smithi016.stdout: "command_send": 0, 2024-09-13T16:34:16.699 INFO:teuthology.orchestra.run.smithi016.stdout: "command_resend": 0, 2024-09-13T16:34:16.699 INFO:teuthology.orchestra.run.smithi016.stdout: "map_epoch": 93, 2024-09-13T16:34:16.699 INFO:teuthology.orchestra.run.smithi016.stdout: "map_full": 0, 2024-09-13T16:34:16.699 INFO:teuthology.orchestra.run.smithi016.stdout: "map_inc": 2, 2024-09-13T16:34:16.699 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_sessions": 6, 2024-09-13T16:34:16.699 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_open": 6, 2024-09-13T16:34:16.699 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_close": 0, 2024-09-13T16:34:16.699 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_laggy": 0, 2024-09-13T16:34:16.700 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_wr": 0, 2024-09-13T16:34:16.700 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_rd": 2, 2024-09-13T16:34:16.700 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_del": 0 2024-09-13T16:34:16.700 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.700 INFO:teuthology.orchestra.run.smithi016.stdout: "oft": { 2024-09-13T16:34:16.700 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_objs": 0, 2024-09-13T16:34:16.700 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_kv_pairs": 0, 2024-09-13T16:34:16.700 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_updates": 0, 2024-09-13T16:34:16.700 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_removes": 0 2024-09-13T16:34:16.700 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.700 INFO:teuthology.orchestra.run.smithi016.stdout: "osd-slow-ops": { 2024-09-13T16:34:16.700 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_ops_count": 0 2024-09-13T16:34:16.700 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.700 INFO:teuthology.orchestra.run.smithi016.stdout: "purge_queue": { 2024-09-13T16:34:16.701 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops": 0, 2024-09-13T16:34:16.701 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops_high_water": 0, 2024-09-13T16:34:16.701 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing": 0, 2024-09-13T16:34:16.701 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_high_water": 0, 2024-09-13T16:34:16.701 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed_ops": 0, 2024-09-13T16:34:16.701 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed": 0, 2024-09-13T16:34:16.701 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_item_in_journal": 0 2024-09-13T16:34:16.701 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.701 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T16:34:16.701 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-13T16:34:16.701 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-13T16:34:16.701 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-13T16:34:16.701 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 962, 2024-09-13T16:34:16.701 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 885462, 2024-09-13T16:34:16.702 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:16.702 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 962, 2024-09-13T16:34:16.702 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-13T16:34:16.702 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-13T16:34:16.702 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 962, 2024-09-13T16:34:16.702 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 885462, 2024-09-13T16:34:16.702 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-13T16:34:16.702 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.702 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.702 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.702 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:16.702 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.702 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_bytes": { 2024-09-13T16:34:16.702 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-13T16:34:16.702 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-13T16:34:16.703 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-13T16:34:16.703 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-09-13T16:34:16.703 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-09-13T16:34:16.703 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:16.703 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:16.703 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 852, 2024-09-13T16:34:16.703 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 2105, 2024-09-13T16:34:16.703 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 852, 2024-09-13T16:34:16.703 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 2105, 2024-09-13T16:34:16.703 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-13T16:34:16.703 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.703 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.703 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.703 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:16.704 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.704 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_ops": { 2024-09-13T16:34:16.704 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-13T16:34:16.704 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 1024, 2024-09-13T16:34:16.704 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-13T16:34:16.704 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-09-13T16:34:16.704 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-09-13T16:34:16.704 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:16.704 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:16.704 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 852, 2024-09-13T16:34:16.704 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 852, 2024-09-13T16:34:16.704 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 852, 2024-09-13T16:34:16.704 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 852, 2024-09-13T16:34:16.704 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-13T16:34:16.704 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.705 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.705 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.705 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:16.705 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.705 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle": { 2024-09-13T16:34:16.705 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-13T16:34:16.705 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-09-13T16:34:16.705 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-13T16:34:16.705 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-09-13T16:34:16.705 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-09-13T16:34:16.705 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:16.705 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:16.705 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-13T16:34:16.705 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-13T16:34:16.706 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-09-13T16:34:16.706 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-09-13T16:34:16.706 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-13T16:34:16.706 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.706 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.706 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.706 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:16.706 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:16.706 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle-0x5558ec656780": { 2024-09-13T16:34:16.706 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-13T16:34:16.706 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-09-13T16:34:16.706 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-13T16:34:16.706 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-09-13T16:34:16.706 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-09-13T16:34:16.707 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:16.707 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:16.707 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-13T16:34:16.707 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-13T16:34:16.707 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-09-13T16:34:16.707 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-09-13T16:34:16.707 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-13T16:34:16.707 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:16.707 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:16.707 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:16.707 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:16.707 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:16.707 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-13T16:34:17.227 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:16 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:34:17.338 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:16 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:34:17.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:16 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T16:34:17.667 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-13T16:34:17.941 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-13T16:34:17.941 DEBUG:tasks.check_counter:Getting stats from j 2024-09-13T16:34:17.941 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-13T16:34:18.150 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T16:34:18.250 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:17 smithi107 ceph-mon[34139]: pgmap v449: 129 pgs: 129 active+clean; 1014 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.4 KiB/s wr, 224 op/s 2024-09-13T16:34:18.251 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:17 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:34:18.251 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:17 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:34:18.251 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:17 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:34:18.272 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:17 smithi016 ceph-mon[31187]: pgmap v449: 129 pgs: 129 active+clean; 1014 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.4 KiB/s wr, 224 op/s 2024-09-13T16:34:18.272 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:17 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:34:18.272 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:17 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:34:18.272 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:17 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:34:18.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:17 smithi111 ceph-mon[34600]: pgmap v449: 129 pgs: 129 active+clean; 1014 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.4 KiB/s wr, 224 op/s 2024-09-13T16:34:18.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:17 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T16:34:18.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:17 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T16:34:18.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:17 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' 2024-09-13T16:34:19.942 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-13T16:34:19.942 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T16:34:19.942 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 108, 2024-09-13T16:34:19.942 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 103, 2024-09-13T16:34:19.943 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 712144, 2024-09-13T16:34:19.943 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 57648, 2024-09-13T16:34:19.943 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-09-13T16:34:19.943 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-09-13T16:34:19.943 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.019612409, 2024-09-13T16:34:19.943 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.006956173, 2024-09-13T16:34:19.943 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.017476871, 2024-09-13T16:34:19.943 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-13T16:34:19.943 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:19.943 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 102, 2024-09-13T16:34:19.943 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.005804312, 2024-09-13T16:34:19.943 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000056905 2024-09-13T16:34:19.943 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:19.943 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:19.944 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:19.944 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:19.944 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:19.944 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:19.944 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 712144, 2024-09-13T16:34:19.944 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 57648 2024-09-13T16:34:19.944 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:19.944 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T16:34:19.944 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 1, 2024-09-13T16:34:19.944 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 77, 2024-09-13T16:34:19.944 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 160, 2024-09-13T16:34:19.944 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 89632, 2024-09-13T16:34:19.944 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 3, 2024-09-13T16:34:19.944 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-09-13T16:34:19.945 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.006698744, 2024-09-13T16:34:19.945 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.004633853, 2024-09-13T16:34:19.945 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.001240456, 2024-09-13T16:34:19.945 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-13T16:34:19.945 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:19.945 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 75, 2024-09-13T16:34:19.945 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.003446305, 2024-09-13T16:34:19.945 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000045950 2024-09-13T16:34:19.945 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:19.945 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:19.945 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:19.946 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:19.946 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:19.946 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:19.946 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-13T16:34:19.946 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 89632 2024-09-13T16:34:19.946 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:19.946 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T16:34:19.946 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 4, 2024-09-13T16:34:19.947 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 4, 2024-09-13T16:34:19.947 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 2960, 2024-09-13T16:34:19.947 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 448, 2024-09-13T16:34:19.947 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-09-13T16:34:19.947 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-09-13T16:34:19.947 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.002384849, 2024-09-13T16:34:19.947 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000039117, 2024-09-13T16:34:19.947 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.002354924, 2024-09-13T16:34:19.947 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-13T16:34:19.947 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:19.947 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3, 2024-09-13T16:34:19.948 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.004178310, 2024-09-13T16:34:19.948 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.001392770 2024-09-13T16:34:19.948 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:19.948 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:19.948 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:19.948 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:19.948 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:19.948 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:19.949 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-09-13T16:34:19.949 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 448 2024-09-13T16:34:19.949 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:19.949 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-09-13T16:34:19.949 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-09-13T16:34:19.949 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-09-13T16:34:19.949 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:19.949 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-09-13T16:34:19.949 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-09-13T16:34:19.949 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-09-13T16:34:19.949 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T16:34:19.950 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-09-13T16:34:19.950 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T16:34:19.950 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-09-13T16:34:19.950 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T16:34:19.950 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T16:34:19.950 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T16:34:19.950 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T16:34:19.950 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T16:34:19.950 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-09-13T16:34:19.951 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T16:34:19.951 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T16:34:19.951 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-09-13T16:34:19.951 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-09-13T16:34:19.951 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-09-13T16:34:19.952 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-09-13T16:34:19.952 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T16:34:19.952 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T16:34:19.952 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T16:34:19.952 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T16:34:19.952 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T16:34:19.952 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-09-13T16:34:19.952 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-09-13T16:34:19.953 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-09-13T16:34:19.953 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T16:34:19.953 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T16:34:19.953 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-09-13T16:34:19.953 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-09-13T16:34:19.953 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-09-13T16:34:19.953 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-09-13T16:34:19.953 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T16:34:19.953 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-09-13T16:34:19.953 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T16:34:19.953 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-09-13T16:34:19.953 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 148823, 2024-09-13T16:34:19.954 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 27, 2024-09-13T16:34:19.954 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-09-13T16:34:19.954 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-09-13T16:34:19.954 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-09-13T16:34:19.954 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-09-13T16:34:19.954 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-09-13T16:34:19.954 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-09-13T16:34:19.954 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-09-13T16:34:19.954 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-09-13T16:34:19.954 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-09-13T16:34:19.954 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-09-13T16:34:19.954 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T16:34:19.954 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-09-13T16:34:19.954 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-09-13T16:34:19.955 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-09-13T16:34:19.955 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 928, 2024-09-13T16:34:19.955 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-09-13T16:34:19.955 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-09-13T16:34:19.955 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-09-13T16:34:19.955 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-09-13T16:34:19.955 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-09-13T16:34:19.955 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-13T16:34:19.955 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T16:34:19.955 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-13T16:34:19.955 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-13T16:34:19.955 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-13T16:34:19.955 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 109, 2024-09-13T16:34:19.955 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 696900, 2024-09-13T16:34:19.955 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:19.956 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 109, 2024-09-13T16:34:19.956 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-13T16:34:19.956 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-13T16:34:19.956 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 109, 2024-09-13T16:34:19.956 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 696900, 2024-09-13T16:34:19.956 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-13T16:34:19.956 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-13T16:34:19.956 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-13T16:34:19.956 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-13T16:34:19.956 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:19.956 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-13T16:34:19.956 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-13T16:34:20.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:19 smithi016 ceph-mon[31187]: pgmap v450: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.1 KiB/s wr, 227 op/s 2024-09-13T16:34:20.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:19 smithi107 ceph-mon[34139]: pgmap v450: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.1 KiB/s wr, 227 op/s 2024-09-13T16:34:20.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:19 smithi111 ceph-mon[34600]: pgmap v450: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.1 KiB/s wr, 227 op/s 2024-09-13T16:34:21.418 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-13T16:34:21.419 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-13T16:34:21.419 DEBUG:tasks.check_counter:Getting stats from b 2024-09-13T16:34:21.419 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-13T16:34:21.629 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T16:34:22.018 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:21 smithi107 ceph-mon[34139]: pgmap v451: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 167 op/s 2024-09-13T16:34:22.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:21 smithi016 ceph-mon[31187]: pgmap v451: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 167 op/s 2024-09-13T16:34:22.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:21 smithi111 ceph-mon[34600]: pgmap v451: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 167 op/s 2024-09-13T16:34:23.452 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-13T16:34:23.453 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T16:34:23.453 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 1864, 2024-09-13T16:34:23.453 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 1750, 2024-09-13T16:34:23.453 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 559050, 2024-09-13T16:34:23.453 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 1147732, 2024-09-13T16:34:23.453 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 23, 2024-09-13T16:34:23.453 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 6, 2024-09-13T16:34:23.453 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.137436795, 2024-09-13T16:34:23.453 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.041701673, 2024-09-13T16:34:23.453 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.056776181, 2024-09-13T16:34:23.453 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.036717618, 2024-09-13T16:34:23.453 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:23.453 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1747, 2024-09-13T16:34:23.454 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.780514577, 2024-09-13T16:34:23.454 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000446774 2024-09-13T16:34:23.454 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.454 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:23.454 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 629, 2024-09-13T16:34:23.454 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000143662, 2024-09-13T16:34:23.454 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000228 2024-09-13T16:34:23.454 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.454 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-09-13T16:34:23.454 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 448 2024-09-13T16:34:23.454 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.454 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T16:34:23.454 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 1698, 2024-09-13T16:34:23.454 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 1596, 2024-09-13T16:34:23.455 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 1215934, 2024-09-13T16:34:23.455 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 565474, 2024-09-13T16:34:23.455 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 9, 2024-09-13T16:34:23.455 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 5, 2024-09-13T16:34:23.455 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.118523637, 2024-09-13T16:34:23.455 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.036434994, 2024-09-13T16:34:23.455 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.084403098, 2024-09-13T16:34:23.455 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.030996701, 2024-09-13T16:34:23.455 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:23.455 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1597, 2024-09-13T16:34:23.455 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.707670912, 2024-09-13T16:34:23.455 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000443125 2024-09-13T16:34:23.455 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.455 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:23.456 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 267, 2024-09-13T16:34:23.456 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000058234, 2024-09-13T16:34:23.456 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000218 2024-09-13T16:34:23.456 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.456 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 778544, 2024-09-13T16:34:23.456 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 62800 2024-09-13T16:34:23.456 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.456 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T16:34:23.456 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 1883, 2024-09-13T16:34:23.456 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 1601, 2024-09-13T16:34:23.456 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 613162, 2024-09-13T16:34:23.456 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 930879, 2024-09-13T16:34:23.456 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 32, 2024-09-13T16:34:23.457 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 5, 2024-09-13T16:34:23.457 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.167968031, 2024-09-13T16:34:23.457 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.046916048, 2024-09-13T16:34:23.457 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.069013280, 2024-09-13T16:34:23.457 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.051306420, 2024-09-13T16:34:23.457 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:23.457 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1599, 2024-09-13T16:34:23.457 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.652846278, 2024-09-13T16:34:23.457 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000408284 2024-09-13T16:34:23.457 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.457 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:23.457 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1093, 2024-09-13T16:34:23.457 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000225828, 2024-09-13T16:34:23.458 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000206 2024-09-13T16:34:23.458 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.458 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 99488, 2024-09-13T16:34:23.458 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 239536 2024-09-13T16:34:23.458 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.458 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-09-13T16:34:23.458 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 1, 2024-09-13T16:34:23.458 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-09-13T16:34:23.458 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.458 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-MDSRank": { 2024-09-13T16:34:23.458 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-09-13T16:34:23.458 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-09-13T16:34:23.458 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 501, 2024-09-13T16:34:23.458 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.093292770, 2024-09-13T16:34:23.459 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000186213 2024-09-13T16:34:23.459 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:23.459 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.459 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-PurgeQueue": { 2024-09-13T16:34:23.459 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-09-13T16:34:23.459 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-09-13T16:34:23.459 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2879, 2024-09-13T16:34:23.459 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.084775567, 2024-09-13T16:34:23.459 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000029446 2024-09-13T16:34:23.459 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:23.459 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.459 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-09-13T16:34:23.459 INFO:teuthology.orchestra.run.smithi107.stdout: "request": 63, 2024-09-13T16:34:23.459 INFO:teuthology.orchestra.run.smithi107.stdout: "reply": 63, 2024-09-13T16:34:23.460 INFO:teuthology.orchestra.run.smithi107.stdout: "reply_latency": { 2024-09-13T16:34:23.460 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 63, 2024-09-13T16:34:23.460 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.065351365, 2024-09-13T16:34:23.460 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.001037323 2024-09-13T16:34:23.460 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.460 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_reply": 0, 2024-09-13T16:34:23.460 INFO:teuthology.orchestra.run.smithi107.stdout: "forward": 0, 2024-09-13T16:34:23.460 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_complete": 0, 2024-09-13T16:34:23.460 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_keys": 0, 2024-09-13T16:34:23.460 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_commit": 90, 2024-09-13T16:34:23.460 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_split": 0, 2024-09-13T16:34:23.460 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_merge": 0, 2024-09-13T16:34:23.460 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes": 23, 2024-09-13T16:34:23.460 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_top": 4, 2024-09-13T16:34:23.461 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_bottom": 19, 2024-09-13T16:34:23.461 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pin_tail": 0, 2024-09-13T16:34:23.461 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pinned": 16, 2024-09-13T16:34:23.461 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_expired": 0, 2024-09-13T16:34:23.461 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_with_caps": 2, 2024-09-13T16:34:23.461 INFO:teuthology.orchestra.run.smithi107.stdout: "caps": 2, 2024-09-13T16:34:23.461 INFO:teuthology.orchestra.run.smithi107.stdout: "subtrees": 2, 2024-09-13T16:34:23.461 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse": 89, 2024-09-13T16:34:23.461 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_hit": 80, 2024-09-13T16:34:23.461 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_forward": 0, 2024-09-13T16:34:23.461 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_discover": 0, 2024-09-13T16:34:23.461 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_dir_fetch": 0, 2024-09-13T16:34:23.462 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_remote_ino": 0, 2024-09-13T16:34:23.462 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_lock": 0, 2024-09-13T16:34:23.462 INFO:teuthology.orchestra.run.smithi107.stdout: "load_cent": 0, 2024-09-13T16:34:23.462 INFO:teuthology.orchestra.run.smithi107.stdout: "q": 0, 2024-09-13T16:34:23.462 INFO:teuthology.orchestra.run.smithi107.stdout: "exported": 0, 2024-09-13T16:34:23.462 INFO:teuthology.orchestra.run.smithi107.stdout: "exported_inodes": 0, 2024-09-13T16:34:23.462 INFO:teuthology.orchestra.run.smithi107.stdout: "imported": 0, 2024-09-13T16:34:23.462 INFO:teuthology.orchestra.run.smithi107.stdout: "imported_inodes": 0, 2024-09-13T16:34:23.462 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_dir_fetch": 0, 2024-09-13T16:34:23.462 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_backtrace_fetch": 0, 2024-09-13T16:34:23.462 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_peer_discover": 0, 2024-09-13T16:34:23.462 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rfiles": 4, 2024-09-13T16:34:23.462 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rbytes": 234, 2024-09-13T16:34:23.462 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rsnaps": 0, 2024-09-13T16:34:23.463 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_fetch": 0, 2024-09-13T16:34:23.463 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_set_tag": 0, 2024-09-13T16:34:23.463 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_repaired": 0, 2024-09-13T16:34:23.463 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_inotable_repaired": 0, 2024-09-13T16:34:23.463 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_inodes": 0, 2024-09-13T16:34:23.463 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_base_inodes": 0, 2024-09-13T16:34:23.463 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dirfrag_rstats": 0, 2024-09-13T16:34:23.463 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_file_inodes": 0, 2024-09-13T16:34:23.463 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_inode_file_caps": 0, 2024-09-13T16:34:23.463 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_revoke": 17, 2024-09-13T16:34:23.463 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_grant": 74, 2024-09-13T16:34:23.463 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_trunc": 0, 2024-09-13T16:34:23.463 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-13T16:34:23.463 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-13T16:34:23.464 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps": 83, 2024-09-13T16:34:23.464 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps_dirty": 59, 2024-09-13T16:34:23.464 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_cap_release": 4, 2024-09-13T16:34:23.464 INFO:teuthology.orchestra.run.smithi107.stdout: "process_request_cap_release": 31 2024-09-13T16:34:23.464 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.464 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cache": { 2024-09-13T16:34:23.464 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update": 0, 2024-09-13T16:34:23.464 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update_receipt": 0, 2024-09-13T16:34:23.464 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_try_discover": 0, 2024-09-13T16:34:23.464 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_send_discover": 0, 2024-09-13T16:34:23.464 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_handle_discover": 10, 2024-09-13T16:34:23.464 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays": 0, 2024-09-13T16:34:23.464 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_delayed": 0, 2024-09-13T16:34:23.464 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_enqueuing": 0, 2024-09-13T16:34:23.465 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_created": 5, 2024-09-13T16:34:23.465 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_enqueued": 5, 2024-09-13T16:34:23.465 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_reintegrated": 0, 2024-09-13T16:34:23.465 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_migrated": 0, 2024-09-13T16:34:23.465 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_processing": 0, 2024-09-13T16:34:23.465 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_enqueued": 0, 2024-09-13T16:34:23.465 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_prioritized": 0, 2024-09-13T16:34:23.465 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_started": 0, 2024-09-13T16:34:23.465 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_completed": 0, 2024-09-13T16:34:23.465 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_enqueue_scrub": 0, 2024-09-13T16:34:23.465 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_exportdir": 0, 2024-09-13T16:34:23.465 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_flush": 0, 2024-09-13T16:34:23.465 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragmentdir": 0, 2024-09-13T16:34:23.465 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragstats": 0, 2024-09-13T16:34:23.466 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_inodestats": 0 2024-09-13T16:34:23.466 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.466 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_log": { 2024-09-13T16:34:23.466 INFO:teuthology.orchestra.run.smithi107.stdout: "evadd": 158, 2024-09-13T16:34:23.466 INFO:teuthology.orchestra.run.smithi107.stdout: "evex": 149, 2024-09-13T16:34:23.466 INFO:teuthology.orchestra.run.smithi107.stdout: "evtrm": 149, 2024-09-13T16:34:23.466 INFO:teuthology.orchestra.run.smithi107.stdout: "ev": 9, 2024-09-13T16:34:23.466 INFO:teuthology.orchestra.run.smithi107.stdout: "evexg": 0, 2024-09-13T16:34:23.466 INFO:teuthology.orchestra.run.smithi107.stdout: "evexd": 2, 2024-09-13T16:34:23.466 INFO:teuthology.orchestra.run.smithi107.stdout: "segadd": 14, 2024-09-13T16:34:23.466 INFO:teuthology.orchestra.run.smithi107.stdout: "segex": 13, 2024-09-13T16:34:23.466 INFO:teuthology.orchestra.run.smithi107.stdout: "segtrm": 13, 2024-09-13T16:34:23.466 INFO:teuthology.orchestra.run.smithi107.stdout: "seg": 2, 2024-09-13T16:34:23.466 INFO:teuthology.orchestra.run.smithi107.stdout: "segexg": 0, 2024-09-13T16:34:23.467 INFO:teuthology.orchestra.run.smithi107.stdout: "segexd": 1, 2024-09-13T16:34:23.467 INFO:teuthology.orchestra.run.smithi107.stdout: "expos": 4608085, 2024-09-13T16:34:23.467 INFO:teuthology.orchestra.run.smithi107.stdout: "wrpos": 4635576, 2024-09-13T16:34:23.467 INFO:teuthology.orchestra.run.smithi107.stdout: "rdpos": 0, 2024-09-13T16:34:23.467 INFO:teuthology.orchestra.run.smithi107.stdout: "jlat": { 2024-09-13T16:34:23.467 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 133, 2024-09-13T16:34:23.467 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.253813229, 2024-09-13T16:34:23.467 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.001908370 2024-09-13T16:34:23.467 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.467 INFO:teuthology.orchestra.run.smithi107.stdout: "replayed": 0 2024-09-13T16:34:23.467 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.467 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_mem": { 2024-09-13T16:34:23.467 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 23, 2024-09-13T16:34:23.467 INFO:teuthology.orchestra.run.smithi107.stdout: "ino+": 28, 2024-09-13T16:34:23.468 INFO:teuthology.orchestra.run.smithi107.stdout: "ino-": 5, 2024-09-13T16:34:23.468 INFO:teuthology.orchestra.run.smithi107.stdout: "dir": 18, 2024-09-13T16:34:23.468 INFO:teuthology.orchestra.run.smithi107.stdout: "dir+": 20, 2024-09-13T16:34:23.468 INFO:teuthology.orchestra.run.smithi107.stdout: "dir-": 2, 2024-09-13T16:34:23.468 INFO:teuthology.orchestra.run.smithi107.stdout: "dn": 23, 2024-09-13T16:34:23.468 INFO:teuthology.orchestra.run.smithi107.stdout: "dn+": 32, 2024-09-13T16:34:23.468 INFO:teuthology.orchestra.run.smithi107.stdout: "dn-": 9, 2024-09-13T16:34:23.468 INFO:teuthology.orchestra.run.smithi107.stdout: "cap": 2, 2024-09-13T16:34:23.468 INFO:teuthology.orchestra.run.smithi107.stdout: "cap+": 20, 2024-09-13T16:34:23.468 INFO:teuthology.orchestra.run.smithi107.stdout: "cap-": 18, 2024-09-13T16:34:23.468 INFO:teuthology.orchestra.run.smithi107.stdout: "rss": 52248, 2024-09-13T16:34:23.468 INFO:teuthology.orchestra.run.smithi107.stdout: "heap": 207124 2024-09-13T16:34:23.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.469 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_server": { 2024-09-13T16:34:23.469 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_client_request": 85, 2024-09-13T16:34:23.469 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_server_request": 1, 2024-09-13T16:34:23.469 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_request": 63, 2024-09-13T16:34:23.469 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_session": 18, 2024-09-13T16:34:23.469 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_peer_request": 2, 2024-09-13T16:34:23.469 INFO:teuthology.orchestra.run.smithi107.stdout: "req_create_latency": { 2024-09-13T16:34:23.469 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 7, 2024-09-13T16:34:23.469 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.007042922, 2024-09-13T16:34:23.469 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.001006131 2024-09-13T16:34:23.469 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.469 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getattr_latency": { 2024-09-13T16:34:23.469 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 25, 2024-09-13T16:34:23.469 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.005188587, 2024-09-13T16:34:23.470 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000207543 2024-09-13T16:34:23.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.470 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getfilelock_latency": { 2024-09-13T16:34:23.470 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:23.470 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:23.470 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:23.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.470 INFO:teuthology.orchestra.run.smithi107.stdout: "req_link_latency": { 2024-09-13T16:34:23.470 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:23.470 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:23.470 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:23.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.470 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookup_latency": { 2024-09-13T16:34:23.470 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:23.471 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:23.471 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:23.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.471 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookuphash_latency": { 2024-09-13T16:34:23.471 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:23.471 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:23.471 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:23.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.471 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupino_latency": { 2024-09-13T16:34:23.471 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:23.471 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:23.471 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:23.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.471 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupname_latency": { 2024-09-13T16:34:23.472 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:23.472 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:23.472 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:23.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.472 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupparent_latency": { 2024-09-13T16:34:23.472 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:23.472 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:23.472 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:23.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.472 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupsnap_latency": { 2024-09-13T16:34:23.472 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:23.472 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:23.472 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:23.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.472 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lssnap_latency": { 2024-09-13T16:34:23.473 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:23.473 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:23.473 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:23.473 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.473 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mkdir_latency": { 2024-09-13T16:34:23.473 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 8, 2024-09-13T16:34:23.473 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.013151827, 2024-09-13T16:34:23.473 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.001643978 2024-09-13T16:34:23.473 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.473 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mknod_latency": { 2024-09-13T16:34:23.473 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:23.473 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:23.473 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:23.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.474 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mksnap_latency": { 2024-09-13T16:34:23.474 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:23.474 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:23.474 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:23.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.474 INFO:teuthology.orchestra.run.smithi107.stdout: "req_open_latency": { 2024-09-13T16:34:23.474 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3, 2024-09-13T16:34:23.474 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.001736662, 2024-09-13T16:34:23.474 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000578887 2024-09-13T16:34:23.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.474 INFO:teuthology.orchestra.run.smithi107.stdout: "req_readdir_latency": { 2024-09-13T16:34:23.474 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2, 2024-09-13T16:34:23.474 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.003621728, 2024-09-13T16:34:23.474 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.001810864 2024-09-13T16:34:23.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.475 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rename_latency": { 2024-09-13T16:34:23.475 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2, 2024-09-13T16:34:23.475 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.001962563, 2024-09-13T16:34:23.475 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000981281 2024-09-13T16:34:23.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.475 INFO:teuthology.orchestra.run.smithi107.stdout: "req_renamesnap_latency": { 2024-09-13T16:34:23.475 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:23.475 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:23.475 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:23.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.475 INFO:teuthology.orchestra.run.smithi107.stdout: "req_snapdiff_latency": { 2024-09-13T16:34:23.475 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:23.476 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:23.476 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:23.476 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.476 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmdir_latency": { 2024-09-13T16:34:23.476 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2, 2024-09-13T16:34:23.476 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.004644962, 2024-09-13T16:34:23.476 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.002322481 2024-09-13T16:34:23.476 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.476 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmsnap_latency": { 2024-09-13T16:34:23.476 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:23.476 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:23.476 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:23.476 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.476 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmxattr_latency": { 2024-09-13T16:34:23.477 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:23.477 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:23.477 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:23.477 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.477 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setattr_latency": { 2024-09-13T16:34:23.477 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2, 2024-09-13T16:34:23.477 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.001904093, 2024-09-13T16:34:23.477 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000952046 2024-09-13T16:34:23.477 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.477 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setdirlayout_latency": { 2024-09-13T16:34:23.477 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:23.477 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:23.477 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:23.477 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.478 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setfilelock_latency": { 2024-09-13T16:34:23.478 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:23.478 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:23.478 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:23.478 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.478 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setlayout_latency": { 2024-09-13T16:34:23.478 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:23.478 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:23.478 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:23.478 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.478 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setxattr_latency": { 2024-09-13T16:34:23.478 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 9, 2024-09-13T16:34:23.478 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.019827170, 2024-09-13T16:34:23.478 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.002203018 2024-09-13T16:34:23.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.479 INFO:teuthology.orchestra.run.smithi107.stdout: "req_symlink_latency": { 2024-09-13T16:34:23.479 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:23.479 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:23.479 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:23.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.479 INFO:teuthology.orchestra.run.smithi107.stdout: "req_unlink_latency": { 2024-09-13T16:34:23.479 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3, 2024-09-13T16:34:23.479 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.006270851, 2024-09-13T16:34:23.479 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.002090283 2024-09-13T16:34:23.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.479 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_revoke_eviction": 0, 2024-09-13T16:34:23.479 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_acquisition_throttle": 0, 2024-09-13T16:34:23.479 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getvxattr_latency": { 2024-09-13T16:34:23.480 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:23.480 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:23.480 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:23.480 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:23.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.480 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_sessions": { 2024-09-13T16:34:23.480 INFO:teuthology.orchestra.run.smithi107.stdout: "session_count": 1, 2024-09-13T16:34:23.480 INFO:teuthology.orchestra.run.smithi107.stdout: "session_add": 4, 2024-09-13T16:34:23.480 INFO:teuthology.orchestra.run.smithi107.stdout: "session_remove": 3, 2024-09-13T16:34:23.480 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_open": 1, 2024-09-13T16:34:23.480 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_stale": 0, 2024-09-13T16:34:23.480 INFO:teuthology.orchestra.run.smithi107.stdout: "total_load": 14, 2024-09-13T16:34:23.480 INFO:teuthology.orchestra.run.smithi107.stdout: "average_load": 14, 2024-09-13T16:34:23.481 INFO:teuthology.orchestra.run.smithi107.stdout: "avg_session_uptime": 194, 2024-09-13T16:34:23.481 INFO:teuthology.orchestra.run.smithi107.stdout: "mdthresh_evicted": 0 2024-09-13T16:34:23.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.481 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-09-13T16:34:23.481 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-09-13T16:34:23.481 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-09-13T16:34:23.481 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T16:34:23.481 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-09-13T16:34:23.481 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T16:34:23.481 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-09-13T16:34:23.481 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T16:34:23.481 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T16:34:23.481 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T16:34:23.482 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T16:34:23.482 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T16:34:23.482 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-09-13T16:34:23.482 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T16:34:23.482 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T16:34:23.482 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-09-13T16:34:23.482 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-09-13T16:34:23.482 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-09-13T16:34:23.482 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-09-13T16:34:23.482 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T16:34:23.482 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T16:34:23.482 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T16:34:23.482 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T16:34:23.482 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T16:34:23.483 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-09-13T16:34:23.483 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-09-13T16:34:23.483 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-09-13T16:34:23.483 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T16:34:23.483 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T16:34:23.483 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-09-13T16:34:23.483 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-09-13T16:34:23.483 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-09-13T16:34:23.483 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-09-13T16:34:23.483 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T16:34:23.483 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-09-13T16:34:23.483 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T16:34:23.483 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-09-13T16:34:23.483 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 307901, 2024-09-13T16:34:23.484 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 128, 2024-09-13T16:34:23.484 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 176, 2024-09-13T16:34:23.484 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 2, 2024-09-13T16:34:23.484 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-09-13T16:34:23.484 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-09-13T16:34:23.484 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-09-13T16:34:23.484 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-09-13T16:34:23.484 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-09-13T16:34:23.484 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-09-13T16:34:23.484 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 8856, 2024-09-13T16:34:23.484 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 166, 2024-09-13T16:34:23.484 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T16:34:23.484 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-09-13T16:34:23.484 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-09-13T16:34:23.485 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-09-13T16:34:23.485 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 93210, 2024-09-13T16:34:23.485 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 619, 2024-09-13T16:34:23.485 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-09-13T16:34:23.485 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-09-13T16:34:23.485 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-09-13T16:34:23.485 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-09-13T16:34:23.485 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.485 INFO:teuthology.orchestra.run.smithi107.stdout: "objecter": { 2024-09-13T16:34:23.485 INFO:teuthology.orchestra.run.smithi107.stdout: "op_active": 0, 2024-09-13T16:34:23.485 INFO:teuthology.orchestra.run.smithi107.stdout: "op_laggy": 0, 2024-09-13T16:34:23.485 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send": 3960, 2024-09-13T16:34:23.485 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send_bytes": 541861, 2024-09-13T16:34:23.486 INFO:teuthology.orchestra.run.smithi107.stdout: "op_resend": 37, 2024-09-13T16:34:23.486 INFO:teuthology.orchestra.run.smithi107.stdout: "op_reply": 3960, 2024-09-13T16:34:23.486 INFO:teuthology.orchestra.run.smithi107.stdout: "op_latency": { 2024-09-13T16:34:23.486 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3960, 2024-09-13T16:34:23.486 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 32.592438800, 2024-09-13T16:34:23.486 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.008230413 2024-09-13T16:34:23.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.486 INFO:teuthology.orchestra.run.smithi107.stdout: "op_inflight": 0, 2024-09-13T16:34:23.486 INFO:teuthology.orchestra.run.smithi107.stdout: "oplen_avg": { 2024-09-13T16:34:23.486 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3960, 2024-09-13T16:34:23.486 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 4207 2024-09-13T16:34:23.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.486 INFO:teuthology.orchestra.run.smithi107.stdout: "op": 3960, 2024-09-13T16:34:23.486 INFO:teuthology.orchestra.run.smithi107.stdout: "op_r": 5, 2024-09-13T16:34:23.487 INFO:teuthology.orchestra.run.smithi107.stdout: "op_w": 3955, 2024-09-13T16:34:23.487 INFO:teuthology.orchestra.run.smithi107.stdout: "op_rmw": 0, 2024-09-13T16:34:23.487 INFO:teuthology.orchestra.run.smithi107.stdout: "op_pg": 0, 2024-09-13T16:34:23.487 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_stat": 70, 2024-09-13T16:34:23.487 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_create": 38, 2024-09-13T16:34:23.487 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_read": 5, 2024-09-13T16:34:23.487 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_write": 138, 2024-09-13T16:34:23.487 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writefull": 59, 2024-09-13T16:34:23.487 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writesame": 0, 2024-09-13T16:34:23.487 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_append": 0, 2024-09-13T16:34:23.487 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_zero": 0, 2024-09-13T16:34:23.487 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_truncate": 0, 2024-09-13T16:34:23.487 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_delete": 3589, 2024-09-13T16:34:23.487 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_mapext": 0, 2024-09-13T16:34:23.488 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_sparse_read": 0, 2024-09-13T16:34:23.488 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_clonerange": 0, 2024-09-13T16:34:23.488 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_getxattr": 0, 2024-09-13T16:34:23.488 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_setxattr": 69, 2024-09-13T16:34:23.488 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_cmpxattr": 0, 2024-09-13T16:34:23.488 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_rmxattr": 0, 2024-09-13T16:34:23.488 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_resetxattrs": 0, 2024-09-13T16:34:23.488 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_call": 0, 2024-09-13T16:34:23.488 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_watch": 0, 2024-09-13T16:34:23.488 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_notify": 0, 2024-09-13T16:34:23.488 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_src_cmpxattr": 0, 2024-09-13T16:34:23.488 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls": 0, 2024-09-13T16:34:23.488 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls_filter": 0, 2024-09-13T16:34:23.488 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_other": 0, 2024-09-13T16:34:23.489 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_active": 0, 2024-09-13T16:34:23.489 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_send": 0, 2024-09-13T16:34:23.489 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_resend": 0, 2024-09-13T16:34:23.489 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_ping": 0, 2024-09-13T16:34:23.489 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_active": 0, 2024-09-13T16:34:23.489 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_send": 0, 2024-09-13T16:34:23.489 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_resend": 0, 2024-09-13T16:34:23.489 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_active": 0, 2024-09-13T16:34:23.489 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_send": 0, 2024-09-13T16:34:23.489 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_resend": 0, 2024-09-13T16:34:23.489 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_active": 0, 2024-09-13T16:34:23.489 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_send": 0, 2024-09-13T16:34:23.489 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_resend": 0, 2024-09-13T16:34:23.489 INFO:teuthology.orchestra.run.smithi107.stdout: "command_active": 0, 2024-09-13T16:34:23.490 INFO:teuthology.orchestra.run.smithi107.stdout: "command_send": 0, 2024-09-13T16:34:23.490 INFO:teuthology.orchestra.run.smithi107.stdout: "command_resend": 0, 2024-09-13T16:34:23.490 INFO:teuthology.orchestra.run.smithi107.stdout: "map_epoch": 93, 2024-09-13T16:34:23.490 INFO:teuthology.orchestra.run.smithi107.stdout: "map_full": 0, 2024-09-13T16:34:23.490 INFO:teuthology.orchestra.run.smithi107.stdout: "map_inc": 9, 2024-09-13T16:34:23.490 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_sessions": 12, 2024-09-13T16:34:23.490 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_open": 12, 2024-09-13T16:34:23.490 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_close": 0, 2024-09-13T16:34:23.490 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_laggy": 0, 2024-09-13T16:34:23.490 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_wr": 220, 2024-09-13T16:34:23.490 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_rd": 0, 2024-09-13T16:34:23.490 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_del": 19 2024-09-13T16:34:23.490 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.490 INFO:teuthology.orchestra.run.smithi107.stdout: "oft": { 2024-09-13T16:34:23.490 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_objs": 1, 2024-09-13T16:34:23.491 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_kv_pairs": 0, 2024-09-13T16:34:23.491 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_updates": 0, 2024-09-13T16:34:23.491 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_removes": 0 2024-09-13T16:34:23.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.491 INFO:teuthology.orchestra.run.smithi107.stdout: "osd-slow-ops": { 2024-09-13T16:34:23.491 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_ops_count": 0 2024-09-13T16:34:23.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.491 INFO:teuthology.orchestra.run.smithi107.stdout: "purge_queue": { 2024-09-13T16:34:23.491 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops": 0, 2024-09-13T16:34:23.491 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops_high_water": 2562, 2024-09-13T16:34:23.491 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing": 0, 2024-09-13T16:34:23.491 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_high_water": 2, 2024-09-13T16:34:23.491 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed_ops": 3077, 2024-09-13T16:34:23.491 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed": 5, 2024-09-13T16:34:23.492 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_item_in_journal": 0 2024-09-13T16:34:23.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.492 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T16:34:23.492 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-13T16:34:23.492 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-09-13T16:34:23.492 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-13T16:34:23.492 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 5441, 2024-09-13T16:34:23.492 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 1876672, 2024-09-13T16:34:23.492 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:23.492 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 5441, 2024-09-13T16:34:23.492 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-13T16:34:23.492 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-13T16:34:23.492 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 5441, 2024-09-13T16:34:23.492 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 1876672, 2024-09-13T16:34:23.493 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-13T16:34:23.493 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:23.493 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:23.493 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:23.493 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:23.493 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.493 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_bytes": { 2024-09-13T16:34:23.493 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-13T16:34:23.493 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-09-13T16:34:23.493 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-13T16:34:23.493 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-09-13T16:34:23.493 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-09-13T16:34:23.493 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:23.493 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:23.493 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 3960, 2024-09-13T16:34:23.494 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 542390, 2024-09-13T16:34:23.494 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 3960, 2024-09-13T16:34:23.494 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 542390, 2024-09-13T16:34:23.494 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-13T16:34:23.494 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:23.494 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:23.494 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:23.494 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:23.494 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.494 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_ops": { 2024-09-13T16:34:23.494 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-13T16:34:23.494 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 1024, 2024-09-13T16:34:23.494 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-13T16:34:23.494 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-09-13T16:34:23.495 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-09-13T16:34:23.495 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:23.495 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:23.495 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 3960, 2024-09-13T16:34:23.495 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 3960, 2024-09-13T16:34:23.495 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 3960, 2024-09-13T16:34:23.495 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 3960, 2024-09-13T16:34:23.495 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-13T16:34:23.495 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:23.495 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:23.495 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:23.495 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:23.495 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.495 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle": { 2024-09-13T16:34:23.495 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-13T16:34:23.496 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-09-13T16:34:23.496 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-13T16:34:23.496 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 5, 2024-09-13T16:34:23.496 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 529, 2024-09-13T16:34:23.496 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:23.496 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 5, 2024-09-13T16:34:23.496 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-13T16:34:23.496 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-13T16:34:23.496 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 5, 2024-09-13T16:34:23.496 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 529, 2024-09-13T16:34:23.496 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-13T16:34:23.496 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:23.496 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:23.496 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:23.497 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:23.497 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:23.497 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle-0x55633ce10500": { 2024-09-13T16:34:23.497 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-13T16:34:23.497 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-09-13T16:34:23.497 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-13T16:34:23.497 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 158, 2024-09-13T16:34:23.497 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 441272, 2024-09-13T16:34:23.497 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:23.497 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 158, 2024-09-13T16:34:23.497 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-13T16:34:23.497 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-13T16:34:23.497 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 133, 2024-09-13T16:34:23.497 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 441272, 2024-09-13T16:34:23.498 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-13T16:34:23.498 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:23.498 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:23.498 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:23.498 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:23.498 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:23.498 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-13T16:34:24.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:23 smithi107 ceph-mon[34139]: pgmap v452: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.5 KiB/s wr, 115 op/s 2024-09-13T16:34:24.169 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-09-13T16:34:24.169 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-13T16:34:24.169 DEBUG:tasks.check_counter:Getting stats from e 2024-09-13T16:34:24.169 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-13T16:34:24.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:23 smithi016 ceph-mon[31187]: pgmap v452: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.5 KiB/s wr, 115 op/s 2024-09-13T16:34:24.371 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T16:34:24.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:23 smithi111 ceph-mon[34600]: pgmap v452: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.5 KiB/s wr, 115 op/s 2024-09-13T16:34:26.120 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-13T16:34:26.120 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T16:34:26.120 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 329, 2024-09-13T16:34:26.120 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 331, 2024-09-13T16:34:26.120 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 88340, 2024-09-13T16:34:26.120 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 102383, 2024-09-13T16:34:26.120 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 6, 2024-09-13T16:34:26.120 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 2, 2024-09-13T16:34:26.120 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.030203813, 2024-09-13T16:34:26.121 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.012124535, 2024-09-13T16:34:26.121 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.007297423, 2024-09-13T16:34:26.121 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.007187736, 2024-09-13T16:34:26.121 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:26.121 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 329, 2024-09-13T16:34:26.121 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.012607681, 2024-09-13T16:34:26.121 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000038321 2024-09-13T16:34:26.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.121 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:26.121 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.121 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.121 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.122 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-13T16:34:26.122 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 0 2024-09-13T16:34:26.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.122 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T16:34:26.122 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 356, 2024-09-13T16:34:26.122 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 345, 2024-09-13T16:34:26.122 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 815772, 2024-09-13T16:34:26.122 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 131652, 2024-09-13T16:34:26.122 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 4, 2024-09-13T16:34:26.122 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 3, 2024-09-13T16:34:26.122 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.040095992, 2024-09-13T16:34:26.122 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.013402706, 2024-09-13T16:34:26.122 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.024232511, 2024-09-13T16:34:26.122 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.007238968, 2024-09-13T16:34:26.123 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:26.123 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 344, 2024-09-13T16:34:26.123 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.014359156, 2024-09-13T16:34:26.123 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000041741 2024-09-13T16:34:26.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.123 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:26.123 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.123 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.123 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.123 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 758432, 2024-09-13T16:34:26.123 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 59184 2024-09-13T16:34:26.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.124 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T16:34:26.124 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 328, 2024-09-13T16:34:26.124 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 401, 2024-09-13T16:34:26.124 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 111980, 2024-09-13T16:34:26.124 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 300606, 2024-09-13T16:34:26.124 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 4, 2024-09-13T16:34:26.124 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 3, 2024-09-13T16:34:26.124 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.045526995, 2024-09-13T16:34:26.124 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.014954315, 2024-09-13T16:34:26.124 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.013313408, 2024-09-13T16:34:26.124 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.013054654, 2024-09-13T16:34:26.124 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:26.124 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 400, 2024-09-13T16:34:26.124 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.025488679, 2024-09-13T16:34:26.125 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000063721 2024-09-13T16:34:26.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.125 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:26.125 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.125 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.125 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.125 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 3120, 2024-09-13T16:34:26.125 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 200464 2024-09-13T16:34:26.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.125 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-09-13T16:34:26.125 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 1, 2024-09-13T16:34:26.125 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-09-13T16:34:26.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.126 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-MDSRank": { 2024-09-13T16:34:26.126 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-09-13T16:34:26.126 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-09-13T16:34:26.126 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1505, 2024-09-13T16:34:26.126 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.042414157, 2024-09-13T16:34:26.126 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000028182 2024-09-13T16:34:26.126 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:26.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.126 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-PurgeQueue": { 2024-09-13T16:34:26.126 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-09-13T16:34:26.126 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-09-13T16:34:26.126 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.126 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.127 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.127 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:26.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.127 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-09-13T16:34:26.127 INFO:teuthology.orchestra.run.smithi107.stdout: "request": 0, 2024-09-13T16:34:26.127 INFO:teuthology.orchestra.run.smithi107.stdout: "reply": 0, 2024-09-13T16:34:26.127 INFO:teuthology.orchestra.run.smithi107.stdout: "reply_latency": { 2024-09-13T16:34:26.127 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.127 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.127 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.127 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_reply": 0, 2024-09-13T16:34:26.127 INFO:teuthology.orchestra.run.smithi107.stdout: "forward": 0, 2024-09-13T16:34:26.127 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_complete": 0, 2024-09-13T16:34:26.128 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_keys": 0, 2024-09-13T16:34:26.128 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_commit": 0, 2024-09-13T16:34:26.128 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_split": 0, 2024-09-13T16:34:26.128 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_merge": 0, 2024-09-13T16:34:26.128 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes": 0, 2024-09-13T16:34:26.128 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_top": 0, 2024-09-13T16:34:26.128 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_bottom": 0, 2024-09-13T16:34:26.128 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pin_tail": 0, 2024-09-13T16:34:26.128 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pinned": 0, 2024-09-13T16:34:26.128 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_expired": 0, 2024-09-13T16:34:26.128 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_with_caps": 0, 2024-09-13T16:34:26.128 INFO:teuthology.orchestra.run.smithi107.stdout: "caps": 0, 2024-09-13T16:34:26.128 INFO:teuthology.orchestra.run.smithi107.stdout: "subtrees": 1, 2024-09-13T16:34:26.128 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse": 0, 2024-09-13T16:34:26.129 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_hit": 0, 2024-09-13T16:34:26.129 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_forward": 0, 2024-09-13T16:34:26.129 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_discover": 0, 2024-09-13T16:34:26.129 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_dir_fetch": 0, 2024-09-13T16:34:26.129 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_remote_ino": 0, 2024-09-13T16:34:26.129 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_lock": 0, 2024-09-13T16:34:26.129 INFO:teuthology.orchestra.run.smithi107.stdout: "load_cent": 0, 2024-09-13T16:34:26.129 INFO:teuthology.orchestra.run.smithi107.stdout: "q": 0, 2024-09-13T16:34:26.129 INFO:teuthology.orchestra.run.smithi107.stdout: "exported": 0, 2024-09-13T16:34:26.129 INFO:teuthology.orchestra.run.smithi107.stdout: "exported_inodes": 0, 2024-09-13T16:34:26.129 INFO:teuthology.orchestra.run.smithi107.stdout: "imported": 0, 2024-09-13T16:34:26.129 INFO:teuthology.orchestra.run.smithi107.stdout: "imported_inodes": 0, 2024-09-13T16:34:26.129 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_dir_fetch": 0, 2024-09-13T16:34:26.130 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_backtrace_fetch": 0, 2024-09-13T16:34:26.130 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_peer_discover": 0, 2024-09-13T16:34:26.130 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rfiles": 0, 2024-09-13T16:34:26.130 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rbytes": 0, 2024-09-13T16:34:26.130 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rsnaps": 0, 2024-09-13T16:34:26.130 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_fetch": 0, 2024-09-13T16:34:26.130 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_set_tag": 0, 2024-09-13T16:34:26.130 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_repaired": 0, 2024-09-13T16:34:26.130 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_inotable_repaired": 0, 2024-09-13T16:34:26.130 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_inodes": 0, 2024-09-13T16:34:26.130 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_base_inodes": 0, 2024-09-13T16:34:26.130 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dirfrag_rstats": 0, 2024-09-13T16:34:26.130 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_file_inodes": 0, 2024-09-13T16:34:26.130 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_inode_file_caps": 0, 2024-09-13T16:34:26.131 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_revoke": 0, 2024-09-13T16:34:26.131 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_grant": 0, 2024-09-13T16:34:26.131 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_trunc": 0, 2024-09-13T16:34:26.131 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-13T16:34:26.131 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-13T16:34:26.131 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps": 0, 2024-09-13T16:34:26.131 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps_dirty": 0, 2024-09-13T16:34:26.131 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_cap_release": 0, 2024-09-13T16:34:26.131 INFO:teuthology.orchestra.run.smithi107.stdout: "process_request_cap_release": 0 2024-09-13T16:34:26.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.131 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cache": { 2024-09-13T16:34:26.131 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update": 0, 2024-09-13T16:34:26.131 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update_receipt": 0, 2024-09-13T16:34:26.131 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_try_discover": 0, 2024-09-13T16:34:26.132 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_send_discover": 0, 2024-09-13T16:34:26.132 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_handle_discover": 0, 2024-09-13T16:34:26.132 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays": 0, 2024-09-13T16:34:26.132 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_delayed": 0, 2024-09-13T16:34:26.132 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_enqueuing": 0, 2024-09-13T16:34:26.132 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_created": 0, 2024-09-13T16:34:26.132 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_enqueued": 0, 2024-09-13T16:34:26.132 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_reintegrated": 0, 2024-09-13T16:34:26.132 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_migrated": 0, 2024-09-13T16:34:26.132 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_processing": 0, 2024-09-13T16:34:26.132 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_enqueued": 0, 2024-09-13T16:34:26.132 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_prioritized": 0, 2024-09-13T16:34:26.132 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_started": 0, 2024-09-13T16:34:26.132 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_completed": 0, 2024-09-13T16:34:26.133 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_enqueue_scrub": 0, 2024-09-13T16:34:26.133 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_exportdir": 0, 2024-09-13T16:34:26.133 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_flush": 0, 2024-09-13T16:34:26.133 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragmentdir": 0, 2024-09-13T16:34:26.133 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragstats": 0, 2024-09-13T16:34:26.133 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_inodestats": 0 2024-09-13T16:34:26.133 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.133 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_log": { 2024-09-13T16:34:26.133 INFO:teuthology.orchestra.run.smithi107.stdout: "evadd": 0, 2024-09-13T16:34:26.133 INFO:teuthology.orchestra.run.smithi107.stdout: "evex": 0, 2024-09-13T16:34:26.133 INFO:teuthology.orchestra.run.smithi107.stdout: "evtrm": 0, 2024-09-13T16:34:26.133 INFO:teuthology.orchestra.run.smithi107.stdout: "ev": 4, 2024-09-13T16:34:26.133 INFO:teuthology.orchestra.run.smithi107.stdout: "evexg": 0, 2024-09-13T16:34:26.133 INFO:teuthology.orchestra.run.smithi107.stdout: "evexd": 0, 2024-09-13T16:34:26.134 INFO:teuthology.orchestra.run.smithi107.stdout: "segadd": 0, 2024-09-13T16:34:26.134 INFO:teuthology.orchestra.run.smithi107.stdout: "segex": 0, 2024-09-13T16:34:26.134 INFO:teuthology.orchestra.run.smithi107.stdout: "segtrm": 0, 2024-09-13T16:34:26.134 INFO:teuthology.orchestra.run.smithi107.stdout: "seg": 1, 2024-09-13T16:34:26.134 INFO:teuthology.orchestra.run.smithi107.stdout: "segexg": 0, 2024-09-13T16:34:26.134 INFO:teuthology.orchestra.run.smithi107.stdout: "segexd": 0, 2024-09-13T16:34:26.134 INFO:teuthology.orchestra.run.smithi107.stdout: "expos": 4194304, 2024-09-13T16:34:26.134 INFO:teuthology.orchestra.run.smithi107.stdout: "wrpos": 4195871, 2024-09-13T16:34:26.134 INFO:teuthology.orchestra.run.smithi107.stdout: "rdpos": 4195377, 2024-09-13T16:34:26.134 INFO:teuthology.orchestra.run.smithi107.stdout: "jlat": { 2024-09-13T16:34:26.134 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.134 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.134 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.134 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.135 INFO:teuthology.orchestra.run.smithi107.stdout: "replayed": 4 2024-09-13T16:34:26.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.135 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_mem": { 2024-09-13T16:34:26.135 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 3, 2024-09-13T16:34:26.135 INFO:teuthology.orchestra.run.smithi107.stdout: "ino+": 3, 2024-09-13T16:34:26.135 INFO:teuthology.orchestra.run.smithi107.stdout: "ino-": 0, 2024-09-13T16:34:26.135 INFO:teuthology.orchestra.run.smithi107.stdout: "dir": 1, 2024-09-13T16:34:26.135 INFO:teuthology.orchestra.run.smithi107.stdout: "dir+": 1, 2024-09-13T16:34:26.135 INFO:teuthology.orchestra.run.smithi107.stdout: "dir-": 0, 2024-09-13T16:34:26.135 INFO:teuthology.orchestra.run.smithi107.stdout: "dn": 0, 2024-09-13T16:34:26.135 INFO:teuthology.orchestra.run.smithi107.stdout: "dn+": 0, 2024-09-13T16:34:26.135 INFO:teuthology.orchestra.run.smithi107.stdout: "dn-": 0, 2024-09-13T16:34:26.135 INFO:teuthology.orchestra.run.smithi107.stdout: "cap": 0, 2024-09-13T16:34:26.135 INFO:teuthology.orchestra.run.smithi107.stdout: "cap+": 0, 2024-09-13T16:34:26.136 INFO:teuthology.orchestra.run.smithi107.stdout: "cap-": 0, 2024-09-13T16:34:26.136 INFO:teuthology.orchestra.run.smithi107.stdout: "rss": 47784, 2024-09-13T16:34:26.136 INFO:teuthology.orchestra.run.smithi107.stdout: "heap": 207124 2024-09-13T16:34:26.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.136 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_server": { 2024-09-13T16:34:26.136 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_client_request": 0, 2024-09-13T16:34:26.136 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_server_request": 0, 2024-09-13T16:34:26.136 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_request": 0, 2024-09-13T16:34:26.137 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_session": 0, 2024-09-13T16:34:26.137 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_peer_request": 0, 2024-09-13T16:34:26.137 INFO:teuthology.orchestra.run.smithi107.stdout: "req_create_latency": { 2024-09-13T16:34:26.137 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.137 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.137 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.137 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getattr_latency": { 2024-09-13T16:34:26.137 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.137 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.137 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.138 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getfilelock_latency": { 2024-09-13T16:34:26.138 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.138 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.138 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.138 INFO:teuthology.orchestra.run.smithi107.stdout: "req_link_latency": { 2024-09-13T16:34:26.138 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.138 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.138 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.138 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookup_latency": { 2024-09-13T16:34:26.138 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.138 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.139 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.139 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookuphash_latency": { 2024-09-13T16:34:26.139 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.139 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.139 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.139 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupino_latency": { 2024-09-13T16:34:26.139 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.139 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.139 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.139 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupname_latency": { 2024-09-13T16:34:26.140 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.140 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.140 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.140 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupparent_latency": { 2024-09-13T16:34:26.140 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.140 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.140 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.140 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupsnap_latency": { 2024-09-13T16:34:26.140 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.140 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.140 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.141 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lssnap_latency": { 2024-09-13T16:34:26.141 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.141 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.141 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.141 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mkdir_latency": { 2024-09-13T16:34:26.141 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.141 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.141 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.141 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mknod_latency": { 2024-09-13T16:34:26.142 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:25 smithi107 ceph-mon[34139]: pgmap v453: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 14 op/s 2024-09-13T16:34:26.142 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.142 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.142 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.143 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mksnap_latency": { 2024-09-13T16:34:26.143 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.143 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.143 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.143 INFO:teuthology.orchestra.run.smithi107.stdout: "req_open_latency": { 2024-09-13T16:34:26.143 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.143 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.143 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.143 INFO:teuthology.orchestra.run.smithi107.stdout: "req_readdir_latency": { 2024-09-13T16:34:26.143 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.143 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.143 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.144 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rename_latency": { 2024-09-13T16:34:26.144 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.144 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.144 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.144 INFO:teuthology.orchestra.run.smithi107.stdout: "req_renamesnap_latency": { 2024-09-13T16:34:26.144 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.144 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.144 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.144 INFO:teuthology.orchestra.run.smithi107.stdout: "req_snapdiff_latency": { 2024-09-13T16:34:26.144 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.144 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.145 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.145 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.145 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmdir_latency": { 2024-09-13T16:34:26.145 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.145 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.145 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.145 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.145 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmsnap_latency": { 2024-09-13T16:34:26.145 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.145 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.145 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.145 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.145 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmxattr_latency": { 2024-09-13T16:34:26.146 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.146 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.146 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.146 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.146 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setattr_latency": { 2024-09-13T16:34:26.146 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.146 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.146 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.146 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.146 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setdirlayout_latency": { 2024-09-13T16:34:26.146 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.146 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.146 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.146 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.147 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setfilelock_latency": { 2024-09-13T16:34:26.147 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.147 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.147 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.147 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.147 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setlayout_latency": { 2024-09-13T16:34:26.147 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.147 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.147 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.147 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.147 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setxattr_latency": { 2024-09-13T16:34:26.147 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.147 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.147 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.147 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.148 INFO:teuthology.orchestra.run.smithi107.stdout: "req_symlink_latency": { 2024-09-13T16:34:26.148 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.148 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.148 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.148 INFO:teuthology.orchestra.run.smithi107.stdout: "req_unlink_latency": { 2024-09-13T16:34:26.148 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.148 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.148 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.148 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_revoke_eviction": 0, 2024-09-13T16:34:26.148 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_acquisition_throttle": 0, 2024-09-13T16:34:26.148 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getvxattr_latency": { 2024-09-13T16:34:26.149 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.149 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.149 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.149 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:26.149 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.149 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_sessions": { 2024-09-13T16:34:26.149 INFO:teuthology.orchestra.run.smithi107.stdout: "session_count": 1, 2024-09-13T16:34:26.149 INFO:teuthology.orchestra.run.smithi107.stdout: "session_add": 2, 2024-09-13T16:34:26.149 INFO:teuthology.orchestra.run.smithi107.stdout: "session_remove": 1, 2024-09-13T16:34:26.149 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_open": 1, 2024-09-13T16:34:26.149 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_stale": 0, 2024-09-13T16:34:26.149 INFO:teuthology.orchestra.run.smithi107.stdout: "total_load": 0, 2024-09-13T16:34:26.149 INFO:teuthology.orchestra.run.smithi107.stdout: "average_load": 0, 2024-09-13T16:34:26.149 INFO:teuthology.orchestra.run.smithi107.stdout: "avg_session_uptime": 1214, 2024-09-13T16:34:26.150 INFO:teuthology.orchestra.run.smithi107.stdout: "mdthresh_evicted": 0 2024-09-13T16:34:26.150 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.150 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-09-13T16:34:26.150 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-09-13T16:34:26.150 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-09-13T16:34:26.150 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T16:34:26.150 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-09-13T16:34:26.150 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T16:34:26.150 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-09-13T16:34:26.150 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T16:34:26.150 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T16:34:26.150 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T16:34:26.150 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T16:34:26.151 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T16:34:26.151 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-09-13T16:34:26.151 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T16:34:26.151 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T16:34:26.151 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-09-13T16:34:26.151 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-09-13T16:34:26.151 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-09-13T16:34:26.151 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-09-13T16:34:26.151 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T16:34:26.151 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T16:34:26.151 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T16:34:26.151 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T16:34:26.151 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T16:34:26.151 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-09-13T16:34:26.152 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-09-13T16:34:26.152 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-09-13T16:34:26.152 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T16:34:26.152 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T16:34:26.152 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-09-13T16:34:26.152 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-09-13T16:34:26.152 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-09-13T16:34:26.152 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-09-13T16:34:26.152 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T16:34:26.152 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-09-13T16:34:26.152 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T16:34:26.152 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-09-13T16:34:26.152 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 189257, 2024-09-13T16:34:26.152 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 49, 2024-09-13T16:34:26.153 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-09-13T16:34:26.153 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-09-13T16:34:26.153 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-09-13T16:34:26.153 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-09-13T16:34:26.153 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-09-13T16:34:26.153 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-09-13T16:34:26.153 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-09-13T16:34:26.153 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-09-13T16:34:26.153 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 8856, 2024-09-13T16:34:26.153 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 166, 2024-09-13T16:34:26.153 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T16:34:26.153 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-09-13T16:34:26.153 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-09-13T16:34:26.153 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-09-13T16:34:26.154 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 8368, 2024-09-13T16:34:26.154 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 13, 2024-09-13T16:34:26.154 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-09-13T16:34:26.154 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-09-13T16:34:26.154 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-09-13T16:34:26.154 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-09-13T16:34:26.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.154 INFO:teuthology.orchestra.run.smithi107.stdout: "objecter": { 2024-09-13T16:34:26.154 INFO:teuthology.orchestra.run.smithi107.stdout: "op_active": 0, 2024-09-13T16:34:26.154 INFO:teuthology.orchestra.run.smithi107.stdout: "op_laggy": 0, 2024-09-13T16:34:26.154 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send": 884, 2024-09-13T16:34:26.154 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send_bytes": 21, 2024-09-13T16:34:26.154 INFO:teuthology.orchestra.run.smithi107.stdout: "op_resend": 3, 2024-09-13T16:34:26.154 INFO:teuthology.orchestra.run.smithi107.stdout: "op_reply": 884, 2024-09-13T16:34:26.155 INFO:teuthology.orchestra.run.smithi107.stdout: "op_latency": { 2024-09-13T16:34:26.155 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 884, 2024-09-13T16:34:26.155 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.427992640, 2024-09-13T16:34:26.155 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000484154 2024-09-13T16:34:26.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.155 INFO:teuthology.orchestra.run.smithi107.stdout: "op_inflight": 0, 2024-09-13T16:34:26.155 INFO:teuthology.orchestra.run.smithi107.stdout: "oplen_avg": { 2024-09-13T16:34:26.155 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 884, 2024-09-13T16:34:26.155 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 885 2024-09-13T16:34:26.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.155 INFO:teuthology.orchestra.run.smithi107.stdout: "op": 884, 2024-09-13T16:34:26.155 INFO:teuthology.orchestra.run.smithi107.stdout: "op_r": 884, 2024-09-13T16:34:26.155 INFO:teuthology.orchestra.run.smithi107.stdout: "op_w": 0, 2024-09-13T16:34:26.155 INFO:teuthology.orchestra.run.smithi107.stdout: "op_rmw": 0, 2024-09-13T16:34:26.155 INFO:teuthology.orchestra.run.smithi107.stdout: "op_pg": 0, 2024-09-13T16:34:26.156 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_stat": 553, 2024-09-13T16:34:26.156 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_create": 0, 2024-09-13T16:34:26.156 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_read": 329, 2024-09-13T16:34:26.156 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_write": 0, 2024-09-13T16:34:26.156 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writefull": 0, 2024-09-13T16:34:26.156 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writesame": 0, 2024-09-13T16:34:26.156 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_append": 0, 2024-09-13T16:34:26.156 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_zero": 0, 2024-09-13T16:34:26.156 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_truncate": 0, 2024-09-13T16:34:26.156 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_delete": 0, 2024-09-13T16:34:26.156 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_mapext": 0, 2024-09-13T16:34:26.156 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_sparse_read": 0, 2024-09-13T16:34:26.156 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_clonerange": 0, 2024-09-13T16:34:26.157 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_getxattr": 1, 2024-09-13T16:34:26.157 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_setxattr": 0, 2024-09-13T16:34:26.157 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_cmpxattr": 0, 2024-09-13T16:34:26.157 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_rmxattr": 0, 2024-09-13T16:34:26.157 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_resetxattrs": 0, 2024-09-13T16:34:26.157 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_call": 0, 2024-09-13T16:34:26.157 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_watch": 0, 2024-09-13T16:34:26.157 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_notify": 0, 2024-09-13T16:34:26.157 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_src_cmpxattr": 0, 2024-09-13T16:34:26.157 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls": 0, 2024-09-13T16:34:26.157 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls_filter": 0, 2024-09-13T16:34:26.157 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_other": 0, 2024-09-13T16:34:26.157 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_active": 0, 2024-09-13T16:34:26.157 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_send": 0, 2024-09-13T16:34:26.157 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_resend": 0, 2024-09-13T16:34:26.158 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_ping": 0, 2024-09-13T16:34:26.158 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_active": 0, 2024-09-13T16:34:26.158 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_send": 0, 2024-09-13T16:34:26.158 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_resend": 0, 2024-09-13T16:34:26.158 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_active": 0, 2024-09-13T16:34:26.158 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_send": 0, 2024-09-13T16:34:26.158 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_resend": 0, 2024-09-13T16:34:26.158 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_active": 0, 2024-09-13T16:34:26.158 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_send": 0, 2024-09-13T16:34:26.158 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_resend": 0, 2024-09-13T16:34:26.158 INFO:teuthology.orchestra.run.smithi107.stdout: "command_active": 0, 2024-09-13T16:34:26.159 INFO:teuthology.orchestra.run.smithi107.stdout: "command_send": 0, 2024-09-13T16:34:26.159 INFO:teuthology.orchestra.run.smithi107.stdout: "command_resend": 0, 2024-09-13T16:34:26.159 INFO:teuthology.orchestra.run.smithi107.stdout: "map_epoch": 93, 2024-09-13T16:34:26.159 INFO:teuthology.orchestra.run.smithi107.stdout: "map_full": 0, 2024-09-13T16:34:26.159 INFO:teuthology.orchestra.run.smithi107.stdout: "map_inc": 4, 2024-09-13T16:34:26.159 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_sessions": 5, 2024-09-13T16:34:26.159 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_open": 5, 2024-09-13T16:34:26.159 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_close": 0, 2024-09-13T16:34:26.159 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_laggy": 0, 2024-09-13T16:34:26.159 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_wr": 0, 2024-09-13T16:34:26.159 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_rd": 2, 2024-09-13T16:34:26.159 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_del": 0 2024-09-13T16:34:26.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.160 INFO:teuthology.orchestra.run.smithi107.stdout: "oft": { 2024-09-13T16:34:26.160 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_objs": 0, 2024-09-13T16:34:26.160 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_kv_pairs": 0, 2024-09-13T16:34:26.160 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_updates": 0, 2024-09-13T16:34:26.160 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_removes": 0 2024-09-13T16:34:26.160 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.160 INFO:teuthology.orchestra.run.smithi107.stdout: "osd-slow-ops": { 2024-09-13T16:34:26.160 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_ops_count": 0 2024-09-13T16:34:26.160 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.160 INFO:teuthology.orchestra.run.smithi107.stdout: "purge_queue": { 2024-09-13T16:34:26.160 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops": 0, 2024-09-13T16:34:26.160 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops_high_water": 0, 2024-09-13T16:34:26.160 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing": 0, 2024-09-13T16:34:26.160 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_high_water": 0, 2024-09-13T16:34:26.161 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed_ops": 0, 2024-09-13T16:34:26.161 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed": 0, 2024-09-13T16:34:26.161 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_item_in_journal": 0 2024-09-13T16:34:26.161 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.161 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T16:34:26.161 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-13T16:34:26.161 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-09-13T16:34:26.161 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-13T16:34:26.161 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 1009, 2024-09-13T16:34:26.161 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 916017, 2024-09-13T16:34:26.161 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:26.161 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 1009, 2024-09-13T16:34:26.161 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-13T16:34:26.162 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-13T16:34:26.162 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 1009, 2024-09-13T16:34:26.162 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 916017, 2024-09-13T16:34:26.162 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-13T16:34:26.162 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.162 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.162 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.162 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:26.162 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.162 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_bytes": { 2024-09-13T16:34:26.162 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-13T16:34:26.162 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-09-13T16:34:26.162 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-13T16:34:26.162 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-09-13T16:34:26.163 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-09-13T16:34:26.163 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:26.163 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:26.163 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 884, 2024-09-13T16:34:26.163 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 1572, 2024-09-13T16:34:26.163 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 884, 2024-09-13T16:34:26.163 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 1572, 2024-09-13T16:34:26.163 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-13T16:34:26.163 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.163 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.163 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.163 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:26.163 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.163 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_ops": { 2024-09-13T16:34:26.164 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-13T16:34:26.164 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 1024, 2024-09-13T16:34:26.164 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-13T16:34:26.164 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-09-13T16:34:26.164 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-09-13T16:34:26.164 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:26.164 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:26.164 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 884, 2024-09-13T16:34:26.164 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 884, 2024-09-13T16:34:26.164 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 884, 2024-09-13T16:34:26.164 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 884, 2024-09-13T16:34:26.164 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-13T16:34:26.164 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.164 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.165 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.165 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:26.165 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.165 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle": { 2024-09-13T16:34:26.165 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-13T16:34:26.165 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-09-13T16:34:26.165 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-13T16:34:26.165 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-09-13T16:34:26.165 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-09-13T16:34:26.165 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:26.165 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:26.165 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-13T16:34:26.165 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-13T16:34:26.165 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 0, 2024-09-13T16:34:26.166 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 0, 2024-09-13T16:34:26.166 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-13T16:34:26.166 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.166 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.166 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.166 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:26.166 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:26.166 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle-0x55bca44a8820": { 2024-09-13T16:34:26.166 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-13T16:34:26.166 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-09-13T16:34:26.166 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-13T16:34:26.166 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-09-13T16:34:26.166 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-09-13T16:34:26.166 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:26.167 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:26.167 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-13T16:34:26.167 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-13T16:34:26.167 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 0, 2024-09-13T16:34:26.167 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 0, 2024-09-13T16:34:26.167 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-13T16:34:26.167 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:26.167 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:26.167 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:26.167 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:26.167 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:26.167 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-13T16:34:26.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:25 smithi016 ceph-mon[31187]: pgmap v453: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 14 op/s 2024-09-13T16:34:26.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:25 smithi111 ceph-mon[34600]: pgmap v453: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 14 op/s 2024-09-13T16:34:27.038 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-13T16:34:27.038 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-13T16:34:27.038 DEBUG:tasks.check_counter:Getting stats from h 2024-09-13T16:34:27.039 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-13T16:34:27.238 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T16:34:28.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:27 smithi107 ceph-mon[34139]: pgmap v454: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-13T16:34:28.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:27 smithi016 ceph-mon[31187]: pgmap v454: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-13T16:34:28.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:27 smithi111 ceph-mon[34600]: pgmap v454: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-13T16:34:29.030 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-13T16:34:29.030 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T16:34:29.030 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 35, 2024-09-13T16:34:29.030 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 37, 2024-09-13T16:34:29.030 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 8332, 2024-09-13T16:34:29.030 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 51818, 2024-09-13T16:34:29.030 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 14, 2024-09-13T16:34:29.030 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 5, 2024-09-13T16:34:29.031 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.007182372, 2024-09-13T16:34:29.031 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.000776169, 2024-09-13T16:34:29.031 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.004516126, 2024-09-13T16:34:29.031 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000432694, 2024-09-13T16:34:29.031 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:29.031 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 35, 2024-09-13T16:34:29.031 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.014592652, 2024-09-13T16:34:29.031 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000416932 2024-09-13T16:34:29.031 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.031 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:29.031 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 23, 2024-09-13T16:34:29.031 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000004386, 2024-09-13T16:34:29.031 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000190 2024-09-13T16:34:29.032 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.032 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 928, 2024-09-13T16:34:29.032 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 736 2024-09-13T16:34:29.032 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.032 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T16:34:29.032 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 166, 2024-09-13T16:34:29.032 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 152, 2024-09-13T16:34:29.032 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 771378, 2024-09-13T16:34:29.032 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 75333, 2024-09-13T16:34:29.032 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 5, 2024-09-13T16:34:29.032 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 4, 2024-09-13T16:34:29.032 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.023260689, 2024-09-13T16:34:29.032 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.007666881, 2024-09-13T16:34:29.032 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.021392531, 2024-09-13T16:34:29.033 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000372994, 2024-09-13T16:34:29.033 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:29.033 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 152, 2024-09-13T16:34:29.033 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.021546287, 2024-09-13T16:34:29.033 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000141751 2024-09-13T16:34:29.033 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.033 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:29.033 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.033 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.033 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.033 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.033 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 767296, 2024-09-13T16:34:29.033 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 62880 2024-09-13T16:34:29.033 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.034 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T16:34:29.034 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 102, 2024-09-13T16:34:29.034 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 260, 2024-09-13T16:34:29.034 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 31999, 2024-09-13T16:34:29.034 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 293766, 2024-09-13T16:34:29.034 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 6, 2024-09-13T16:34:29.034 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 4, 2024-09-13T16:34:29.034 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.025700796, 2024-09-13T16:34:29.034 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.012877588, 2024-09-13T16:34:29.034 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.008874290, 2024-09-13T16:34:29.034 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.001717273, 2024-09-13T16:34:29.034 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:29.034 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 258, 2024-09-13T16:34:29.034 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.018821129, 2024-09-13T16:34:29.035 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000072950 2024-09-13T16:34:29.035 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.035 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:29.035 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 271, 2024-09-13T16:34:29.035 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000072908, 2024-09-13T16:34:29.035 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000269 2024-09-13T16:34:29.035 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.035 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-13T16:34:29.035 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 201424 2024-09-13T16:34:29.035 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.035 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-09-13T16:34:29.035 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 1, 2024-09-13T16:34:29.035 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-09-13T16:34:29.035 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.036 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-MDSRank": { 2024-09-13T16:34:29.036 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-09-13T16:34:29.036 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-09-13T16:34:29.036 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 54, 2024-09-13T16:34:29.036 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.001461573, 2024-09-13T16:34:29.036 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000027066 2024-09-13T16:34:29.036 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:29.036 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.036 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-PurgeQueue": { 2024-09-13T16:34:29.036 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-09-13T16:34:29.036 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-09-13T16:34:29.036 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2, 2024-09-13T16:34:29.036 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000013494, 2024-09-13T16:34:29.037 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000006747 2024-09-13T16:34:29.037 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:29.037 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.037 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-09-13T16:34:29.037 INFO:teuthology.orchestra.run.smithi107.stdout: "request": 1, 2024-09-13T16:34:29.037 INFO:teuthology.orchestra.run.smithi107.stdout: "reply": 1, 2024-09-13T16:34:29.037 INFO:teuthology.orchestra.run.smithi107.stdout: "reply_latency": { 2024-09-13T16:34:29.037 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1, 2024-09-13T16:34:29.037 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000523217, 2024-09-13T16:34:29.037 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000523217 2024-09-13T16:34:29.038 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.038 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_reply": 0, 2024-09-13T16:34:29.038 INFO:teuthology.orchestra.run.smithi107.stdout: "forward": 0, 2024-09-13T16:34:29.038 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_complete": 0, 2024-09-13T16:34:29.038 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_keys": 0, 2024-09-13T16:34:29.038 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_commit": 11, 2024-09-13T16:34:29.038 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_split": 0, 2024-09-13T16:34:29.038 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_merge": 0, 2024-09-13T16:34:29.038 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes": 11, 2024-09-13T16:34:29.038 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_top": 0, 2024-09-13T16:34:29.038 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_bottom": 11, 2024-09-13T16:34:29.038 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pin_tail": 0, 2024-09-13T16:34:29.038 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pinned": 11, 2024-09-13T16:34:29.038 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_expired": 0, 2024-09-13T16:34:29.039 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_with_caps": 1, 2024-09-13T16:34:29.039 INFO:teuthology.orchestra.run.smithi107.stdout: "caps": 1, 2024-09-13T16:34:29.039 INFO:teuthology.orchestra.run.smithi107.stdout: "subtrees": 2, 2024-09-13T16:34:29.039 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse": 2, 2024-09-13T16:34:29.039 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_hit": 1, 2024-09-13T16:34:29.039 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_forward": 0, 2024-09-13T16:34:29.039 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_discover": 1, 2024-09-13T16:34:29.039 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_dir_fetch": 0, 2024-09-13T16:34:29.039 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_remote_ino": 0, 2024-09-13T16:34:29.039 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_lock": 0, 2024-09-13T16:34:29.039 INFO:teuthology.orchestra.run.smithi107.stdout: "load_cent": 0, 2024-09-13T16:34:29.039 INFO:teuthology.orchestra.run.smithi107.stdout: "q": 0, 2024-09-13T16:34:29.039 INFO:teuthology.orchestra.run.smithi107.stdout: "exported": 0, 2024-09-13T16:34:29.039 INFO:teuthology.orchestra.run.smithi107.stdout: "exported_inodes": 0, 2024-09-13T16:34:29.040 INFO:teuthology.orchestra.run.smithi107.stdout: "imported": 0, 2024-09-13T16:34:29.040 INFO:teuthology.orchestra.run.smithi107.stdout: "imported_inodes": 0, 2024-09-13T16:34:29.040 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_dir_fetch": 0, 2024-09-13T16:34:29.040 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_backtrace_fetch": 0, 2024-09-13T16:34:29.040 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_peer_discover": 0, 2024-09-13T16:34:29.040 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rfiles": 5, 2024-09-13T16:34:29.040 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rbytes": 5026218218, 2024-09-13T16:34:29.040 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rsnaps": 0, 2024-09-13T16:34:29.040 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_fetch": 0, 2024-09-13T16:34:29.040 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_set_tag": 0, 2024-09-13T16:34:29.040 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_repaired": 0, 2024-09-13T16:34:29.040 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_inotable_repaired": 0, 2024-09-13T16:34:29.040 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_inodes": 0, 2024-09-13T16:34:29.040 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_base_inodes": 0, 2024-09-13T16:34:29.041 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dirfrag_rstats": 0, 2024-09-13T16:34:29.041 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_file_inodes": 0, 2024-09-13T16:34:29.041 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_inode_file_caps": 0, 2024-09-13T16:34:29.041 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_revoke": 0, 2024-09-13T16:34:29.041 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_grant": 0, 2024-09-13T16:34:29.041 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_trunc": 0, 2024-09-13T16:34:29.041 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-13T16:34:29.041 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-13T16:34:29.041 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps": 0, 2024-09-13T16:34:29.041 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps_dirty": 0, 2024-09-13T16:34:29.041 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_cap_release": 0, 2024-09-13T16:34:29.044 INFO:teuthology.orchestra.run.smithi107.stdout: "process_request_cap_release": 0 2024-09-13T16:34:29.044 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.044 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cache": { 2024-09-13T16:34:29.044 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update": 0, 2024-09-13T16:34:29.044 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update_receipt": 0, 2024-09-13T16:34:29.044 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_try_discover": 0, 2024-09-13T16:34:29.044 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_send_discover": 2, 2024-09-13T16:34:29.044 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_handle_discover": 0, 2024-09-13T16:34:29.044 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays": 0, 2024-09-13T16:34:29.044 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_delayed": 0, 2024-09-13T16:34:29.044 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_enqueuing": 0, 2024-09-13T16:34:29.044 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_created": 0, 2024-09-13T16:34:29.045 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_enqueued": 0, 2024-09-13T16:34:29.045 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_reintegrated": 0, 2024-09-13T16:34:29.045 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_migrated": 0, 2024-09-13T16:34:29.045 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_processing": 0, 2024-09-13T16:34:29.045 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_enqueued": 0, 2024-09-13T16:34:29.045 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_prioritized": 0, 2024-09-13T16:34:29.045 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_started": 0, 2024-09-13T16:34:29.045 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_completed": 0, 2024-09-13T16:34:29.045 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_enqueue_scrub": 0, 2024-09-13T16:34:29.045 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_exportdir": 0, 2024-09-13T16:34:29.045 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_flush": 0, 2024-09-13T16:34:29.045 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragmentdir": 0, 2024-09-13T16:34:29.045 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragstats": 0, 2024-09-13T16:34:29.045 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_inodestats": 0 2024-09-13T16:34:29.046 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.046 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_log": { 2024-09-13T16:34:29.046 INFO:teuthology.orchestra.run.smithi107.stdout: "evadd": 4, 2024-09-13T16:34:29.046 INFO:teuthology.orchestra.run.smithi107.stdout: "evex": 0, 2024-09-13T16:34:29.046 INFO:teuthology.orchestra.run.smithi107.stdout: "evtrm": 0, 2024-09-13T16:34:29.046 INFO:teuthology.orchestra.run.smithi107.stdout: "ev": 4, 2024-09-13T16:34:29.046 INFO:teuthology.orchestra.run.smithi107.stdout: "evexg": 0, 2024-09-13T16:34:29.046 INFO:teuthology.orchestra.run.smithi107.stdout: "evexd": 0, 2024-09-13T16:34:29.046 INFO:teuthology.orchestra.run.smithi107.stdout: "segadd": 1, 2024-09-13T16:34:29.046 INFO:teuthology.orchestra.run.smithi107.stdout: "segex": 0, 2024-09-13T16:34:29.046 INFO:teuthology.orchestra.run.smithi107.stdout: "segtrm": 0, 2024-09-13T16:34:29.046 INFO:teuthology.orchestra.run.smithi107.stdout: "seg": 1, 2024-09-13T16:34:29.046 INFO:teuthology.orchestra.run.smithi107.stdout: "segexg": 0, 2024-09-13T16:34:29.046 INFO:teuthology.orchestra.run.smithi107.stdout: "segexd": 0, 2024-09-13T16:34:29.046 INFO:teuthology.orchestra.run.smithi107.stdout: "expos": 4194304, 2024-09-13T16:34:29.047 INFO:teuthology.orchestra.run.smithi107.stdout: "wrpos": 4195871, 2024-09-13T16:34:29.047 INFO:teuthology.orchestra.run.smithi107.stdout: "rdpos": 0, 2024-09-13T16:34:29.047 INFO:teuthology.orchestra.run.smithi107.stdout: "jlat": { 2024-09-13T16:34:29.047 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 4, 2024-09-13T16:34:29.047 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.007676400, 2024-09-13T16:34:29.047 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.001919100 2024-09-13T16:34:29.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.047 INFO:teuthology.orchestra.run.smithi107.stdout: "replayed": 0 2024-09-13T16:34:29.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.047 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_mem": { 2024-09-13T16:34:29.048 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 14, 2024-09-13T16:34:29.048 INFO:teuthology.orchestra.run.smithi107.stdout: "ino+": 14, 2024-09-13T16:34:29.048 INFO:teuthology.orchestra.run.smithi107.stdout: "ino-": 0, 2024-09-13T16:34:29.048 INFO:teuthology.orchestra.run.smithi107.stdout: "dir": 12, 2024-09-13T16:34:29.048 INFO:teuthology.orchestra.run.smithi107.stdout: "dir+": 13, 2024-09-13T16:34:29.048 INFO:teuthology.orchestra.run.smithi107.stdout: "dir-": 1, 2024-09-13T16:34:29.048 INFO:teuthology.orchestra.run.smithi107.stdout: "dn": 11, 2024-09-13T16:34:29.048 INFO:teuthology.orchestra.run.smithi107.stdout: "dn+": 11, 2024-09-13T16:34:29.048 INFO:teuthology.orchestra.run.smithi107.stdout: "dn-": 0, 2024-09-13T16:34:29.048 INFO:teuthology.orchestra.run.smithi107.stdout: "cap": 1, 2024-09-13T16:34:29.048 INFO:teuthology.orchestra.run.smithi107.stdout: "cap+": 1, 2024-09-13T16:34:29.048 INFO:teuthology.orchestra.run.smithi107.stdout: "cap-": 0, 2024-09-13T16:34:29.048 INFO:teuthology.orchestra.run.smithi107.stdout: "rss": 47284, 2024-09-13T16:34:29.048 INFO:teuthology.orchestra.run.smithi107.stdout: "heap": 198932 2024-09-13T16:34:29.049 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.049 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_server": { 2024-09-13T16:34:29.049 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_client_request": 2, 2024-09-13T16:34:29.049 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_server_request": 0, 2024-09-13T16:34:29.049 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_request": 1, 2024-09-13T16:34:29.049 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_session": 9, 2024-09-13T16:34:29.049 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_peer_request": 0, 2024-09-13T16:34:29.049 INFO:teuthology.orchestra.run.smithi107.stdout: "req_create_latency": { 2024-09-13T16:34:29.049 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.049 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.049 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.049 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.049 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getattr_latency": { 2024-09-13T16:34:29.049 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1, 2024-09-13T16:34:29.049 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000523217, 2024-09-13T16:34:29.050 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000523217 2024-09-13T16:34:29.050 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.050 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getfilelock_latency": { 2024-09-13T16:34:29.050 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.050 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.050 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.050 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.050 INFO:teuthology.orchestra.run.smithi107.stdout: "req_link_latency": { 2024-09-13T16:34:29.050 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.050 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.050 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.050 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.050 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookup_latency": { 2024-09-13T16:34:29.050 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.051 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.051 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.051 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.051 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookuphash_latency": { 2024-09-13T16:34:29.051 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.051 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.051 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.051 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.051 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupino_latency": { 2024-09-13T16:34:29.051 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.051 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.051 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.051 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.051 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupname_latency": { 2024-09-13T16:34:29.052 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.052 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.052 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.052 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupparent_latency": { 2024-09-13T16:34:29.052 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.052 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.052 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.052 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupsnap_latency": { 2024-09-13T16:34:29.052 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.052 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.052 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.053 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lssnap_latency": { 2024-09-13T16:34:29.053 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.053 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.053 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.053 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.053 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mkdir_latency": { 2024-09-13T16:34:29.053 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.053 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.053 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.053 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.053 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mknod_latency": { 2024-09-13T16:34:29.053 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.053 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.054 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.054 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.054 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mksnap_latency": { 2024-09-13T16:34:29.054 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.054 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.054 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.054 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.054 INFO:teuthology.orchestra.run.smithi107.stdout: "req_open_latency": { 2024-09-13T16:34:29.054 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.054 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.054 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.054 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.054 INFO:teuthology.orchestra.run.smithi107.stdout: "req_readdir_latency": { 2024-09-13T16:34:29.054 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.055 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.055 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.055 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rename_latency": { 2024-09-13T16:34:29.055 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.055 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.055 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.055 INFO:teuthology.orchestra.run.smithi107.stdout: "req_renamesnap_latency": { 2024-09-13T16:34:29.055 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.055 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.055 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.055 INFO:teuthology.orchestra.run.smithi107.stdout: "req_snapdiff_latency": { 2024-09-13T16:34:29.056 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.056 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.056 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.056 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.056 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmdir_latency": { 2024-09-13T16:34:29.056 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.056 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.056 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.056 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.056 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmsnap_latency": { 2024-09-13T16:34:29.056 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.056 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.056 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.056 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.057 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmxattr_latency": { 2024-09-13T16:34:29.057 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.057 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.057 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.057 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setattr_latency": { 2024-09-13T16:34:29.057 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.057 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.057 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.057 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setdirlayout_latency": { 2024-09-13T16:34:29.057 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.057 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.057 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.058 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.058 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setfilelock_latency": { 2024-09-13T16:34:29.058 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.058 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.058 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.058 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.058 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setlayout_latency": { 2024-09-13T16:34:29.058 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.058 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.058 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.058 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.058 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setxattr_latency": { 2024-09-13T16:34:29.058 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.058 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.059 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.059 INFO:teuthology.orchestra.run.smithi107.stdout: "req_symlink_latency": { 2024-09-13T16:34:29.059 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.059 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.059 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.059 INFO:teuthology.orchestra.run.smithi107.stdout: "req_unlink_latency": { 2024-09-13T16:34:29.060 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.060 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.060 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.060 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_revoke_eviction": 0, 2024-09-13T16:34:29.060 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_acquisition_throttle": 0, 2024-09-13T16:34:29.060 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getvxattr_latency": { 2024-09-13T16:34:29.060 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.060 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.060 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.060 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:29.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.061 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_sessions": { 2024-09-13T16:34:29.061 INFO:teuthology.orchestra.run.smithi107.stdout: "session_count": 1, 2024-09-13T16:34:29.061 INFO:teuthology.orchestra.run.smithi107.stdout: "session_add": 2, 2024-09-13T16:34:29.061 INFO:teuthology.orchestra.run.smithi107.stdout: "session_remove": 1, 2024-09-13T16:34:29.061 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_open": 1, 2024-09-13T16:34:29.061 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_stale": 0, 2024-09-13T16:34:29.061 INFO:teuthology.orchestra.run.smithi107.stdout: "total_load": 1, 2024-09-13T16:34:29.061 INFO:teuthology.orchestra.run.smithi107.stdout: "average_load": 1, 2024-09-13T16:34:29.061 INFO:teuthology.orchestra.run.smithi107.stdout: "avg_session_uptime": 198, 2024-09-13T16:34:29.061 INFO:teuthology.orchestra.run.smithi107.stdout: "mdthresh_evicted": 0 2024-09-13T16:34:29.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.061 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-09-13T16:34:29.061 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-09-13T16:34:29.062 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-09-13T16:34:29.062 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T16:34:29.062 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-09-13T16:34:29.062 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T16:34:29.062 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-09-13T16:34:29.062 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T16:34:29.062 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T16:34:29.062 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T16:34:29.062 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T16:34:29.062 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T16:34:29.062 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-09-13T16:34:29.062 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T16:34:29.062 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T16:34:29.062 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-09-13T16:34:29.063 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-09-13T16:34:29.063 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-09-13T16:34:29.063 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-09-13T16:34:29.063 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T16:34:29.063 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T16:34:29.063 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T16:34:29.063 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T16:34:29.063 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T16:34:29.063 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-09-13T16:34:29.063 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-09-13T16:34:29.063 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-09-13T16:34:29.063 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T16:34:29.063 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T16:34:29.063 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-09-13T16:34:29.064 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-09-13T16:34:29.064 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-09-13T16:34:29.064 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-09-13T16:34:29.064 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T16:34:29.064 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-09-13T16:34:29.064 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T16:34:29.064 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-09-13T16:34:29.064 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 238310, 2024-09-13T16:34:29.064 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 77, 2024-09-13T16:34:29.064 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-09-13T16:34:29.064 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-09-13T16:34:29.064 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-09-13T16:34:29.064 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-09-13T16:34:29.064 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-09-13T16:34:29.065 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-09-13T16:34:29.065 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-09-13T16:34:29.065 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-09-13T16:34:29.065 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 8856, 2024-09-13T16:34:29.065 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 166, 2024-09-13T16:34:29.065 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T16:34:29.065 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-09-13T16:34:29.065 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-09-13T16:34:29.065 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-09-13T16:34:29.065 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 54776, 2024-09-13T16:34:29.065 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 215, 2024-09-13T16:34:29.065 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-09-13T16:34:29.065 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-09-13T16:34:29.065 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-09-13T16:34:29.066 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-09-13T16:34:29.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.066 INFO:teuthology.orchestra.run.smithi107.stdout: "objecter": { 2024-09-13T16:34:29.066 INFO:teuthology.orchestra.run.smithi107.stdout: "op_active": 0, 2024-09-13T16:34:29.066 INFO:teuthology.orchestra.run.smithi107.stdout: "op_laggy": 0, 2024-09-13T16:34:29.066 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send": 41, 2024-09-13T16:34:29.066 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send_bytes": 11963, 2024-09-13T16:34:29.066 INFO:teuthology.orchestra.run.smithi107.stdout: "op_resend": 27, 2024-09-13T16:34:29.066 INFO:teuthology.orchestra.run.smithi107.stdout: "op_reply": 41, 2024-09-13T16:34:29.066 INFO:teuthology.orchestra.run.smithi107.stdout: "op_latency": { 2024-09-13T16:34:29.066 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 41, 2024-09-13T16:34:29.066 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.220996219, 2024-09-13T16:34:29.066 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.005390151 2024-09-13T16:34:29.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.067 INFO:teuthology.orchestra.run.smithi107.stdout: "op_inflight": 0, 2024-09-13T16:34:29.067 INFO:teuthology.orchestra.run.smithi107.stdout: "oplen_avg": { 2024-09-13T16:34:29.067 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 41, 2024-09-13T16:34:29.067 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 64 2024-09-13T16:34:29.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.067 INFO:teuthology.orchestra.run.smithi107.stdout: "op": 41, 2024-09-13T16:34:29.067 INFO:teuthology.orchestra.run.smithi107.stdout: "op_r": 0, 2024-09-13T16:34:29.067 INFO:teuthology.orchestra.run.smithi107.stdout: "op_w": 41, 2024-09-13T16:34:29.067 INFO:teuthology.orchestra.run.smithi107.stdout: "op_rmw": 0, 2024-09-13T16:34:29.069 INFO:teuthology.orchestra.run.smithi107.stdout: "op_pg": 0, 2024-09-13T16:34:29.069 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_stat": 0, 2024-09-13T16:34:29.069 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_create": 10, 2024-09-13T16:34:29.069 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_read": 0, 2024-09-13T16:34:29.069 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_write": 4, 2024-09-13T16:34:29.069 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writefull": 7, 2024-09-13T16:34:29.069 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writesame": 0, 2024-09-13T16:34:29.069 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_append": 0, 2024-09-13T16:34:29.069 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_zero": 0, 2024-09-13T16:34:29.069 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_truncate": 0, 2024-09-13T16:34:29.069 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_delete": 6, 2024-09-13T16:34:29.069 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_mapext": 0, 2024-09-13T16:34:29.070 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_sparse_read": 0, 2024-09-13T16:34:29.070 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_clonerange": 0, 2024-09-13T16:34:29.070 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_getxattr": 0, 2024-09-13T16:34:29.070 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_setxattr": 20, 2024-09-13T16:34:29.070 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_cmpxattr": 0, 2024-09-13T16:34:29.070 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_rmxattr": 0, 2024-09-13T16:34:29.070 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_resetxattrs": 0, 2024-09-13T16:34:29.070 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_call": 0, 2024-09-13T16:34:29.070 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_watch": 0, 2024-09-13T16:34:29.070 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_notify": 0, 2024-09-13T16:34:29.070 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_src_cmpxattr": 0, 2024-09-13T16:34:29.070 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls": 0, 2024-09-13T16:34:29.070 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls_filter": 0, 2024-09-13T16:34:29.070 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_other": 0, 2024-09-13T16:34:29.071 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_active": 0, 2024-09-13T16:34:29.071 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_send": 0, 2024-09-13T16:34:29.071 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_resend": 0, 2024-09-13T16:34:29.071 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_ping": 0, 2024-09-13T16:34:29.071 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_active": 0, 2024-09-13T16:34:29.071 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_send": 0, 2024-09-13T16:34:29.071 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_resend": 0, 2024-09-13T16:34:29.071 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_active": 0, 2024-09-13T16:34:29.071 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_send": 0, 2024-09-13T16:34:29.071 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_resend": 0, 2024-09-13T16:34:29.071 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_active": 0, 2024-09-13T16:34:29.071 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_send": 0, 2024-09-13T16:34:29.071 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_resend": 0, 2024-09-13T16:34:29.071 INFO:teuthology.orchestra.run.smithi107.stdout: "command_active": 0, 2024-09-13T16:34:29.072 INFO:teuthology.orchestra.run.smithi107.stdout: "command_send": 0, 2024-09-13T16:34:29.072 INFO:teuthology.orchestra.run.smithi107.stdout: "command_resend": 0, 2024-09-13T16:34:29.072 INFO:teuthology.orchestra.run.smithi107.stdout: "map_epoch": 93, 2024-09-13T16:34:29.072 INFO:teuthology.orchestra.run.smithi107.stdout: "map_full": 0, 2024-09-13T16:34:29.072 INFO:teuthology.orchestra.run.smithi107.stdout: "map_inc": 4, 2024-09-13T16:34:29.072 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_sessions": 8, 2024-09-13T16:34:29.072 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_open": 8, 2024-09-13T16:34:29.072 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_close": 0, 2024-09-13T16:34:29.072 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_laggy": 0, 2024-09-13T16:34:29.072 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_wr": 16, 2024-09-13T16:34:29.072 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_rd": 0, 2024-09-13T16:34:29.072 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_del": 1 2024-09-13T16:34:29.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.073 INFO:teuthology.orchestra.run.smithi107.stdout: "oft": { 2024-09-13T16:34:29.073 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_objs": 1, 2024-09-13T16:34:29.073 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_kv_pairs": 0, 2024-09-13T16:34:29.073 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_updates": 0, 2024-09-13T16:34:29.073 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_removes": 0 2024-09-13T16:34:29.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.073 INFO:teuthology.orchestra.run.smithi107.stdout: "osd-slow-ops": { 2024-09-13T16:34:29.073 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_ops_count": 0 2024-09-13T16:34:29.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.073 INFO:teuthology.orchestra.run.smithi107.stdout: "purge_queue": { 2024-09-13T16:34:29.073 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops": 0, 2024-09-13T16:34:29.073 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops_high_water": 0, 2024-09-13T16:34:29.073 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing": 0, 2024-09-13T16:34:29.073 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_high_water": 0, 2024-09-13T16:34:29.074 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed_ops": 0, 2024-09-13T16:34:29.074 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed": 0, 2024-09-13T16:34:29.074 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_item_in_journal": 0 2024-09-13T16:34:29.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.074 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T16:34:29.074 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-13T16:34:29.074 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-09-13T16:34:29.074 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-13T16:34:29.074 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 299, 2024-09-13T16:34:29.074 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 774263, 2024-09-13T16:34:29.074 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:29.074 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 299, 2024-09-13T16:34:29.074 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-13T16:34:29.075 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-13T16:34:29.075 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 299, 2024-09-13T16:34:29.075 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 774263, 2024-09-13T16:34:29.075 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-13T16:34:29.075 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.075 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.075 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.075 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:29.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.075 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_bytes": { 2024-09-13T16:34:29.075 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-13T16:34:29.075 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-09-13T16:34:29.075 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-13T16:34:29.075 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-09-13T16:34:29.076 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-09-13T16:34:29.076 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:29.076 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:29.076 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 41, 2024-09-13T16:34:29.076 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 11963, 2024-09-13T16:34:29.076 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 41, 2024-09-13T16:34:29.076 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 11963, 2024-09-13T16:34:29.076 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-13T16:34:29.076 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.076 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.076 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.076 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:29.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.077 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_ops": { 2024-09-13T16:34:29.077 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-13T16:34:29.077 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 1024, 2024-09-13T16:34:29.077 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-13T16:34:29.077 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-09-13T16:34:29.077 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-09-13T16:34:29.077 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:29.077 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:29.077 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 41, 2024-09-13T16:34:29.077 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 41, 2024-09-13T16:34:29.077 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 41, 2024-09-13T16:34:29.077 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 41, 2024-09-13T16:34:29.077 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-13T16:34:29.077 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.078 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.078 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.078 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:29.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.078 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle": { 2024-09-13T16:34:29.078 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-13T16:34:29.078 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-09-13T16:34:29.078 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-13T16:34:29.078 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-09-13T16:34:29.078 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-09-13T16:34:29.078 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:29.078 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:29.078 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-13T16:34:29.078 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-13T16:34:29.079 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 0, 2024-09-13T16:34:29.079 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 0, 2024-09-13T16:34:29.079 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-13T16:34:29.079 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.079 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.079 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.079 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:29.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:29.079 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle-0x559f60e70500": { 2024-09-13T16:34:29.079 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-13T16:34:29.079 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-09-13T16:34:29.079 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-13T16:34:29.079 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 4, 2024-09-13T16:34:29.079 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 1567, 2024-09-13T16:34:29.080 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:29.080 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 4, 2024-09-13T16:34:29.080 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-13T16:34:29.080 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-13T16:34:29.080 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 4, 2024-09-13T16:34:29.080 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 1567, 2024-09-13T16:34:29.080 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-13T16:34:29.080 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:29.080 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:29.080 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:29.080 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:29.080 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:29.080 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-13T16:34:29.917 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-13T16:34:29.917 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-13T16:34:29.917 DEBUG:tasks.check_counter:Getting stats from k 2024-09-13T16:34:29.918 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-13T16:34:30.116 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T16:34:30.236 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:29 smithi107 ceph-mon[34139]: pgmap v455: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-13T16:34:30.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:29 smithi016 ceph-mon[31187]: pgmap v455: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-13T16:34:30.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:29 smithi111 ceph-mon[34600]: pgmap v455: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-13T16:34:31.941 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-13T16:34:31.942 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T16:34:31.942 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 325, 2024-09-13T16:34:31.942 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 359, 2024-09-13T16:34:31.942 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 814436, 2024-09-13T16:34:31.942 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 154562, 2024-09-13T16:34:31.942 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 7, 2024-09-13T16:34:31.942 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 2, 2024-09-13T16:34:31.942 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.050272047, 2024-09-13T16:34:31.942 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.018876128, 2024-09-13T16:34:31.942 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.035588249, 2024-09-13T16:34:31.942 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.001914990, 2024-09-13T16:34:31.942 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:31.942 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 361, 2024-09-13T16:34:31.942 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.039930731, 2024-09-13T16:34:31.943 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000110611 2024-09-13T16:34:31.943 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.943 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:31.943 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 343, 2024-09-13T16:34:31.943 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000081109, 2024-09-13T16:34:31.943 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000236 2024-09-13T16:34:31.943 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.943 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 763376, 2024-09-13T16:34:31.943 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 66800 2024-09-13T16:34:31.943 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.943 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T16:34:31.943 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 13, 2024-09-13T16:34:31.943 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 92, 2024-09-13T16:34:31.943 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 3451, 2024-09-13T16:34:31.944 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 209299, 2024-09-13T16:34:31.944 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 6, 2024-09-13T16:34:31.944 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 4, 2024-09-13T16:34:31.944 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.007247475, 2024-09-13T16:34:31.944 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.003375770, 2024-09-13T16:34:31.944 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.002697008, 2024-09-13T16:34:31.944 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000299331, 2024-09-13T16:34:31.944 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:31.944 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 90, 2024-09-13T16:34:31.944 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.018292732, 2024-09-13T16:34:31.944 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000203252 2024-09-13T16:34:31.944 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.944 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:31.945 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.945 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.945 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.945 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.945 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-13T16:34:31.945 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 203712 2024-09-13T16:34:31.945 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.945 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T16:34:31.945 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 24, 2024-09-13T16:34:31.945 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 28, 2024-09-13T16:34:31.945 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 5771, 2024-09-13T16:34:31.945 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 52359, 2024-09-13T16:34:31.945 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 11, 2024-09-13T16:34:31.946 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 7, 2024-09-13T16:34:31.946 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.006367317, 2024-09-13T16:34:31.946 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.000878790, 2024-09-13T16:34:31.946 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.004932101, 2024-09-13T16:34:31.946 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000282292, 2024-09-13T16:34:31.946 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:31.946 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 24, 2024-09-13T16:34:31.946 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.029606789, 2024-09-13T16:34:31.946 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.001233616 2024-09-13T16:34:31.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.947 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:31.947 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 20, 2024-09-13T16:34:31.947 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000003361, 2024-09-13T16:34:31.947 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000168 2024-09-13T16:34:31.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.947 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-13T16:34:31.947 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 0 2024-09-13T16:34:31.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.947 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-09-13T16:34:31.947 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 1, 2024-09-13T16:34:31.948 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-09-13T16:34:31.948 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.948 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-MDSRank": { 2024-09-13T16:34:31.948 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-09-13T16:34:31.948 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-09-13T16:34:31.948 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 41, 2024-09-13T16:34:31.948 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.002109818, 2024-09-13T16:34:31.948 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000051458 2024-09-13T16:34:31.948 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:31.948 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.948 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-PurgeQueue": { 2024-09-13T16:34:31.948 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-09-13T16:34:31.948 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-09-13T16:34:31.949 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2, 2024-09-13T16:34:31.949 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000011320, 2024-09-13T16:34:31.949 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000005660 2024-09-13T16:34:31.949 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:31.949 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.949 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-09-13T16:34:31.949 INFO:teuthology.orchestra.run.smithi107.stdout: "request": 2, 2024-09-13T16:34:31.949 INFO:teuthology.orchestra.run.smithi107.stdout: "reply": 1, 2024-09-13T16:34:31.949 INFO:teuthology.orchestra.run.smithi107.stdout: "reply_latency": { 2024-09-13T16:34:31.949 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1, 2024-09-13T16:34:31.950 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000796221, 2024-09-13T16:34:31.950 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000796221 2024-09-13T16:34:31.950 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.950 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_reply": 0, 2024-09-13T16:34:31.950 INFO:teuthology.orchestra.run.smithi107.stdout: "forward": 1, 2024-09-13T16:34:31.950 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_complete": 0, 2024-09-13T16:34:31.950 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_keys": 0, 2024-09-13T16:34:31.950 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_commit": 11, 2024-09-13T16:34:31.950 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_split": 0, 2024-09-13T16:34:31.950 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_merge": 0, 2024-09-13T16:34:31.950 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes": 14, 2024-09-13T16:34:31.950 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_top": 0, 2024-09-13T16:34:31.950 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_bottom": 14, 2024-09-13T16:34:31.950 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pin_tail": 0, 2024-09-13T16:34:31.951 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pinned": 14, 2024-09-13T16:34:31.951 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_expired": 0, 2024-09-13T16:34:31.951 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_with_caps": 1, 2024-09-13T16:34:31.951 INFO:teuthology.orchestra.run.smithi107.stdout: "caps": 1, 2024-09-13T16:34:31.951 INFO:teuthology.orchestra.run.smithi107.stdout: "subtrees": 2, 2024-09-13T16:34:31.951 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse": 3, 2024-09-13T16:34:31.951 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_hit": 1, 2024-09-13T16:34:31.951 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_forward": 0, 2024-09-13T16:34:31.951 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_discover": 1, 2024-09-13T16:34:31.951 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_dir_fetch": 0, 2024-09-13T16:34:31.951 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_remote_ino": 0, 2024-09-13T16:34:31.951 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_lock": 0, 2024-09-13T16:34:31.951 INFO:teuthology.orchestra.run.smithi107.stdout: "load_cent": 0, 2024-09-13T16:34:31.952 INFO:teuthology.orchestra.run.smithi107.stdout: "q": 0, 2024-09-13T16:34:31.952 INFO:teuthology.orchestra.run.smithi107.stdout: "exported": 0, 2024-09-13T16:34:31.952 INFO:teuthology.orchestra.run.smithi107.stdout: "exported_inodes": 0, 2024-09-13T16:34:31.952 INFO:teuthology.orchestra.run.smithi107.stdout: "imported": 0, 2024-09-13T16:34:31.952 INFO:teuthology.orchestra.run.smithi107.stdout: "imported_inodes": 0, 2024-09-13T16:34:31.952 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_dir_fetch": 0, 2024-09-13T16:34:31.952 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_backtrace_fetch": 0, 2024-09-13T16:34:31.952 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_peer_discover": 0, 2024-09-13T16:34:31.952 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rfiles": 5, 2024-09-13T16:34:31.952 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rbytes": 5026218218, 2024-09-13T16:34:31.952 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rsnaps": 0, 2024-09-13T16:34:31.952 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_fetch": 0, 2024-09-13T16:34:31.952 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_set_tag": 0, 2024-09-13T16:34:31.952 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_repaired": 0, 2024-09-13T16:34:31.953 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_inotable_repaired": 0, 2024-09-13T16:34:31.953 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_inodes": 0, 2024-09-13T16:34:31.953 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_base_inodes": 0, 2024-09-13T16:34:31.953 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dirfrag_rstats": 0, 2024-09-13T16:34:31.953 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_file_inodes": 0, 2024-09-13T16:34:31.953 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_inode_file_caps": 0, 2024-09-13T16:34:31.953 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_revoke": 1, 2024-09-13T16:34:31.953 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_grant": 0, 2024-09-13T16:34:31.953 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_trunc": 0, 2024-09-13T16:34:31.953 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-13T16:34:31.953 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-13T16:34:31.953 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps": 1, 2024-09-13T16:34:31.953 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps_dirty": 0, 2024-09-13T16:34:31.953 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_cap_release": 0, 2024-09-13T16:34:31.954 INFO:teuthology.orchestra.run.smithi107.stdout: "process_request_cap_release": 1 2024-09-13T16:34:31.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.954 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cache": { 2024-09-13T16:34:31.954 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update": 0, 2024-09-13T16:34:31.954 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update_receipt": 0, 2024-09-13T16:34:31.954 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_try_discover": 0, 2024-09-13T16:34:31.954 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_send_discover": 2, 2024-09-13T16:34:31.954 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_handle_discover": 0, 2024-09-13T16:34:31.954 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays": 0, 2024-09-13T16:34:31.954 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_delayed": 0, 2024-09-13T16:34:31.954 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_enqueuing": 0, 2024-09-13T16:34:31.954 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_created": 0, 2024-09-13T16:34:31.954 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_enqueued": 0, 2024-09-13T16:34:31.954 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_reintegrated": 0, 2024-09-13T16:34:31.955 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_migrated": 0, 2024-09-13T16:34:31.955 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_processing": 0, 2024-09-13T16:34:31.955 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_enqueued": 0, 2024-09-13T16:34:31.955 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_prioritized": 0, 2024-09-13T16:34:31.955 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_started": 0, 2024-09-13T16:34:31.955 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_completed": 0, 2024-09-13T16:34:31.955 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_enqueue_scrub": 0, 2024-09-13T16:34:31.955 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_exportdir": 0, 2024-09-13T16:34:31.955 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_flush": 0, 2024-09-13T16:34:31.955 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragmentdir": 0, 2024-09-13T16:34:31.955 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragstats": 0, 2024-09-13T16:34:31.955 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_inodestats": 0 2024-09-13T16:34:31.955 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.955 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_log": { 2024-09-13T16:34:31.955 INFO:teuthology.orchestra.run.smithi107.stdout: "evadd": 2, 2024-09-13T16:34:31.956 INFO:teuthology.orchestra.run.smithi107.stdout: "evex": 0, 2024-09-13T16:34:31.956 INFO:teuthology.orchestra.run.smithi107.stdout: "evtrm": 0, 2024-09-13T16:34:31.956 INFO:teuthology.orchestra.run.smithi107.stdout: "ev": 2, 2024-09-13T16:34:31.956 INFO:teuthology.orchestra.run.smithi107.stdout: "evexg": 0, 2024-09-13T16:34:31.956 INFO:teuthology.orchestra.run.smithi107.stdout: "evexd": 0, 2024-09-13T16:34:31.956 INFO:teuthology.orchestra.run.smithi107.stdout: "segadd": 1, 2024-09-13T16:34:31.956 INFO:teuthology.orchestra.run.smithi107.stdout: "segex": 0, 2024-09-13T16:34:31.956 INFO:teuthology.orchestra.run.smithi107.stdout: "segtrm": 0, 2024-09-13T16:34:31.957 INFO:teuthology.orchestra.run.smithi107.stdout: "seg": 1, 2024-09-13T16:34:31.957 INFO:teuthology.orchestra.run.smithi107.stdout: "segexg": 0, 2024-09-13T16:34:31.957 INFO:teuthology.orchestra.run.smithi107.stdout: "segexd": 0, 2024-09-13T16:34:31.957 INFO:teuthology.orchestra.run.smithi107.stdout: "expos": 4194304, 2024-09-13T16:34:31.957 INFO:teuthology.orchestra.run.smithi107.stdout: "wrpos": 4195338, 2024-09-13T16:34:31.957 INFO:teuthology.orchestra.run.smithi107.stdout: "rdpos": 0, 2024-09-13T16:34:31.957 INFO:teuthology.orchestra.run.smithi107.stdout: "jlat": { 2024-09-13T16:34:31.957 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2, 2024-09-13T16:34:31.957 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.002938640, 2024-09-13T16:34:31.957 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.001469320 2024-09-13T16:34:31.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.957 INFO:teuthology.orchestra.run.smithi107.stdout: "replayed": 0 2024-09-13T16:34:31.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.957 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_mem": { 2024-09-13T16:34:31.958 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 17, 2024-09-13T16:34:31.958 INFO:teuthology.orchestra.run.smithi107.stdout: "ino+": 17, 2024-09-13T16:34:31.958 INFO:teuthology.orchestra.run.smithi107.stdout: "ino-": 0, 2024-09-13T16:34:31.958 INFO:teuthology.orchestra.run.smithi107.stdout: "dir": 15, 2024-09-13T16:34:31.958 INFO:teuthology.orchestra.run.smithi107.stdout: "dir+": 16, 2024-09-13T16:34:31.958 INFO:teuthology.orchestra.run.smithi107.stdout: "dir-": 1, 2024-09-13T16:34:31.958 INFO:teuthology.orchestra.run.smithi107.stdout: "dn": 14, 2024-09-13T16:34:31.958 INFO:teuthology.orchestra.run.smithi107.stdout: "dn+": 14, 2024-09-13T16:34:31.958 INFO:teuthology.orchestra.run.smithi107.stdout: "dn-": 0, 2024-09-13T16:34:31.958 INFO:teuthology.orchestra.run.smithi107.stdout: "cap": 1, 2024-09-13T16:34:31.958 INFO:teuthology.orchestra.run.smithi107.stdout: "cap+": 1, 2024-09-13T16:34:31.959 INFO:teuthology.orchestra.run.smithi107.stdout: "cap-": 0, 2024-09-13T16:34:31.959 INFO:teuthology.orchestra.run.smithi107.stdout: "rss": 50612, 2024-09-13T16:34:31.959 INFO:teuthology.orchestra.run.smithi107.stdout: "heap": 198932 2024-09-13T16:34:31.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.959 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_server": { 2024-09-13T16:34:31.959 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_client_request": 3, 2024-09-13T16:34:31.959 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_server_request": 0, 2024-09-13T16:34:31.959 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_request": 2, 2024-09-13T16:34:31.959 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_session": 5, 2024-09-13T16:34:31.959 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_peer_request": 0, 2024-09-13T16:34:31.959 INFO:teuthology.orchestra.run.smithi107.stdout: "req_create_latency": { 2024-09-13T16:34:31.959 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.959 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.959 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.960 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getattr_latency": { 2024-09-13T16:34:31.960 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1, 2024-09-13T16:34:31.960 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000796221, 2024-09-13T16:34:31.960 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000796221 2024-09-13T16:34:31.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.960 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getfilelock_latency": { 2024-09-13T16:34:31.960 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.960 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.961 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.961 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.961 INFO:teuthology.orchestra.run.smithi107.stdout: "req_link_latency": { 2024-09-13T16:34:31.961 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.961 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.961 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.961 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.961 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookup_latency": { 2024-09-13T16:34:31.961 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.961 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.961 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.962 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.962 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookuphash_latency": { 2024-09-13T16:34:31.962 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.962 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.962 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.962 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.962 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupino_latency": { 2024-09-13T16:34:31.962 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.962 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.962 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.962 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.962 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupname_latency": { 2024-09-13T16:34:31.962 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.962 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.963 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.963 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupparent_latency": { 2024-09-13T16:34:31.963 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.963 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.963 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.963 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupsnap_latency": { 2024-09-13T16:34:31.963 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.963 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.963 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.963 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lssnap_latency": { 2024-09-13T16:34:31.963 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.963 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.964 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.964 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mkdir_latency": { 2024-09-13T16:34:31.964 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.964 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.964 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.964 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mknod_latency": { 2024-09-13T16:34:31.964 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.964 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.964 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.964 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mksnap_latency": { 2024-09-13T16:34:31.965 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.965 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.965 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.965 INFO:teuthology.orchestra.run.smithi107.stdout: "req_open_latency": { 2024-09-13T16:34:31.965 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.965 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.965 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.965 INFO:teuthology.orchestra.run.smithi107.stdout: "req_readdir_latency": { 2024-09-13T16:34:31.965 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.965 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.965 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.966 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rename_latency": { 2024-09-13T16:34:31.966 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.966 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.966 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.966 INFO:teuthology.orchestra.run.smithi107.stdout: "req_renamesnap_latency": { 2024-09-13T16:34:31.966 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.966 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.966 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.967 INFO:teuthology.orchestra.run.smithi107.stdout: "req_snapdiff_latency": { 2024-09-13T16:34:31.967 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.967 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.967 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.967 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmdir_latency": { 2024-09-13T16:34:31.967 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.967 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.967 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.967 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmsnap_latency": { 2024-09-13T16:34:31.967 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.967 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.968 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.968 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.968 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmxattr_latency": { 2024-09-13T16:34:31.968 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.968 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.968 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.968 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.968 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setattr_latency": { 2024-09-13T16:34:31.968 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.968 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.968 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.968 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.968 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setdirlayout_latency": { 2024-09-13T16:34:31.968 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.969 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.969 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.969 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setfilelock_latency": { 2024-09-13T16:34:31.969 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.969 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.969 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.969 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setlayout_latency": { 2024-09-13T16:34:31.969 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.969 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.969 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.969 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setxattr_latency": { 2024-09-13T16:34:31.970 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.970 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.970 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.970 INFO:teuthology.orchestra.run.smithi107.stdout: "req_symlink_latency": { 2024-09-13T16:34:31.970 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.970 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.970 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.971 INFO:teuthology.orchestra.run.smithi107.stdout: "req_unlink_latency": { 2024-09-13T16:34:31.971 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.971 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.971 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.971 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_revoke_eviction": 0, 2024-09-13T16:34:31.971 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_acquisition_throttle": 0, 2024-09-13T16:34:31.971 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getvxattr_latency": { 2024-09-13T16:34:31.971 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.971 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.971 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.971 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:31.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.971 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_sessions": { 2024-09-13T16:34:31.972 INFO:teuthology.orchestra.run.smithi107.stdout: "session_count": 1, 2024-09-13T16:34:31.972 INFO:teuthology.orchestra.run.smithi107.stdout: "session_add": 1, 2024-09-13T16:34:31.972 INFO:teuthology.orchestra.run.smithi107.stdout: "session_remove": 0, 2024-09-13T16:34:31.972 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_open": 1, 2024-09-13T16:34:31.972 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_stale": 0, 2024-09-13T16:34:31.972 INFO:teuthology.orchestra.run.smithi107.stdout: "total_load": 1, 2024-09-13T16:34:31.972 INFO:teuthology.orchestra.run.smithi107.stdout: "average_load": 1, 2024-09-13T16:34:31.972 INFO:teuthology.orchestra.run.smithi107.stdout: "avg_session_uptime": 203, 2024-09-13T16:34:31.972 INFO:teuthology.orchestra.run.smithi107.stdout: "mdthresh_evicted": 0 2024-09-13T16:34:31.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.973 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-09-13T16:34:31.973 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-09-13T16:34:31.973 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-09-13T16:34:31.973 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T16:34:31.973 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-09-13T16:34:31.973 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T16:34:31.973 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-09-13T16:34:31.973 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T16:34:31.973 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T16:34:31.973 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T16:34:31.973 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T16:34:31.974 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T16:34:31.974 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-09-13T16:34:31.974 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T16:34:31.974 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T16:34:31.974 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-09-13T16:34:31.974 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-09-13T16:34:31.974 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-09-13T16:34:31.974 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-09-13T16:34:31.974 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T16:34:31.974 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T16:34:31.974 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T16:34:31.974 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T16:34:31.974 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T16:34:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-09-13T16:34:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-09-13T16:34:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-09-13T16:34:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T16:34:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T16:34:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-09-13T16:34:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-09-13T16:34:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-09-13T16:34:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-09-13T16:34:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T16:34:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-09-13T16:34:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T16:34:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-09-13T16:34:31.975 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 238383, 2024-09-13T16:34:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 74, 2024-09-13T16:34:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-09-13T16:34:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-09-13T16:34:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-09-13T16:34:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-09-13T16:34:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-09-13T16:34:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-09-13T16:34:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-09-13T16:34:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-09-13T16:34:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 8856, 2024-09-13T16:34:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 166, 2024-09-13T16:34:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T16:34:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-09-13T16:34:31.976 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-09-13T16:34:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-09-13T16:34:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 67221, 2024-09-13T16:34:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 288, 2024-09-13T16:34:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-09-13T16:34:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-09-13T16:34:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-09-13T16:34:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-09-13T16:34:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: "objecter": { 2024-09-13T16:34:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: "op_active": 0, 2024-09-13T16:34:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: "op_laggy": 0, 2024-09-13T16:34:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send": 38, 2024-09-13T16:34:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send_bytes": 11515, 2024-09-13T16:34:31.977 INFO:teuthology.orchestra.run.smithi107.stdout: "op_resend": 33, 2024-09-13T16:34:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: "op_reply": 38, 2024-09-13T16:34:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: "op_latency": { 2024-09-13T16:34:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 38, 2024-09-13T16:34:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.192996719, 2024-09-13T16:34:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.005078861 2024-09-13T16:34:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: "op_inflight": 0, 2024-09-13T16:34:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: "oplen_avg": { 2024-09-13T16:34:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 38, 2024-09-13T16:34:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 61 2024-09-13T16:34:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: "op": 38, 2024-09-13T16:34:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: "op_r": 0, 2024-09-13T16:34:31.978 INFO:teuthology.orchestra.run.smithi107.stdout: "op_w": 38, 2024-09-13T16:34:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: "op_rmw": 0, 2024-09-13T16:34:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: "op_pg": 0, 2024-09-13T16:34:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_stat": 0, 2024-09-13T16:34:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_create": 10, 2024-09-13T16:34:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_read": 0, 2024-09-13T16:34:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_write": 2, 2024-09-13T16:34:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writefull": 6, 2024-09-13T16:34:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writesame": 0, 2024-09-13T16:34:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_append": 0, 2024-09-13T16:34:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_zero": 0, 2024-09-13T16:34:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_truncate": 0, 2024-09-13T16:34:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_delete": 6, 2024-09-13T16:34:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_mapext": 0, 2024-09-13T16:34:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_sparse_read": 0, 2024-09-13T16:34:31.979 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_clonerange": 0, 2024-09-13T16:34:31.980 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_getxattr": 0, 2024-09-13T16:34:31.980 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_setxattr": 20, 2024-09-13T16:34:31.980 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_cmpxattr": 0, 2024-09-13T16:34:31.980 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_rmxattr": 0, 2024-09-13T16:34:31.980 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_resetxattrs": 0, 2024-09-13T16:34:31.980 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_call": 0, 2024-09-13T16:34:31.980 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_watch": 0, 2024-09-13T16:34:31.980 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_notify": 0, 2024-09-13T16:34:31.980 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_src_cmpxattr": 0, 2024-09-13T16:34:31.980 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls": 0, 2024-09-13T16:34:31.980 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls_filter": 0, 2024-09-13T16:34:31.980 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_other": 0, 2024-09-13T16:34:31.980 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_active": 0, 2024-09-13T16:34:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_send": 0, 2024-09-13T16:34:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_resend": 0, 2024-09-13T16:34:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_ping": 0, 2024-09-13T16:34:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_active": 0, 2024-09-13T16:34:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_send": 0, 2024-09-13T16:34:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_resend": 0, 2024-09-13T16:34:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_active": 0, 2024-09-13T16:34:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_send": 0, 2024-09-13T16:34:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_resend": 0, 2024-09-13T16:34:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_active": 0, 2024-09-13T16:34:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_send": 0, 2024-09-13T16:34:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_resend": 0, 2024-09-13T16:34:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: "command_active": 0, 2024-09-13T16:34:31.981 INFO:teuthology.orchestra.run.smithi107.stdout: "command_send": 0, 2024-09-13T16:34:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: "command_resend": 0, 2024-09-13T16:34:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: "map_epoch": 93, 2024-09-13T16:34:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: "map_full": 0, 2024-09-13T16:34:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: "map_inc": 4, 2024-09-13T16:34:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_sessions": 8, 2024-09-13T16:34:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_open": 8, 2024-09-13T16:34:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_close": 0, 2024-09-13T16:34:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_laggy": 0, 2024-09-13T16:34:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_wr": 16, 2024-09-13T16:34:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_rd": 0, 2024-09-13T16:34:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_del": 1 2024-09-13T16:34:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: "oft": { 2024-09-13T16:34:31.982 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_objs": 1, 2024-09-13T16:34:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_kv_pairs": 0, 2024-09-13T16:34:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_updates": 0, 2024-09-13T16:34:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_removes": 0 2024-09-13T16:34:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: "osd-slow-ops": { 2024-09-13T16:34:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_ops_count": 0 2024-09-13T16:34:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: "purge_queue": { 2024-09-13T16:34:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops": 0, 2024-09-13T16:34:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops_high_water": 0, 2024-09-13T16:34:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing": 0, 2024-09-13T16:34:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_high_water": 0, 2024-09-13T16:34:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed_ops": 0, 2024-09-13T16:34:31.983 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed": 0, 2024-09-13T16:34:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_item_in_journal": 0 2024-09-13T16:34:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T16:34:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-13T16:34:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-09-13T16:34:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-13T16:34:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 358, 2024-09-13T16:34:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 780638, 2024-09-13T16:34:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 358, 2024-09-13T16:34:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-13T16:34:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-13T16:34:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 358, 2024-09-13T16:34:31.984 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 780638, 2024-09-13T16:34:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-13T16:34:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_bytes": { 2024-09-13T16:34:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-13T16:34:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-09-13T16:34:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-13T16:34:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-09-13T16:34:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-09-13T16:34:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 38, 2024-09-13T16:34:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 11515, 2024-09-13T16:34:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 38, 2024-09-13T16:34:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 11515, 2024-09-13T16:34:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-13T16:34:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_ops": { 2024-09-13T16:34:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-13T16:34:31.986 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 1024, 2024-09-13T16:34:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-13T16:34:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-09-13T16:34:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-09-13T16:34:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 38, 2024-09-13T16:34:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 38, 2024-09-13T16:34:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 38, 2024-09-13T16:34:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 38, 2024-09-13T16:34:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-13T16:34:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.987 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:31.988 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.988 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle": { 2024-09-13T16:34:31.988 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-13T16:34:31.988 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-09-13T16:34:31.988 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-13T16:34:31.988 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-09-13T16:34:31.988 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-09-13T16:34:31.988 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:31.988 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:31.988 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-13T16:34:31.988 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-13T16:34:31.988 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 0, 2024-09-13T16:34:31.988 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 0, 2024-09-13T16:34:31.988 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-13T16:34:31.989 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.989 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.989 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.989 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:31.989 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-13T16:34:31.989 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle-0x556cf4b6e500": { 2024-09-13T16:34:31.989 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-13T16:34:31.989 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-09-13T16:34:31.989 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-13T16:34:31.989 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 2, 2024-09-13T16:34:31.989 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 1034, 2024-09-13T16:34:31.989 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:31.989 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 2, 2024-09-13T16:34:31.990 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-13T16:34:31.990 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-13T16:34:31.990 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 2, 2024-09-13T16:34:31.990 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 1034, 2024-09-13T16:34:31.990 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-13T16:34:31.990 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-13T16:34:31.990 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-13T16:34:31.990 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-13T16:34:31.990 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:31.990 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-13T16:34:31.990 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-13T16:34:32.253 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:31 smithi107 ceph-mon[34139]: pgmap v456: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T16:34:32.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:31 smithi016 ceph-mon[31187]: pgmap v456: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T16:34:32.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:31 smithi111 ceph-mon[34600]: pgmap v456: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T16:34:32.743 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-13T16:34:32.743 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-13T16:34:32.743 DEBUG:tasks.check_counter:Getting stats from c 2024-09-13T16:34:32.744 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-13T16:34:32.953 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T16:34:34.331 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:34 smithi111 ceph-mon[34600]: pgmap v457: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-13T16:34:34.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:34 smithi107 ceph-mon[34139]: pgmap v457: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-13T16:34:34.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:34 smithi016 ceph-mon[31187]: pgmap v457: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-13T16:34:34.691 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-13T16:34:34.691 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T16:34:34.691 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 725, 2024-09-13T16:34:34.692 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 727, 2024-09-13T16:34:34.692 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 655329, 2024-09-13T16:34:34.692 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 225485, 2024-09-13T16:34:34.692 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 6, 2024-09-13T16:34:34.692 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 2, 2024-09-13T16:34:34.692 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.066516807, 2024-09-13T16:34:34.692 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.020897542, 2024-09-13T16:34:34.692 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.018110056, 2024-09-13T16:34:34.692 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.020719851, 2024-09-13T16:34:34.692 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:34.692 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 725, 2024-09-13T16:34:34.692 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.026682669, 2024-09-13T16:34:34.693 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000036803 2024-09-13T16:34:34.693 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.693 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:34.693 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.693 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.693 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.693 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.693 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-13T16:34:34.693 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 0 2024-09-13T16:34:34.693 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.693 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T16:34:34.693 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 5, 2024-09-13T16:34:34.694 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 82, 2024-09-13T16:34:34.694 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 1816, 2024-09-13T16:34:34.694 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 204670, 2024-09-13T16:34:34.694 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 5, 2024-09-13T16:34:34.694 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 4, 2024-09-13T16:34:34.694 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.006341524, 2024-09-13T16:34:34.694 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.003245233, 2024-09-13T16:34:34.694 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.002110614, 2024-09-13T16:34:34.694 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000106064, 2024-09-13T16:34:34.694 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:34.694 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 80, 2024-09-13T16:34:34.694 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.007321748, 2024-09-13T16:34:34.695 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000091521 2024-09-13T16:34:34.695 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.695 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:34.695 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.695 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.695 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.695 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.695 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-13T16:34:34.695 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 203360 2024-09-13T16:34:34.695 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.695 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T16:34:34.695 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 459, 2024-09-13T16:34:34.695 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 446, 2024-09-13T16:34:34.696 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 843075, 2024-09-13T16:34:34.696 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 163015, 2024-09-13T16:34:34.696 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 3, 2024-09-13T16:34:34.696 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 2, 2024-09-13T16:34:34.696 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.053150689, 2024-09-13T16:34:34.696 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.018289515, 2024-09-13T16:34:34.696 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.029539980, 2024-09-13T16:34:34.696 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.010104434, 2024-09-13T16:34:34.696 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:34.696 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 446, 2024-09-13T16:34:34.696 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.022819447, 2024-09-13T16:34:34.696 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000051164 2024-09-13T16:34:34.696 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.697 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:34.697 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.697 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.697 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.697 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.697 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 761824, 2024-09-13T16:34:34.697 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 60384 2024-09-13T16:34:34.697 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.697 INFO:teuthology.orchestra.run.smithi111.stdout: "cct": { 2024-09-13T16:34:34.697 INFO:teuthology.orchestra.run.smithi111.stdout: "total_workers": 1, 2024-09-13T16:34:34.697 INFO:teuthology.orchestra.run.smithi111.stdout: "unhealthy_workers": 0 2024-09-13T16:34:34.697 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.697 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-MDSRank": { 2024-09-13T16:34:34.698 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-09-13T16:34:34.698 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-09-13T16:34:34.698 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 1747, 2024-09-13T16:34:34.698 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.046250664, 2024-09-13T16:34:34.698 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000026474 2024-09-13T16:34:34.698 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:34.698 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.698 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-PurgeQueue": { 2024-09-13T16:34:34.698 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-09-13T16:34:34.698 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-09-13T16:34:34.698 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.698 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.698 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.698 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:34.699 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.699 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": { 2024-09-13T16:34:34.699 INFO:teuthology.orchestra.run.smithi111.stdout: "request": 0, 2024-09-13T16:34:34.699 INFO:teuthology.orchestra.run.smithi111.stdout: "reply": 0, 2024-09-13T16:34:34.699 INFO:teuthology.orchestra.run.smithi111.stdout: "reply_latency": { 2024-09-13T16:34:34.699 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.699 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.699 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.699 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.699 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_reply": 0, 2024-09-13T16:34:34.699 INFO:teuthology.orchestra.run.smithi111.stdout: "forward": 0, 2024-09-13T16:34:34.699 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_complete": 0, 2024-09-13T16:34:34.699 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_keys": 0, 2024-09-13T16:34:34.700 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_commit": 0, 2024-09-13T16:34:34.700 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_split": 0, 2024-09-13T16:34:34.700 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_merge": 0, 2024-09-13T16:34:34.700 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes": 21, 2024-09-13T16:34:34.700 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_top": 6, 2024-09-13T16:34:34.700 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_bottom": 15, 2024-09-13T16:34:34.700 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pin_tail": 0, 2024-09-13T16:34:34.700 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pinned": 12, 2024-09-13T16:34:34.700 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_expired": 0, 2024-09-13T16:34:34.700 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_with_caps": 0, 2024-09-13T16:34:34.700 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": 0, 2024-09-13T16:34:34.700 INFO:teuthology.orchestra.run.smithi111.stdout: "subtrees": 2, 2024-09-13T16:34:34.700 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse": 0, 2024-09-13T16:34:34.701 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_hit": 0, 2024-09-13T16:34:34.701 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_forward": 0, 2024-09-13T16:34:34.701 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_discover": 0, 2024-09-13T16:34:34.701 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_dir_fetch": 0, 2024-09-13T16:34:34.701 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_remote_ino": 0, 2024-09-13T16:34:34.701 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_lock": 0, 2024-09-13T16:34:34.701 INFO:teuthology.orchestra.run.smithi111.stdout: "load_cent": 0, 2024-09-13T16:34:34.701 INFO:teuthology.orchestra.run.smithi111.stdout: "q": 0, 2024-09-13T16:34:34.701 INFO:teuthology.orchestra.run.smithi111.stdout: "exported": 0, 2024-09-13T16:34:34.701 INFO:teuthology.orchestra.run.smithi111.stdout: "exported_inodes": 0, 2024-09-13T16:34:34.701 INFO:teuthology.orchestra.run.smithi111.stdout: "imported": 0, 2024-09-13T16:34:34.701 INFO:teuthology.orchestra.run.smithi111.stdout: "imported_inodes": 0, 2024-09-13T16:34:34.702 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_dir_fetch": 0, 2024-09-13T16:34:34.702 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_backtrace_fetch": 0, 2024-09-13T16:34:34.702 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_peer_discover": 0, 2024-09-13T16:34:34.702 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rfiles": 4, 2024-09-13T16:34:34.702 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rbytes": 234, 2024-09-13T16:34:34.702 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rsnaps": 0, 2024-09-13T16:34:34.702 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_fetch": 0, 2024-09-13T16:34:34.702 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_set_tag": 0, 2024-09-13T16:34:34.702 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_repaired": 0, 2024-09-13T16:34:34.702 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_inotable_repaired": 0, 2024-09-13T16:34:34.702 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_inodes": 0, 2024-09-13T16:34:34.702 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_base_inodes": 0, 2024-09-13T16:34:34.702 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dirfrag_rstats": 0, 2024-09-13T16:34:34.703 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_file_inodes": 0, 2024-09-13T16:34:34.703 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_inode_file_caps": 0, 2024-09-13T16:34:34.703 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_revoke": 0, 2024-09-13T16:34:34.703 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_grant": 0, 2024-09-13T16:34:34.703 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_trunc": 0, 2024-09-13T16:34:34.703 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-13T16:34:34.703 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-13T16:34:34.703 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps": 0, 2024-09-13T16:34:34.703 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps_dirty": 0, 2024-09-13T16:34:34.703 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_cap_release": 0, 2024-09-13T16:34:34.703 INFO:teuthology.orchestra.run.smithi111.stdout: "process_request_cap_release": 0 2024-09-13T16:34:34.703 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.703 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_cache": { 2024-09-13T16:34:34.704 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update": 0, 2024-09-13T16:34:34.704 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update_receipt": 0, 2024-09-13T16:34:34.704 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_try_discover": 0, 2024-09-13T16:34:34.704 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_send_discover": 0, 2024-09-13T16:34:34.704 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_handle_discover": 0, 2024-09-13T16:34:34.704 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays": 0, 2024-09-13T16:34:34.704 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_delayed": 0, 2024-09-13T16:34:34.704 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_enqueuing": 0, 2024-09-13T16:34:34.704 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_created": 0, 2024-09-13T16:34:34.704 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_enqueued": 0, 2024-09-13T16:34:34.704 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_reintegrated": 0, 2024-09-13T16:34:34.704 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_migrated": 0, 2024-09-13T16:34:34.704 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_processing": 0, 2024-09-13T16:34:34.705 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_enqueued": 0, 2024-09-13T16:34:34.705 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_prioritized": 0, 2024-09-13T16:34:34.705 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_started": 0, 2024-09-13T16:34:34.705 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_completed": 0, 2024-09-13T16:34:34.705 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_enqueue_scrub": 0, 2024-09-13T16:34:34.705 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_exportdir": 0, 2024-09-13T16:34:34.705 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_flush": 0, 2024-09-13T16:34:34.705 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragmentdir": 0, 2024-09-13T16:34:34.705 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragstats": 0, 2024-09-13T16:34:34.705 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_inodestats": 0 2024-09-13T16:34:34.705 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.705 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_log": { 2024-09-13T16:34:34.705 INFO:teuthology.orchestra.run.smithi111.stdout: "evadd": 0, 2024-09-13T16:34:34.706 INFO:teuthology.orchestra.run.smithi111.stdout: "evex": 0, 2024-09-13T16:34:34.706 INFO:teuthology.orchestra.run.smithi111.stdout: "evtrm": 0, 2024-09-13T16:34:34.706 INFO:teuthology.orchestra.run.smithi111.stdout: "ev": 158, 2024-09-13T16:34:34.706 INFO:teuthology.orchestra.run.smithi111.stdout: "evexg": 0, 2024-09-13T16:34:34.706 INFO:teuthology.orchestra.run.smithi111.stdout: "evexd": 0, 2024-09-13T16:34:34.706 INFO:teuthology.orchestra.run.smithi111.stdout: "segadd": 0, 2024-09-13T16:34:34.706 INFO:teuthology.orchestra.run.smithi111.stdout: "segex": 0, 2024-09-13T16:34:34.706 INFO:teuthology.orchestra.run.smithi111.stdout: "segtrm": 0, 2024-09-13T16:34:34.706 INFO:teuthology.orchestra.run.smithi111.stdout: "seg": 2, 2024-09-13T16:34:34.706 INFO:teuthology.orchestra.run.smithi111.stdout: "segexg": 0, 2024-09-13T16:34:34.707 INFO:teuthology.orchestra.run.smithi111.stdout: "segexd": 0, 2024-09-13T16:34:34.707 INFO:teuthology.orchestra.run.smithi111.stdout: "expos": 4608085, 2024-09-13T16:34:34.707 INFO:teuthology.orchestra.run.smithi111.stdout: "wrpos": 4635576, 2024-09-13T16:34:34.707 INFO:teuthology.orchestra.run.smithi111.stdout: "rdpos": 4631818, 2024-09-13T16:34:34.707 INFO:teuthology.orchestra.run.smithi111.stdout: "jlat": { 2024-09-13T16:34:34.707 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.707 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.707 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.707 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.707 INFO:teuthology.orchestra.run.smithi111.stdout: "replayed": 158 2024-09-13T16:34:34.707 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.707 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_mem": { 2024-09-13T16:34:34.707 INFO:teuthology.orchestra.run.smithi111.stdout: "ino": 16, 2024-09-13T16:34:34.708 INFO:teuthology.orchestra.run.smithi111.stdout: "ino+": 21, 2024-09-13T16:34:34.708 INFO:teuthology.orchestra.run.smithi111.stdout: "ino-": 5, 2024-09-13T16:34:34.708 INFO:teuthology.orchestra.run.smithi111.stdout: "dir": 11, 2024-09-13T16:34:34.708 INFO:teuthology.orchestra.run.smithi111.stdout: "dir+": 13, 2024-09-13T16:34:34.708 INFO:teuthology.orchestra.run.smithi111.stdout: "dir-": 2, 2024-09-13T16:34:34.708 INFO:teuthology.orchestra.run.smithi111.stdout: "dn": 21, 2024-09-13T16:34:34.708 INFO:teuthology.orchestra.run.smithi111.stdout: "dn+": 25, 2024-09-13T16:34:34.708 INFO:teuthology.orchestra.run.smithi111.stdout: "dn-": 4, 2024-09-13T16:34:34.708 INFO:teuthology.orchestra.run.smithi111.stdout: "cap": 0, 2024-09-13T16:34:34.709 INFO:teuthology.orchestra.run.smithi111.stdout: "cap+": 0, 2024-09-13T16:34:34.709 INFO:teuthology.orchestra.run.smithi111.stdout: "cap-": 0, 2024-09-13T16:34:34.709 INFO:teuthology.orchestra.run.smithi111.stdout: "rss": 48028, 2024-09-13T16:34:34.709 INFO:teuthology.orchestra.run.smithi111.stdout: "heap": 207124 2024-09-13T16:34:34.709 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.709 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_server": { 2024-09-13T16:34:34.709 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_client_request": 0, 2024-09-13T16:34:34.709 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_server_request": 0, 2024-09-13T16:34:34.709 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_request": 0, 2024-09-13T16:34:34.709 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_session": 0, 2024-09-13T16:34:34.709 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_peer_request": 0, 2024-09-13T16:34:34.709 INFO:teuthology.orchestra.run.smithi111.stdout: "req_create_latency": { 2024-09-13T16:34:34.710 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.710 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.710 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.710 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.710 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getattr_latency": { 2024-09-13T16:34:34.710 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.710 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.710 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.710 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.710 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getfilelock_latency": { 2024-09-13T16:34:34.710 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.710 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.710 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.710 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.711 INFO:teuthology.orchestra.run.smithi111.stdout: "req_link_latency": { 2024-09-13T16:34:34.711 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.711 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.711 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.711 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.711 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookup_latency": { 2024-09-13T16:34:34.711 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.711 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.711 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.711 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.711 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookuphash_latency": { 2024-09-13T16:34:34.711 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.711 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.712 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.712 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.712 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupino_latency": { 2024-09-13T16:34:34.712 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.712 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.712 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.712 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.712 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupname_latency": { 2024-09-13T16:34:34.712 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.712 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.712 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.712 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.712 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupparent_latency": { 2024-09-13T16:34:34.713 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.713 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.713 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.713 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.713 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupsnap_latency": { 2024-09-13T16:34:34.713 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.713 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.713 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.713 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.713 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lssnap_latency": { 2024-09-13T16:34:34.713 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.713 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.714 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.714 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.714 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mkdir_latency": { 2024-09-13T16:34:34.714 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.714 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.714 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.714 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.714 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mknod_latency": { 2024-09-13T16:34:34.714 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.714 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.714 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.714 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.714 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mksnap_latency": { 2024-09-13T16:34:34.715 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.715 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.715 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.715 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.715 INFO:teuthology.orchestra.run.smithi111.stdout: "req_open_latency": { 2024-09-13T16:34:34.715 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.715 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.715 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.715 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.715 INFO:teuthology.orchestra.run.smithi111.stdout: "req_readdir_latency": { 2024-09-13T16:34:34.715 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.715 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.715 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.715 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.716 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rename_latency": { 2024-09-13T16:34:34.716 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.716 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.716 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.716 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.716 INFO:teuthology.orchestra.run.smithi111.stdout: "req_renamesnap_latency": { 2024-09-13T16:34:34.716 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.716 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.716 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.716 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.716 INFO:teuthology.orchestra.run.smithi111.stdout: "req_snapdiff_latency": { 2024-09-13T16:34:34.716 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.716 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.717 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.717 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.717 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmdir_latency": { 2024-09-13T16:34:34.717 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.717 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.717 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.717 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.717 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmsnap_latency": { 2024-09-13T16:34:34.717 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.717 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.717 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.717 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.717 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmxattr_latency": { 2024-09-13T16:34:34.718 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.718 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.718 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.718 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.718 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setattr_latency": { 2024-09-13T16:34:34.718 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.718 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.718 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.718 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.718 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setdirlayout_latency": { 2024-09-13T16:34:34.718 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.718 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.719 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.719 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.719 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setfilelock_latency": { 2024-09-13T16:34:34.719 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.719 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.719 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.719 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.719 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setlayout_latency": { 2024-09-13T16:34:34.719 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.719 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.719 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.719 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.719 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setxattr_latency": { 2024-09-13T16:34:34.720 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.720 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.720 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.720 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.720 INFO:teuthology.orchestra.run.smithi111.stdout: "req_symlink_latency": { 2024-09-13T16:34:34.720 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.720 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.720 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.720 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.720 INFO:teuthology.orchestra.run.smithi111.stdout: "req_unlink_latency": { 2024-09-13T16:34:34.720 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.720 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.720 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.721 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.721 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_revoke_eviction": 0, 2024-09-13T16:34:34.721 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_acquisition_throttle": 0, 2024-09-13T16:34:34.721 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getvxattr_latency": { 2024-09-13T16:34:34.721 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.721 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.721 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.721 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:34.721 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.721 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_sessions": { 2024-09-13T16:34:34.721 INFO:teuthology.orchestra.run.smithi111.stdout: "session_count": 1, 2024-09-13T16:34:34.721 INFO:teuthology.orchestra.run.smithi111.stdout: "session_add": 4, 2024-09-13T16:34:34.721 INFO:teuthology.orchestra.run.smithi111.stdout: "session_remove": 3, 2024-09-13T16:34:34.722 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_open": 2, 2024-09-13T16:34:34.722 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_stale": 0, 2024-09-13T16:34:34.722 INFO:teuthology.orchestra.run.smithi111.stdout: "total_load": 0, 2024-09-13T16:34:34.722 INFO:teuthology.orchestra.run.smithi111.stdout: "average_load": 0, 2024-09-13T16:34:34.722 INFO:teuthology.orchestra.run.smithi111.stdout: "avg_session_uptime": 2230, 2024-09-13T16:34:34.722 INFO:teuthology.orchestra.run.smithi111.stdout: "mdthresh_evicted": 0 2024-09-13T16:34:34.722 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.722 INFO:teuthology.orchestra.run.smithi111.stdout: "mempool": { 2024-09-13T16:34:34.722 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_bytes": 0, 2024-09-13T16:34:34.722 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_items": 0, 2024-09-13T16:34:34.722 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T16:34:34.722 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_items": 0, 2024-09-13T16:34:34.722 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T16:34:34.722 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_items": 0, 2024-09-13T16:34:34.723 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T16:34:34.723 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T16:34:34.723 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T16:34:34.723 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T16:34:34.723 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T16:34:34.723 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_items": 0, 2024-09-13T16:34:34.723 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T16:34:34.723 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T16:34:34.723 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_bytes": 0, 2024-09-13T16:34:34.723 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_items": 0, 2024-09-13T16:34:34.723 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_bytes": 0, 2024-09-13T16:34:34.723 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_items": 0, 2024-09-13T16:34:34.723 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T16:34:34.724 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T16:34:34.724 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T16:34:34.724 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T16:34:34.724 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T16:34:34.724 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_items": 0, 2024-09-13T16:34:34.724 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_bytes": 0, 2024-09-13T16:34:34.724 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_items": 0, 2024-09-13T16:34:34.724 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T16:34:34.724 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T16:34:34.724 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_bytes": 0, 2024-09-13T16:34:34.724 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_items": 0, 2024-09-13T16:34:34.724 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_bytes": 0, 2024-09-13T16:34:34.724 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_items": 0, 2024-09-13T16:34:34.725 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T16:34:34.725 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_items": 0, 2024-09-13T16:34:34.725 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T16:34:34.725 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_items": 0, 2024-09-13T16:34:34.725 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_bytes": 189239, 2024-09-13T16:34:34.725 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_items": 48, 2024-09-13T16:34:34.725 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_bytes": 0, 2024-09-13T16:34:34.725 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_items": 0, 2024-09-13T16:34:34.725 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_bytes": 0, 2024-09-13T16:34:34.725 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_items": 0, 2024-09-13T16:34:34.725 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_bytes": 0, 2024-09-13T16:34:34.725 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_items": 0, 2024-09-13T16:34:34.725 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_bytes": 0, 2024-09-13T16:34:34.726 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_items": 0, 2024-09-13T16:34:34.726 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_bytes": 8856, 2024-09-13T16:34:34.726 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_items": 166, 2024-09-13T16:34:34.726 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T16:34:34.726 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_items": 0, 2024-09-13T16:34:34.726 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_bytes": 0, 2024-09-13T16:34:34.726 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_items": 0, 2024-09-13T16:34:34.726 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_bytes": 64042, 2024-09-13T16:34:34.726 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_items": 300, 2024-09-13T16:34:34.726 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_bytes": 0, 2024-09-13T16:34:34.726 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_items": 0, 2024-09-13T16:34:34.726 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_bytes": 0, 2024-09-13T16:34:34.726 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_items": 0 2024-09-13T16:34:34.726 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.727 INFO:teuthology.orchestra.run.smithi111.stdout: "objecter": { 2024-09-13T16:34:34.727 INFO:teuthology.orchestra.run.smithi111.stdout: "op_active": 0, 2024-09-13T16:34:34.727 INFO:teuthology.orchestra.run.smithi111.stdout: "op_laggy": 0, 2024-09-13T16:34:34.727 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send": 1059, 2024-09-13T16:34:34.727 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send_bytes": 26, 2024-09-13T16:34:34.727 INFO:teuthology.orchestra.run.smithi111.stdout: "op_resend": 4, 2024-09-13T16:34:34.727 INFO:teuthology.orchestra.run.smithi111.stdout: "op_reply": 1059, 2024-09-13T16:34:34.727 INFO:teuthology.orchestra.run.smithi111.stdout: "op_latency": { 2024-09-13T16:34:34.727 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 1059, 2024-09-13T16:34:34.727 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.492991145, 2024-09-13T16:34:34.727 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000465525 2024-09-13T16:34:34.727 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.727 INFO:teuthology.orchestra.run.smithi111.stdout: "op_inflight": 0, 2024-09-13T16:34:34.727 INFO:teuthology.orchestra.run.smithi111.stdout: "oplen_avg": { 2024-09-13T16:34:34.728 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 1059, 2024-09-13T16:34:34.728 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 1060 2024-09-13T16:34:34.728 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.728 INFO:teuthology.orchestra.run.smithi111.stdout: "op": 1059, 2024-09-13T16:34:34.728 INFO:teuthology.orchestra.run.smithi111.stdout: "op_r": 1059, 2024-09-13T16:34:34.728 INFO:teuthology.orchestra.run.smithi111.stdout: "op_w": 0, 2024-09-13T16:34:34.728 INFO:teuthology.orchestra.run.smithi111.stdout: "op_rmw": 0, 2024-09-13T16:34:34.728 INFO:teuthology.orchestra.run.smithi111.stdout: "op_pg": 0, 2024-09-13T16:34:34.728 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_stat": 660, 2024-09-13T16:34:34.728 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_create": 0, 2024-09-13T16:34:34.728 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_read": 396, 2024-09-13T16:34:34.728 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_write": 0, 2024-09-13T16:34:34.728 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writefull": 0, 2024-09-13T16:34:34.729 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writesame": 0, 2024-09-13T16:34:34.729 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_append": 0, 2024-09-13T16:34:34.729 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_zero": 0, 2024-09-13T16:34:34.729 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_truncate": 0, 2024-09-13T16:34:34.729 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_delete": 0, 2024-09-13T16:34:34.729 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_mapext": 0, 2024-09-13T16:34:34.729 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_sparse_read": 0, 2024-09-13T16:34:34.729 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_clonerange": 0, 2024-09-13T16:34:34.729 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_getxattr": 2, 2024-09-13T16:34:34.729 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_setxattr": 0, 2024-09-13T16:34:34.729 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_cmpxattr": 0, 2024-09-13T16:34:34.729 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_rmxattr": 0, 2024-09-13T16:34:34.729 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_resetxattrs": 0, 2024-09-13T16:34:34.730 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_call": 0, 2024-09-13T16:34:34.730 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_watch": 0, 2024-09-13T16:34:34.730 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_notify": 0, 2024-09-13T16:34:34.730 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_src_cmpxattr": 0, 2024-09-13T16:34:34.730 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls": 0, 2024-09-13T16:34:34.730 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls_filter": 0, 2024-09-13T16:34:34.730 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_other": 0, 2024-09-13T16:34:34.730 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_active": 0, 2024-09-13T16:34:34.730 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_send": 0, 2024-09-13T16:34:34.730 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_resend": 0, 2024-09-13T16:34:34.730 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_ping": 0, 2024-09-13T16:34:34.730 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_active": 0, 2024-09-13T16:34:34.730 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_send": 0, 2024-09-13T16:34:34.730 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_resend": 0, 2024-09-13T16:34:34.731 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_active": 0, 2024-09-13T16:34:34.731 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_send": 0, 2024-09-13T16:34:34.731 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_resend": 0, 2024-09-13T16:34:34.731 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_active": 0, 2024-09-13T16:34:34.731 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_send": 0, 2024-09-13T16:34:34.731 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_resend": 0, 2024-09-13T16:34:34.731 INFO:teuthology.orchestra.run.smithi111.stdout: "command_active": 0, 2024-09-13T16:34:34.731 INFO:teuthology.orchestra.run.smithi111.stdout: "command_send": 0, 2024-09-13T16:34:34.731 INFO:teuthology.orchestra.run.smithi111.stdout: "command_resend": 0, 2024-09-13T16:34:34.731 INFO:teuthology.orchestra.run.smithi111.stdout: "map_epoch": 93, 2024-09-13T16:34:34.731 INFO:teuthology.orchestra.run.smithi111.stdout: "map_full": 0, 2024-09-13T16:34:34.731 INFO:teuthology.orchestra.run.smithi111.stdout: "map_inc": 4, 2024-09-13T16:34:34.731 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_sessions": 5, 2024-09-13T16:34:34.731 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_open": 5, 2024-09-13T16:34:34.732 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_close": 0, 2024-09-13T16:34:34.732 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_laggy": 0, 2024-09-13T16:34:34.732 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_wr": 0, 2024-09-13T16:34:34.732 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_rd": 2, 2024-09-13T16:34:34.732 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_del": 0 2024-09-13T16:34:34.732 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.732 INFO:teuthology.orchestra.run.smithi111.stdout: "oft": { 2024-09-13T16:34:34.732 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_objs": 0, 2024-09-13T16:34:34.732 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_kv_pairs": 0, 2024-09-13T16:34:34.732 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_updates": 0, 2024-09-13T16:34:34.732 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_removes": 0 2024-09-13T16:34:34.732 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.732 INFO:teuthology.orchestra.run.smithi111.stdout: "osd-slow-ops": { 2024-09-13T16:34:34.732 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_ops_count": 0 2024-09-13T16:34:34.733 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.733 INFO:teuthology.orchestra.run.smithi111.stdout: "purge_queue": { 2024-09-13T16:34:34.733 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops": 0, 2024-09-13T16:34:34.733 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops_high_water": 0, 2024-09-13T16:34:34.733 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing": 0, 2024-09-13T16:34:34.733 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_high_water": 0, 2024-09-13T16:34:34.733 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed_ops": 0, 2024-09-13T16:34:34.733 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed": 0, 2024-09-13T16:34:34.733 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_item_in_journal": 0 2024-09-13T16:34:34.733 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.733 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T16:34:34.733 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-13T16:34:34.733 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-09-13T16:34:34.734 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-13T16:34:34.734 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 1185, 2024-09-13T16:34:34.734 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 1384207, 2024-09-13T16:34:34.734 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:34.734 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 1185, 2024-09-13T16:34:34.734 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-13T16:34:34.734 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-13T16:34:34.734 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 1185, 2024-09-13T16:34:34.734 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 1384207, 2024-09-13T16:34:34.734 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-13T16:34:34.734 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.734 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.734 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.734 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:34.735 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.735 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_bytes": { 2024-09-13T16:34:34.735 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-13T16:34:34.735 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-09-13T16:34:34.735 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-13T16:34:34.735 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-13T16:34:34.735 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-13T16:34:34.735 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:34.735 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:34.735 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 1059, 2024-09-13T16:34:34.735 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 441282, 2024-09-13T16:34:34.735 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 1059, 2024-09-13T16:34:34.735 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 441282, 2024-09-13T16:34:34.736 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-13T16:34:34.736 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.736 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.736 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.736 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:34.736 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.736 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_ops": { 2024-09-13T16:34:34.737 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-13T16:34:34.737 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 1024, 2024-09-13T16:34:34.737 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-13T16:34:34.737 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-13T16:34:34.737 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-13T16:34:34.737 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:34.737 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:34.737 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 1059, 2024-09-13T16:34:34.737 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 1059, 2024-09-13T16:34:34.737 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 1059, 2024-09-13T16:34:34.737 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 1059, 2024-09-13T16:34:34.737 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-13T16:34:34.737 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.738 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.738 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.738 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:34.738 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.738 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle": { 2024-09-13T16:34:34.738 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-13T16:34:34.738 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-09-13T16:34:34.738 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-13T16:34:34.738 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-13T16:34:34.738 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-13T16:34:34.738 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:34.738 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:34.738 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-13T16:34:34.739 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-13T16:34:34.739 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 0, 2024-09-13T16:34:34.739 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 0, 2024-09-13T16:34:34.739 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-13T16:34:34.739 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.739 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.739 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.739 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:34.739 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:34.739 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle-0x56511451e820": { 2024-09-13T16:34:34.739 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-13T16:34:34.739 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-09-13T16:34:34.739 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-13T16:34:34.740 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-13T16:34:34.740 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-13T16:34:34.740 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:34.740 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:34.740 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-13T16:34:34.740 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-13T16:34:34.740 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 0, 2024-09-13T16:34:34.740 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 0, 2024-09-13T16:34:34.740 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-13T16:34:34.740 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:34.740 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:34.740 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:34.741 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:34.741 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:34.741 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-13T16:34:35.519 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-13T16:34:35.519 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-13T16:34:35.519 DEBUG:tasks.check_counter:Getting stats from f 2024-09-13T16:34:35.519 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-13T16:34:35.725 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T16:34:36.111 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:36 smithi111 ceph-mon[34600]: pgmap v458: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T16:34:36.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:36 smithi107 ceph-mon[34139]: pgmap v458: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T16:34:36.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:36 smithi016 ceph-mon[31187]: pgmap v458: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T16:34:37.419 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-13T16:34:37.419 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T16:34:37.419 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 919, 2024-09-13T16:34:37.419 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 921, 2024-09-13T16:34:37.419 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 264201, 2024-09-13T16:34:37.420 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 285816, 2024-09-13T16:34:37.420 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 6, 2024-09-13T16:34:37.420 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 2, 2024-09-13T16:34:37.420 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.081945173, 2024-09-13T16:34:37.420 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.026926743, 2024-09-13T16:34:37.420 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.021394159, 2024-09-13T16:34:37.420 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.026378637, 2024-09-13T16:34:37.420 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:37.420 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 919, 2024-09-13T16:34:37.420 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.035623931, 2024-09-13T16:34:37.420 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000038763 2024-09-13T16:34:37.420 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.420 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:37.421 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.421 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.421 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.421 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.421 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-13T16:34:37.421 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 0 2024-09-13T16:34:37.421 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.421 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T16:34:37.421 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 131, 2024-09-13T16:34:37.421 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 118, 2024-09-13T16:34:37.421 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 762492, 2024-09-13T16:34:37.421 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 61500, 2024-09-13T16:34:37.422 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 3, 2024-09-13T16:34:37.422 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 2, 2024-09-13T16:34:37.422 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.022888962, 2024-09-13T16:34:37.422 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.007441517, 2024-09-13T16:34:37.422 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.021889321, 2024-09-13T16:34:37.422 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000055699, 2024-09-13T16:34:37.422 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:37.422 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 118, 2024-09-13T16:34:37.422 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.011372094, 2024-09-13T16:34:37.422 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000096373 2024-09-13T16:34:37.423 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.423 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:37.423 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.423 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.423 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.423 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.423 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 761936, 2024-09-13T16:34:37.423 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 60816 2024-09-13T16:34:37.423 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.423 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T16:34:37.423 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 2, 2024-09-13T16:34:37.423 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 80, 2024-09-13T16:34:37.423 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 428, 2024-09-13T16:34:37.424 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 205431, 2024-09-13T16:34:37.424 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 4, 2024-09-13T16:34:37.424 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 3, 2024-09-13T16:34:37.424 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.005526255, 2024-09-13T16:34:37.424 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.003564210, 2024-09-13T16:34:37.424 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.001067749, 2024-09-13T16:34:37.424 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000022617, 2024-09-13T16:34:37.424 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:37.424 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 78, 2024-09-13T16:34:37.424 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.005716604, 2024-09-13T16:34:37.425 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000073289 2024-09-13T16:34:37.425 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.425 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:37.425 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.425 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.425 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.425 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.425 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-13T16:34:37.425 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 205120 2024-09-13T16:34:37.425 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.425 INFO:teuthology.orchestra.run.smithi111.stdout: "cct": { 2024-09-13T16:34:37.425 INFO:teuthology.orchestra.run.smithi111.stdout: "total_workers": 1, 2024-09-13T16:34:37.425 INFO:teuthology.orchestra.run.smithi111.stdout: "unhealthy_workers": 0 2024-09-13T16:34:37.426 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.426 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-MDSRank": { 2024-09-13T16:34:37.426 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-09-13T16:34:37.426 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-09-13T16:34:37.426 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 1592, 2024-09-13T16:34:37.426 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.042399671, 2024-09-13T16:34:37.426 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000026632 2024-09-13T16:34:37.426 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:37.426 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.426 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-PurgeQueue": { 2024-09-13T16:34:37.427 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-09-13T16:34:37.427 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-09-13T16:34:37.427 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.427 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.427 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.427 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:37.427 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.428 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": { 2024-09-13T16:34:37.428 INFO:teuthology.orchestra.run.smithi111.stdout: "request": 0, 2024-09-13T16:34:37.428 INFO:teuthology.orchestra.run.smithi111.stdout: "reply": 0, 2024-09-13T16:34:37.428 INFO:teuthology.orchestra.run.smithi111.stdout: "reply_latency": { 2024-09-13T16:34:37.428 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.428 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.428 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.428 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.428 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_reply": 0, 2024-09-13T16:34:37.428 INFO:teuthology.orchestra.run.smithi111.stdout: "forward": 0, 2024-09-13T16:34:37.428 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_complete": 0, 2024-09-13T16:34:37.428 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_keys": 0, 2024-09-13T16:34:37.429 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_commit": 0, 2024-09-13T16:34:37.429 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_split": 0, 2024-09-13T16:34:37.429 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_merge": 0, 2024-09-13T16:34:37.429 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes": 0, 2024-09-13T16:34:37.429 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_top": 0, 2024-09-13T16:34:37.429 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_bottom": 0, 2024-09-13T16:34:37.429 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pin_tail": 0, 2024-09-13T16:34:37.429 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pinned": 0, 2024-09-13T16:34:37.429 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_expired": 0, 2024-09-13T16:34:37.429 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_with_caps": 0, 2024-09-13T16:34:37.429 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": 0, 2024-09-13T16:34:37.429 INFO:teuthology.orchestra.run.smithi111.stdout: "subtrees": 1, 2024-09-13T16:34:37.429 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse": 0, 2024-09-13T16:34:37.430 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_hit": 0, 2024-09-13T16:34:37.430 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_forward": 0, 2024-09-13T16:34:37.430 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_discover": 0, 2024-09-13T16:34:37.430 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_dir_fetch": 0, 2024-09-13T16:34:37.430 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_remote_ino": 0, 2024-09-13T16:34:37.430 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_lock": 0, 2024-09-13T16:34:37.430 INFO:teuthology.orchestra.run.smithi111.stdout: "load_cent": 0, 2024-09-13T16:34:37.430 INFO:teuthology.orchestra.run.smithi111.stdout: "q": 0, 2024-09-13T16:34:37.430 INFO:teuthology.orchestra.run.smithi111.stdout: "exported": 0, 2024-09-13T16:34:37.430 INFO:teuthology.orchestra.run.smithi111.stdout: "exported_inodes": 0, 2024-09-13T16:34:37.430 INFO:teuthology.orchestra.run.smithi111.stdout: "imported": 0, 2024-09-13T16:34:37.430 INFO:teuthology.orchestra.run.smithi111.stdout: "imported_inodes": 0, 2024-09-13T16:34:37.430 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_dir_fetch": 0, 2024-09-13T16:34:37.431 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_backtrace_fetch": 0, 2024-09-13T16:34:37.431 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_peer_discover": 0, 2024-09-13T16:34:37.431 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rfiles": 0, 2024-09-13T16:34:37.431 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rbytes": 0, 2024-09-13T16:34:37.431 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rsnaps": 0, 2024-09-13T16:34:37.431 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_fetch": 0, 2024-09-13T16:34:37.431 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_set_tag": 0, 2024-09-13T16:34:37.431 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_repaired": 0, 2024-09-13T16:34:37.431 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_inotable_repaired": 0, 2024-09-13T16:34:37.431 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_inodes": 0, 2024-09-13T16:34:37.431 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_base_inodes": 0, 2024-09-13T16:34:37.431 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dirfrag_rstats": 0, 2024-09-13T16:34:37.431 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_file_inodes": 0, 2024-09-13T16:34:37.432 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_inode_file_caps": 0, 2024-09-13T16:34:37.432 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_revoke": 0, 2024-09-13T16:34:37.432 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_grant": 0, 2024-09-13T16:34:37.432 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_trunc": 0, 2024-09-13T16:34:37.432 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-13T16:34:37.432 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-13T16:34:37.432 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps": 0, 2024-09-13T16:34:37.432 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps_dirty": 0, 2024-09-13T16:34:37.432 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_cap_release": 0, 2024-09-13T16:34:37.432 INFO:teuthology.orchestra.run.smithi111.stdout: "process_request_cap_release": 0 2024-09-13T16:34:37.432 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.432 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_cache": { 2024-09-13T16:34:37.432 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update": 0, 2024-09-13T16:34:37.432 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update_receipt": 0, 2024-09-13T16:34:37.436 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_try_discover": 0, 2024-09-13T16:34:37.436 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_send_discover": 0, 2024-09-13T16:34:37.436 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_handle_discover": 0, 2024-09-13T16:34:37.436 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays": 0, 2024-09-13T16:34:37.436 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_delayed": 0, 2024-09-13T16:34:37.436 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_enqueuing": 0, 2024-09-13T16:34:37.436 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_created": 0, 2024-09-13T16:34:37.437 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_enqueued": 0, 2024-09-13T16:34:37.437 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_reintegrated": 0, 2024-09-13T16:34:37.437 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_migrated": 0, 2024-09-13T16:34:37.437 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_processing": 0, 2024-09-13T16:34:37.437 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_enqueued": 0, 2024-09-13T16:34:37.437 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_prioritized": 0, 2024-09-13T16:34:37.437 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_started": 0, 2024-09-13T16:34:37.437 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_completed": 0, 2024-09-13T16:34:37.437 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_enqueue_scrub": 0, 2024-09-13T16:34:37.437 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_exportdir": 0, 2024-09-13T16:34:37.437 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_flush": 0, 2024-09-13T16:34:37.437 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragmentdir": 0, 2024-09-13T16:34:37.437 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragstats": 0, 2024-09-13T16:34:37.438 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_inodestats": 0 2024-09-13T16:34:37.438 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.438 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_log": { 2024-09-13T16:34:37.438 INFO:teuthology.orchestra.run.smithi111.stdout: "evadd": 0, 2024-09-13T16:34:37.438 INFO:teuthology.orchestra.run.smithi111.stdout: "evex": 0, 2024-09-13T16:34:37.438 INFO:teuthology.orchestra.run.smithi111.stdout: "evtrm": 0, 2024-09-13T16:34:37.438 INFO:teuthology.orchestra.run.smithi111.stdout: "ev": 6, 2024-09-13T16:34:37.438 INFO:teuthology.orchestra.run.smithi111.stdout: "evexg": 0, 2024-09-13T16:34:37.438 INFO:teuthology.orchestra.run.smithi111.stdout: "evexd": 0, 2024-09-13T16:34:37.438 INFO:teuthology.orchestra.run.smithi111.stdout: "segadd": 0, 2024-09-13T16:34:37.438 INFO:teuthology.orchestra.run.smithi111.stdout: "segex": 0, 2024-09-13T16:34:37.438 INFO:teuthology.orchestra.run.smithi111.stdout: "segtrm": 0, 2024-09-13T16:34:37.438 INFO:teuthology.orchestra.run.smithi111.stdout: "seg": 1, 2024-09-13T16:34:37.439 INFO:teuthology.orchestra.run.smithi111.stdout: "segexg": 0, 2024-09-13T16:34:37.439 INFO:teuthology.orchestra.run.smithi111.stdout: "segexd": 0, 2024-09-13T16:34:37.439 INFO:teuthology.orchestra.run.smithi111.stdout: "expos": 4194304, 2024-09-13T16:34:37.439 INFO:teuthology.orchestra.run.smithi111.stdout: "wrpos": 4196404, 2024-09-13T16:34:37.439 INFO:teuthology.orchestra.run.smithi111.stdout: "rdpos": 4196263, 2024-09-13T16:34:37.439 INFO:teuthology.orchestra.run.smithi111.stdout: "jlat": { 2024-09-13T16:34:37.439 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.439 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.439 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.439 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.439 INFO:teuthology.orchestra.run.smithi111.stdout: "replayed": 6 2024-09-13T16:34:37.439 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.439 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_mem": { 2024-09-13T16:34:37.440 INFO:teuthology.orchestra.run.smithi111.stdout: "ino": 3, 2024-09-13T16:34:37.440 INFO:teuthology.orchestra.run.smithi111.stdout: "ino+": 3, 2024-09-13T16:34:37.440 INFO:teuthology.orchestra.run.smithi111.stdout: "ino-": 0, 2024-09-13T16:34:37.440 INFO:teuthology.orchestra.run.smithi111.stdout: "dir": 1, 2024-09-13T16:34:37.440 INFO:teuthology.orchestra.run.smithi111.stdout: "dir+": 1, 2024-09-13T16:34:37.440 INFO:teuthology.orchestra.run.smithi111.stdout: "dir-": 0, 2024-09-13T16:34:37.440 INFO:teuthology.orchestra.run.smithi111.stdout: "dn": 0, 2024-09-13T16:34:37.440 INFO:teuthology.orchestra.run.smithi111.stdout: "dn+": 0, 2024-09-13T16:34:37.440 INFO:teuthology.orchestra.run.smithi111.stdout: "dn-": 0, 2024-09-13T16:34:37.440 INFO:teuthology.orchestra.run.smithi111.stdout: "cap": 0, 2024-09-13T16:34:37.440 INFO:teuthology.orchestra.run.smithi111.stdout: "cap+": 0, 2024-09-13T16:34:37.441 INFO:teuthology.orchestra.run.smithi111.stdout: "cap-": 0, 2024-09-13T16:34:37.441 INFO:teuthology.orchestra.run.smithi111.stdout: "rss": 47144, 2024-09-13T16:34:37.441 INFO:teuthology.orchestra.run.smithi111.stdout: "heap": 207124 2024-09-13T16:34:37.441 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.441 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_server": { 2024-09-13T16:34:37.441 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_client_request": 0, 2024-09-13T16:34:37.441 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_server_request": 0, 2024-09-13T16:34:37.441 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_request": 0, 2024-09-13T16:34:37.441 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_session": 0, 2024-09-13T16:34:37.441 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_peer_request": 0, 2024-09-13T16:34:37.441 INFO:teuthology.orchestra.run.smithi111.stdout: "req_create_latency": { 2024-09-13T16:34:37.441 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.442 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.442 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.442 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.442 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getattr_latency": { 2024-09-13T16:34:37.442 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.442 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.442 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.442 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.443 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getfilelock_latency": { 2024-09-13T16:34:37.443 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.443 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.443 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.443 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.443 INFO:teuthology.orchestra.run.smithi111.stdout: "req_link_latency": { 2024-09-13T16:34:37.443 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.443 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.443 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.443 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.444 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookup_latency": { 2024-09-13T16:34:37.444 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.444 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.444 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.444 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.444 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookuphash_latency": { 2024-09-13T16:34:37.444 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.444 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.444 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.444 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.444 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupino_latency": { 2024-09-13T16:34:37.444 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.445 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.445 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.445 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.445 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupname_latency": { 2024-09-13T16:34:37.445 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.445 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.445 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.445 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.445 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupparent_latency": { 2024-09-13T16:34:37.445 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.446 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.446 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.446 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.446 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupsnap_latency": { 2024-09-13T16:34:37.446 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.446 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.446 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.447 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.447 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lssnap_latency": { 2024-09-13T16:34:37.447 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.447 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.447 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.447 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.447 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mkdir_latency": { 2024-09-13T16:34:37.447 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.447 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.447 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.447 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.448 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mknod_latency": { 2024-09-13T16:34:37.448 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.449 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.449 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.449 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.449 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mksnap_latency": { 2024-09-13T16:34:37.449 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.449 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.449 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.449 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.449 INFO:teuthology.orchestra.run.smithi111.stdout: "req_open_latency": { 2024-09-13T16:34:37.449 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.450 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.450 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.450 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.450 INFO:teuthology.orchestra.run.smithi111.stdout: "req_readdir_latency": { 2024-09-13T16:34:37.450 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.450 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.450 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.450 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.450 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rename_latency": { 2024-09-13T16:34:37.451 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.451 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.451 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.451 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.451 INFO:teuthology.orchestra.run.smithi111.stdout: "req_renamesnap_latency": { 2024-09-13T16:34:37.451 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.451 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.451 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.451 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.451 INFO:teuthology.orchestra.run.smithi111.stdout: "req_snapdiff_latency": { 2024-09-13T16:34:37.452 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.452 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.452 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.452 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.452 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmdir_latency": { 2024-09-13T16:34:37.452 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.452 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.452 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.452 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.452 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmsnap_latency": { 2024-09-13T16:34:37.452 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.452 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.452 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.453 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.453 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmxattr_latency": { 2024-09-13T16:34:37.453 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.453 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.453 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.453 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.453 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setattr_latency": { 2024-09-13T16:34:37.453 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.453 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.453 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.453 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.453 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setdirlayout_latency": { 2024-09-13T16:34:37.453 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.454 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.454 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.454 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.454 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setfilelock_latency": { 2024-09-13T16:34:37.454 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.454 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.454 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.454 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.454 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setlayout_latency": { 2024-09-13T16:34:37.454 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.454 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.454 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.454 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.454 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setxattr_latency": { 2024-09-13T16:34:37.455 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.455 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.455 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.455 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.455 INFO:teuthology.orchestra.run.smithi111.stdout: "req_symlink_latency": { 2024-09-13T16:34:37.455 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.455 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.455 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.455 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.455 INFO:teuthology.orchestra.run.smithi111.stdout: "req_unlink_latency": { 2024-09-13T16:34:37.455 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.455 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.455 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.455 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.456 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_revoke_eviction": 0, 2024-09-13T16:34:37.456 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_acquisition_throttle": 0, 2024-09-13T16:34:37.456 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getvxattr_latency": { 2024-09-13T16:34:37.456 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.456 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.456 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.456 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:37.456 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.456 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_sessions": { 2024-09-13T16:34:37.456 INFO:teuthology.orchestra.run.smithi111.stdout: "session_count": 1, 2024-09-13T16:34:37.456 INFO:teuthology.orchestra.run.smithi111.stdout: "session_add": 3, 2024-09-13T16:34:37.456 INFO:teuthology.orchestra.run.smithi111.stdout: "session_remove": 2, 2024-09-13T16:34:37.456 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_open": 2, 2024-09-13T16:34:37.457 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_stale": 0, 2024-09-13T16:34:37.457 INFO:teuthology.orchestra.run.smithi111.stdout: "total_load": 0, 2024-09-13T16:34:37.457 INFO:teuthology.orchestra.run.smithi111.stdout: "average_load": 0, 2024-09-13T16:34:37.457 INFO:teuthology.orchestra.run.smithi111.stdout: "avg_session_uptime": 2232, 2024-09-13T16:34:37.457 INFO:teuthology.orchestra.run.smithi111.stdout: "mdthresh_evicted": 0 2024-09-13T16:34:37.457 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.457 INFO:teuthology.orchestra.run.smithi111.stdout: "mempool": { 2024-09-13T16:34:37.457 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_bytes": 0, 2024-09-13T16:34:37.457 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_items": 0, 2024-09-13T16:34:37.457 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T16:34:37.457 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_items": 0, 2024-09-13T16:34:37.457 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T16:34:37.457 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_items": 0, 2024-09-13T16:34:37.458 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T16:34:37.458 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T16:34:37.458 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T16:34:37.458 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T16:34:37.458 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T16:34:37.458 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_items": 0, 2024-09-13T16:34:37.458 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T16:34:37.458 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T16:34:37.458 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_bytes": 0, 2024-09-13T16:34:37.458 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_items": 0, 2024-09-13T16:34:37.458 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_bytes": 0, 2024-09-13T16:34:37.458 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_items": 0, 2024-09-13T16:34:37.458 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T16:34:37.458 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T16:34:37.459 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T16:34:37.459 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T16:34:37.459 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T16:34:37.459 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_items": 0, 2024-09-13T16:34:37.459 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_bytes": 0, 2024-09-13T16:34:37.459 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_items": 0, 2024-09-13T16:34:37.459 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T16:34:37.459 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T16:34:37.459 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_bytes": 0, 2024-09-13T16:34:37.459 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_items": 0, 2024-09-13T16:34:37.459 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_bytes": 0, 2024-09-13T16:34:37.459 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_items": 0, 2024-09-13T16:34:37.459 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T16:34:37.459 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_items": 0, 2024-09-13T16:34:37.460 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T16:34:37.460 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_items": 0, 2024-09-13T16:34:37.460 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_bytes": 181159, 2024-09-13T16:34:37.460 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_items": 44, 2024-09-13T16:34:37.460 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_bytes": 0, 2024-09-13T16:34:37.460 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_items": 0, 2024-09-13T16:34:37.460 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_bytes": 0, 2024-09-13T16:34:37.460 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_items": 0, 2024-09-13T16:34:37.460 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_bytes": 0, 2024-09-13T16:34:37.460 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_items": 0, 2024-09-13T16:34:37.460 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_bytes": 0, 2024-09-13T16:34:37.460 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_items": 0, 2024-09-13T16:34:37.460 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_bytes": 8856, 2024-09-13T16:34:37.461 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_items": 166, 2024-09-13T16:34:37.461 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T16:34:37.461 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_items": 0, 2024-09-13T16:34:37.461 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_bytes": 0, 2024-09-13T16:34:37.461 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_items": 0, 2024-09-13T16:34:37.461 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_bytes": 8368, 2024-09-13T16:34:37.461 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_items": 13, 2024-09-13T16:34:37.461 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_bytes": 0, 2024-09-13T16:34:37.461 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_items": 0, 2024-09-13T16:34:37.461 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_bytes": 0, 2024-09-13T16:34:37.461 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_items": 0 2024-09-13T16:34:37.462 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.462 INFO:teuthology.orchestra.run.smithi111.stdout: "objecter": { 2024-09-13T16:34:37.462 INFO:teuthology.orchestra.run.smithi111.stdout: "op_active": 0, 2024-09-13T16:34:37.462 INFO:teuthology.orchestra.run.smithi111.stdout: "op_laggy": 0, 2024-09-13T16:34:37.462 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send": 922, 2024-09-13T16:34:37.462 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send_bytes": 21, 2024-09-13T16:34:37.462 INFO:teuthology.orchestra.run.smithi111.stdout: "op_resend": 3, 2024-09-13T16:34:37.462 INFO:teuthology.orchestra.run.smithi111.stdout: "op_reply": 922, 2024-09-13T16:34:37.462 INFO:teuthology.orchestra.run.smithi111.stdout: "op_latency": { 2024-09-13T16:34:37.463 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 922, 2024-09-13T16:34:37.463 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.466991603, 2024-09-13T16:34:37.463 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000506498 2024-09-13T16:34:37.463 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.463 INFO:teuthology.orchestra.run.smithi111.stdout: "op_inflight": 0, 2024-09-13T16:34:37.463 INFO:teuthology.orchestra.run.smithi111.stdout: "oplen_avg": { 2024-09-13T16:34:37.463 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 922, 2024-09-13T16:34:37.463 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 923 2024-09-13T16:34:37.463 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.463 INFO:teuthology.orchestra.run.smithi111.stdout: "op": 922, 2024-09-13T16:34:37.463 INFO:teuthology.orchestra.run.smithi111.stdout: "op_r": 922, 2024-09-13T16:34:37.463 INFO:teuthology.orchestra.run.smithi111.stdout: "op_w": 0, 2024-09-13T16:34:37.464 INFO:teuthology.orchestra.run.smithi111.stdout: "op_rmw": 0, 2024-09-13T16:34:37.464 INFO:teuthology.orchestra.run.smithi111.stdout: "op_pg": 0, 2024-09-13T16:34:37.464 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_stat": 577, 2024-09-13T16:34:37.464 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_create": 0, 2024-09-13T16:34:37.464 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_read": 343, 2024-09-13T16:34:37.464 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_write": 0, 2024-09-13T16:34:37.464 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writefull": 0, 2024-09-13T16:34:37.464 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writesame": 0, 2024-09-13T16:34:37.464 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_append": 0, 2024-09-13T16:34:37.464 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_zero": 0, 2024-09-13T16:34:37.465 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_truncate": 0, 2024-09-13T16:34:37.465 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_delete": 0, 2024-09-13T16:34:37.465 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_mapext": 0, 2024-09-13T16:34:37.465 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_sparse_read": 0, 2024-09-13T16:34:37.465 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_clonerange": 0, 2024-09-13T16:34:37.465 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_getxattr": 1, 2024-09-13T16:34:37.465 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_setxattr": 0, 2024-09-13T16:34:37.466 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_cmpxattr": 0, 2024-09-13T16:34:37.466 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_rmxattr": 0, 2024-09-13T16:34:37.466 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_resetxattrs": 0, 2024-09-13T16:34:37.466 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_call": 0, 2024-09-13T16:34:37.466 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_watch": 0, 2024-09-13T16:34:37.466 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_notify": 0, 2024-09-13T16:34:37.466 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_src_cmpxattr": 0, 2024-09-13T16:34:37.466 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls": 0, 2024-09-13T16:34:37.466 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls_filter": 0, 2024-09-13T16:34:37.466 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_other": 0, 2024-09-13T16:34:37.466 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_active": 0, 2024-09-13T16:34:37.466 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_send": 0, 2024-09-13T16:34:37.466 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_resend": 0, 2024-09-13T16:34:37.467 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_ping": 0, 2024-09-13T16:34:37.467 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_active": 0, 2024-09-13T16:34:37.467 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_send": 0, 2024-09-13T16:34:37.467 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_resend": 0, 2024-09-13T16:34:37.467 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_active": 0, 2024-09-13T16:34:37.467 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_send": 0, 2024-09-13T16:34:37.467 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_resend": 0, 2024-09-13T16:34:37.467 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_active": 0, 2024-09-13T16:34:37.467 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_send": 0, 2024-09-13T16:34:37.467 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_resend": 0, 2024-09-13T16:34:37.468 INFO:teuthology.orchestra.run.smithi111.stdout: "command_active": 0, 2024-09-13T16:34:37.468 INFO:teuthology.orchestra.run.smithi111.stdout: "command_send": 0, 2024-09-13T16:34:37.468 INFO:teuthology.orchestra.run.smithi111.stdout: "command_resend": 0, 2024-09-13T16:34:37.468 INFO:teuthology.orchestra.run.smithi111.stdout: "map_epoch": 93, 2024-09-13T16:34:37.468 INFO:teuthology.orchestra.run.smithi111.stdout: "map_full": 0, 2024-09-13T16:34:37.468 INFO:teuthology.orchestra.run.smithi111.stdout: "map_inc": 4, 2024-09-13T16:34:37.468 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_sessions": 4, 2024-09-13T16:34:37.468 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_open": 4, 2024-09-13T16:34:37.468 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_close": 0, 2024-09-13T16:34:37.468 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_laggy": 0, 2024-09-13T16:34:37.468 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_wr": 0, 2024-09-13T16:34:37.468 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_rd": 2, 2024-09-13T16:34:37.469 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_del": 0 2024-09-13T16:34:37.469 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.469 INFO:teuthology.orchestra.run.smithi111.stdout: "oft": { 2024-09-13T16:34:37.469 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_objs": 0, 2024-09-13T16:34:37.469 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_kv_pairs": 0, 2024-09-13T16:34:37.469 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_updates": 0, 2024-09-13T16:34:37.469 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_removes": 0 2024-09-13T16:34:37.469 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.469 INFO:teuthology.orchestra.run.smithi111.stdout: "osd-slow-ops": { 2024-09-13T16:34:37.469 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_ops_count": 0 2024-09-13T16:34:37.469 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.470 INFO:teuthology.orchestra.run.smithi111.stdout: "purge_queue": { 2024-09-13T16:34:37.470 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops": 0, 2024-09-13T16:34:37.470 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops_high_water": 0, 2024-09-13T16:34:37.470 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing": 0, 2024-09-13T16:34:37.470 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_high_water": 0, 2024-09-13T16:34:37.470 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed_ops": 0, 2024-09-13T16:34:37.470 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed": 0, 2024-09-13T16:34:37.470 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_item_in_journal": 0 2024-09-13T16:34:37.470 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.470 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T16:34:37.470 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-13T16:34:37.470 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-09-13T16:34:37.470 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-13T16:34:37.470 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 1048, 2024-09-13T16:34:37.471 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 923432, 2024-09-13T16:34:37.471 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:37.471 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 1048, 2024-09-13T16:34:37.471 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-13T16:34:37.471 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-13T16:34:37.471 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 1048, 2024-09-13T16:34:37.471 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 923432, 2024-09-13T16:34:37.471 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-13T16:34:37.471 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.471 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.471 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.471 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:37.471 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.472 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_bytes": { 2024-09-13T16:34:37.472 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-13T16:34:37.472 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-09-13T16:34:37.472 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-13T16:34:37.472 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-13T16:34:37.472 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-13T16:34:37.472 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:37.472 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:37.472 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 922, 2024-09-13T16:34:37.472 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 2105, 2024-09-13T16:34:37.472 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 922, 2024-09-13T16:34:37.472 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 2105, 2024-09-13T16:34:37.472 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-13T16:34:37.472 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.473 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.473 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.473 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:37.473 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.473 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_ops": { 2024-09-13T16:34:37.473 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-13T16:34:37.473 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 1024, 2024-09-13T16:34:37.473 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-13T16:34:37.473 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-13T16:34:37.473 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-13T16:34:37.473 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:37.473 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:37.473 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 922, 2024-09-13T16:34:37.473 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 922, 2024-09-13T16:34:37.474 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 922, 2024-09-13T16:34:37.474 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 922, 2024-09-13T16:34:37.474 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-13T16:34:37.474 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.474 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.474 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.474 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:37.474 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.474 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle": { 2024-09-13T16:34:37.474 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-13T16:34:37.474 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-09-13T16:34:37.474 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-13T16:34:37.474 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-13T16:34:37.474 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-13T16:34:37.475 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:37.475 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:37.475 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-13T16:34:37.475 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-13T16:34:37.475 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 0, 2024-09-13T16:34:37.475 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 0, 2024-09-13T16:34:37.475 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-13T16:34:37.475 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.475 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.475 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.475 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:37.475 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:37.475 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle-0x5566d7ec4780": { 2024-09-13T16:34:37.475 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-13T16:34:37.476 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-09-13T16:34:37.476 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-13T16:34:37.476 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-13T16:34:37.476 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-13T16:34:37.476 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:37.476 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:37.476 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-13T16:34:37.476 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-13T16:34:37.476 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 0, 2024-09-13T16:34:37.476 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 0, 2024-09-13T16:34:37.476 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-13T16:34:37.477 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:37.477 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:37.477 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:37.477 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:37.477 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:37.477 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-13T16:34:38.166 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:38 smithi111 ceph-mon[34600]: pgmap v459: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-13T16:34:38.188 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-13T16:34:38.189 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-13T16:34:38.189 DEBUG:tasks.check_counter:Getting stats from i 2024-09-13T16:34:38.189 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-13T16:34:38.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:38 smithi107 ceph-mon[34139]: pgmap v459: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-13T16:34:38.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:38 smithi016 ceph-mon[31187]: pgmap v459: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-13T16:34:38.386 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T16:34:40.080 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-13T16:34:40.081 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T16:34:40.081 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 152, 2024-09-13T16:34:40.081 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 223, 2024-09-13T16:34:40.081 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 41431, 2024-09-13T16:34:40.081 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 116323, 2024-09-13T16:34:40.081 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 8, 2024-09-13T16:34:40.081 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 3, 2024-09-13T16:34:40.081 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.024863728, 2024-09-13T16:34:40.081 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.008292706, 2024-09-13T16:34:40.082 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.010755942, 2024-09-13T16:34:40.082 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.002655848, 2024-09-13T16:34:40.082 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:40.082 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 221, 2024-09-13T16:34:40.082 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.033171552, 2024-09-13T16:34:40.082 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000150097 2024-09-13T16:34:40.082 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.082 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:40.082 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 320, 2024-09-13T16:34:40.082 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000085527, 2024-09-13T16:34:40.082 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000267 2024-09-13T16:34:40.082 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.082 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-13T16:34:40.083 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 0 2024-09-13T16:34:40.083 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.083 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T16:34:40.083 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 178, 2024-09-13T16:34:40.083 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 164, 2024-09-13T16:34:40.083 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 774919, 2024-09-13T16:34:40.083 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 110913, 2024-09-13T16:34:40.083 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 11, 2024-09-13T16:34:40.083 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 7, 2024-09-13T16:34:40.084 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.024975664, 2024-09-13T16:34:40.084 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.008041431, 2024-09-13T16:34:40.084 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.022598021, 2024-09-13T16:34:40.084 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000382376, 2024-09-13T16:34:40.084 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:40.084 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 163, 2024-09-13T16:34:40.084 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.016855053, 2024-09-13T16:34:40.084 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000103405 2024-09-13T16:34:40.085 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.085 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:40.085 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 29, 2024-09-13T16:34:40.085 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000004671, 2024-09-13T16:34:40.085 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000161 2024-09-13T16:34:40.085 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.085 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 770048, 2024-09-13T16:34:40.085 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 65056 2024-09-13T16:34:40.086 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.086 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T16:34:40.086 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 18, 2024-09-13T16:34:40.086 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 93, 2024-09-13T16:34:40.086 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 6567, 2024-09-13T16:34:40.086 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 217223, 2024-09-13T16:34:40.086 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 7, 2024-09-13T16:34:40.086 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 5, 2024-09-13T16:34:40.086 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.009627551, 2024-09-13T16:34:40.086 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.003388478, 2024-09-13T16:34:40.086 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.005016808, 2024-09-13T16:34:40.087 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000220491, 2024-09-13T16:34:40.087 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:40.087 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 92, 2024-09-13T16:34:40.087 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.017640145, 2024-09-13T16:34:40.087 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000191740 2024-09-13T16:34:40.087 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.087 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:40.087 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 7, 2024-09-13T16:34:40.087 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000001093, 2024-09-13T16:34:40.087 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000156 2024-09-13T16:34:40.087 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.088 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 4048, 2024-09-13T16:34:40.088 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 207712 2024-09-13T16:34:40.088 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.088 INFO:teuthology.orchestra.run.smithi111.stdout: "cct": { 2024-09-13T16:34:40.088 INFO:teuthology.orchestra.run.smithi111.stdout: "total_workers": 1, 2024-09-13T16:34:40.088 INFO:teuthology.orchestra.run.smithi111.stdout: "unhealthy_workers": 0 2024-09-13T16:34:40.088 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.088 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-MDSRank": { 2024-09-13T16:34:40.088 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-09-13T16:34:40.088 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-09-13T16:34:40.088 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 56, 2024-09-13T16:34:40.088 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.001793125, 2024-09-13T16:34:40.089 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000032020 2024-09-13T16:34:40.089 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:40.089 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.089 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-PurgeQueue": { 2024-09-13T16:34:40.089 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-09-13T16:34:40.089 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-09-13T16:34:40.089 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2, 2024-09-13T16:34:40.089 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000012601, 2024-09-13T16:34:40.089 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000006300 2024-09-13T16:34:40.089 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:40.089 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.089 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": { 2024-09-13T16:34:40.089 INFO:teuthology.orchestra.run.smithi111.stdout: "request": 4, 2024-09-13T16:34:40.090 INFO:teuthology.orchestra.run.smithi111.stdout: "reply": 3, 2024-09-13T16:34:40.090 INFO:teuthology.orchestra.run.smithi111.stdout: "reply_latency": { 2024-09-13T16:34:40.090 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 3, 2024-09-13T16:34:40.090 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.001829378, 2024-09-13T16:34:40.090 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000609792 2024-09-13T16:34:40.090 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.090 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_reply": 0, 2024-09-13T16:34:40.090 INFO:teuthology.orchestra.run.smithi111.stdout: "forward": 1, 2024-09-13T16:34:40.090 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_complete": 0, 2024-09-13T16:34:40.090 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_keys": 0, 2024-09-13T16:34:40.090 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_commit": 11, 2024-09-13T16:34:40.090 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_split": 0, 2024-09-13T16:34:40.091 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_merge": 0, 2024-09-13T16:34:40.091 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes": 12, 2024-09-13T16:34:40.091 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_top": 0, 2024-09-13T16:34:40.091 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_bottom": 12, 2024-09-13T16:34:40.091 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pin_tail": 0, 2024-09-13T16:34:40.091 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pinned": 12, 2024-09-13T16:34:40.091 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_expired": 0, 2024-09-13T16:34:40.091 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_with_caps": 2, 2024-09-13T16:34:40.091 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": 2, 2024-09-13T16:34:40.091 INFO:teuthology.orchestra.run.smithi111.stdout: "subtrees": 2, 2024-09-13T16:34:40.091 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse": 8, 2024-09-13T16:34:40.091 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_hit": 5, 2024-09-13T16:34:40.092 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_forward": 0, 2024-09-13T16:34:40.092 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_discover": 2, 2024-09-13T16:34:40.092 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_dir_fetch": 0, 2024-09-13T16:34:40.092 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_remote_ino": 0, 2024-09-13T16:34:40.092 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_lock": 0, 2024-09-13T16:34:40.092 INFO:teuthology.orchestra.run.smithi111.stdout: "load_cent": 0, 2024-09-13T16:34:40.092 INFO:teuthology.orchestra.run.smithi111.stdout: "q": 0, 2024-09-13T16:34:40.092 INFO:teuthology.orchestra.run.smithi111.stdout: "exported": 0, 2024-09-13T16:34:40.092 INFO:teuthology.orchestra.run.smithi111.stdout: "exported_inodes": 0, 2024-09-13T16:34:40.092 INFO:teuthology.orchestra.run.smithi111.stdout: "imported": 0, 2024-09-13T16:34:40.092 INFO:teuthology.orchestra.run.smithi111.stdout: "imported_inodes": 0, 2024-09-13T16:34:40.092 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_dir_fetch": 0, 2024-09-13T16:34:40.093 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_backtrace_fetch": 0, 2024-09-13T16:34:40.093 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_peer_discover": 0, 2024-09-13T16:34:40.093 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rfiles": 5, 2024-09-13T16:34:40.093 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rbytes": 5026218218, 2024-09-13T16:34:40.093 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rsnaps": 0, 2024-09-13T16:34:40.093 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_fetch": 0, 2024-09-13T16:34:40.093 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_set_tag": 0, 2024-09-13T16:34:40.093 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_repaired": 0, 2024-09-13T16:34:40.093 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_inotable_repaired": 0, 2024-09-13T16:34:40.093 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_inodes": 0, 2024-09-13T16:34:40.093 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_base_inodes": 0, 2024-09-13T16:34:40.093 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dirfrag_rstats": 0, 2024-09-13T16:34:40.093 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_file_inodes": 0, 2024-09-13T16:34:40.094 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_inode_file_caps": 0, 2024-09-13T16:34:40.094 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_revoke": 0, 2024-09-13T16:34:40.094 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_grant": 0, 2024-09-13T16:34:40.094 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_trunc": 0, 2024-09-13T16:34:40.094 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-13T16:34:40.094 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-13T16:34:40.094 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps": 0, 2024-09-13T16:34:40.095 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps_dirty": 0, 2024-09-13T16:34:40.095 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_cap_release": 0, 2024-09-13T16:34:40.095 INFO:teuthology.orchestra.run.smithi111.stdout: "process_request_cap_release": 1 2024-09-13T16:34:40.095 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.095 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_cache": { 2024-09-13T16:34:40.095 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update": 0, 2024-09-13T16:34:40.095 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update_receipt": 0, 2024-09-13T16:34:40.095 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_try_discover": 0, 2024-09-13T16:34:40.095 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_send_discover": 3, 2024-09-13T16:34:40.095 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_handle_discover": 0, 2024-09-13T16:34:40.095 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays": 0, 2024-09-13T16:34:40.095 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_delayed": 0, 2024-09-13T16:34:40.095 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_enqueuing": 0, 2024-09-13T16:34:40.095 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_created": 0, 2024-09-13T16:34:40.096 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_enqueued": 0, 2024-09-13T16:34:40.096 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_reintegrated": 0, 2024-09-13T16:34:40.096 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_migrated": 0, 2024-09-13T16:34:40.096 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_processing": 0, 2024-09-13T16:34:40.096 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_enqueued": 0, 2024-09-13T16:34:40.096 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_prioritized": 0, 2024-09-13T16:34:40.096 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_started": 0, 2024-09-13T16:34:40.096 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_completed": 0, 2024-09-13T16:34:40.096 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_enqueue_scrub": 0, 2024-09-13T16:34:40.096 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_exportdir": 0, 2024-09-13T16:34:40.096 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_flush": 0, 2024-09-13T16:34:40.096 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragmentdir": 0, 2024-09-13T16:34:40.096 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragstats": 0, 2024-09-13T16:34:40.097 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_inodestats": 0 2024-09-13T16:34:40.097 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.097 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_log": { 2024-09-13T16:34:40.097 INFO:teuthology.orchestra.run.smithi111.stdout: "evadd": 6, 2024-09-13T16:34:40.097 INFO:teuthology.orchestra.run.smithi111.stdout: "evex": 0, 2024-09-13T16:34:40.097 INFO:teuthology.orchestra.run.smithi111.stdout: "evtrm": 0, 2024-09-13T16:34:40.097 INFO:teuthology.orchestra.run.smithi111.stdout: "ev": 6, 2024-09-13T16:34:40.097 INFO:teuthology.orchestra.run.smithi111.stdout: "evexg": 0, 2024-09-13T16:34:40.097 INFO:teuthology.orchestra.run.smithi111.stdout: "evexd": 0, 2024-09-13T16:34:40.097 INFO:teuthology.orchestra.run.smithi111.stdout: "segadd": 1, 2024-09-13T16:34:40.097 INFO:teuthology.orchestra.run.smithi111.stdout: "segex": 0, 2024-09-13T16:34:40.097 INFO:teuthology.orchestra.run.smithi111.stdout: "segtrm": 0, 2024-09-13T16:34:40.098 INFO:teuthology.orchestra.run.smithi111.stdout: "seg": 1, 2024-09-13T16:34:40.098 INFO:teuthology.orchestra.run.smithi111.stdout: "segexg": 0, 2024-09-13T16:34:40.098 INFO:teuthology.orchestra.run.smithi111.stdout: "segexd": 0, 2024-09-13T16:34:40.098 INFO:teuthology.orchestra.run.smithi111.stdout: "expos": 4194304, 2024-09-13T16:34:40.098 INFO:teuthology.orchestra.run.smithi111.stdout: "wrpos": 4196404, 2024-09-13T16:34:40.098 INFO:teuthology.orchestra.run.smithi111.stdout: "rdpos": 0, 2024-09-13T16:34:40.098 INFO:teuthology.orchestra.run.smithi111.stdout: "jlat": { 2024-09-13T16:34:40.098 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 6, 2024-09-13T16:34:40.098 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.010177749, 2024-09-13T16:34:40.098 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.001696291 2024-09-13T16:34:40.098 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.098 INFO:teuthology.orchestra.run.smithi111.stdout: "replayed": 0 2024-09-13T16:34:40.099 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.099 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_mem": { 2024-09-13T16:34:40.099 INFO:teuthology.orchestra.run.smithi111.stdout: "ino": 15, 2024-09-13T16:34:40.099 INFO:teuthology.orchestra.run.smithi111.stdout: "ino+": 15, 2024-09-13T16:34:40.099 INFO:teuthology.orchestra.run.smithi111.stdout: "ino-": 0, 2024-09-13T16:34:40.099 INFO:teuthology.orchestra.run.smithi111.stdout: "dir": 13, 2024-09-13T16:34:40.099 INFO:teuthology.orchestra.run.smithi111.stdout: "dir+": 14, 2024-09-13T16:34:40.099 INFO:teuthology.orchestra.run.smithi111.stdout: "dir-": 1, 2024-09-13T16:34:40.099 INFO:teuthology.orchestra.run.smithi111.stdout: "dn": 12, 2024-09-13T16:34:40.100 INFO:teuthology.orchestra.run.smithi111.stdout: "dn+": 12, 2024-09-13T16:34:40.100 INFO:teuthology.orchestra.run.smithi111.stdout: "dn-": 0, 2024-09-13T16:34:40.100 INFO:teuthology.orchestra.run.smithi111.stdout: "cap": 2, 2024-09-13T16:34:40.100 INFO:teuthology.orchestra.run.smithi111.stdout: "cap+": 3, 2024-09-13T16:34:40.100 INFO:teuthology.orchestra.run.smithi111.stdout: "cap-": 1, 2024-09-13T16:34:40.100 INFO:teuthology.orchestra.run.smithi111.stdout: "rss": 49192, 2024-09-13T16:34:40.100 INFO:teuthology.orchestra.run.smithi111.stdout: "heap": 198932 2024-09-13T16:34:40.100 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.100 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_server": { 2024-09-13T16:34:40.100 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_client_request": 9, 2024-09-13T16:34:40.100 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_server_request": 0, 2024-09-13T16:34:40.100 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_request": 4, 2024-09-13T16:34:40.100 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_session": 13, 2024-09-13T16:34:40.100 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_peer_request": 1, 2024-09-13T16:34:40.101 INFO:teuthology.orchestra.run.smithi111.stdout: "req_create_latency": { 2024-09-13T16:34:40.101 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.101 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.101 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.101 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.101 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getattr_latency": { 2024-09-13T16:34:40.101 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 3, 2024-09-13T16:34:40.101 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.001829378, 2024-09-13T16:34:40.101 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000609792 2024-09-13T16:34:40.101 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.101 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getfilelock_latency": { 2024-09-13T16:34:40.102 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.102 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.102 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.102 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.102 INFO:teuthology.orchestra.run.smithi111.stdout: "req_link_latency": { 2024-09-13T16:34:40.102 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.102 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.102 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.102 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.103 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookup_latency": { 2024-09-13T16:34:40.103 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.103 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.103 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.103 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.103 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookuphash_latency": { 2024-09-13T16:34:40.103 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.103 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.103 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.103 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.103 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupino_latency": { 2024-09-13T16:34:40.103 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.104 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.104 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.104 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.104 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupname_latency": { 2024-09-13T16:34:40.104 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.104 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.104 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.104 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.104 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupparent_latency": { 2024-09-13T16:34:40.104 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.104 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.104 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.105 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.105 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupsnap_latency": { 2024-09-13T16:34:40.105 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.105 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.105 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.105 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.105 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lssnap_latency": { 2024-09-13T16:34:40.105 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.105 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.105 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.105 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.106 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mkdir_latency": { 2024-09-13T16:34:40.106 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.106 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.106 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.106 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.107 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mknod_latency": { 2024-09-13T16:34:40.107 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.107 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.107 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.107 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.108 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mksnap_latency": { 2024-09-13T16:34:40.108 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.108 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.108 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.108 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.108 INFO:teuthology.orchestra.run.smithi111.stdout: "req_open_latency": { 2024-09-13T16:34:40.108 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.108 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.108 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.109 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.109 INFO:teuthology.orchestra.run.smithi111.stdout: "req_readdir_latency": { 2024-09-13T16:34:40.109 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.109 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.109 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.109 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.109 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rename_latency": { 2024-09-13T16:34:40.109 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.109 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.109 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.109 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.110 INFO:teuthology.orchestra.run.smithi111.stdout: "req_renamesnap_latency": { 2024-09-13T16:34:40.110 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.110 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.110 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.110 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.110 INFO:teuthology.orchestra.run.smithi111.stdout: "req_snapdiff_latency": { 2024-09-13T16:34:40.110 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.110 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.110 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.110 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.110 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmdir_latency": { 2024-09-13T16:34:40.110 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.110 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.111 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.111 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.111 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmsnap_latency": { 2024-09-13T16:34:40.111 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.111 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.111 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.111 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.111 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmxattr_latency": { 2024-09-13T16:34:40.111 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.111 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.111 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.111 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.111 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setattr_latency": { 2024-09-13T16:34:40.112 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.112 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.112 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.112 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.112 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setdirlayout_latency": { 2024-09-13T16:34:40.112 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.112 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.112 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.112 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.112 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setfilelock_latency": { 2024-09-13T16:34:40.112 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.112 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.112 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.112 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.113 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setlayout_latency": { 2024-09-13T16:34:40.113 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.113 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.113 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.113 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.113 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setxattr_latency": { 2024-09-13T16:34:40.113 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.113 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.113 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.113 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.113 INFO:teuthology.orchestra.run.smithi111.stdout: "req_symlink_latency": { 2024-09-13T16:34:40.113 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.113 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.113 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.114 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.114 INFO:teuthology.orchestra.run.smithi111.stdout: "req_unlink_latency": { 2024-09-13T16:34:40.114 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.114 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.114 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.114 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.114 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_revoke_eviction": 0, 2024-09-13T16:34:40.114 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_acquisition_throttle": 0, 2024-09-13T16:34:40.114 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getvxattr_latency": { 2024-09-13T16:34:40.114 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.114 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.114 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.114 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:40.114 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.115 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_sessions": { 2024-09-13T16:34:40.115 INFO:teuthology.orchestra.run.smithi111.stdout: "session_count": 1, 2024-09-13T16:34:40.115 INFO:teuthology.orchestra.run.smithi111.stdout: "session_add": 3, 2024-09-13T16:34:40.115 INFO:teuthology.orchestra.run.smithi111.stdout: "session_remove": 2, 2024-09-13T16:34:40.115 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_open": 1, 2024-09-13T16:34:40.115 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_stale": 0, 2024-09-13T16:34:40.115 INFO:teuthology.orchestra.run.smithi111.stdout: "total_load": 2, 2024-09-13T16:34:40.115 INFO:teuthology.orchestra.run.smithi111.stdout: "average_load": 2, 2024-09-13T16:34:40.115 INFO:teuthology.orchestra.run.smithi111.stdout: "avg_session_uptime": 211, 2024-09-13T16:34:40.115 INFO:teuthology.orchestra.run.smithi111.stdout: "mdthresh_evicted": 0 2024-09-13T16:34:40.115 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.115 INFO:teuthology.orchestra.run.smithi111.stdout: "mempool": { 2024-09-13T16:34:40.115 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_bytes": 0, 2024-09-13T16:34:40.115 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_items": 0, 2024-09-13T16:34:40.116 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T16:34:40.116 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_items": 0, 2024-09-13T16:34:40.116 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T16:34:40.116 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_items": 0, 2024-09-13T16:34:40.116 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T16:34:40.116 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T16:34:40.116 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T16:34:40.116 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T16:34:40.116 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T16:34:40.117 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_items": 0, 2024-09-13T16:34:40.117 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T16:34:40.117 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T16:34:40.117 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_bytes": 0, 2024-09-13T16:34:40.117 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_items": 0, 2024-09-13T16:34:40.117 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_bytes": 0, 2024-09-13T16:34:40.117 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_items": 0, 2024-09-13T16:34:40.117 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T16:34:40.117 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T16:34:40.117 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T16:34:40.117 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T16:34:40.117 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T16:34:40.117 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_items": 0, 2024-09-13T16:34:40.117 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_bytes": 0, 2024-09-13T16:34:40.118 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_items": 0, 2024-09-13T16:34:40.118 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T16:34:40.118 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T16:34:40.118 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_bytes": 0, 2024-09-13T16:34:40.118 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_items": 0, 2024-09-13T16:34:40.118 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_bytes": 0, 2024-09-13T16:34:40.118 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_items": 0, 2024-09-13T16:34:40.118 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T16:34:40.118 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_items": 0, 2024-09-13T16:34:40.118 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T16:34:40.118 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_items": 0, 2024-09-13T16:34:40.118 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_bytes": 247233, 2024-09-13T16:34:40.118 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_items": 86, 2024-09-13T16:34:40.119 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_bytes": 0, 2024-09-13T16:34:40.119 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_items": 0, 2024-09-13T16:34:40.119 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_bytes": 0, 2024-09-13T16:34:40.119 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_items": 0, 2024-09-13T16:34:40.119 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_bytes": 0, 2024-09-13T16:34:40.119 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_items": 0, 2024-09-13T16:34:40.119 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_bytes": 0, 2024-09-13T16:34:40.119 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_items": 0, 2024-09-13T16:34:40.119 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_bytes": 8856, 2024-09-13T16:34:40.119 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_items": 166, 2024-09-13T16:34:40.119 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T16:34:40.119 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_items": 0, 2024-09-13T16:34:40.119 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_bytes": 0, 2024-09-13T16:34:40.120 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_items": 0, 2024-09-13T16:34:40.120 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_bytes": 59328, 2024-09-13T16:34:40.120 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_items": 235, 2024-09-13T16:34:40.120 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_bytes": 0, 2024-09-13T16:34:40.120 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_items": 0, 2024-09-13T16:34:40.120 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_bytes": 0, 2024-09-13T16:34:40.120 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_items": 0 2024-09-13T16:34:40.120 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.120 INFO:teuthology.orchestra.run.smithi111.stdout: "objecter": { 2024-09-13T16:34:40.120 INFO:teuthology.orchestra.run.smithi111.stdout: "op_active": 0, 2024-09-13T16:34:40.120 INFO:teuthology.orchestra.run.smithi111.stdout: "op_laggy": 0, 2024-09-13T16:34:40.120 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send": 44, 2024-09-13T16:34:40.120 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send_bytes": 12647, 2024-09-13T16:34:40.120 INFO:teuthology.orchestra.run.smithi111.stdout: "op_resend": 27, 2024-09-13T16:34:40.121 INFO:teuthology.orchestra.run.smithi111.stdout: "op_reply": 44, 2024-09-13T16:34:40.121 INFO:teuthology.orchestra.run.smithi111.stdout: "op_latency": { 2024-09-13T16:34:40.121 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 44, 2024-09-13T16:34:40.121 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.215996148, 2024-09-13T16:34:40.121 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.004909003 2024-09-13T16:34:40.121 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.121 INFO:teuthology.orchestra.run.smithi111.stdout: "op_inflight": 0, 2024-09-13T16:34:40.121 INFO:teuthology.orchestra.run.smithi111.stdout: "oplen_avg": { 2024-09-13T16:34:40.121 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 44, 2024-09-13T16:34:40.121 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 67 2024-09-13T16:34:40.121 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.121 INFO:teuthology.orchestra.run.smithi111.stdout: "op": 44, 2024-09-13T16:34:40.121 INFO:teuthology.orchestra.run.smithi111.stdout: "op_r": 0, 2024-09-13T16:34:40.121 INFO:teuthology.orchestra.run.smithi111.stdout: "op_w": 44, 2024-09-13T16:34:40.122 INFO:teuthology.orchestra.run.smithi111.stdout: "op_rmw": 0, 2024-09-13T16:34:40.122 INFO:teuthology.orchestra.run.smithi111.stdout: "op_pg": 0, 2024-09-13T16:34:40.122 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_stat": 0, 2024-09-13T16:34:40.122 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_create": 10, 2024-09-13T16:34:40.122 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_read": 0, 2024-09-13T16:34:40.122 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_write": 6, 2024-09-13T16:34:40.122 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writefull": 8, 2024-09-13T16:34:40.122 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writesame": 0, 2024-09-13T16:34:40.122 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_append": 0, 2024-09-13T16:34:40.122 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_zero": 0, 2024-09-13T16:34:40.122 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_truncate": 0, 2024-09-13T16:34:40.122 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_delete": 6, 2024-09-13T16:34:40.122 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_mapext": 0, 2024-09-13T16:34:40.122 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_sparse_read": 0, 2024-09-13T16:34:40.123 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_clonerange": 0, 2024-09-13T16:34:40.123 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_getxattr": 0, 2024-09-13T16:34:40.123 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_setxattr": 20, 2024-09-13T16:34:40.123 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_cmpxattr": 0, 2024-09-13T16:34:40.123 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_rmxattr": 0, 2024-09-13T16:34:40.123 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_resetxattrs": 0, 2024-09-13T16:34:40.123 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_call": 0, 2024-09-13T16:34:40.123 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_watch": 0, 2024-09-13T16:34:40.123 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_notify": 0, 2024-09-13T16:34:40.123 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_src_cmpxattr": 0, 2024-09-13T16:34:40.123 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls": 0, 2024-09-13T16:34:40.123 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls_filter": 0, 2024-09-13T16:34:40.123 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_other": 0, 2024-09-13T16:34:40.123 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_active": 0, 2024-09-13T16:34:40.124 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_send": 0, 2024-09-13T16:34:40.124 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_resend": 0, 2024-09-13T16:34:40.124 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_ping": 0, 2024-09-13T16:34:40.124 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_active": 0, 2024-09-13T16:34:40.124 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_send": 0, 2024-09-13T16:34:40.124 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_resend": 0, 2024-09-13T16:34:40.124 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_active": 0, 2024-09-13T16:34:40.124 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_send": 0, 2024-09-13T16:34:40.124 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_resend": 0, 2024-09-13T16:34:40.124 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_active": 0, 2024-09-13T16:34:40.124 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_send": 0, 2024-09-13T16:34:40.124 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_resend": 0, 2024-09-13T16:34:40.124 INFO:teuthology.orchestra.run.smithi111.stdout: "command_active": 0, 2024-09-13T16:34:40.124 INFO:teuthology.orchestra.run.smithi111.stdout: "command_send": 0, 2024-09-13T16:34:40.125 INFO:teuthology.orchestra.run.smithi111.stdout: "command_resend": 0, 2024-09-13T16:34:40.125 INFO:teuthology.orchestra.run.smithi111.stdout: "map_epoch": 93, 2024-09-13T16:34:40.125 INFO:teuthology.orchestra.run.smithi111.stdout: "map_full": 0, 2024-09-13T16:34:40.125 INFO:teuthology.orchestra.run.smithi111.stdout: "map_inc": 4, 2024-09-13T16:34:40.125 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_sessions": 9, 2024-09-13T16:34:40.125 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_open": 9, 2024-09-13T16:34:40.125 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_close": 0, 2024-09-13T16:34:40.125 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_laggy": 0, 2024-09-13T16:34:40.125 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_wr": 16, 2024-09-13T16:34:40.125 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_rd": 0, 2024-09-13T16:34:40.125 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_del": 1 2024-09-13T16:34:40.125 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.125 INFO:teuthology.orchestra.run.smithi111.stdout: "oft": { 2024-09-13T16:34:40.125 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_objs": 1, 2024-09-13T16:34:40.125 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_kv_pairs": 0, 2024-09-13T16:34:40.126 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_updates": 0, 2024-09-13T16:34:40.126 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_removes": 0 2024-09-13T16:34:40.126 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.126 INFO:teuthology.orchestra.run.smithi111.stdout: "osd-slow-ops": { 2024-09-13T16:34:40.126 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_ops_count": 0 2024-09-13T16:34:40.126 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.126 INFO:teuthology.orchestra.run.smithi111.stdout: "purge_queue": { 2024-09-13T16:34:40.126 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops": 0, 2024-09-13T16:34:40.126 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops_high_water": 0, 2024-09-13T16:34:40.126 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing": 0, 2024-09-13T16:34:40.126 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_high_water": 0, 2024-09-13T16:34:40.127 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed_ops": 0, 2024-09-13T16:34:40.127 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed": 0, 2024-09-13T16:34:40.127 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_item_in_journal": 0 2024-09-13T16:34:40.127 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.127 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T16:34:40.127 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-13T16:34:40.127 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-09-13T16:34:40.127 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-13T16:34:40.127 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 344, 2024-09-13T16:34:40.127 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 780959, 2024-09-13T16:34:40.127 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:40.128 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 344, 2024-09-13T16:34:40.128 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-13T16:34:40.128 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-13T16:34:40.128 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 344, 2024-09-13T16:34:40.128 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 780959, 2024-09-13T16:34:40.128 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-13T16:34:40.128 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.128 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.128 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.128 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:40.128 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.128 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_bytes": { 2024-09-13T16:34:40.128 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-13T16:34:40.128 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-09-13T16:34:40.129 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-13T16:34:40.129 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-13T16:34:40.129 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-13T16:34:40.129 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:40.129 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:40.129 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 44, 2024-09-13T16:34:40.129 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 12647, 2024-09-13T16:34:40.129 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 44, 2024-09-13T16:34:40.129 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 12647, 2024-09-13T16:34:40.129 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-13T16:34:40.129 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.129 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.129 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.129 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:40.130 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.130 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_ops": { 2024-09-13T16:34:40.130 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-13T16:34:40.130 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 1024, 2024-09-13T16:34:40.130 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-13T16:34:40.130 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-13T16:34:40.130 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-13T16:34:40.130 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:40.130 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:40.130 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 44, 2024-09-13T16:34:40.130 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 44, 2024-09-13T16:34:40.130 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 44, 2024-09-13T16:34:40.130 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 44, 2024-09-13T16:34:40.130 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-13T16:34:40.131 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.131 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.131 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.131 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:40.131 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.131 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle": { 2024-09-13T16:34:40.131 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-13T16:34:40.131 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-09-13T16:34:40.131 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-13T16:34:40.131 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-13T16:34:40.131 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-13T16:34:40.131 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:40.131 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:40.131 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-13T16:34:40.131 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-13T16:34:40.132 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 0, 2024-09-13T16:34:40.132 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 0, 2024-09-13T16:34:40.132 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-13T16:34:40.132 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.132 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.132 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.132 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:40.132 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:40.132 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle-0x559af8cc0500": { 2024-09-13T16:34:40.132 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-13T16:34:40.132 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-09-13T16:34:40.132 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-13T16:34:40.132 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 6, 2024-09-13T16:34:40.132 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 2100, 2024-09-13T16:34:40.133 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:40.133 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 6, 2024-09-13T16:34:40.133 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-13T16:34:40.133 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-13T16:34:40.133 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 6, 2024-09-13T16:34:40.133 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 2100, 2024-09-13T16:34:40.133 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-13T16:34:40.133 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:40.133 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:40.133 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:40.133 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:40.133 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:40.133 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-13T16:34:40.134 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:40 smithi111 ceph-mon[34600]: pgmap v460: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-13T16:34:40.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:40 smithi107 ceph-mon[34139]: pgmap v460: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-13T16:34:40.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:40 smithi016 ceph-mon[31187]: pgmap v460: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-13T16:34:40.771 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-13T16:34:40.771 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-13T16:34:40.771 DEBUG:tasks.check_counter:Getting stats from l 2024-09-13T16:34:40.772 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361 shell --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-13T16:34:40.969 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T16:34:42.091 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:42 smithi111 ceph-mon[34600]: pgmap v461: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T16:34:42.352 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:42 smithi107 ceph-mon[34139]: pgmap v461: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T16:34:42.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:42 smithi016 ceph-mon[31187]: pgmap v461: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T16:34:42.627 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-13T16:34:42.628 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T16:34:42.628 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 34, 2024-09-13T16:34:42.628 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 33, 2024-09-13T16:34:42.628 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 10343, 2024-09-13T16:34:42.628 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 31021, 2024-09-13T16:34:42.628 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 11, 2024-09-13T16:34:42.628 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 6, 2024-09-13T16:34:42.628 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.009415848, 2024-09-13T16:34:42.628 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.000631530, 2024-09-13T16:34:42.628 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.006958338, 2024-09-13T16:34:42.628 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000421360, 2024-09-13T16:34:42.629 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:42.629 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 32, 2024-09-13T16:34:42.629 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.041673353, 2024-09-13T16:34:42.629 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.001302292 2024-09-13T16:34:42.629 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.629 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:42.629 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 17, 2024-09-13T16:34:42.629 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000003001, 2024-09-13T16:34:42.629 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000176 2024-09-13T16:34:42.629 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.629 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 3728, 2024-09-13T16:34:42.629 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 1024 2024-09-13T16:34:42.629 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.630 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T16:34:42.630 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 20, 2024-09-13T16:34:42.630 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 104, 2024-09-13T16:34:42.630 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 5131, 2024-09-13T16:34:42.630 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 240061, 2024-09-13T16:34:42.630 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 10, 2024-09-13T16:34:42.630 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 6, 2024-09-13T16:34:42.630 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.011267710, 2024-09-13T16:34:42.630 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.003964869, 2024-09-13T16:34:42.630 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.005628943, 2024-09-13T16:34:42.630 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000623487, 2024-09-13T16:34:42.630 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:42.630 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 100, 2024-09-13T16:34:42.630 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.038343003, 2024-09-13T16:34:42.631 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000383430 2024-09-13T16:34:42.631 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.631 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:42.631 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 17, 2024-09-13T16:34:42.631 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000002340, 2024-09-13T16:34:42.631 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000137 2024-09-13T16:34:42.631 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.631 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 2336, 2024-09-13T16:34:42.631 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 210736 2024-09-13T16:34:42.631 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.631 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T16:34:42.631 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 289, 2024-09-13T16:34:42.631 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 352, 2024-09-13T16:34:42.631 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 793900, 2024-09-13T16:34:42.632 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 180466, 2024-09-13T16:34:42.632 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 6, 2024-09-13T16:34:42.632 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 4, 2024-09-13T16:34:42.632 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.047066803, 2024-09-13T16:34:42.632 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.017054310, 2024-09-13T16:34:42.632 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.043928963, 2024-09-13T16:34:42.632 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.002521890, 2024-09-13T16:34:42.632 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T16:34:42.632 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 353, 2024-09-13T16:34:42.632 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.073569684, 2024-09-13T16:34:42.633 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000208412 2024-09-13T16:34:42.633 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.633 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-13T16:34:42.633 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 299, 2024-09-13T16:34:42.633 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000097361, 2024-09-13T16:34:42.633 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000325 2024-09-13T16:34:42.633 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.633 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 751904, 2024-09-13T16:34:42.633 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 67952 2024-09-13T16:34:42.633 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.633 INFO:teuthology.orchestra.run.smithi111.stdout: "cct": { 2024-09-13T16:34:42.633 INFO:teuthology.orchestra.run.smithi111.stdout: "total_workers": 1, 2024-09-13T16:34:42.633 INFO:teuthology.orchestra.run.smithi111.stdout: "unhealthy_workers": 0 2024-09-13T16:34:42.634 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.634 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-MDSRank": { 2024-09-13T16:34:42.634 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-09-13T16:34:42.634 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-09-13T16:34:42.634 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 51, 2024-09-13T16:34:42.634 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.002793920, 2024-09-13T16:34:42.634 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000054782 2024-09-13T16:34:42.634 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:42.634 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.634 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-PurgeQueue": { 2024-09-13T16:34:42.634 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-09-13T16:34:42.634 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-09-13T16:34:42.634 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2, 2024-09-13T16:34:42.634 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000019400, 2024-09-13T16:34:42.635 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000009700 2024-09-13T16:34:42.635 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:42.635 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.635 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": { 2024-09-13T16:34:42.635 INFO:teuthology.orchestra.run.smithi111.stdout: "request": 4, 2024-09-13T16:34:42.635 INFO:teuthology.orchestra.run.smithi111.stdout: "reply": 3, 2024-09-13T16:34:42.635 INFO:teuthology.orchestra.run.smithi111.stdout: "reply_latency": { 2024-09-13T16:34:42.635 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 3, 2024-09-13T16:34:42.635 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.001712259, 2024-09-13T16:34:42.635 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000570753 2024-09-13T16:34:42.635 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.635 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_reply": 0, 2024-09-13T16:34:42.635 INFO:teuthology.orchestra.run.smithi111.stdout: "forward": 1, 2024-09-13T16:34:42.635 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_complete": 0, 2024-09-13T16:34:42.636 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_keys": 0, 2024-09-13T16:34:42.636 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_commit": 11, 2024-09-13T16:34:42.636 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_split": 0, 2024-09-13T16:34:42.636 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_merge": 0, 2024-09-13T16:34:42.636 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes": 12, 2024-09-13T16:34:42.636 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_top": 0, 2024-09-13T16:34:42.636 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_bottom": 12, 2024-09-13T16:34:42.636 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pin_tail": 0, 2024-09-13T16:34:42.636 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pinned": 11, 2024-09-13T16:34:42.636 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_expired": 0, 2024-09-13T16:34:42.636 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_with_caps": 0, 2024-09-13T16:34:42.636 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": 0, 2024-09-13T16:34:42.636 INFO:teuthology.orchestra.run.smithi111.stdout: "subtrees": 2, 2024-09-13T16:34:42.637 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse": 8, 2024-09-13T16:34:42.637 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_hit": 6, 2024-09-13T16:34:42.637 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_forward": 0, 2024-09-13T16:34:42.637 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_discover": 2, 2024-09-13T16:34:42.637 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_dir_fetch": 0, 2024-09-13T16:34:42.637 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_remote_ino": 0, 2024-09-13T16:34:42.637 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_lock": 0, 2024-09-13T16:34:42.637 INFO:teuthology.orchestra.run.smithi111.stdout: "load_cent": 0, 2024-09-13T16:34:42.637 INFO:teuthology.orchestra.run.smithi111.stdout: "q": 0, 2024-09-13T16:34:42.637 INFO:teuthology.orchestra.run.smithi111.stdout: "exported": 0, 2024-09-13T16:34:42.637 INFO:teuthology.orchestra.run.smithi111.stdout: "exported_inodes": 0, 2024-09-13T16:34:42.637 INFO:teuthology.orchestra.run.smithi111.stdout: "imported": 0, 2024-09-13T16:34:42.637 INFO:teuthology.orchestra.run.smithi111.stdout: "imported_inodes": 0, 2024-09-13T16:34:42.638 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_dir_fetch": 0, 2024-09-13T16:34:42.638 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_backtrace_fetch": 0, 2024-09-13T16:34:42.638 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_peer_discover": 0, 2024-09-13T16:34:42.638 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rfiles": 5, 2024-09-13T16:34:42.638 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rbytes": 5026218218, 2024-09-13T16:34:42.638 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rsnaps": 0, 2024-09-13T16:34:42.638 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_fetch": 0, 2024-09-13T16:34:42.638 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_set_tag": 0, 2024-09-13T16:34:42.638 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_repaired": 0, 2024-09-13T16:34:42.638 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_inotable_repaired": 0, 2024-09-13T16:34:42.638 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_inodes": 0, 2024-09-13T16:34:42.638 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_base_inodes": 0, 2024-09-13T16:34:42.638 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dirfrag_rstats": 0, 2024-09-13T16:34:42.638 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_file_inodes": 0, 2024-09-13T16:34:42.639 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_inode_file_caps": 0, 2024-09-13T16:34:42.639 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_revoke": 0, 2024-09-13T16:34:42.639 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_grant": 0, 2024-09-13T16:34:42.639 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_trunc": 0, 2024-09-13T16:34:42.639 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-13T16:34:42.639 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-13T16:34:42.639 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps": 0, 2024-09-13T16:34:42.639 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps_dirty": 0, 2024-09-13T16:34:42.639 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_cap_release": 0, 2024-09-13T16:34:42.639 INFO:teuthology.orchestra.run.smithi111.stdout: "process_request_cap_release": 0 2024-09-13T16:34:42.639 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.639 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_cache": { 2024-09-13T16:34:42.639 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update": 0, 2024-09-13T16:34:42.639 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update_receipt": 0, 2024-09-13T16:34:42.640 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_try_discover": 0, 2024-09-13T16:34:42.640 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_send_discover": 3, 2024-09-13T16:34:42.640 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_handle_discover": 0, 2024-09-13T16:34:42.640 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays": 0, 2024-09-13T16:34:42.640 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_delayed": 0, 2024-09-13T16:34:42.640 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_enqueuing": 0, 2024-09-13T16:34:42.640 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_created": 0, 2024-09-13T16:34:42.640 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_enqueued": 0, 2024-09-13T16:34:42.640 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_reintegrated": 0, 2024-09-13T16:34:42.640 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_migrated": 0, 2024-09-13T16:34:42.640 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_processing": 0, 2024-09-13T16:34:42.640 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_enqueued": 0, 2024-09-13T16:34:42.640 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_prioritized": 0, 2024-09-13T16:34:42.640 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_started": 0, 2024-09-13T16:34:42.641 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_completed": 0, 2024-09-13T16:34:42.641 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_enqueue_scrub": 0, 2024-09-13T16:34:42.641 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_exportdir": 0, 2024-09-13T16:34:42.641 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_flush": 0, 2024-09-13T16:34:42.641 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragmentdir": 0, 2024-09-13T16:34:42.641 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragstats": 0, 2024-09-13T16:34:42.641 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_inodestats": 0 2024-09-13T16:34:42.641 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.641 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_log": { 2024-09-13T16:34:42.641 INFO:teuthology.orchestra.run.smithi111.stdout: "evadd": 6, 2024-09-13T16:34:42.641 INFO:teuthology.orchestra.run.smithi111.stdout: "evex": 0, 2024-09-13T16:34:42.641 INFO:teuthology.orchestra.run.smithi111.stdout: "evtrm": 0, 2024-09-13T16:34:42.641 INFO:teuthology.orchestra.run.smithi111.stdout: "ev": 6, 2024-09-13T16:34:42.642 INFO:teuthology.orchestra.run.smithi111.stdout: "evexg": 0, 2024-09-13T16:34:42.642 INFO:teuthology.orchestra.run.smithi111.stdout: "evexd": 0, 2024-09-13T16:34:42.642 INFO:teuthology.orchestra.run.smithi111.stdout: "segadd": 1, 2024-09-13T16:34:42.642 INFO:teuthology.orchestra.run.smithi111.stdout: "segex": 0, 2024-09-13T16:34:42.642 INFO:teuthology.orchestra.run.smithi111.stdout: "segtrm": 0, 2024-09-13T16:34:42.642 INFO:teuthology.orchestra.run.smithi111.stdout: "seg": 1, 2024-09-13T16:34:42.642 INFO:teuthology.orchestra.run.smithi111.stdout: "segexg": 0, 2024-09-13T16:34:42.642 INFO:teuthology.orchestra.run.smithi111.stdout: "segexd": 0, 2024-09-13T16:34:42.642 INFO:teuthology.orchestra.run.smithi111.stdout: "expos": 4194304, 2024-09-13T16:34:42.642 INFO:teuthology.orchestra.run.smithi111.stdout: "wrpos": 4196404, 2024-09-13T16:34:42.642 INFO:teuthology.orchestra.run.smithi111.stdout: "rdpos": 0, 2024-09-13T16:34:42.642 INFO:teuthology.orchestra.run.smithi111.stdout: "jlat": { 2024-09-13T16:34:42.642 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 6, 2024-09-13T16:34:42.642 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.011129290, 2024-09-13T16:34:42.642 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.001854881 2024-09-13T16:34:42.643 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.643 INFO:teuthology.orchestra.run.smithi111.stdout: "replayed": 0 2024-09-13T16:34:42.643 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.643 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_mem": { 2024-09-13T16:34:42.643 INFO:teuthology.orchestra.run.smithi111.stdout: "ino": 15, 2024-09-13T16:34:42.643 INFO:teuthology.orchestra.run.smithi111.stdout: "ino+": 15, 2024-09-13T16:34:42.643 INFO:teuthology.orchestra.run.smithi111.stdout: "ino-": 0, 2024-09-13T16:34:42.643 INFO:teuthology.orchestra.run.smithi111.stdout: "dir": 13, 2024-09-13T16:34:42.643 INFO:teuthology.orchestra.run.smithi111.stdout: "dir+": 14, 2024-09-13T16:34:42.643 INFO:teuthology.orchestra.run.smithi111.stdout: "dir-": 1, 2024-09-13T16:34:42.643 INFO:teuthology.orchestra.run.smithi111.stdout: "dn": 12, 2024-09-13T16:34:42.643 INFO:teuthology.orchestra.run.smithi111.stdout: "dn+": 12, 2024-09-13T16:34:42.643 INFO:teuthology.orchestra.run.smithi111.stdout: "dn-": 0, 2024-09-13T16:34:42.643 INFO:teuthology.orchestra.run.smithi111.stdout: "cap": 0, 2024-09-13T16:34:42.643 INFO:teuthology.orchestra.run.smithi111.stdout: "cap+": 3, 2024-09-13T16:34:42.644 INFO:teuthology.orchestra.run.smithi111.stdout: "cap-": 3, 2024-09-13T16:34:42.644 INFO:teuthology.orchestra.run.smithi111.stdout: "rss": 48284, 2024-09-13T16:34:42.644 INFO:teuthology.orchestra.run.smithi111.stdout: "heap": 198932 2024-09-13T16:34:42.644 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.644 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_server": { 2024-09-13T16:34:42.644 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_client_request": 6, 2024-09-13T16:34:42.644 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_server_request": 0, 2024-09-13T16:34:42.644 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_request": 4, 2024-09-13T16:34:42.644 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_session": 12, 2024-09-13T16:34:42.644 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_peer_request": 0, 2024-09-13T16:34:42.644 INFO:teuthology.orchestra.run.smithi111.stdout: "req_create_latency": { 2024-09-13T16:34:42.644 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.644 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.645 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.645 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.645 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getattr_latency": { 2024-09-13T16:34:42.645 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 1, 2024-09-13T16:34:42.645 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000267646, 2024-09-13T16:34:42.645 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000267646 2024-09-13T16:34:42.645 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.645 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getfilelock_latency": { 2024-09-13T16:34:42.645 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.645 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.645 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.645 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.645 INFO:teuthology.orchestra.run.smithi111.stdout: "req_link_latency": { 2024-09-13T16:34:42.645 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.646 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.646 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.646 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.646 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookup_latency": { 2024-09-13T16:34:42.646 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2, 2024-09-13T16:34:42.646 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.001444613, 2024-09-13T16:34:42.646 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000722306 2024-09-13T16:34:42.646 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.646 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookuphash_latency": { 2024-09-13T16:34:42.646 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.646 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.646 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.646 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.647 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupino_latency": { 2024-09-13T16:34:42.647 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.647 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.647 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.647 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.647 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupname_latency": { 2024-09-13T16:34:42.647 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.647 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.647 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.647 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.647 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupparent_latency": { 2024-09-13T16:34:42.647 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.647 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.647 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.648 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.648 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupsnap_latency": { 2024-09-13T16:34:42.648 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.648 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.648 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.648 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.648 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lssnap_latency": { 2024-09-13T16:34:42.648 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.648 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.648 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.648 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.648 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mkdir_latency": { 2024-09-13T16:34:42.648 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.648 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.649 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.649 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.649 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mknod_latency": { 2024-09-13T16:34:42.649 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.649 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.649 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.649 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.650 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mksnap_latency": { 2024-09-13T16:34:42.650 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.650 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.650 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.650 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.650 INFO:teuthology.orchestra.run.smithi111.stdout: "req_open_latency": { 2024-09-13T16:34:42.650 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.650 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.650 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.650 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.650 INFO:teuthology.orchestra.run.smithi111.stdout: "req_readdir_latency": { 2024-09-13T16:34:42.650 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.650 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.650 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.651 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.651 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rename_latency": { 2024-09-13T16:34:42.651 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.651 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.651 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.651 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.651 INFO:teuthology.orchestra.run.smithi111.stdout: "req_renamesnap_latency": { 2024-09-13T16:34:42.651 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.651 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.651 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.651 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.651 INFO:teuthology.orchestra.run.smithi111.stdout: "req_snapdiff_latency": { 2024-09-13T16:34:42.651 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.651 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.652 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.652 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.652 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmdir_latency": { 2024-09-13T16:34:42.652 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.652 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.652 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.652 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.652 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmsnap_latency": { 2024-09-13T16:34:42.652 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.652 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.652 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.652 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.652 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmxattr_latency": { 2024-09-13T16:34:42.653 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.653 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.653 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.653 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.653 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setattr_latency": { 2024-09-13T16:34:42.653 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.653 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.653 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.653 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.653 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setdirlayout_latency": { 2024-09-13T16:34:42.653 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.653 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.653 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.653 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.654 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setfilelock_latency": { 2024-09-13T16:34:42.654 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.654 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.654 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.654 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.654 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setlayout_latency": { 2024-09-13T16:34:42.654 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.654 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.654 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.654 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.654 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setxattr_latency": { 2024-09-13T16:34:42.654 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.654 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.654 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.655 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.655 INFO:teuthology.orchestra.run.smithi111.stdout: "req_symlink_latency": { 2024-09-13T16:34:42.655 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.655 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.655 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.655 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.655 INFO:teuthology.orchestra.run.smithi111.stdout: "req_unlink_latency": { 2024-09-13T16:34:42.655 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.655 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.655 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.655 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.655 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_revoke_eviction": 0, 2024-09-13T16:34:42.655 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_acquisition_throttle": 0, 2024-09-13T16:34:42.656 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getvxattr_latency": { 2024-09-13T16:34:42.656 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.656 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.656 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.656 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:42.656 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.656 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_sessions": { 2024-09-13T16:34:42.656 INFO:teuthology.orchestra.run.smithi111.stdout: "session_count": 1, 2024-09-13T16:34:42.656 INFO:teuthology.orchestra.run.smithi111.stdout: "session_add": 3, 2024-09-13T16:34:42.656 INFO:teuthology.orchestra.run.smithi111.stdout: "session_remove": 2, 2024-09-13T16:34:42.656 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_open": 1, 2024-09-13T16:34:42.656 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_stale": 0, 2024-09-13T16:34:42.656 INFO:teuthology.orchestra.run.smithi111.stdout: "total_load": 2, 2024-09-13T16:34:42.657 INFO:teuthology.orchestra.run.smithi111.stdout: "average_load": 1, 2024-09-13T16:34:42.657 INFO:teuthology.orchestra.run.smithi111.stdout: "avg_session_uptime": 215, 2024-09-13T16:34:42.657 INFO:teuthology.orchestra.run.smithi111.stdout: "mdthresh_evicted": 0 2024-09-13T16:34:42.657 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.657 INFO:teuthology.orchestra.run.smithi111.stdout: "mempool": { 2024-09-13T16:34:42.657 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_bytes": 0, 2024-09-13T16:34:42.657 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_items": 0, 2024-09-13T16:34:42.657 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T16:34:42.657 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_items": 0, 2024-09-13T16:34:42.657 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T16:34:42.657 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_items": 0, 2024-09-13T16:34:42.657 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T16:34:42.657 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T16:34:42.658 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T16:34:42.658 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T16:34:42.658 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T16:34:42.658 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_items": 0, 2024-09-13T16:34:42.658 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T16:34:42.658 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T16:34:42.658 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_bytes": 0, 2024-09-13T16:34:42.658 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_items": 0, 2024-09-13T16:34:42.658 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_bytes": 0, 2024-09-13T16:34:42.658 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_items": 0, 2024-09-13T16:34:42.658 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T16:34:42.658 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T16:34:42.658 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T16:34:42.659 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T16:34:42.659 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T16:34:42.659 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_items": 0, 2024-09-13T16:34:42.659 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_bytes": 0, 2024-09-13T16:34:42.659 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_items": 0, 2024-09-13T16:34:42.659 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T16:34:42.659 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T16:34:42.659 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_bytes": 0, 2024-09-13T16:34:42.659 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_items": 0, 2024-09-13T16:34:42.659 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_bytes": 0, 2024-09-13T16:34:42.659 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_items": 0, 2024-09-13T16:34:42.659 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T16:34:42.659 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_items": 0, 2024-09-13T16:34:42.660 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T16:34:42.660 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_items": 0, 2024-09-13T16:34:42.660 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_bytes": 239159, 2024-09-13T16:34:42.660 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_items": 83, 2024-09-13T16:34:42.660 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_bytes": 88, 2024-09-13T16:34:42.660 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_items": 1, 2024-09-13T16:34:42.660 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_bytes": 0, 2024-09-13T16:34:42.660 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_items": 0, 2024-09-13T16:34:42.660 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_bytes": 0, 2024-09-13T16:34:42.660 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_items": 0, 2024-09-13T16:34:42.660 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_bytes": 0, 2024-09-13T16:34:42.660 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_items": 0, 2024-09-13T16:34:42.661 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_bytes": 8856, 2024-09-13T16:34:42.661 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_items": 166, 2024-09-13T16:34:42.661 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T16:34:42.661 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_items": 0, 2024-09-13T16:34:42.661 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_bytes": 0, 2024-09-13T16:34:42.661 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_items": 0, 2024-09-13T16:34:42.661 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_bytes": 58616, 2024-09-13T16:34:42.661 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_items": 234, 2024-09-13T16:34:42.661 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_bytes": 0, 2024-09-13T16:34:42.661 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_items": 0, 2024-09-13T16:34:42.661 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_bytes": 0, 2024-09-13T16:34:42.661 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_items": 0 2024-09-13T16:34:42.661 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.661 INFO:teuthology.orchestra.run.smithi111.stdout: "objecter": { 2024-09-13T16:34:42.662 INFO:teuthology.orchestra.run.smithi111.stdout: "op_active": 0, 2024-09-13T16:34:42.662 INFO:teuthology.orchestra.run.smithi111.stdout: "op_laggy": 0, 2024-09-13T16:34:42.662 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send": 45, 2024-09-13T16:34:42.662 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send_bytes": 12717, 2024-09-13T16:34:42.662 INFO:teuthology.orchestra.run.smithi111.stdout: "op_resend": 33, 2024-09-13T16:34:42.662 INFO:teuthology.orchestra.run.smithi111.stdout: "op_reply": 45, 2024-09-13T16:34:42.662 INFO:teuthology.orchestra.run.smithi111.stdout: "op_latency": { 2024-09-13T16:34:42.662 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 45, 2024-09-13T16:34:42.662 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 8.278852095, 2024-09-13T16:34:42.662 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.183974491 2024-09-13T16:34:42.662 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.662 INFO:teuthology.orchestra.run.smithi111.stdout: "op_inflight": 0, 2024-09-13T16:34:42.662 INFO:teuthology.orchestra.run.smithi111.stdout: "oplen_avg": { 2024-09-13T16:34:42.662 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 45, 2024-09-13T16:34:42.663 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 69 2024-09-13T16:34:42.663 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.663 INFO:teuthology.orchestra.run.smithi111.stdout: "op": 45, 2024-09-13T16:34:42.663 INFO:teuthology.orchestra.run.smithi111.stdout: "op_r": 0, 2024-09-13T16:34:42.663 INFO:teuthology.orchestra.run.smithi111.stdout: "op_w": 45, 2024-09-13T16:34:42.663 INFO:teuthology.orchestra.run.smithi111.stdout: "op_rmw": 0, 2024-09-13T16:34:42.663 INFO:teuthology.orchestra.run.smithi111.stdout: "op_pg": 0, 2024-09-13T16:34:42.663 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_stat": 0, 2024-09-13T16:34:42.663 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_create": 10, 2024-09-13T16:34:42.663 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_read": 0, 2024-09-13T16:34:42.663 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_write": 6, 2024-09-13T16:34:42.663 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writefull": 8, 2024-09-13T16:34:42.663 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writesame": 0, 2024-09-13T16:34:42.663 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_append": 0, 2024-09-13T16:34:42.664 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_zero": 0, 2024-09-13T16:34:42.664 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_truncate": 0, 2024-09-13T16:34:42.664 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_delete": 6, 2024-09-13T16:34:42.664 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_mapext": 0, 2024-09-13T16:34:42.664 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_sparse_read": 0, 2024-09-13T16:34:42.664 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_clonerange": 0, 2024-09-13T16:34:42.664 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_getxattr": 0, 2024-09-13T16:34:42.664 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_setxattr": 20, 2024-09-13T16:34:42.664 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_cmpxattr": 0, 2024-09-13T16:34:42.664 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_rmxattr": 0, 2024-09-13T16:34:42.664 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_resetxattrs": 0, 2024-09-13T16:34:42.664 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_call": 0, 2024-09-13T16:34:42.664 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_watch": 0, 2024-09-13T16:34:42.664 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_notify": 0, 2024-09-13T16:34:42.665 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_src_cmpxattr": 0, 2024-09-13T16:34:42.665 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls": 0, 2024-09-13T16:34:42.665 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls_filter": 0, 2024-09-13T16:34:42.665 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_other": 0, 2024-09-13T16:34:42.665 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_active": 0, 2024-09-13T16:34:42.665 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_send": 0, 2024-09-13T16:34:42.665 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_resend": 0, 2024-09-13T16:34:42.665 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_ping": 0, 2024-09-13T16:34:42.665 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_active": 0, 2024-09-13T16:34:42.665 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_send": 0, 2024-09-13T16:34:42.665 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_resend": 0, 2024-09-13T16:34:42.665 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_active": 0, 2024-09-13T16:34:42.665 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_send": 0, 2024-09-13T16:34:42.666 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_resend": 0, 2024-09-13T16:34:42.666 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_active": 0, 2024-09-13T16:34:42.666 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_send": 0, 2024-09-13T16:34:42.666 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_resend": 0, 2024-09-13T16:34:42.666 INFO:teuthology.orchestra.run.smithi111.stdout: "command_active": 0, 2024-09-13T16:34:42.666 INFO:teuthology.orchestra.run.smithi111.stdout: "command_send": 0, 2024-09-13T16:34:42.666 INFO:teuthology.orchestra.run.smithi111.stdout: "command_resend": 0, 2024-09-13T16:34:42.666 INFO:teuthology.orchestra.run.smithi111.stdout: "map_epoch": 93, 2024-09-13T16:34:42.666 INFO:teuthology.orchestra.run.smithi111.stdout: "map_full": 0, 2024-09-13T16:34:42.666 INFO:teuthology.orchestra.run.smithi111.stdout: "map_inc": 3, 2024-09-13T16:34:42.666 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_sessions": 9, 2024-09-13T16:34:42.666 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_open": 9, 2024-09-13T16:34:42.666 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_close": 0, 2024-09-13T16:34:42.666 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_laggy": 0, 2024-09-13T16:34:42.667 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_wr": 17, 2024-09-13T16:34:42.667 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_rd": 0, 2024-09-13T16:34:42.667 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_del": 2 2024-09-13T16:34:42.667 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.667 INFO:teuthology.orchestra.run.smithi111.stdout: "oft": { 2024-09-13T16:34:42.667 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_objs": 1, 2024-09-13T16:34:42.667 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_kv_pairs": 0, 2024-09-13T16:34:42.667 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_updates": 0, 2024-09-13T16:34:42.667 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_removes": 0 2024-09-13T16:34:42.667 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.667 INFO:teuthology.orchestra.run.smithi111.stdout: "osd-slow-ops": { 2024-09-13T16:34:42.667 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_ops_count": 0 2024-09-13T16:34:42.667 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.668 INFO:teuthology.orchestra.run.smithi111.stdout: "purge_queue": { 2024-09-13T16:34:42.668 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops": 0, 2024-09-13T16:34:42.668 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops_high_water": 0, 2024-09-13T16:34:42.668 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing": 0, 2024-09-13T16:34:42.668 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_high_water": 0, 2024-09-13T16:34:42.668 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed_ops": 0, 2024-09-13T16:34:42.668 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed": 0, 2024-09-13T16:34:42.668 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_item_in_journal": 0 2024-09-13T16:34:42.668 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.668 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T16:34:42.668 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-13T16:34:42.668 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-09-13T16:34:42.668 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-13T16:34:42.669 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 339, 2024-09-13T16:34:42.669 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 767581, 2024-09-13T16:34:42.669 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:42.669 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 339, 2024-09-13T16:34:42.669 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-13T16:34:42.669 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-13T16:34:42.669 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 339, 2024-09-13T16:34:42.669 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 767581, 2024-09-13T16:34:42.669 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-13T16:34:42.669 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.669 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.669 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.669 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:42.669 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.669 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_bytes": { 2024-09-13T16:34:42.670 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-13T16:34:42.670 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-09-13T16:34:42.670 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-13T16:34:42.670 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-13T16:34:42.670 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-13T16:34:42.670 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:42.670 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:42.670 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 45, 2024-09-13T16:34:42.670 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 12717, 2024-09-13T16:34:42.670 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 45, 2024-09-13T16:34:42.670 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 12717, 2024-09-13T16:34:42.670 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-13T16:34:42.670 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.670 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.670 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.671 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:42.671 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.671 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_ops": { 2024-09-13T16:34:42.671 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-13T16:34:42.671 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 1024, 2024-09-13T16:34:42.671 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-13T16:34:42.671 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-13T16:34:42.671 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-13T16:34:42.671 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:42.671 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:42.671 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 45, 2024-09-13T16:34:42.671 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 45, 2024-09-13T16:34:42.671 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 45, 2024-09-13T16:34:42.671 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 45, 2024-09-13T16:34:42.672 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-13T16:34:42.672 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.672 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.672 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.672 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:42.672 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.672 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle": { 2024-09-13T16:34:42.672 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-13T16:34:42.672 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-09-13T16:34:42.672 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-13T16:34:42.672 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-13T16:34:42.672 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-13T16:34:42.672 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:42.672 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-13T16:34:42.673 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-13T16:34:42.673 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-13T16:34:42.673 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 0, 2024-09-13T16:34:42.673 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 0, 2024-09-13T16:34:42.673 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-13T16:34:42.673 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.673 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.673 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.673 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:42.673 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-13T16:34:42.673 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle-0x5622cf678500": { 2024-09-13T16:34:42.673 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-13T16:34:42.673 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-09-13T16:34:42.673 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-13T16:34:42.674 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 6, 2024-09-13T16:34:42.674 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 2100, 2024-09-13T16:34:42.674 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-13T16:34:42.674 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 6, 2024-09-13T16:34:42.674 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-13T16:34:42.674 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-13T16:34:42.674 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 6, 2024-09-13T16:34:42.674 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 2100, 2024-09-13T16:34:42.674 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-13T16:34:42.674 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-13T16:34:42.674 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-13T16:34:42.674 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-13T16:34:42.674 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:42.674 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-13T16:34:42.674 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-13T16:34:43.421 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-13T16:34:43.421 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-13T16:34:43.422 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-13T16:34:43.431 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-13T16:34:43.432 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-13T16:34:43.432 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:34:43.455 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-13T16:34:43.455 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:34:43.514 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:34:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.667+0000 7fc817501640 1 -- 172.21.15.16:0/1969551156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8100ff920 msgr2=0x7fc8100ffdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:43.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.667+0000 7fc817501640 1 --2- 172.21.15.16:0/1969551156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8100ff920 0x7fc8100ffdd0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc808009ca0 tx=0x7fc80802a8c0 comp rx=0 tx=0).stop 2024-09-13T16:34:43.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.667+0000 7fc817501640 1 -- 172.21.15.16:0/1969551156 shutdown_connections 2024-09-13T16:34:43.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.667+0000 7fc817501640 1 --2- 172.21.15.16:0/1969551156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8100ff920 0x7fc8100ffdd0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:43.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.667+0000 7fc817501640 1 --2- 172.21.15.16:0/1969551156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8100fef80 0x7fc8100ff3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:43.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.667+0000 7fc817501640 1 --2- 172.21.15.16:0/1969551156 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8100fe640 0x7fc8100fea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:43.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.667+0000 7fc817501640 1 -- 172.21.15.16:0/1969551156 >> 172.21.15.16:0/1969551156 conn(0x7fc8100fa430 msgr2=0x7fc8100fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:43.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.668+0000 7fc817501640 1 -- 172.21.15.16:0/1969551156 shutdown_connections 2024-09-13T16:34:43.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.668+0000 7fc817501640 1 -- 172.21.15.16:0/1969551156 wait complete. 2024-09-13T16:34:43.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.668+0000 7fc817501640 1 Processor -- start 2024-09-13T16:34:43.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.668+0000 7fc817501640 1 -- start start 2024-09-13T16:34:43.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.668+0000 7fc817501640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8100fe640 0x7fc810197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.668+0000 7fc817501640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8100fef80 0x7fc810198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.668+0000 7fc817501640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8100ff920 0x7fc81019f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.668+0000 7fc817501640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81019f8c0 con 0x7fc8100fef80 2024-09-13T16:34:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.668+0000 7fc817501640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81019fa30 con 0x7fc8100fe640 2024-09-13T16:34:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.668+0000 7fc817501640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81019fbf0 con 0x7fc8100ff920 2024-09-13T16:34:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.669+0000 7fc814a9c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8100fef80 0x7fc810198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.669+0000 7fc814a9c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8100fef80 0x7fc810198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42132/0 (socket says 172.21.15.16:42132) 2024-09-13T16:34:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.669+0000 7fc814a9c640 1 -- 172.21.15.16:0/495007092 learned_addr learned my addr 172.21.15.16:0/495007092 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:34:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.669+0000 7fc81529d640 1 --2- 172.21.15.16:0/495007092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8100fe640 0x7fc810197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:43.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.669+0000 7fc815a9e640 1 --2- 172.21.15.16:0/495007092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8100ff920 0x7fc81019f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:43.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.669+0000 7fc814a9c640 1 -- 172.21.15.16:0/495007092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8100ff920 msgr2=0x7fc81019f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:43.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.669+0000 7fc814a9c640 1 --2- 172.21.15.16:0/495007092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8100ff920 0x7fc81019f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:43.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.669+0000 7fc814a9c640 1 -- 172.21.15.16:0/495007092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8100fe640 msgr2=0x7fc810197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:43.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.669+0000 7fc814a9c640 1 --2- 172.21.15.16:0/495007092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8100fe640 0x7fc810197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:43.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.669+0000 7fc814a9c640 1 -- 172.21.15.16:0/495007092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc808009900 con 0x7fc8100fef80 2024-09-13T16:34:43.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.669+0000 7fc814a9c640 1 --2- 172.21.15.16:0/495007092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8100fef80 0x7fc810198310 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc80000d950 tx=0x7fc80000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:43.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.669+0000 7fc8067fc640 1 -- 172.21.15.16:0/495007092 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc800009c30 con 0x7fc8100fef80 2024-09-13T16:34:43.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.669+0000 7fc8067fc640 1 -- 172.21.15.16:0/495007092 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc80000de80 con 0x7fc8100fef80 2024-09-13T16:34:43.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.669+0000 7fc8067fc640 1 -- 172.21.15.16:0/495007092 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80000f620 con 0x7fc8100fef80 2024-09-13T16:34:43.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.669+0000 7fc817501640 1 -- 172.21.15.16:0/495007092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc81019fed0 con 0x7fc8100fef80 2024-09-13T16:34:43.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.669+0000 7fc817501640 1 -- 172.21.15.16:0/495007092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8101a0300 con 0x7fc8100fef80 2024-09-13T16:34:43.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.670+0000 7fc817501640 1 -- 172.21.15.16:0/495007092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc810109570 con 0x7fc8100fef80 2024-09-13T16:34:43.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.671+0000 7fc8067fc640 1 -- 172.21.15.16:0/495007092 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fc800010460 con 0x7fc8100fef80 2024-09-13T16:34:43.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.672+0000 7fc8067fc640 1 --2- 172.21.15.16:0/495007092 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc7fc0a1090 0x7fc7fc0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:43.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.672+0000 7fc81529d640 1 --2- 172.21.15.16:0/495007092 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc7fc0a1090 0x7fc7fc0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:43.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.673+0000 7fc8067fc640 1 -- 172.21.15.16:0/495007092 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc8000cc8a0 con 0x7fc8100fef80 2024-09-13T16:34:43.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.673+0000 7fc81529d640 1 --2- 172.21.15.16:0/495007092 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc7fc0a1090 0x7fc7fc0a3550 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00abe0 tx=0x7fc80c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:43.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.677+0000 7fc8067fc640 1 -- 172.21.15.16:0/495007092 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc800095c80 con 0x7fc8100fef80 2024-09-13T16:34:43.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.852+0000 7fc817501640 1 -- 172.21.15.16:0/495007092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc81002c480 con 0x7fc8100fef80 2024-09-13T16:34:43.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.854+0000 7fc8067fc640 1 -- 172.21.15.16:0/495007092 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24593 (secure 0 0 0) 0x7fc8000992a0 con 0x7fc8100fef80 2024-09-13T16:34:43.856 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:34:43.856 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":93,"fsid":"0d241f6a-71ec-11ef-bceb-c7b262605968","created":"2024-09-13T16:21:17.971738+0000","modified":"2024-09-13T16:30:05.681521+0000","last_up_change":"2024-09-13T16:27:54.673413+0000","last_in_change":"2024-09-13T16:27:35.718390+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:24:22.981019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:28:37.746027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:28:38.602731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:28:41.633322+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e11ef8f-08d6-4e74-afd6-f49f4bd40032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6803","nonce":2079394186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6805","nonce":2079394186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6809","nonce":2079394186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6807","nonce":2079394186}]},"public_addr":"172.21.15.16:6803/2079394186","cluster_addr":"172.21.15.16:6805/2079394186","heartbeat_back_addr":"172.21.15.16:6809/2079394186","heartbeat_front_addr":"172.21.15.16:6807/2079394186","state":["exists","up"]},{"osd":1,"uuid":"f2cd3694-9c9e-4da7-976e-f80a8df120f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6801","nonce":4021360958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6803","nonce":4021360958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6807","nonce":4021360958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6805","nonce":4021360958}]},"public_addr":"172.21.15.107:6801/4021360958","cluster_addr":"172.21.15.107:6803/4021360958","heartbeat_back_addr":"172.21.15.107:6807/4021360958","heartbeat_front_addr":"172.21.15.107:6805/4021360958","state":["exists","up"]},{"osd":2,"uuid":"a5bbf2eb-6599-44d5-ad96-68bb89ee19f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6801","nonce":3227179825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6803","nonce":3227179825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6807","nonce":3227179825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6805","nonce":3227179825}]},"public_addr":"172.21.15.111:6801/3227179825","cluster_addr":"172.21.15.111:6803/3227179825","heartbeat_back_addr":"172.21.15.111:6807/3227179825","heartbeat_front_addr":"172.21.15.111:6805/3227179825","state":["exists","up"]},{"osd":3,"uuid":"d6295ffe-11bc-4964-a250-9fecca870b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6811","nonce":2390375937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6813","nonce":2390375937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6817","nonce":2390375937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6815","nonce":2390375937}]},"public_addr":"172.21.15.16:6811/2390375937","cluster_addr":"172.21.15.16:6813/2390375937","heartbeat_back_addr":"172.21.15.16:6817/2390375937","heartbeat_front_addr":"172.21.15.16:6815/2390375937","state":["exists","up"]},{"osd":4,"uuid":"0a8446e0-fc86-4646-beb6-a7b9d433093d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6809","nonce":1936582048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6811","nonce":1936582048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6815","nonce":1936582048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6813","nonce":1936582048}]},"public_addr":"172.21.15.107:6809/1936582048","cluster_addr":"172.21.15.107:6811/1936582048","heartbeat_back_addr":"172.21.15.107:6815/1936582048","heartbeat_front_addr":"172.21.15.107:6813/1936582048","state":["exists","up"]},{"osd":5,"uuid":"2c97a446-357e-4975-9e51-9c95fe84a663","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6809","nonce":483858448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6811","nonce":483858448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6815","nonce":483858448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6813","nonce":483858448}]},"public_addr":"172.21.15.111:6809/483858448","cluster_addr":"172.21.15.111:6811/483858448","heartbeat_back_addr":"172.21.15.111:6815/483858448","heartbeat_front_addr":"172.21.15.111:6813/483858448","state":["exists","up"]},{"osd":6,"uuid":"890239e6-e6fb-4f9c-b973-05cc66d199bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6819","nonce":1330214141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6821","nonce":1330214141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6825","nonce":1330214141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6823","nonce":1330214141}]},"public_addr":"172.21.15.16:6819/1330214141","cluster_addr":"172.21.15.16:6821/1330214141","heartbeat_back_addr":"172.21.15.16:6825/1330214141","heartbeat_front_addr":"172.21.15.16:6823/1330214141","state":["exists","up"]},{"osd":7,"uuid":"ee826a9c-f441-4903-9b1b-66b627f7b712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6817","nonce":1133876754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6819","nonce":1133876754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6823","nonce":1133876754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6821","nonce":1133876754}]},"public_addr":"172.21.15.107:6817/1133876754","cluster_addr":"172.21.15.107:6819/1133876754","heartbeat_back_addr":"172.21.15.107:6823/1133876754","heartbeat_front_addr":"172.21.15.107:6821/1133876754","state":["exists","up"]},{"osd":8,"uuid":"e19773f2-dec8-44e0-9d33-0252e07d6415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6817","nonce":837548190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6819","nonce":837548190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6823","nonce":837548190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6821","nonce":837548190}]},"public_addr":"172.21.15.111:6817/837548190","cluster_addr":"172.21.15.111:6819/837548190","heartbeat_back_addr":"172.21.15.111:6823/837548190","heartbeat_front_addr":"172.21.15.111:6821/837548190","state":["exists","up"]},{"osd":9,"uuid":"bae4d6db-8b90-4d02-8e83-7d8f2a6f8fdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6827","nonce":4175383686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6829","nonce":4175383686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6833","nonce":4175383686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6831","nonce":4175383686}]},"public_addr":"172.21.15.16:6827/4175383686","cluster_addr":"172.21.15.16:6829/4175383686","heartbeat_back_addr":"172.21.15.16:6833/4175383686","heartbeat_front_addr":"172.21.15.16:6831/4175383686","state":["exists","up"]},{"osd":10,"uuid":"fd0e7740-76c8-4c99-a349-7077b4905143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6825","nonce":2659550426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6827","nonce":2659550426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6831","nonce":2659550426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6829","nonce":2659550426}]},"public_addr":"172.21.15.107:6825/2659550426","cluster_addr":"172.21.15.107:6827/2659550426","heartbeat_back_addr":"172.21.15.107:6831/2659550426","heartbeat_front_addr":"172.21.15.107:6829/2659550426","state":["exists","up"]},{"osd":11,"uuid":"94a1751d-b00c-4e23-815d-3a8bc3dea146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6825","nonce":1633548683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6827","nonce":1633548683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6831","nonce":1633548683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6829","nonce":1633548683}]},"public_addr":"172.21.15.111:6825/1633548683","cluster_addr":"172.21.15.111:6827/1633548683","heartbeat_back_addr":"172.21.15.111:6831/1633548683","heartbeat_front_addr":"172.21.15.111:6829/1633548683","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:36.359852+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:58.014202+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:19.753672+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:43.404817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:06.001677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:29.361587+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:52.554847+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:15.957553+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:39.581093+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:04.911389+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:28.971280+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:53.473561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/2098378667":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/3536229016":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6801/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/2684687547":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6800/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/368410612":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/12158160":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6800/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6801/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/3754423379":"2024-09-14T16:21:40.126459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:34:43.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.856+0000 7fc817501640 1 -- 172.21.15.16:0/495007092 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc7fc0a1090 msgr2=0x7fc7fc0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:43.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.856+0000 7fc817501640 1 --2- 172.21.15.16:0/495007092 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc7fc0a1090 0x7fc7fc0a3550 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00abe0 tx=0x7fc80c00a2d0 comp rx=0 tx=0).stop 2024-09-13T16:34:43.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.857+0000 7fc817501640 1 -- 172.21.15.16:0/495007092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8100fef80 msgr2=0x7fc810198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:43.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.857+0000 7fc817501640 1 --2- 172.21.15.16:0/495007092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8100fef80 0x7fc810198310 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc80000d950 tx=0x7fc80000dd50 comp rx=0 tx=0).stop 2024-09-13T16:34:43.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.857+0000 7fc817501640 1 -- 172.21.15.16:0/495007092 shutdown_connections 2024-09-13T16:34:43.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.857+0000 7fc817501640 1 --2- 172.21.15.16:0/495007092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8100ff920 0x7fc81019f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:43.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.857+0000 7fc817501640 1 --2- 172.21.15.16:0/495007092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8100fef80 0x7fc810198310 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:43.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.857+0000 7fc817501640 1 --2- 172.21.15.16:0/495007092 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fc7fc0a1090 0x7fc7fc0a3550 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:43.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.857+0000 7fc817501640 1 --2- 172.21.15.16:0/495007092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8100fe640 0x7fc810197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:43.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.857+0000 7fc817501640 1 -- 172.21.15.16:0/495007092 >> 172.21.15.16:0/495007092 conn(0x7fc8100fa430 msgr2=0x7fc8100fbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:43.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.857+0000 7fc817501640 1 -- 172.21.15.16:0/495007092 shutdown_connections 2024-09-13T16:34:43.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.857+0000 7fc817501640 1 -- 172.21.15.16:0/495007092 wait complete. 2024-09-13T16:34:43.869 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:24:22.981019+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T16:28:37.746027+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T16:28:38.602731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T16:28:41.633322+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T16:34:43.870 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:34:43.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.975+0000 7faa3afb5640 1 -- 172.21.15.16:0/3917510018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa34100f60 msgr2=0x7faa341013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:43.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.975+0000 7faa3afb5640 1 --2- 172.21.15.16:0/3917510018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa34100f60 0x7faa341013e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7faa20009900 tx=0x7faa2002a410 comp rx=0 tx=0).stop 2024-09-13T16:34:43.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.975+0000 7faa3afb5640 1 -- 172.21.15.16:0/3917510018 shutdown_connections 2024-09-13T16:34:43.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.975+0000 7faa3afb5640 1 --2- 172.21.15.16:0/3917510018 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa34101920 0x7faa341081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:43.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.975+0000 7faa3afb5640 1 --2- 172.21.15.16:0/3917510018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa34100f60 0x7faa341013e0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:43.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.975+0000 7faa3afb5640 1 --2- 172.21.15.16:0/3917510018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa340ffd70 0x7faa34100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:43.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.975+0000 7faa3afb5640 1 -- 172.21.15.16:0/3917510018 >> 172.21.15.16:0/3917510018 conn(0x7faa340fb4d0 msgr2=0x7faa340fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:43.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.975+0000 7faa3afb5640 1 -- 172.21.15.16:0/3917510018 shutdown_connections 2024-09-13T16:34:43.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.975+0000 7faa3afb5640 1 -- 172.21.15.16:0/3917510018 wait complete. 2024-09-13T16:34:43.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.976+0000 7faa3afb5640 1 Processor -- start 2024-09-13T16:34:43.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.976+0000 7faa3afb5640 1 -- start start 2024-09-13T16:34:43.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.976+0000 7faa3afb5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa340ffd70 0x7faa34195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:43.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.976+0000 7faa3afb5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa34100f60 0x7faa341961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:43.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.976+0000 7faa3afb5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa34101920 0x7faa3419d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:43.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.976+0000 7faa3afb5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3419d790 con 0x7faa34101920 2024-09-13T16:34:43.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.976+0000 7faa3afb5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3419d900 con 0x7faa340ffd70 2024-09-13T16:34:43.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.976+0000 7faa3afb5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3419dac0 con 0x7faa34100f60 2024-09-13T16:34:43.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.976+0000 7faa39552640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa34101920 0x7faa3419d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:43.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.976+0000 7faa39552640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa34101920 0x7faa3419d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42150/0 (socket says 172.21.15.16:42150) 2024-09-13T16:34:43.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.976+0000 7faa39552640 1 -- 172.21.15.16:0/137273946 learned_addr learned my addr 172.21.15.16:0/137273946 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:34:43.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.976+0000 7faa38d51640 1 --2- 172.21.15.16:0/137273946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa340ffd70 0x7faa34195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:43.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.976+0000 7faa33fff640 1 --2- 172.21.15.16:0/137273946 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa34100f60 0x7faa341961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:43.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.976+0000 7faa39552640 1 -- 172.21.15.16:0/137273946 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa34100f60 msgr2=0x7faa341961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:43.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.976+0000 7faa39552640 1 --2- 172.21.15.16:0/137273946 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa34100f60 0x7faa341961c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:43.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.976+0000 7faa39552640 1 -- 172.21.15.16:0/137273946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa340ffd70 msgr2=0x7faa34195c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:43.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.976+0000 7faa39552640 1 --2- 172.21.15.16:0/137273946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa340ffd70 0x7faa34195c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:43.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.976+0000 7faa39552640 1 -- 172.21.15.16:0/137273946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa20009560 con 0x7faa34101920 2024-09-13T16:34:43.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.976+0000 7faa33fff640 1 --2- 172.21.15.16:0/137273946 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa34100f60 0x7faa341961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:34:43.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.976+0000 7faa38d51640 1 --2- 172.21.15.16:0/137273946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa340ffd70 0x7faa34195c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:34:43.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.976+0000 7faa39552640 1 --2- 172.21.15.16:0/137273946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa34101920 0x7faa3419d220 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00cad0 tx=0x7faa2c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:43.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.976+0000 7faa31ffb640 1 -- 172.21.15.16:0/137273946 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c00ae20 con 0x7faa34101920 2024-09-13T16:34:43.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.977+0000 7faa31ffb640 1 -- 172.21.15.16:0/137273946 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faa2c005020 con 0x7faa34101920 2024-09-13T16:34:43.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.977+0000 7faa3afb5640 1 -- 172.21.15.16:0/137273946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa3419dda0 con 0x7faa34101920 2024-09-13T16:34:43.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.977+0000 7faa31ffb640 1 -- 172.21.15.16:0/137273946 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c00f6b0 con 0x7faa34101920 2024-09-13T16:34:43.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.977+0000 7faa3afb5640 1 -- 172.21.15.16:0/137273946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa3419e200 con 0x7faa34101920 2024-09-13T16:34:43.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.978+0000 7faa3afb5640 1 -- 172.21.15.16:0/137273946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa34107340 con 0x7faa34101920 2024-09-13T16:34:43.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.979+0000 7faa31ffb640 1 -- 172.21.15.16:0/137273946 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7faa2c005190 con 0x7faa34101920 2024-09-13T16:34:43.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.980+0000 7faa31ffb640 1 --2- 172.21.15.16:0/137273946 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7faa240a1090 0x7faa240a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:43.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.980+0000 7faa38d51640 1 --2- 172.21.15.16:0/137273946 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7faa240a1090 0x7faa240a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:43.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.980+0000 7faa31ffb640 1 -- 172.21.15.16:0/137273946 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7faa2c0ccd70 con 0x7faa34101920 2024-09-13T16:34:43.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.980+0000 7faa38d51640 1 --2- 172.21.15.16:0/137273946 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7faa240a1090 0x7faa240a3550 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7faa34196b50 tx=0x7faa28005e90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:43.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:43.984+0000 7faa31ffb640 1 -- 172.21.15.16:0/137273946 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7faa2c096150 con 0x7faa34101920 2024-09-13T16:34:44.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.154+0000 7faa3afb5640 1 -- 172.21.15.16:0/137273946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faa3402c480 con 0x7faa34101920 2024-09-13T16:34:44.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.154+0000 7faa31ffb640 1 -- 172.21.15.16:0/137273946 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7faa2c01d020 con 0x7faa34101920 2024-09-13T16:34:44.155 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-13T16:34:44.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.156+0000 7faa3afb5640 1 -- 172.21.15.16:0/137273946 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7faa240a1090 msgr2=0x7faa240a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:44.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.156+0000 7faa3afb5640 1 --2- 172.21.15.16:0/137273946 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7faa240a1090 0x7faa240a3550 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7faa34196b50 tx=0x7faa28005e90 comp rx=0 tx=0).stop 2024-09-13T16:34:44.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.156+0000 7faa3afb5640 1 -- 172.21.15.16:0/137273946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa34101920 msgr2=0x7faa3419d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:44.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.156+0000 7faa3afb5640 1 --2- 172.21.15.16:0/137273946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa34101920 0x7faa3419d220 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00cad0 tx=0x7faa2c00cf90 comp rx=0 tx=0).stop 2024-09-13T16:34:44.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.156+0000 7faa3afb5640 1 -- 172.21.15.16:0/137273946 shutdown_connections 2024-09-13T16:34:44.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.156+0000 7faa3afb5640 1 --2- 172.21.15.16:0/137273946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa34101920 0x7faa3419d220 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:44.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.156+0000 7faa3afb5640 1 --2- 172.21.15.16:0/137273946 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa34100f60 0x7faa341961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:44.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.156+0000 7faa3afb5640 1 --2- 172.21.15.16:0/137273946 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7faa240a1090 0x7faa240a3550 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:44.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.156+0000 7faa3afb5640 1 --2- 172.21.15.16:0/137273946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa340ffd70 0x7faa34195c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:44.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.156+0000 7faa3afb5640 1 -- 172.21.15.16:0/137273946 >> 172.21.15.16:0/137273946 conn(0x7faa340fb4d0 msgr2=0x7faa34104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:44.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.156+0000 7faa3afb5640 1 -- 172.21.15.16:0/137273946 shutdown_connections 2024-09-13T16:34:44.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.156+0000 7faa3afb5640 1 -- 172.21.15.16:0/137273946 wait complete. 2024-09-13T16:34:44.167 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:44 smithi016 ceph-mon[31187]: pgmap v462: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-13T16:34:44.167 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:44 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/495007092' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:34:44.168 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T16:34:44.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.273+0000 7f48a5918640 1 -- 172.21.15.16:0/3370795226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48a0105ae0 msgr2=0x7f48a0107ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:44.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.273+0000 7f48a5918640 1 --2- 172.21.15.16:0/3370795226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48a0105ae0 0x7f48a0107ed0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4898009ca0 tx=0x7f489802a8c0 comp rx=0 tx=0).stop 2024-09-13T16:34:44.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.273+0000 7f48a5918640 1 -- 172.21.15.16:0/3370795226 shutdown_connections 2024-09-13T16:34:44.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.273+0000 7f48a5918640 1 --2- 172.21.15.16:0/3370795226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48a0105ae0 0x7f48a0107ed0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:44.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.273+0000 7f48a5918640 1 --2- 172.21.15.16:0/3370795226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48a01031b0 0x7f48a01055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:44.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.273+0000 7f48a5918640 1 --2- 172.21.15.16:0/3370795226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48a0100880 0x7f48a0102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:44.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.273+0000 7f48a5918640 1 -- 172.21.15.16:0/3370795226 >> 172.21.15.16:0/3370795226 conn(0x7f48a00fa3e0 msgr2=0x7f48a00fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:44.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.273+0000 7f48a5918640 1 -- 172.21.15.16:0/3370795226 shutdown_connections 2024-09-13T16:34:44.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.273+0000 7f48a5918640 1 -- 172.21.15.16:0/3370795226 wait complete. 2024-09-13T16:34:44.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.274+0000 7f48a5918640 1 Processor -- start 2024-09-13T16:34:44.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.274+0000 7f48a5918640 1 -- start start 2024-09-13T16:34:44.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.274+0000 7f48a5918640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48a0100880 0x7f48a0197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:44.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.274+0000 7f48a5918640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48a01031b0 0x7f48a0198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:44.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.274+0000 7f48a5918640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48a0105ae0 0x7f48a019f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:44.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.274+0000 7f48a5918640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a019f9c0 con 0x7f48a0100880 2024-09-13T16:34:44.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.274+0000 7f48a5918640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a019fb30 con 0x7f48a01031b0 2024-09-13T16:34:44.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.274+0000 7f48a5918640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a019fcf0 con 0x7f48a0105ae0 2024-09-13T16:34:44.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.274+0000 7f489effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48a0100880 0x7f48a0197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:44.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.274+0000 7f489effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48a0100880 0x7f48a0197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42164/0 (socket says 172.21.15.16:42164) 2024-09-13T16:34:44.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.275+0000 7f489effd640 1 -- 172.21.15.16:0/1086567860 learned_addr learned my addr 172.21.15.16:0/1086567860 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:34:44.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.275+0000 7f489f7fe640 1 --2- 172.21.15.16:0/1086567860 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48a0105ae0 0x7f48a019f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:44.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.275+0000 7f489effd640 1 -- 172.21.15.16:0/1086567860 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48a0105ae0 msgr2=0x7f48a019f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:44.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.275+0000 7f489effd640 1 --2- 172.21.15.16:0/1086567860 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48a0105ae0 0x7f48a019f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:44.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.275+0000 7f489effd640 1 -- 172.21.15.16:0/1086567860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48a01031b0 msgr2=0x7f48a0198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:44.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.275+0000 7f489e7fc640 1 --2- 172.21.15.16:0/1086567860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48a01031b0 0x7f48a0198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:44.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.275+0000 7f489effd640 1 --2- 172.21.15.16:0/1086567860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48a01031b0 0x7f48a0198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:44.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.275+0000 7f489effd640 1 -- 172.21.15.16:0/1086567860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4898009900 con 0x7f48a0100880 2024-09-13T16:34:44.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.275+0000 7f489effd640 1 --2- 172.21.15.16:0/1086567860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48a0100880 0x7f48a0197ed0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f489000d950 tx=0x7f489000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:44.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.275+0000 7f48a4916640 1 -- 172.21.15.16:0/1086567860 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4890009a30 con 0x7f48a0100880 2024-09-13T16:34:44.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.275+0000 7f48a4916640 1 -- 172.21.15.16:0/1086567860 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f48900044b0 con 0x7f48a0100880 2024-09-13T16:34:44.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.275+0000 7f48a5918640 1 -- 172.21.15.16:0/1086567860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a019ffd0 con 0x7f48a0100880 2024-09-13T16:34:44.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.275+0000 7f48a4916640 1 -- 172.21.15.16:0/1086567860 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48900047a0 con 0x7f48a0100880 2024-09-13T16:34:44.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.275+0000 7f48a5918640 1 -- 172.21.15.16:0/1086567860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a01a03f0 con 0x7f48a0100880 2024-09-13T16:34:44.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.276+0000 7f48a5918640 1 -- 172.21.15.16:0/1086567860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a0103590 con 0x7f48a0100880 2024-09-13T16:34:44.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.277+0000 7f48a4916640 1 -- 172.21.15.16:0/1086567860 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f4890004020 con 0x7f48a0100880 2024-09-13T16:34:44.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.278+0000 7f48a4916640 1 --2- 172.21.15.16:0/1086567860 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f48880a1090 0x7f48880a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.278+0000 7f489e7fc640 1 --2- 172.21.15.16:0/1086567860 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f48880a1090 0x7f48880a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.278+0000 7f48a4916640 1 -- 172.21.15.16:0/1086567860 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f48900cc7f0 con 0x7f48a0100880 2024-09-13T16:34:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.279+0000 7f489e7fc640 1 --2- 172.21.15.16:0/1086567860 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f48880a1090 0x7f48880a3550 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f48a01993e0 tx=0x7f4894005be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:44.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.282+0000 7f48a4916640 1 -- 172.21.15.16:0/1086567860 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f4890095bd0 con 0x7f48a0100880 2024-09-13T16:34:44.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:44 smithi107 ceph-mon[34139]: pgmap v462: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-13T16:34:44.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:44 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/495007092' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:34:44.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:44 smithi111 ceph-mon[34600]: pgmap v462: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-13T16:34:44.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:44 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/495007092' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:34:44.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.455+0000 7f48a5918640 1 -- 172.21.15.16:0/1086567860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f48a010b7c0 con 0x7f48a0100880 2024-09-13T16:34:44.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.455+0000 7f48a4916640 1 -- 172.21.15.16:0/1086567860 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f48900991f0 con 0x7f48a0100880 2024-09-13T16:34:44.456 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-13T16:34:44.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.457+0000 7f48a5918640 1 -- 172.21.15.16:0/1086567860 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f48880a1090 msgr2=0x7f48880a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:44.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.457+0000 7f48a5918640 1 --2- 172.21.15.16:0/1086567860 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f48880a1090 0x7f48880a3550 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f48a01993e0 tx=0x7f4894005be0 comp rx=0 tx=0).stop 2024-09-13T16:34:44.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.457+0000 7f48a5918640 1 -- 172.21.15.16:0/1086567860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48a0100880 msgr2=0x7f48a0197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:44.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.457+0000 7f48a5918640 1 --2- 172.21.15.16:0/1086567860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48a0100880 0x7f48a0197ed0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f489000d950 tx=0x7f489000de10 comp rx=0 tx=0).stop 2024-09-13T16:34:44.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.457+0000 7f48a5918640 1 -- 172.21.15.16:0/1086567860 shutdown_connections 2024-09-13T16:34:44.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.457+0000 7f48a5918640 1 --2- 172.21.15.16:0/1086567860 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48a0105ae0 0x7f48a019f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:44.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.457+0000 7f48a5918640 1 --2- 172.21.15.16:0/1086567860 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f48880a1090 0x7f48880a3550 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:44.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.457+0000 7f48a5918640 1 --2- 172.21.15.16:0/1086567860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48a01031b0 0x7f48a0198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:44.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.457+0000 7f48a5918640 1 --2- 172.21.15.16:0/1086567860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48a0100880 0x7f48a0197ed0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:44.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.457+0000 7f48a5918640 1 -- 172.21.15.16:0/1086567860 >> 172.21.15.16:0/1086567860 conn(0x7f48a00fa3e0 msgr2=0x7f48a0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:44.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.457+0000 7f48a5918640 1 -- 172.21.15.16:0/1086567860 shutdown_connections 2024-09-13T16:34:44.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.457+0000 7f48a5918640 1 -- 172.21.15.16:0/1086567860 wait complete. 2024-09-13T16:34:44.469 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T16:34:44.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.574+0000 7fae15661640 1 -- 172.21.15.16:0/2950154839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae100ffda0 msgr2=0x7fae101001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:44.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.574+0000 7fae15661640 1 --2- 172.21.15.16:0/2950154839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae100ffda0 0x7fae101001a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fae00009900 tx=0x7fae0002a410 comp rx=0 tx=0).stop 2024-09-13T16:34:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.574+0000 7fae15661640 1 -- 172.21.15.16:0/2950154839 shutdown_connections 2024-09-13T16:34:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.574+0000 7fae15661640 1 --2- 172.21.15.16:0/2950154839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae10101950 0x7fae101081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.574+0000 7fae15661640 1 --2- 172.21.15.16:0/2950154839 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae10100f90 0x7fae10101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.574+0000 7fae15661640 1 --2- 172.21.15.16:0/2950154839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae100ffda0 0x7fae101001a0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.574+0000 7fae15661640 1 -- 172.21.15.16:0/2950154839 >> 172.21.15.16:0/2950154839 conn(0x7fae100fb520 msgr2=0x7fae100fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.574+0000 7fae15661640 1 -- 172.21.15.16:0/2950154839 shutdown_connections 2024-09-13T16:34:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.574+0000 7fae15661640 1 -- 172.21.15.16:0/2950154839 wait complete. 2024-09-13T16:34:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.575+0000 7fae15661640 1 Processor -- start 2024-09-13T16:34:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.575+0000 7fae15661640 1 -- start start 2024-09-13T16:34:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.575+0000 7fae15661640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae100ffda0 0x7fae10074260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.575+0000 7fae15661640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae10100f90 0x7fae100728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.575+0000 7fae15661640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae10101950 0x7fae10072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.575+0000 7fae15661640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae10073540 con 0x7fae10101950 2024-09-13T16:34:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.575+0000 7fae15661640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae100736b0 con 0x7fae100ffda0 2024-09-13T16:34:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.575+0000 7fae15661640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1019fd20 con 0x7fae10100f90 2024-09-13T16:34:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.575+0000 7fae0f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae10101950 0x7fae10072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.575+0000 7fae0f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae10101950 0x7fae10072de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42176/0 (socket says 172.21.15.16:42176) 2024-09-13T16:34:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.575+0000 7fae0f7fe640 1 -- 172.21.15.16:0/3394242884 learned_addr learned my addr 172.21.15.16:0/3394242884 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:34:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.575+0000 7fae0effd640 1 --2- 172.21.15.16:0/3394242884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae100ffda0 0x7fae10074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.575+0000 7fae0e7fc640 1 --2- 172.21.15.16:0/3394242884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae10100f90 0x7fae100728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.575+0000 7fae0f7fe640 1 -- 172.21.15.16:0/3394242884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae10100f90 msgr2=0x7fae100728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.575+0000 7fae0f7fe640 1 --2- 172.21.15.16:0/3394242884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae10100f90 0x7fae100728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.575+0000 7fae0f7fe640 1 -- 172.21.15.16:0/3394242884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae100ffda0 msgr2=0x7fae10074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.575+0000 7fae0f7fe640 1 --2- 172.21.15.16:0/3394242884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae100ffda0 0x7fae10074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.575+0000 7fae0f7fe640 1 -- 172.21.15.16:0/3394242884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae00009560 con 0x7fae10101950 2024-09-13T16:34:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.576+0000 7fae0f7fe640 1 --2- 172.21.15.16:0/3394242884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae10101950 0x7fae10072de0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fae0800db20 tx=0x7fae0800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:44.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.576+0000 7fadf7fff640 1 -- 172.21.15.16:0/3394242884 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0800bea0 con 0x7fae10101950 2024-09-13T16:34:44.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.576+0000 7fadf7fff640 1 -- 172.21.15.16:0/3394242884 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fae08009940 con 0x7fae10101950 2024-09-13T16:34:44.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.576+0000 7fae15661640 1 -- 172.21.15.16:0/3394242884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae1019ffd0 con 0x7fae10101950 2024-09-13T16:34:44.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.576+0000 7fadf7fff640 1 -- 172.21.15.16:0/3394242884 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae08010460 con 0x7fae10101950 2024-09-13T16:34:44.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.576+0000 7fae15661640 1 -- 172.21.15.16:0/3394242884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae101a0400 con 0x7fae10101950 2024-09-13T16:34:44.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.577+0000 7fae15661640 1 -- 172.21.15.16:0/3394242884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae101072d0 con 0x7fae10101950 2024-09-13T16:34:44.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.578+0000 7fadf7fff640 1 -- 172.21.15.16:0/3394242884 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fae08004a40 con 0x7fae10101950 2024-09-13T16:34:44.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.579+0000 7fadf7fff640 1 --2- 172.21.15.16:0/3394242884 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fadf80a1090 0x7fadf80a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:44.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.579+0000 7fae0effd640 1 --2- 172.21.15.16:0/3394242884 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fadf80a1090 0x7fadf80a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:44.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.579+0000 7fadf7fff640 1 -- 172.21.15.16:0/3394242884 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fae080cc840 con 0x7fae10101950 2024-09-13T16:34:44.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.579+0000 7fae0effd640 1 --2- 172.21.15.16:0/3394242884 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fadf80a1090 0x7fadf80a3550 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fae000057c0 tx=0x7fae00005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:44.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.582+0000 7fadf7fff640 1 -- 172.21.15.16:0/3394242884 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fae08095cd0 con 0x7fae10101950 2024-09-13T16:34:44.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.755+0000 7fae15661640 1 -- 172.21.15.16:0/3394242884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fae1002c410 con 0x7fae10101950 2024-09-13T16:34:44.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.755+0000 7fadf7fff640 1 -- 172.21.15.16:0/3394242884 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7fae080170a0 con 0x7fae10101950 2024-09-13T16:34:44.757 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-09-13T16:34:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.757+0000 7fae15661640 1 -- 172.21.15.16:0/3394242884 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fadf80a1090 msgr2=0x7fadf80a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.757+0000 7fae15661640 1 --2- 172.21.15.16:0/3394242884 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fadf80a1090 0x7fadf80a3550 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fae000057c0 tx=0x7fae00005a20 comp rx=0 tx=0).stop 2024-09-13T16:34:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.757+0000 7fae15661640 1 -- 172.21.15.16:0/3394242884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae10101950 msgr2=0x7fae10072de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.757+0000 7fae15661640 1 --2- 172.21.15.16:0/3394242884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae10101950 0x7fae10072de0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fae0800db20 tx=0x7fae0800b260 comp rx=0 tx=0).stop 2024-09-13T16:34:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.757+0000 7fae15661640 1 -- 172.21.15.16:0/3394242884 shutdown_connections 2024-09-13T16:34:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.757+0000 7fae15661640 1 --2- 172.21.15.16:0/3394242884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae10101950 0x7fae10072de0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.757+0000 7fae15661640 1 --2- 172.21.15.16:0/3394242884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae10100f90 0x7fae100728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.757+0000 7fae15661640 1 --2- 172.21.15.16:0/3394242884 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fadf80a1090 0x7fadf80a3550 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.757+0000 7fae15661640 1 --2- 172.21.15.16:0/3394242884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae100ffda0 0x7fae10074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.757+0000 7fae15661640 1 -- 172.21.15.16:0/3394242884 >> 172.21.15.16:0/3394242884 conn(0x7fae100fb520 msgr2=0x7fae101044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.757+0000 7fae15661640 1 -- 172.21.15.16:0/3394242884 shutdown_connections 2024-09-13T16:34:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.758+0000 7fae15661640 1 -- 172.21.15.16:0/3394242884 wait complete. 2024-09-13T16:34:44.770 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T16:34:44.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.874+0000 7f7504053640 1 -- 172.21.15.16:0/3212595962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74fc100840 msgr2=0x7f74fc1070d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:44.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.874+0000 7f7504053640 1 --2- 172.21.15.16:0/3212595962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74fc100840 0x7f74fc1070d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f74f4009ca0 tx=0x7f74f402a8c0 comp rx=0 tx=0).stop 2024-09-13T16:34:44.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.874+0000 7f7504053640 1 -- 172.21.15.16:0/3212595962 shutdown_connections 2024-09-13T16:34:44.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.874+0000 7f7504053640 1 --2- 172.21.15.16:0/3212595962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74fc100840 0x7f74fc1070d0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:44.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.875+0000 7f7504053640 1 --2- 172.21.15.16:0/3212595962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74fc0ffe80 0x7f74fc100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:44.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.875+0000 7f7504053640 1 --2- 172.21.15.16:0/3212595962 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74fc0fec90 0x7f74fc0ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:44.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.875+0000 7f7504053640 1 -- 172.21.15.16:0/3212595962 >> 172.21.15.16:0/3212595962 conn(0x7f74fc0fa3f0 msgr2=0x7f74fc0fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:44.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.875+0000 7f7504053640 1 -- 172.21.15.16:0/3212595962 shutdown_connections 2024-09-13T16:34:44.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.875+0000 7f7504053640 1 -- 172.21.15.16:0/3212595962 wait complete. 2024-09-13T16:34:44.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.875+0000 7f7504053640 1 Processor -- start 2024-09-13T16:34:44.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.875+0000 7f7504053640 1 -- start start 2024-09-13T16:34:44.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.875+0000 7f7504053640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74fc0fec90 0x7f74fc195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:44.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.875+0000 7f7504053640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74fc0ffe80 0x7f74fc196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:44.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.875+0000 7f7504053640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74fc100840 0x7f74fc19d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:44.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.875+0000 7f7504053640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74fc19d7f0 con 0x7f74fc0ffe80 2024-09-13T16:34:44.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.875+0000 7f7504053640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74fc19d960 con 0x7f74fc0fec90 2024-09-13T16:34:44.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.875+0000 7f7504053640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74fc19db20 con 0x7f74fc100840 2024-09-13T16:34:44.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.876+0000 7f75015ee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74fc0ffe80 0x7f74fc196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:44.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.876+0000 7f75015ee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74fc0ffe80 0x7f74fc196250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42184/0 (socket says 172.21.15.16:42184) 2024-09-13T16:34:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.876+0000 7f75015ee640 1 -- 172.21.15.16:0/893011296 learned_addr learned my addr 172.21.15.16:0/893011296 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:34:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.876+0000 7f7501def640 1 --2- 172.21.15.16:0/893011296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74fc0fec90 0x7f74fc195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.876+0000 7f75025f0640 1 --2- 172.21.15.16:0/893011296 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74fc100840 0x7f74fc19d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.876+0000 7f75015ee640 1 -- 172.21.15.16:0/893011296 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74fc100840 msgr2=0x7f74fc19d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.876+0000 7f75015ee640 1 --2- 172.21.15.16:0/893011296 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74fc100840 0x7f74fc19d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.876+0000 7f75015ee640 1 -- 172.21.15.16:0/893011296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74fc0fec90 msgr2=0x7f74fc195d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.876+0000 7f75015ee640 1 --2- 172.21.15.16:0/893011296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74fc0fec90 0x7f74fc195d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.876+0000 7f75015ee640 1 -- 172.21.15.16:0/893011296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f4009900 con 0x7f74fc0ffe80 2024-09-13T16:34:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.876+0000 7f7501def640 1 --2- 172.21.15.16:0/893011296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74fc0fec90 0x7f74fc195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:34:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.876+0000 7f75015ee640 1 --2- 172.21.15.16:0/893011296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74fc0ffe80 0x7f74fc196250 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f74ec00c890 tx=0x7f74ec00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.876+0000 7f74f2ffd640 1 -- 172.21.15.16:0/893011296 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ec004220 con 0x7f74fc0ffe80 2024-09-13T16:34:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.876+0000 7f74f2ffd640 1 -- 172.21.15.16:0/893011296 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74ec007e40 con 0x7f74fc0ffe80 2024-09-13T16:34:44.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.876+0000 7f7504053640 1 -- 172.21.15.16:0/893011296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74fc19de00 con 0x7f74fc0ffe80 2024-09-13T16:34:44.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.876+0000 7f74f2ffd640 1 -- 172.21.15.16:0/893011296 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ec004a00 con 0x7f74fc0ffe80 2024-09-13T16:34:44.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.876+0000 7f7504053640 1 -- 172.21.15.16:0/893011296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74fc19e280 con 0x7f74fc0ffe80 2024-09-13T16:34:44.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.877+0000 7f7504053640 1 -- 172.21.15.16:0/893011296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74fc10a480 con 0x7f74fc0ffe80 2024-09-13T16:34:44.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.879+0000 7f74f2ffd640 1 -- 172.21.15.16:0/893011296 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f74ec0079b0 con 0x7f74fc0ffe80 2024-09-13T16:34:44.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.879+0000 7f74f2ffd640 1 --2- 172.21.15.16:0/893011296 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f74e80a1090 0x7f74e80a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:44.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.879+0000 7f7501def640 1 --2- 172.21.15.16:0/893011296 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f74e80a1090 0x7f74e80a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:44.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.879+0000 7f74f2ffd640 1 -- 172.21.15.16:0/893011296 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f74ec0cce30 con 0x7f74fc0ffe80 2024-09-13T16:34:44.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.880+0000 7f7501def640 1 --2- 172.21.15.16:0/893011296 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f74e80a1090 0x7f74e80a3550 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f74fc1971e0 tx=0x7f74f8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:44.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:44.883+0000 7f74f2ffd640 1 -- 172.21.15.16:0/893011296 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f74ec0961e0 con 0x7f74fc0ffe80 2024-09-13T16:34:45.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.059+0000 7f7504053640 1 -- 172.21.15.16:0/893011296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f74fc19e4d0 con 0x7f74fc0ffe80 2024-09-13T16:34:45.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.060+0000 7f74f2ffd640 1 -- 172.21.15.16:0/893011296 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f74ec01d020 con 0x7f74fc0ffe80 2024-09-13T16:34:45.061 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-13T16:34:45.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.062+0000 7f7504053640 1 -- 172.21.15.16:0/893011296 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f74e80a1090 msgr2=0x7f74e80a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:45.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.062+0000 7f7504053640 1 --2- 172.21.15.16:0/893011296 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f74e80a1090 0x7f74e80a3550 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f74fc1971e0 tx=0x7f74f8008040 comp rx=0 tx=0).stop 2024-09-13T16:34:45.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.062+0000 7f7504053640 1 -- 172.21.15.16:0/893011296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74fc0ffe80 msgr2=0x7f74fc196250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:45.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.062+0000 7f7504053640 1 --2- 172.21.15.16:0/893011296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74fc0ffe80 0x7f74fc196250 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f74ec00c890 tx=0x7f74ec00cd50 comp rx=0 tx=0).stop 2024-09-13T16:34:45.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.062+0000 7f7504053640 1 -- 172.21.15.16:0/893011296 shutdown_connections 2024-09-13T16:34:45.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.062+0000 7f7504053640 1 --2- 172.21.15.16:0/893011296 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74fc100840 0x7f74fc19d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:45.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.062+0000 7f7504053640 1 --2- 172.21.15.16:0/893011296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74fc0ffe80 0x7f74fc196250 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:45.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.062+0000 7f7504053640 1 --2- 172.21.15.16:0/893011296 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f74e80a1090 0x7f74e80a3550 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:45.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.062+0000 7f7504053640 1 --2- 172.21.15.16:0/893011296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74fc0fec90 0x7f74fc195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:45.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.062+0000 7f7504053640 1 -- 172.21.15.16:0/893011296 >> 172.21.15.16:0/893011296 conn(0x7f74fc0fa3f0 msgr2=0x7f74fc068620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:45.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.062+0000 7f7504053640 1 -- 172.21.15.16:0/893011296 shutdown_connections 2024-09-13T16:34:45.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.062+0000 7f7504053640 1 -- 172.21.15.16:0/893011296 wait complete. 2024-09-13T16:34:45.074 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T16:34:45.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.177+0000 7efcf5bbc640 1 -- 172.21.15.16:0/4245231260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcf0100880 msgr2=0x7efcf0107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:45.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.177+0000 7efcf5bbc640 1 --2- 172.21.15.16:0/4245231260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcf0100880 0x7efcf0107110 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7efce8009ca0 tx=0x7efce802a8c0 comp rx=0 tx=0).stop 2024-09-13T16:34:45.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.177+0000 7efcf5bbc640 1 -- 172.21.15.16:0/4245231260 shutdown_connections 2024-09-13T16:34:45.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.177+0000 7efcf5bbc640 1 --2- 172.21.15.16:0/4245231260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcf0100880 0x7efcf0107110 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:45.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.177+0000 7efcf5bbc640 1 --2- 172.21.15.16:0/4245231260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcf00ffec0 0x7efcf0100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:45.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.177+0000 7efcf5bbc640 1 --2- 172.21.15.16:0/4245231260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcf00fecd0 0x7efcf00ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:45.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.177+0000 7efcf5bbc640 1 -- 172.21.15.16:0/4245231260 >> 172.21.15.16:0/4245231260 conn(0x7efcf00fa430 msgr2=0x7efcf00fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:45.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.177+0000 7efcf5bbc640 1 -- 172.21.15.16:0/4245231260 shutdown_connections 2024-09-13T16:34:45.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.177+0000 7efcf5bbc640 1 -- 172.21.15.16:0/4245231260 wait complete. 2024-09-13T16:34:45.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.178+0000 7efcf5bbc640 1 Processor -- start 2024-09-13T16:34:45.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.178+0000 7efcf5bbc640 1 -- start start 2024-09-13T16:34:45.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.178+0000 7efcf5bbc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcf00fecd0 0x7efcf0194b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:45.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.178+0000 7efcf5bbc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcf00ffec0 0x7efcf0195080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:45.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.178+0000 7efcf5bbc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcf0100880 0x7efcf019c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:45.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.178+0000 7efcf5bbc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf019c630 con 0x7efcf00ffec0 2024-09-13T16:34:45.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.178+0000 7efcf5bbc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf019c7a0 con 0x7efcf0100880 2024-09-13T16:34:45.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.178+0000 7efcf5bbc640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf019c960 con 0x7efcf00fecd0 2024-09-13T16:34:45.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.178+0000 7efceeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcf00ffec0 0x7efcf0195080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:45.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.178+0000 7efceeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcf00ffec0 0x7efcf0195080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42200/0 (socket says 172.21.15.16:42200) 2024-09-13T16:34:45.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.178+0000 7efceeffd640 1 -- 172.21.15.16:0/1515141357 learned_addr learned my addr 172.21.15.16:0/1515141357 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:34:45.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.179+0000 7efceffff640 1 --2- 172.21.15.16:0/1515141357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcf0100880 0x7efcf019c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:45.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.179+0000 7efcef7fe640 1 --2- 172.21.15.16:0/1515141357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcf00fecd0 0x7efcf0194b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:45.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.179+0000 7efceeffd640 1 -- 172.21.15.16:0/1515141357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcf00fecd0 msgr2=0x7efcf0194b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:45.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.179+0000 7efceeffd640 1 --2- 172.21.15.16:0/1515141357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcf00fecd0 0x7efcf0194b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:45.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.179+0000 7efceeffd640 1 -- 172.21.15.16:0/1515141357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcf0100880 msgr2=0x7efcf019c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:45.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.179+0000 7efceeffd640 1 --2- 172.21.15.16:0/1515141357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcf0100880 0x7efcf019c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:45.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.179+0000 7efceeffd640 1 -- 172.21.15.16:0/1515141357 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce8009900 con 0x7efcf00ffec0 2024-09-13T16:34:45.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.179+0000 7efcef7fe640 1 --2- 172.21.15.16:0/1515141357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcf00fecd0 0x7efcf0194b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:34:45.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.179+0000 7efceeffd640 1 --2- 172.21.15.16:0/1515141357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcf00ffec0 0x7efcf0195080 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7efce400e9f0 tx=0x7efce400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:45.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.179+0000 7efcecff9640 1 -- 172.21.15.16:0/1515141357 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce4009700 con 0x7efcf00ffec0 2024-09-13T16:34:45.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.179+0000 7efcecff9640 1 -- 172.21.15.16:0/1515141357 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efce400cd80 con 0x7efcf00ffec0 2024-09-13T16:34:45.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.179+0000 7efcf5bbc640 1 -- 172.21.15.16:0/1515141357 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcf019cc40 con 0x7efcf00ffec0 2024-09-13T16:34:45.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.179+0000 7efcecff9640 1 -- 172.21.15.16:0/1515141357 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce40187e0 con 0x7efcf00ffec0 2024-09-13T16:34:45.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.179+0000 7efcf5bbc640 1 -- 172.21.15.16:0/1515141357 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcf019d150 con 0x7efcf00ffec0 2024-09-13T16:34:45.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.180+0000 7efcf5bbc640 1 -- 172.21.15.16:0/1515141357 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcf0106160 con 0x7efcf00ffec0 2024-09-13T16:34:45.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.181+0000 7efcecff9640 1 -- 172.21.15.16:0/1515141357 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7efce4010430 con 0x7efcf00ffec0 2024-09-13T16:34:45.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.182+0000 7efcecff9640 1 --2- 172.21.15.16:0/1515141357 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7efcd80a1090 0x7efcd80a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:45.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.182+0000 7efcef7fe640 1 --2- 172.21.15.16:0/1515141357 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7efcd80a1090 0x7efcd80a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:45.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.182+0000 7efcecff9640 1 -- 172.21.15.16:0/1515141357 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7efce4014070 con 0x7efcf00ffec0 2024-09-13T16:34:45.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.183+0000 7efcef7fe640 1 --2- 172.21.15.16:0/1515141357 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7efcd80a1090 0x7efcd80a3550 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efcf00ffd20 tx=0x7efce0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:45.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.187+0000 7efcecff9640 1 -- 172.21.15.16:0/1515141357 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7efce4052e20 con 0x7efcf00ffec0 2024-09-13T16:34:45.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:45 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/137273946' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:34:45.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:45 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1086567860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T16:34:45.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:45 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3394242884' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T16:34:45.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:45 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/137273946' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:34:45.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:45 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1086567860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T16:34:45.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:45 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3394242884' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T16:34:45.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.373+0000 7efcf5bbc640 1 -- 172.21.15.16:0/1515141357 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7efcf010a440 con 0x7efcf00ffec0 2024-09-13T16:34:45.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.373+0000 7efcecff9640 1 -- 172.21.15.16:0/1515141357 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v93) v1 ==== 81+0+589 (secure 0 0 0) 0x7efce40964c0 con 0x7efcf00ffec0 2024-09-13T16:34:45.374 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2098378667 2024-09-14T16:22:04.485277+0000 2024-09-13T16:34:45.374 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3536229016 2024-09-14T16:22:04.485277+0000 2024-09-13T16:34:45.374 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/1756940569 2024-09-14T16:22:04.485277+0000 2024-09-13T16:34:45.374 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2684687547 2024-09-14T16:22:04.485277+0000 2024-09-13T16:34:45.375 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/1756940569 2024-09-14T16:22:04.485277+0000 2024-09-13T16:34:45.375 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/368410612 2024-09-14T16:21:40.126459+0000 2024-09-13T16:34:45.375 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/12158160 2024-09-14T16:21:40.126459+0000 2024-09-13T16:34:45.375 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/1147765163 2024-09-14T16:21:40.126459+0000 2024-09-13T16:34:45.375 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/1147765163 2024-09-14T16:21:40.126459+0000 2024-09-13T16:34:45.375 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3754423379 2024-09-14T16:21:40.126459+0000 2024-09-13T16:34:45.375 INFO:teuthology.orchestra.run.smithi016.stderr:listed 10 entries 2024-09-13T16:34:45.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.375+0000 7efcf5bbc640 1 -- 172.21.15.16:0/1515141357 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7efcd80a1090 msgr2=0x7efcd80a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:45.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.375+0000 7efcf5bbc640 1 --2- 172.21.15.16:0/1515141357 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7efcd80a1090 0x7efcd80a3550 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efcf00ffd20 tx=0x7efce0005f90 comp rx=0 tx=0).stop 2024-09-13T16:34:45.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.375+0000 7efcf5bbc640 1 -- 172.21.15.16:0/1515141357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcf00ffec0 msgr2=0x7efcf0195080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:45.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.375+0000 7efcf5bbc640 1 --2- 172.21.15.16:0/1515141357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcf00ffec0 0x7efcf0195080 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7efce400e9f0 tx=0x7efce400eeb0 comp rx=0 tx=0).stop 2024-09-13T16:34:45.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.375+0000 7efcf5bbc640 1 -- 172.21.15.16:0/1515141357 shutdown_connections 2024-09-13T16:34:45.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.375+0000 7efcf5bbc640 1 --2- 172.21.15.16:0/1515141357 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7efcd80a1090 0x7efcd80a3550 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:45.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.375+0000 7efcf5bbc640 1 --2- 172.21.15.16:0/1515141357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcf0100880 0x7efcf019c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:45.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.375+0000 7efcf5bbc640 1 --2- 172.21.15.16:0/1515141357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcf00ffec0 0x7efcf0195080 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:45.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.375+0000 7efcf5bbc640 1 --2- 172.21.15.16:0/1515141357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcf00fecd0 0x7efcf0194b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:45.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.375+0000 7efcf5bbc640 1 -- 172.21.15.16:0/1515141357 >> 172.21.15.16:0/1515141357 conn(0x7efcf00fa430 msgr2=0x7efcf0103330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:45.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.375+0000 7efcf5bbc640 1 -- 172.21.15.16:0/1515141357 shutdown_connections 2024-09-13T16:34:45.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.375+0000 7efcf5bbc640 1 -- 172.21.15.16:0/1515141357 wait complete. 2024-09-13T16:34:45.387 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-13T16:34:45.387 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T16:34:45.405 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:34:45.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:45 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/137273946' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:34:45.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:45 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1086567860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T16:34:45.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:45 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3394242884' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T16:34:45.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.546+0000 7fd069e32640 1 -- 172.21.15.16:0/83876324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd064100850 msgr2=0x7fd064102c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:45.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.546+0000 7fd069e32640 1 --2- 172.21.15.16:0/83876324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd064100850 0x7fd064102c40 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd054009900 tx=0x7fd05402a410 comp rx=0 tx=0).stop 2024-09-13T16:34:45.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.546+0000 7fd069e32640 1 -- 172.21.15.16:0/83876324 shutdown_connections 2024-09-13T16:34:45.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.546+0000 7fd069e32640 1 --2- 172.21.15.16:0/83876324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd064105ab0 0x7fd064107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:45.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.546+0000 7fd069e32640 1 --2- 172.21.15.16:0/83876324 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd064103180 0x7fd064105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:45.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.546+0000 7fd069e32640 1 --2- 172.21.15.16:0/83876324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd064100850 0x7fd064102c40 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:45.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.546+0000 7fd069e32640 1 -- 172.21.15.16:0/83876324 >> 172.21.15.16:0/83876324 conn(0x7fd0640fa430 msgr2=0x7fd0640fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:45.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.546+0000 7fd069e32640 1 -- 172.21.15.16:0/83876324 shutdown_connections 2024-09-13T16:34:45.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.546+0000 7fd069e32640 1 -- 172.21.15.16:0/83876324 wait complete. 2024-09-13T16:34:45.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.547+0000 7fd069e32640 1 Processor -- start 2024-09-13T16:34:45.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.547+0000 7fd069e32640 1 -- start start 2024-09-13T16:34:45.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.547+0000 7fd069e32640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd064100850 0x7fd0641001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:45.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.547+0000 7fd069e32640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd064103180 0x7fd0640fe800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:45.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.547+0000 7fd069e32640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd064105ab0 0x7fd0640fed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:45.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.547+0000 7fd069e32640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0640ff4a0 con 0x7fd064103180 2024-09-13T16:34:45.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.547+0000 7fd069e32640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0640ff610 con 0x7fd064100850 2024-09-13T16:34:45.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.547+0000 7fd069e32640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06419fe70 con 0x7fd064105ab0 2024-09-13T16:34:45.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.547+0000 7fd05affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd064103180 0x7fd0640fe800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:45.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.548+0000 7fd05affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd064103180 0x7fd0640fe800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42232/0 (socket says 172.21.15.16:42232) 2024-09-13T16:34:45.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.548+0000 7fd05affd640 1 -- 172.21.15.16:0/2190663327 learned_addr learned my addr 172.21.15.16:0/2190663327 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:34:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.548+0000 7fd0637fe640 1 --2- 172.21.15.16:0/2190663327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd064100850 0x7fd0641001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.548+0000 7fd063fff640 1 --2- 172.21.15.16:0/2190663327 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd064105ab0 0x7fd0640fed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.548+0000 7fd05affd640 1 -- 172.21.15.16:0/2190663327 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd064105ab0 msgr2=0x7fd0640fed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.548+0000 7fd05affd640 1 --2- 172.21.15.16:0/2190663327 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd064105ab0 0x7fd0640fed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.548+0000 7fd05affd640 1 -- 172.21.15.16:0/2190663327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd064100850 msgr2=0x7fd0641001c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.548+0000 7fd05affd640 1 --2- 172.21.15.16:0/2190663327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd064100850 0x7fd0641001c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.548+0000 7fd05affd640 1 -- 172.21.15.16:0/2190663327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd054009560 con 0x7fd064103180 2024-09-13T16:34:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.548+0000 7fd05affd640 1 --2- 172.21.15.16:0/2190663327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd064103180 0x7fd0640fe800 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00b480 tx=0x7fd04c00b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.548+0000 7fd0617fa640 1 -- 172.21.15.16:0/2190663327 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c010870 con 0x7fd064103180 2024-09-13T16:34:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.548+0000 7fd0617fa640 1 -- 172.21.15.16:0/2190663327 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd04c010e60 con 0x7fd064103180 2024-09-13T16:34:45.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.548+0000 7fd069e32640 1 -- 172.21.15.16:0/2190663327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0641a0120 con 0x7fd064103180 2024-09-13T16:34:45.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.548+0000 7fd0617fa640 1 -- 172.21.15.16:0/2190663327 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c017360 con 0x7fd064103180 2024-09-13T16:34:45.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.548+0000 7fd069e32640 1 -- 172.21.15.16:0/2190663327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0641a0630 con 0x7fd064103180 2024-09-13T16:34:45.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.549+0000 7fd069e32640 1 -- 172.21.15.16:0/2190663327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0640fbf20 con 0x7fd064103180 2024-09-13T16:34:45.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.550+0000 7fd0617fa640 1 -- 172.21.15.16:0/2190663327 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd04c0109d0 con 0x7fd064103180 2024-09-13T16:34:45.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.551+0000 7fd0617fa640 1 --2- 172.21.15.16:0/2190663327 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd0500a1090 0x7fd0500a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:45.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.551+0000 7fd0637fe640 1 --2- 172.21.15.16:0/2190663327 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd0500a1090 0x7fd0500a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:45.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.551+0000 7fd0617fa640 1 -- 172.21.15.16:0/2190663327 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd04c0cca60 con 0x7fd064103180 2024-09-13T16:34:45.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.552+0000 7fd0637fe640 1 --2- 172.21.15.16:0/2190663327 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd0500a1090 0x7fd0500a3550 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd05402aa00 tx=0x7fd054005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:45.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.554+0000 7fd0617fa640 1 -- 172.21.15.16:0/2190663327 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd04c095ed0 con 0x7fd064103180 2024-09-13T16:34:45.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.726+0000 7fd069e32640 1 -- 172.21.15.16:0/2190663327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd06410b5f0 con 0x7fd064103180 2024-09-13T16:34:45.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.728+0000 7fd0617fa640 1 -- 172.21.15.16:0/2190663327 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24593 (secure 0 0 0) 0x7fd04c0994f0 con 0x7fd064103180 2024-09-13T16:34:45.730 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:34:45.731 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":93,"fsid":"0d241f6a-71ec-11ef-bceb-c7b262605968","created":"2024-09-13T16:21:17.971738+0000","modified":"2024-09-13T16:30:05.681521+0000","last_up_change":"2024-09-13T16:27:54.673413+0000","last_in_change":"2024-09-13T16:27:35.718390+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:24:22.981019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:28:37.746027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:28:38.602731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:28:41.633322+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e11ef8f-08d6-4e74-afd6-f49f4bd40032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6803","nonce":2079394186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6805","nonce":2079394186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6809","nonce":2079394186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6807","nonce":2079394186}]},"public_addr":"172.21.15.16:6803/2079394186","cluster_addr":"172.21.15.16:6805/2079394186","heartbeat_back_addr":"172.21.15.16:6809/2079394186","heartbeat_front_addr":"172.21.15.16:6807/2079394186","state":["exists","up"]},{"osd":1,"uuid":"f2cd3694-9c9e-4da7-976e-f80a8df120f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6801","nonce":4021360958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6803","nonce":4021360958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6807","nonce":4021360958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6805","nonce":4021360958}]},"public_addr":"172.21.15.107:6801/4021360958","cluster_addr":"172.21.15.107:6803/4021360958","heartbeat_back_addr":"172.21.15.107:6807/4021360958","heartbeat_front_addr":"172.21.15.107:6805/4021360958","state":["exists","up"]},{"osd":2,"uuid":"a5bbf2eb-6599-44d5-ad96-68bb89ee19f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6801","nonce":3227179825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6803","nonce":3227179825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6807","nonce":3227179825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6805","nonce":3227179825}]},"public_addr":"172.21.15.111:6801/3227179825","cluster_addr":"172.21.15.111:6803/3227179825","heartbeat_back_addr":"172.21.15.111:6807/3227179825","heartbeat_front_addr":"172.21.15.111:6805/3227179825","state":["exists","up"]},{"osd":3,"uuid":"d6295ffe-11bc-4964-a250-9fecca870b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6811","nonce":2390375937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6813","nonce":2390375937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6817","nonce":2390375937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6815","nonce":2390375937}]},"public_addr":"172.21.15.16:6811/2390375937","cluster_addr":"172.21.15.16:6813/2390375937","heartbeat_back_addr":"172.21.15.16:6817/2390375937","heartbeat_front_addr":"172.21.15.16:6815/2390375937","state":["exists","up"]},{"osd":4,"uuid":"0a8446e0-fc86-4646-beb6-a7b9d433093d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6809","nonce":1936582048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6811","nonce":1936582048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6815","nonce":1936582048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6813","nonce":1936582048}]},"public_addr":"172.21.15.107:6809/1936582048","cluster_addr":"172.21.15.107:6811/1936582048","heartbeat_back_addr":"172.21.15.107:6815/1936582048","heartbeat_front_addr":"172.21.15.107:6813/1936582048","state":["exists","up"]},{"osd":5,"uuid":"2c97a446-357e-4975-9e51-9c95fe84a663","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6809","nonce":483858448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6811","nonce":483858448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6815","nonce":483858448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6813","nonce":483858448}]},"public_addr":"172.21.15.111:6809/483858448","cluster_addr":"172.21.15.111:6811/483858448","heartbeat_back_addr":"172.21.15.111:6815/483858448","heartbeat_front_addr":"172.21.15.111:6813/483858448","state":["exists","up"]},{"osd":6,"uuid":"890239e6-e6fb-4f9c-b973-05cc66d199bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6819","nonce":1330214141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6821","nonce":1330214141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6825","nonce":1330214141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6823","nonce":1330214141}]},"public_addr":"172.21.15.16:6819/1330214141","cluster_addr":"172.21.15.16:6821/1330214141","heartbeat_back_addr":"172.21.15.16:6825/1330214141","heartbeat_front_addr":"172.21.15.16:6823/1330214141","state":["exists","up"]},{"osd":7,"uuid":"ee826a9c-f441-4903-9b1b-66b627f7b712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6817","nonce":1133876754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6819","nonce":1133876754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6823","nonce":1133876754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6821","nonce":1133876754}]},"public_addr":"172.21.15.107:6817/1133876754","cluster_addr":"172.21.15.107:6819/1133876754","heartbeat_back_addr":"172.21.15.107:6823/1133876754","heartbeat_front_addr":"172.21.15.107:6821/1133876754","state":["exists","up"]},{"osd":8,"uuid":"e19773f2-dec8-44e0-9d33-0252e07d6415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6817","nonce":837548190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6819","nonce":837548190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6823","nonce":837548190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6821","nonce":837548190}]},"public_addr":"172.21.15.111:6817/837548190","cluster_addr":"172.21.15.111:6819/837548190","heartbeat_back_addr":"172.21.15.111:6823/837548190","heartbeat_front_addr":"172.21.15.111:6821/837548190","state":["exists","up"]},{"osd":9,"uuid":"bae4d6db-8b90-4d02-8e83-7d8f2a6f8fdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6827","nonce":4175383686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6829","nonce":4175383686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6833","nonce":4175383686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6831","nonce":4175383686}]},"public_addr":"172.21.15.16:6827/4175383686","cluster_addr":"172.21.15.16:6829/4175383686","heartbeat_back_addr":"172.21.15.16:6833/4175383686","heartbeat_front_addr":"172.21.15.16:6831/4175383686","state":["exists","up"]},{"osd":10,"uuid":"fd0e7740-76c8-4c99-a349-7077b4905143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6825","nonce":2659550426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6827","nonce":2659550426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6831","nonce":2659550426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6829","nonce":2659550426}]},"public_addr":"172.21.15.107:6825/2659550426","cluster_addr":"172.21.15.107:6827/2659550426","heartbeat_back_addr":"172.21.15.107:6831/2659550426","heartbeat_front_addr":"172.21.15.107:6829/2659550426","state":["exists","up"]},{"osd":11,"uuid":"94a1751d-b00c-4e23-815d-3a8bc3dea146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6825","nonce":1633548683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6827","nonce":1633548683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6831","nonce":1633548683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6829","nonce":1633548683}]},"public_addr":"172.21.15.111:6825/1633548683","cluster_addr":"172.21.15.111:6827/1633548683","heartbeat_back_addr":"172.21.15.111:6831/1633548683","heartbeat_front_addr":"172.21.15.111:6829/1633548683","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:36.359852+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:58.014202+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:19.753672+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:43.404817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:06.001677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:29.361587+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:52.554847+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:15.957553+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:39.581093+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:04.911389+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:28.971280+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:53.473561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/2098378667":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/3536229016":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6801/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/2684687547":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6800/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/368410612":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/12158160":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6800/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6801/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/3754423379":"2024-09-14T16:21:40.126459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:34:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.730+0000 7fd069e32640 1 -- 172.21.15.16:0/2190663327 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd0500a1090 msgr2=0x7fd0500a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.730+0000 7fd069e32640 1 --2- 172.21.15.16:0/2190663327 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd0500a1090 0x7fd0500a3550 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd05402aa00 tx=0x7fd054005a20 comp rx=0 tx=0).stop 2024-09-13T16:34:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.730+0000 7fd069e32640 1 -- 172.21.15.16:0/2190663327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd064103180 msgr2=0x7fd0640fe800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.730+0000 7fd069e32640 1 --2- 172.21.15.16:0/2190663327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd064103180 0x7fd0640fe800 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00b480 tx=0x7fd04c00b940 comp rx=0 tx=0).stop 2024-09-13T16:34:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.730+0000 7fd069e32640 1 -- 172.21.15.16:0/2190663327 shutdown_connections 2024-09-13T16:34:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.730+0000 7fd069e32640 1 --2- 172.21.15.16:0/2190663327 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd064105ab0 0x7fd0640fed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.730+0000 7fd069e32640 1 --2- 172.21.15.16:0/2190663327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd064103180 0x7fd0640fe800 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.730+0000 7fd069e32640 1 --2- 172.21.15.16:0/2190663327 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fd0500a1090 0x7fd0500a3550 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.730+0000 7fd069e32640 1 --2- 172.21.15.16:0/2190663327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd064100850 0x7fd0641001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:45.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.730+0000 7fd069e32640 1 -- 172.21.15.16:0/2190663327 >> 172.21.15.16:0/2190663327 conn(0x7fd0640fa430 msgr2=0x7fd0641011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:45.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.730+0000 7fd069e32640 1 -- 172.21.15.16:0/2190663327 shutdown_connections 2024-09-13T16:34:45.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.730+0000 7fd069e32640 1 -- 172.21.15.16:0/2190663327 wait complete. 2024-09-13T16:34:45.742 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:24:22.981019+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T16:28:37.746027+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T16:28:38.602731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T16:28:41.633322+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T16:34:45.744 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:34:45.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.848+0000 7f7f7fca3640 1 -- 172.21.15.16:0/1004543238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f78105a70 msgr2=0x7f7f78107ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:45.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.848+0000 7f7f7fca3640 1 --2- 172.21.15.16:0/1004543238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f78105a70 0x7f7f78107ee0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7f70009ca0 tx=0x7f7f7002a8c0 comp rx=0 tx=0).stop 2024-09-13T16:34:45.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.848+0000 7f7f7fca3640 1 -- 172.21.15.16:0/1004543238 shutdown_connections 2024-09-13T16:34:45.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.848+0000 7f7f7fca3640 1 --2- 172.21.15.16:0/1004543238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f78105a70 0x7f7f78107ee0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:45.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.848+0000 7f7f7fca3640 1 --2- 172.21.15.16:0/1004543238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f78103140 0x7f7f78105530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:45.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.848+0000 7f7f7fca3640 1 --2- 172.21.15.16:0/1004543238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f78100810 0x7f7f78102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:45.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.848+0000 7f7f7fca3640 1 -- 172.21.15.16:0/1004543238 >> 172.21.15.16:0/1004543238 conn(0x7f7f780fa3f0 msgr2=0x7f7f780fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:45.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.848+0000 7f7f7fca3640 1 -- 172.21.15.16:0/1004543238 shutdown_connections 2024-09-13T16:34:45.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.848+0000 7f7f7fca3640 1 -- 172.21.15.16:0/1004543238 wait complete. 2024-09-13T16:34:45.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.849+0000 7f7f7fca3640 1 Processor -- start 2024-09-13T16:34:45.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.849+0000 7f7f7fca3640 1 -- start start 2024-09-13T16:34:45.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.849+0000 7f7f7fca3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f78100810 0x7f7f78195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:45.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.849+0000 7f7f7fca3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f78103140 0x7f7f781961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:45.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.849+0000 7f7f7fca3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f78105a70 0x7f7f7819d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:45.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.849+0000 7f7f7fca3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7819d790 con 0x7f7f78100810 2024-09-13T16:34:45.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.849+0000 7f7f7fca3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7819d900 con 0x7f7f78105a70 2024-09-13T16:34:45.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.849+0000 7f7f7fca3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7819dac0 con 0x7f7f78103140 2024-09-13T16:34:45.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.849+0000 7f7f7da3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f78100810 0x7f7f78195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:45.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.849+0000 7f7f7da3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f78100810 0x7f7f78195c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42252/0 (socket says 172.21.15.16:42252) 2024-09-13T16:34:45.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.849+0000 7f7f7da3f640 1 -- 172.21.15.16:0/2689136211 learned_addr learned my addr 172.21.15.16:0/2689136211 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:34:45.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.850+0000 7f7f7e240640 1 --2- 172.21.15.16:0/2689136211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f78105a70 0x7f7f7819d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:45.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.850+0000 7f7f7d23e640 1 --2- 172.21.15.16:0/2689136211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f78103140 0x7f7f781961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:45.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.850+0000 7f7f7da3f640 1 -- 172.21.15.16:0/2689136211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f78103140 msgr2=0x7f7f781961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:45.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.850+0000 7f7f7da3f640 1 --2- 172.21.15.16:0/2689136211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f78103140 0x7f7f781961c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:45.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.850+0000 7f7f7da3f640 1 -- 172.21.15.16:0/2689136211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f78105a70 msgr2=0x7f7f7819d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:45.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.850+0000 7f7f7da3f640 1 --2- 172.21.15.16:0/2689136211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f78105a70 0x7f7f7819d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:45.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.850+0000 7f7f7da3f640 1 -- 172.21.15.16:0/2689136211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f70009900 con 0x7f7f78100810 2024-09-13T16:34:45.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.850+0000 7f7f7d23e640 1 --2- 172.21.15.16:0/2689136211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f78103140 0x7f7f781961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:34:45.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.850+0000 7f7f7da3f640 1 --2- 172.21.15.16:0/2689136211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f78100810 0x7f7f78195c80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00e9f0 tx=0x7f7f6c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:45.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.850+0000 7f7f6affd640 1 -- 172.21.15.16:0/2689136211 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c009700 con 0x7f7f78100810 2024-09-13T16:34:45.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.850+0000 7f7f6affd640 1 -- 172.21.15.16:0/2689136211 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7f6c00cd80 con 0x7f7f78100810 2024-09-13T16:34:45.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.850+0000 7f7f7fca3640 1 -- 172.21.15.16:0/2689136211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f7819dda0 con 0x7f7f78100810 2024-09-13T16:34:45.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.850+0000 7f7f6affd640 1 -- 172.21.15.16:0/2689136211 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c0187e0 con 0x7f7f78100810 2024-09-13T16:34:45.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.850+0000 7f7f7fca3640 1 -- 172.21.15.16:0/2689136211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f7819e1f0 con 0x7f7f78100810 2024-09-13T16:34:45.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.851+0000 7f7f7fca3640 1 -- 172.21.15.16:0/2689136211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f780fbee0 con 0x7f7f78100810 2024-09-13T16:34:45.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.852+0000 7f7f6affd640 1 -- 172.21.15.16:0/2689136211 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7f6c010430 con 0x7f7f78100810 2024-09-13T16:34:45.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.853+0000 7f7f6affd640 1 --2- 172.21.15.16:0/2689136211 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7f640a1090 0x7f7f640a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:45.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.853+0000 7f7f7d23e640 1 --2- 172.21.15.16:0/2689136211 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7f640a1090 0x7f7f640a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:45.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.853+0000 7f7f6affd640 1 -- 172.21.15.16:0/2689136211 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f7f6c014070 con 0x7f7f78100810 2024-09-13T16:34:45.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.853+0000 7f7f7d23e640 1 --2- 172.21.15.16:0/2689136211 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7f640a1090 0x7f7f640a3550 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7f78197180 tx=0x7f7f74005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:45.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:45.857+0000 7f7f6affd640 1 -- 172.21.15.16:0/2689136211 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7f6c052e20 con 0x7f7f78100810 2024-09-13T16:34:46.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.028+0000 7f7f7fca3640 1 -- 172.21.15.16:0/2689136211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7f7802c480 con 0x7f7f78100810 2024-09-13T16:34:46.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.028+0000 7f7f6affd640 1 -- 172.21.15.16:0/2689136211 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7f6c0ce020 con 0x7f7f78100810 2024-09-13T16:34:46.029 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-13T16:34:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.030+0000 7f7f7fca3640 1 -- 172.21.15.16:0/2689136211 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7f640a1090 msgr2=0x7f7f640a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.030+0000 7f7f7fca3640 1 --2- 172.21.15.16:0/2689136211 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7f640a1090 0x7f7f640a3550 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7f78197180 tx=0x7f7f74005e50 comp rx=0 tx=0).stop 2024-09-13T16:34:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.030+0000 7f7f7fca3640 1 -- 172.21.15.16:0/2689136211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f78100810 msgr2=0x7f7f78195c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.030+0000 7f7f7fca3640 1 --2- 172.21.15.16:0/2689136211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f78100810 0x7f7f78195c80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00e9f0 tx=0x7f7f6c00eeb0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.031+0000 7f7f7fca3640 1 -- 172.21.15.16:0/2689136211 shutdown_connections 2024-09-13T16:34:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.031+0000 7f7f7fca3640 1 --2- 172.21.15.16:0/2689136211 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7f640a1090 0x7f7f640a3550 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.031+0000 7f7f7fca3640 1 --2- 172.21.15.16:0/2689136211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f78105a70 0x7f7f7819d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.031+0000 7f7f7fca3640 1 --2- 172.21.15.16:0/2689136211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f78103140 0x7f7f781961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.031+0000 7f7f7fca3640 1 --2- 172.21.15.16:0/2689136211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f78100810 0x7f7f78195c80 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.031+0000 7f7f7fca3640 1 -- 172.21.15.16:0/2689136211 >> 172.21.15.16:0/2689136211 conn(0x7f7f780fa3f0 msgr2=0x7f7f78101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.031+0000 7f7f7fca3640 1 -- 172.21.15.16:0/2689136211 shutdown_connections 2024-09-13T16:34:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.031+0000 7f7f7fca3640 1 -- 172.21.15.16:0/2689136211 wait complete. 2024-09-13T16:34:46.043 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T16:34:46.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.147+0000 7f5f33419640 1 -- 172.21.15.16:0/3829721894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f2c105b10 msgr2=0x7f5f2c107f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:46.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.147+0000 7f5f33419640 1 --2- 172.21.15.16:0/3829721894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f2c105b10 0x7f5f2c107f00 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009ca0 tx=0x7f5f2402a8c0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.147+0000 7f5f33419640 1 -- 172.21.15.16:0/3829721894 shutdown_connections 2024-09-13T16:34:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.147+0000 7f5f33419640 1 --2- 172.21.15.16:0/3829721894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f2c105b10 0x7f5f2c107f00 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.147+0000 7f5f33419640 1 --2- 172.21.15.16:0/3829721894 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f2c1031e0 0x7f5f2c1055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.147+0000 7f5f33419640 1 --2- 172.21.15.16:0/3829721894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f2c1008b0 0x7f5f2c102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.147+0000 7f5f33419640 1 -- 172.21.15.16:0/3829721894 >> 172.21.15.16:0/3829721894 conn(0x7f5f2c0fa3d0 msgr2=0x7f5f2c0fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.147+0000 7f5f33419640 1 -- 172.21.15.16:0/3829721894 shutdown_connections 2024-09-13T16:34:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.147+0000 7f5f33419640 1 -- 172.21.15.16:0/3829721894 wait complete. 2024-09-13T16:34:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.148+0000 7f5f33419640 1 Processor -- start 2024-09-13T16:34:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.148+0000 7f5f33419640 1 -- start start 2024-09-13T16:34:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.148+0000 7f5f33419640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f2c1008b0 0x7f5f2c100220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:46.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.148+0000 7f5f33419640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f2c1031e0 0x7f5f2c0fe860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:46.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.148+0000 7f5f33419640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f2c105b10 0x7f5f2c0feda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:46.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.148+0000 7f5f33419640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c0ff500 con 0x7f5f2c105b10 2024-09-13T16:34:46.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.148+0000 7f5f33419640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c0ff670 con 0x7f5f2c1008b0 2024-09-13T16:34:46.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.148+0000 7f5f33419640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c19fd30 con 0x7f5f2c1031e0 2024-09-13T16:34:46.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.148+0000 7f5f319b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f2c105b10 0x7f5f2c0feda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:46.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.148+0000 7f5f319b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f2c105b10 0x7f5f2c0feda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42276/0 (socket says 172.21.15.16:42276) 2024-09-13T16:34:46.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.148+0000 7f5f319b6640 1 -- 172.21.15.16:0/2789931431 learned_addr learned my addr 172.21.15.16:0/2789931431 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:34:46.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.148+0000 7f5f311b5640 1 --2- 172.21.15.16:0/2789931431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f2c1008b0 0x7f5f2c100220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:46.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.148+0000 7f5f309b4640 1 --2- 172.21.15.16:0/2789931431 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f2c1031e0 0x7f5f2c0fe860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:46.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.149+0000 7f5f319b6640 1 -- 172.21.15.16:0/2789931431 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f2c1031e0 msgr2=0x7f5f2c0fe860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:46.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.149+0000 7f5f319b6640 1 --2- 172.21.15.16:0/2789931431 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f2c1031e0 0x7f5f2c0fe860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.149+0000 7f5f319b6640 1 -- 172.21.15.16:0/2789931431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f2c1008b0 msgr2=0x7f5f2c100220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.149+0000 7f5f319b6640 1 --2- 172.21.15.16:0/2789931431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f2c1008b0 0x7f5f2c100220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.149+0000 7f5f319b6640 1 -- 172.21.15.16:0/2789931431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f24009900 con 0x7f5f2c105b10 2024-09-13T16:34:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.149+0000 7f5f311b5640 1 --2- 172.21.15.16:0/2789931431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f2c1008b0 0x7f5f2c100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:34:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.149+0000 7f5f319b6640 1 --2- 172.21.15.16:0/2789931431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f2c105b10 0x7f5f2c0feda0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5f2402ade0 tx=0x7f5f24003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.149+0000 7f5f227fc640 1 -- 172.21.15.16:0/2789931431 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f240307c0 con 0x7f5f2c105b10 2024-09-13T16:34:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.149+0000 7f5f227fc640 1 -- 172.21.15.16:0/2789931431 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f24040070 con 0x7f5f2c105b10 2024-09-13T16:34:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.149+0000 7f5f33419640 1 -- 172.21.15.16:0/2789931431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f2c19ff80 con 0x7f5f2c105b10 2024-09-13T16:34:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.149+0000 7f5f227fc640 1 -- 172.21.15.16:0/2789931431 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f24004460 con 0x7f5f2c105b10 2024-09-13T16:34:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.149+0000 7f5f33419640 1 -- 172.21.15.16:0/2789931431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f2c1a0400 con 0x7f5f2c105b10 2024-09-13T16:34:46.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.150+0000 7f5f33419640 1 -- 172.21.15.16:0/2789931431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f2c0fbec0 con 0x7f5f2c105b10 2024-09-13T16:34:46.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.151+0000 7f5f227fc640 1 -- 172.21.15.16:0/2789931431 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f5f24042030 con 0x7f5f2c105b10 2024-09-13T16:34:46.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.152+0000 7f5f227fc640 1 --2- 172.21.15.16:0/2789931431 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f5f180a1090 0x7f5f180a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:46.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.152+0000 7f5f311b5640 1 --2- 172.21.15.16:0/2789931431 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f5f180a1090 0x7f5f180a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:46.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.152+0000 7f5f227fc640 1 -- 172.21.15.16:0/2789931431 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f5f240ee150 con 0x7f5f2c105b10 2024-09-13T16:34:46.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.152+0000 7f5f311b5640 1 --2- 172.21.15.16:0/2789931431 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f5f180a1090 0x7f5f180a3550 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5f28009fc0 tx=0x7f5f28005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:46.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.156+0000 7f5f227fc640 1 -- 172.21.15.16:0/2789931431 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f5f240b7590 con 0x7f5f2c105b10 2024-09-13T16:34:46.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.326+0000 7f5f33419640 1 -- 172.21.15.16:0/2789931431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5f2c02c3e0 con 0x7f5f2c105b10 2024-09-13T16:34:46.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.326+0000 7f5f227fc640 1 -- 172.21.15.16:0/2789931431 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5f240b5e60 con 0x7f5f2c105b10 2024-09-13T16:34:46.328 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-13T16:34:46.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.328+0000 7f5f33419640 1 -- 172.21.15.16:0/2789931431 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f5f180a1090 msgr2=0x7f5f180a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:46.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.328+0000 7f5f33419640 1 --2- 172.21.15.16:0/2789931431 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f5f180a1090 0x7f5f180a3550 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5f28009fc0 tx=0x7f5f28005f90 comp rx=0 tx=0).stop 2024-09-13T16:34:46.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.328+0000 7f5f33419640 1 -- 172.21.15.16:0/2789931431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f2c105b10 msgr2=0x7f5f2c0feda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:46.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.328+0000 7f5f33419640 1 --2- 172.21.15.16:0/2789931431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f2c105b10 0x7f5f2c0feda0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5f2402ade0 tx=0x7f5f24003c20 comp rx=0 tx=0).stop 2024-09-13T16:34:46.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.328+0000 7f5f33419640 1 -- 172.21.15.16:0/2789931431 shutdown_connections 2024-09-13T16:34:46.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.328+0000 7f5f33419640 1 --2- 172.21.15.16:0/2789931431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f2c105b10 0x7f5f2c0feda0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.328+0000 7f5f33419640 1 --2- 172.21.15.16:0/2789931431 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f2c1031e0 0x7f5f2c0fe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.328+0000 7f5f33419640 1 --2- 172.21.15.16:0/2789931431 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f5f180a1090 0x7f5f180a3550 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.328+0000 7f5f33419640 1 --2- 172.21.15.16:0/2789931431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f2c1008b0 0x7f5f2c100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.328+0000 7f5f33419640 1 -- 172.21.15.16:0/2789931431 >> 172.21.15.16:0/2789931431 conn(0x7f5f2c0fa3d0 msgr2=0x7f5f2c101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:46.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.328+0000 7f5f33419640 1 -- 172.21.15.16:0/2789931431 shutdown_connections 2024-09-13T16:34:46.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.329+0000 7f5f33419640 1 -- 172.21.15.16:0/2789931431 wait complete. 2024-09-13T16:34:46.339 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:46 smithi016 ceph-mon[31187]: pgmap v463: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T16:34:46.339 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:46 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/893011296' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T16:34:46.339 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:46 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1515141357' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T16:34:46.339 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:46 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2190663327' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:34:46.340 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T16:34:46.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:46 smithi107 ceph-mon[34139]: pgmap v463: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T16:34:46.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:46 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/893011296' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T16:34:46.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:46 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1515141357' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T16:34:46.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:46 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/2190663327' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:34:46.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:46 smithi111 ceph-mon[34600]: pgmap v463: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T16:34:46.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:46 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/893011296' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T16:34:46.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:46 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1515141357' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T16:34:46.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:46 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/2190663327' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:34:46.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.444+0000 7f11dd59d640 1 -- 172.21.15.16:0/1866693877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11d80fdfc0 msgr2=0x7f11d80fe3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:46.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.444+0000 7f11dd59d640 1 --2- 172.21.15.16:0/1866693877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11d80fdfc0 0x7f11d80fe3c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f11c8009900 tx=0x7f11c802a410 comp rx=0 tx=0).stop 2024-09-13T16:34:46.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.444+0000 7f11dd59d640 1 -- 172.21.15.16:0/1866693877 shutdown_connections 2024-09-13T16:34:46.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.444+0000 7f11dd59d640 1 --2- 172.21.15.16:0/1866693877 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11d81019f0 0x7f11d8108280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.444+0000 7f11dd59d640 1 --2- 172.21.15.16:0/1866693877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11d8101030 0x7f11d81014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.444+0000 7f11dd59d640 1 --2- 172.21.15.16:0/1866693877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11d80fdfc0 0x7f11d80fe3c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.444+0000 7f11dd59d640 1 -- 172.21.15.16:0/1866693877 >> 172.21.15.16:0/1866693877 conn(0x7f11d80f9aa0 msgr2=0x7f11d80fbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:46.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.444+0000 7f11dd59d640 1 -- 172.21.15.16:0/1866693877 shutdown_connections 2024-09-13T16:34:46.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.444+0000 7f11dd59d640 1 -- 172.21.15.16:0/1866693877 wait complete. 2024-09-13T16:34:46.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.445+0000 7f11dd59d640 1 Processor -- start 2024-09-13T16:34:46.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.445+0000 7f11dd59d640 1 -- start start 2024-09-13T16:34:46.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.445+0000 7f11dd59d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11d80fdfc0 0x7f11d8195b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:46.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.445+0000 7f11dd59d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11d8101030 0x7f11d81960b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:46.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.445+0000 7f11dd59d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11d81019f0 0x7f11d819d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:46.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.445+0000 7f11dd59d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d819d660 con 0x7f11d81019f0 2024-09-13T16:34:46.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.445+0000 7f11dd59d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d819d7d0 con 0x7f11d80fdfc0 2024-09-13T16:34:46.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.445+0000 7f11dd59d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d819d990 con 0x7f11d8101030 2024-09-13T16:34:46.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.445+0000 7f11dcd9c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11d81019f0 0x7f11d819d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:46.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.445+0000 7f11dcd9c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11d81019f0 0x7f11d819d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42292/0 (socket says 172.21.15.16:42292) 2024-09-13T16:34:46.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.445+0000 7f11dcd9c640 1 -- 172.21.15.16:0/3469966867 learned_addr learned my addr 172.21.15.16:0/3469966867 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:34:46.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.446+0000 7f11d7fff640 1 --2- 172.21.15.16:0/3469966867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11d80fdfc0 0x7f11d8195b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:46.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.446+0000 7f11cffff640 1 --2- 172.21.15.16:0/3469966867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11d8101030 0x7f11d81960b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:46.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.446+0000 7f11dcd9c640 1 -- 172.21.15.16:0/3469966867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11d8101030 msgr2=0x7f11d81960b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:46.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.446+0000 7f11dcd9c640 1 --2- 172.21.15.16:0/3469966867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11d8101030 0x7f11d81960b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.446+0000 7f11dcd9c640 1 -- 172.21.15.16:0/3469966867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11d80fdfc0 msgr2=0x7f11d8195b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:46.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.446+0000 7f11dcd9c640 1 --2- 172.21.15.16:0/3469966867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11d80fdfc0 0x7f11d8195b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.446+0000 7f11dcd9c640 1 -- 172.21.15.16:0/3469966867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c8009560 con 0x7f11d81019f0 2024-09-13T16:34:46.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.446+0000 7f11dcd9c640 1 --2- 172.21.15.16:0/3469966867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11d81019f0 0x7f11d819d120 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f11d000eaa0 tx=0x7f11d000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:46.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.446+0000 7f11d5ffb640 1 -- 172.21.15.16:0/3469966867 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d000cdb0 con 0x7f11d81019f0 2024-09-13T16:34:46.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.446+0000 7f11d5ffb640 1 -- 172.21.15.16:0/3469966867 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f11d0005020 con 0x7f11d81019f0 2024-09-13T16:34:46.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.446+0000 7f11dd59d640 1 -- 172.21.15.16:0/3469966867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d819dc70 con 0x7f11d81019f0 2024-09-13T16:34:46.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.446+0000 7f11d5ffb640 1 -- 172.21.15.16:0/3469966867 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d00116d0 con 0x7f11d81019f0 2024-09-13T16:34:46.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.446+0000 7f11dd59d640 1 -- 172.21.15.16:0/3469966867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d819e1d0 con 0x7f11d81019f0 2024-09-13T16:34:46.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.447+0000 7f11dd59d640 1 -- 172.21.15.16:0/3469966867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11d8107370 con 0x7f11d81019f0 2024-09-13T16:34:46.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.448+0000 7f11d5ffb640 1 -- 172.21.15.16:0/3469966867 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f11d0005190 con 0x7f11d81019f0 2024-09-13T16:34:46.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.449+0000 7f11d5ffb640 1 --2- 172.21.15.16:0/3469966867 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f11c40a1110 0x7f11c40a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:46.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.449+0000 7f11d7fff640 1 --2- 172.21.15.16:0/3469966867 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f11c40a1110 0x7f11c40a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:46.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.449+0000 7f11d5ffb640 1 -- 172.21.15.16:0/3469966867 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f11d00cce10 con 0x7f11d81019f0 2024-09-13T16:34:46.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.449+0000 7f11d7fff640 1 --2- 172.21.15.16:0/3469966867 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f11c40a1110 0x7f11c40a35d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f11c8006010 tx=0x7f11c8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:46.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.453+0000 7f11d5ffb640 1 -- 172.21.15.16:0/3469966867 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f11d0096250 con 0x7f11d81019f0 2024-09-13T16:34:46.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.625+0000 7f11dd59d640 1 -- 172.21.15.16:0/3469966867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f11d8106de0 con 0x7f11d81019f0 2024-09-13T16:34:46.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.626+0000 7f11d5ffb640 1 -- 172.21.15.16:0/3469966867 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f11d001f020 con 0x7f11d81019f0 2024-09-13T16:34:46.627 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-09-13T16:34:46.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.628+0000 7f11dd59d640 1 -- 172.21.15.16:0/3469966867 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f11c40a1110 msgr2=0x7f11c40a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:46.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.628+0000 7f11dd59d640 1 --2- 172.21.15.16:0/3469966867 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f11c40a1110 0x7f11c40a35d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f11c8006010 tx=0x7f11c8005a20 comp rx=0 tx=0).stop 2024-09-13T16:34:46.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.628+0000 7f11dd59d640 1 -- 172.21.15.16:0/3469966867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11d81019f0 msgr2=0x7f11d819d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:46.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.628+0000 7f11dd59d640 1 --2- 172.21.15.16:0/3469966867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11d81019f0 0x7f11d819d120 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f11d000eaa0 tx=0x7f11d000ef60 comp rx=0 tx=0).stop 2024-09-13T16:34:46.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.628+0000 7f11dd59d640 1 -- 172.21.15.16:0/3469966867 shutdown_connections 2024-09-13T16:34:46.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.628+0000 7f11dd59d640 1 --2- 172.21.15.16:0/3469966867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11d81019f0 0x7f11d819d120 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.628+0000 7f11dd59d640 1 --2- 172.21.15.16:0/3469966867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11d8101030 0x7f11d81960b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.628+0000 7f11dd59d640 1 --2- 172.21.15.16:0/3469966867 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f11c40a1110 0x7f11c40a35d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.628+0000 7f11dd59d640 1 --2- 172.21.15.16:0/3469966867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11d80fdfc0 0x7f11d8195b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.628+0000 7f11dd59d640 1 -- 172.21.15.16:0/3469966867 >> 172.21.15.16:0/3469966867 conn(0x7f11d80f9aa0 msgr2=0x7f11d8104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:46.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.628+0000 7f11dd59d640 1 -- 172.21.15.16:0/3469966867 shutdown_connections 2024-09-13T16:34:46.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.628+0000 7f11dd59d640 1 -- 172.21.15.16:0/3469966867 wait complete. 2024-09-13T16:34:46.640 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T16:34:46.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.745+0000 7f1905b5e640 1 -- 172.21.15.16:0/1691367402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1900103110 msgr2=0x7f1900105500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:46.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.745+0000 7f1905b5e640 1 --2- 172.21.15.16:0/1691367402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1900103110 0x7f1900105500 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f18e8009900 tx=0x7f18e802a410 comp rx=0 tx=0).stop 2024-09-13T16:34:46.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.745+0000 7f1905b5e640 1 -- 172.21.15.16:0/1691367402 shutdown_connections 2024-09-13T16:34:46.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.745+0000 7f1905b5e640 1 --2- 172.21.15.16:0/1691367402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1900105a40 0x7f1900107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.745+0000 7f1905b5e640 1 --2- 172.21.15.16:0/1691367402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1900103110 0x7f1900105500 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.745+0000 7f1905b5e640 1 --2- 172.21.15.16:0/1691367402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19001007e0 0x7f1900102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.745+0000 7f1905b5e640 1 -- 172.21.15.16:0/1691367402 >> 172.21.15.16:0/1691367402 conn(0x7f19000fa3e0 msgr2=0x7f19000fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:46.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.745+0000 7f1905b5e640 1 -- 172.21.15.16:0/1691367402 shutdown_connections 2024-09-13T16:34:46.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.745+0000 7f1905b5e640 1 -- 172.21.15.16:0/1691367402 wait complete. 2024-09-13T16:34:46.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.745+0000 7f1905b5e640 1 Processor -- start 2024-09-13T16:34:46.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.745+0000 7f1905b5e640 1 -- start start 2024-09-13T16:34:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.746+0000 7f1905b5e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19001007e0 0x7f1900195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.746+0000 7f1905b5e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1900103110 0x7f1900196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.746+0000 7f1905b5e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1900105a40 0x7f190019d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.746+0000 7f1905b5e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190019d800 con 0x7f1900105a40 2024-09-13T16:34:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.746+0000 7f1905b5e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190019d970 con 0x7f19001007e0 2024-09-13T16:34:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.746+0000 7f1905b5e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190019db30 con 0x7f1900103110 2024-09-13T16:34:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.746+0000 7f18fffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1900105a40 0x7f190019d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.746+0000 7f18fffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1900105a40 0x7f190019d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42318/0 (socket says 172.21.15.16:42318) 2024-09-13T16:34:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.746+0000 7f18fffff640 1 -- 172.21.15.16:0/2635339635 learned_addr learned my addr 172.21.15.16:0/2635339635 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:34:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.746+0000 7f18ff7fe640 1 --2- 172.21.15.16:0/2635339635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19001007e0 0x7f1900195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.746+0000 7f18f6ffd640 1 --2- 172.21.15.16:0/2635339635 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1900103110 0x7f1900196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.746+0000 7f18fffff640 1 -- 172.21.15.16:0/2635339635 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1900103110 msgr2=0x7f1900196260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.746+0000 7f18fffff640 1 --2- 172.21.15.16:0/2635339635 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1900103110 0x7f1900196260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.746+0000 7f18fffff640 1 -- 172.21.15.16:0/2635339635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19001007e0 msgr2=0x7f1900195d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.746+0000 7f18fffff640 1 --2- 172.21.15.16:0/2635339635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19001007e0 0x7f1900195d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.746+0000 7f18fffff640 1 -- 172.21.15.16:0/2635339635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e8009560 con 0x7f1900105a40 2024-09-13T16:34:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.746+0000 7f18ff7fe640 1 --2- 172.21.15.16:0/2635339635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19001007e0 0x7f1900195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:34:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.746+0000 7f18fffff640 1 --2- 172.21.15.16:0/2635339635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1900105a40 0x7f190019d290 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f18f800dac0 tx=0x7f18f800df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.746+0000 7f18fd7fa640 1 -- 172.21.15.16:0/2635339635 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f800be30 con 0x7f1900105a40 2024-09-13T16:34:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.746+0000 7f18fd7fa640 1 -- 172.21.15.16:0/2635339635 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f18f8003e00 con 0x7f1900105a40 2024-09-13T16:34:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.746+0000 7f1905b5e640 1 -- 172.21.15.16:0/2635339635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f190019de10 con 0x7f1900105a40 2024-09-13T16:34:46.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.747+0000 7f18fd7fa640 1 -- 172.21.15.16:0/2635339635 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f80106c0 con 0x7f1900105a40 2024-09-13T16:34:46.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.747+0000 7f1905b5e640 1 -- 172.21.15.16:0/2635339635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190019e230 con 0x7f1900105a40 2024-09-13T16:34:46.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.747+0000 7f1905b5e640 1 -- 172.21.15.16:0/2635339635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19001034f0 con 0x7f1900105a40 2024-09-13T16:34:46.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.749+0000 7f18fd7fa640 1 -- 172.21.15.16:0/2635339635 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f18f8003970 con 0x7f1900105a40 2024-09-13T16:34:46.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.749+0000 7f18fd7fa640 1 --2- 172.21.15.16:0/2635339635 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f18ec0a1110 0x7f18ec0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:46.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.749+0000 7f18fd7fa640 1 -- 172.21.15.16:0/2635339635 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f18f80cc610 con 0x7f1900105a40 2024-09-13T16:34:46.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.749+0000 7f18ff7fe640 1 --2- 172.21.15.16:0/2635339635 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f18ec0a1110 0x7f18ec0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:46.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.750+0000 7f18ff7fe640 1 --2- 172.21.15.16:0/2635339635 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f18ec0a1110 0x7f18ec0a35d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f18f0005d00 tx=0x7f18f0005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:46.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.754+0000 7f18fd7fa640 1 -- 172.21.15.16:0/2635339635 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f18f80959c0 con 0x7f1900105a40 2024-09-13T16:34:46.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.926+0000 7f1905b5e640 1 -- 172.21.15.16:0/2635339635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f190010b4e0 con 0x7f1900105a40 2024-09-13T16:34:46.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.926+0000 7f18fd7fa640 1 -- 172.21.15.16:0/2635339635 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f18f80978f0 con 0x7f1900105a40 2024-09-13T16:34:46.928 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-13T16:34:46.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.928+0000 7f1905b5e640 1 -- 172.21.15.16:0/2635339635 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f18ec0a1110 msgr2=0x7f18ec0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:46.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.928+0000 7f1905b5e640 1 --2- 172.21.15.16:0/2635339635 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f18ec0a1110 0x7f18ec0a35d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f18f0005d00 tx=0x7f18f0005c90 comp rx=0 tx=0).stop 2024-09-13T16:34:46.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.928+0000 7f1905b5e640 1 -- 172.21.15.16:0/2635339635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1900105a40 msgr2=0x7f190019d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:46.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.928+0000 7f1905b5e640 1 --2- 172.21.15.16:0/2635339635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1900105a40 0x7f190019d290 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f18f800dac0 tx=0x7f18f800df80 comp rx=0 tx=0).stop 2024-09-13T16:34:46.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.928+0000 7f1905b5e640 1 -- 172.21.15.16:0/2635339635 shutdown_connections 2024-09-13T16:34:46.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.928+0000 7f1905b5e640 1 --2- 172.21.15.16:0/2635339635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1900105a40 0x7f190019d290 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.928+0000 7f1905b5e640 1 --2- 172.21.15.16:0/2635339635 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1900103110 0x7f1900196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.928+0000 7f1905b5e640 1 --2- 172.21.15.16:0/2635339635 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f18ec0a1110 0x7f18ec0a35d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.928+0000 7f1905b5e640 1 --2- 172.21.15.16:0/2635339635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19001007e0 0x7f1900195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:46.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.928+0000 7f1905b5e640 1 -- 172.21.15.16:0/2635339635 >> 172.21.15.16:0/2635339635 conn(0x7f19000fa3e0 msgr2=0x7f19001012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:46.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.929+0000 7f1905b5e640 1 -- 172.21.15.16:0/2635339635 shutdown_connections 2024-09-13T16:34:46.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:46.929+0000 7f1905b5e640 1 -- 172.21.15.16:0/2635339635 wait complete. 2024-09-13T16:34:46.941 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T16:34:47.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.044+0000 7f407c9d4640 1 -- 172.21.15.16:0/1941296579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4078105af0 msgr2=0x7f4078107ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:47.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.044+0000 7f407c9d4640 1 --2- 172.21.15.16:0/1941296579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4078105af0 0x7f4078107ee0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4070009ca0 tx=0x7f407002a8c0 comp rx=0 tx=0).stop 2024-09-13T16:34:47.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.045+0000 7f407c9d4640 1 -- 172.21.15.16:0/1941296579 shutdown_connections 2024-09-13T16:34:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.045+0000 7f407c9d4640 1 --2- 172.21.15.16:0/1941296579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4078105af0 0x7f4078107ee0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.045+0000 7f407c9d4640 1 --2- 172.21.15.16:0/1941296579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40781031c0 0x7f40781055b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.045+0000 7f407c9d4640 1 --2- 172.21.15.16:0/1941296579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4078100890 0x7f4078102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.045+0000 7f407c9d4640 1 -- 172.21.15.16:0/1941296579 >> 172.21.15.16:0/1941296579 conn(0x7f40780fa410 msgr2=0x7f40780fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.045+0000 7f407c9d4640 1 -- 172.21.15.16:0/1941296579 shutdown_connections 2024-09-13T16:34:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.045+0000 7f407c9d4640 1 -- 172.21.15.16:0/1941296579 wait complete. 2024-09-13T16:34:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.045+0000 7f407c9d4640 1 Processor -- start 2024-09-13T16:34:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.045+0000 7f407c9d4640 1 -- start start 2024-09-13T16:34:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.046+0000 7f407c9d4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4078100890 0x7f4078195c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.046+0000 7f407c9d4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40781031c0 0x7f4078196180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.046+0000 7f407c9d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4078105af0 0x7f407819d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.046+0000 7f407c9d4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407819d720 con 0x7f4078105af0 2024-09-13T16:34:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.046+0000 7f407c9d4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407819d890 con 0x7f4078100890 2024-09-13T16:34:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.046+0000 7f407c9d4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407819da50 con 0x7f40781031c0 2024-09-13T16:34:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.046+0000 7f4076d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4078105af0 0x7f407819d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.046+0000 7f4076d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4078105af0 0x7f407819d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42340/0 (socket says 172.21.15.16:42340) 2024-09-13T16:34:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.046+0000 7f4076d9d640 1 -- 172.21.15.16:0/2566074376 learned_addr learned my addr 172.21.15.16:0/2566074376 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:34:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.046+0000 7f4075d9b640 1 --2- 172.21.15.16:0/2566074376 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40781031c0 0x7f4078196180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.046+0000 7f407659c640 1 --2- 172.21.15.16:0/2566074376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4078100890 0x7f4078195c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.046+0000 7f4076d9d640 1 -- 172.21.15.16:0/2566074376 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40781031c0 msgr2=0x7f4078196180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.046+0000 7f4076d9d640 1 --2- 172.21.15.16:0/2566074376 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40781031c0 0x7f4078196180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.046+0000 7f4076d9d640 1 -- 172.21.15.16:0/2566074376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4078100890 msgr2=0x7f4078195c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.046+0000 7f4076d9d640 1 --2- 172.21.15.16:0/2566074376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4078100890 0x7f4078195c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.046+0000 7f4076d9d640 1 -- 172.21.15.16:0/2566074376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4070009900 con 0x7f4078105af0 2024-09-13T16:34:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.046+0000 7f407659c640 1 --2- 172.21.15.16:0/2566074376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4078100890 0x7f4078195c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:34:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.046+0000 7f4076d9d640 1 --2- 172.21.15.16:0/2566074376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4078105af0 0x7f407819d1b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f407002ade0 tx=0x7f4070003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.046+0000 7f40677fe640 1 -- 172.21.15.16:0/2566074376 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40700317c0 con 0x7f4078105af0 2024-09-13T16:34:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.046+0000 7f40677fe640 1 -- 172.21.15.16:0/2566074376 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4070040070 con 0x7f4078105af0 2024-09-13T16:34:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.046+0000 7f407c9d4640 1 -- 172.21.15.16:0/2566074376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f407819dcd0 con 0x7f4078105af0 2024-09-13T16:34:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.047+0000 7f40677fe640 1 -- 172.21.15.16:0/2566074376 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4070004610 con 0x7f4078105af0 2024-09-13T16:34:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.047+0000 7f407c9d4640 1 -- 172.21.15.16:0/2566074376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f407819e180 con 0x7f4078105af0 2024-09-13T16:34:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.047+0000 7f407c9d4640 1 -- 172.21.15.16:0/2566074376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40780fbf00 con 0x7f4078105af0 2024-09-13T16:34:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.049+0000 7f40677fe640 1 -- 172.21.15.16:0/2566074376 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f4070042030 con 0x7f4078105af0 2024-09-13T16:34:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.050+0000 7f40677fe640 1 --2- 172.21.15.16:0/2566074376 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f40600a1110 0x7f40600a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.050+0000 7f407659c640 1 --2- 172.21.15.16:0/2566074376 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f40600a1110 0x7f40600a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.050+0000 7f40677fe640 1 -- 172.21.15.16:0/2566074376 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f40700ee620 con 0x7f4078105af0 2024-09-13T16:34:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.050+0000 7f407659c640 1 --2- 172.21.15.16:0/2566074376 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f40600a1110 0x7f40600a35d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4068005fd0 tx=0x7f4068005ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.053+0000 7f40677fe640 1 -- 172.21.15.16:0/2566074376 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f40700b79d0 con 0x7f4078105af0 2024-09-13T16:34:47.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:47 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/2689136211' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:34:47.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:47 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/2789931431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T16:34:47.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:47 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3469966867' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T16:34:47.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:47 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/2635339635' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T16:34:47.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.243+0000 7f407c9d4640 1 -- 172.21.15.16:0/2566074376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f407810b520 con 0x7f4078105af0 2024-09-13T16:34:47.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.243+0000 7f40677fe640 1 -- 172.21.15.16:0/2566074376 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v93) v1 ==== 81+0+589 (secure 0 0 0) 0x7f40700b9900 con 0x7f4078105af0 2024-09-13T16:34:47.244 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2098378667 2024-09-14T16:22:04.485277+0000 2024-09-13T16:34:47.244 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3536229016 2024-09-14T16:22:04.485277+0000 2024-09-13T16:34:47.244 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/1756940569 2024-09-14T16:22:04.485277+0000 2024-09-13T16:34:47.244 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2684687547 2024-09-14T16:22:04.485277+0000 2024-09-13T16:34:47.244 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/1756940569 2024-09-14T16:22:04.485277+0000 2024-09-13T16:34:47.245 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/368410612 2024-09-14T16:21:40.126459+0000 2024-09-13T16:34:47.245 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/12158160 2024-09-14T16:21:40.126459+0000 2024-09-13T16:34:47.245 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/1147765163 2024-09-14T16:21:40.126459+0000 2024-09-13T16:34:47.245 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/1147765163 2024-09-14T16:21:40.126459+0000 2024-09-13T16:34:47.245 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3754423379 2024-09-14T16:21:40.126459+0000 2024-09-13T16:34:47.245 INFO:teuthology.orchestra.run.smithi016.stderr:listed 10 entries 2024-09-13T16:34:47.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.245+0000 7f407c9d4640 1 -- 172.21.15.16:0/2566074376 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f40600a1110 msgr2=0x7f40600a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:47.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.245+0000 7f407c9d4640 1 --2- 172.21.15.16:0/2566074376 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f40600a1110 0x7f40600a35d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4068005fd0 tx=0x7f4068005ea0 comp rx=0 tx=0).stop 2024-09-13T16:34:47.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.245+0000 7f407c9d4640 1 -- 172.21.15.16:0/2566074376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4078105af0 msgr2=0x7f407819d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:47.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.245+0000 7f407c9d4640 1 --2- 172.21.15.16:0/2566074376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4078105af0 0x7f407819d1b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f407002ade0 tx=0x7f4070003ba0 comp rx=0 tx=0).stop 2024-09-13T16:34:47.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.245+0000 7f407c9d4640 1 -- 172.21.15.16:0/2566074376 shutdown_connections 2024-09-13T16:34:47.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.245+0000 7f407c9d4640 1 --2- 172.21.15.16:0/2566074376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4078105af0 0x7f407819d1b0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:47.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.245+0000 7f407c9d4640 1 --2- 172.21.15.16:0/2566074376 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40781031c0 0x7f4078196180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:47.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.245+0000 7f407c9d4640 1 --2- 172.21.15.16:0/2566074376 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f40600a1110 0x7f40600a35d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:47.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.245+0000 7f407c9d4640 1 --2- 172.21.15.16:0/2566074376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4078100890 0x7f4078195c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:47.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.245+0000 7f407c9d4640 1 -- 172.21.15.16:0/2566074376 >> 172.21.15.16:0/2566074376 conn(0x7f40780fa410 msgr2=0x7f4078101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:47.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.245+0000 7f407c9d4640 1 -- 172.21.15.16:0/2566074376 shutdown_connections 2024-09-13T16:34:47.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:47.245+0000 7f407c9d4640 1 -- 172.21.15.16:0/2566074376 wait complete. 2024-09-13T16:34:47.257 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:47 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2689136211' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:34:47.257 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:47 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2789931431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T16:34:47.257 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:47 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3469966867' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T16:34:47.257 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:47 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2635339635' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T16:34:47.258 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2024-09-13T16:34:47.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:34:47.258 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-13T16:34:47.282 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[85149]: fuse finished with error 0 and tester_r 0 2024-09-13T16:34:47.284 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:34:47.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:47 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/2689136211' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T16:34:47.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:47 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/2789931431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T16:34:47.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:47 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3469966867' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T16:34:47.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:47 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/2635339635' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T16:34:48.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:48 smithi016 ceph-mon[31187]: pgmap v464: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-13T16:34:48.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:48 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2566074376' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T16:34:48.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:48 smithi107 ceph-mon[34139]: pgmap v464: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-13T16:34:48.946 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:48 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/2566074376' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T16:34:48.947 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:48 smithi111 ceph-mon[34600]: pgmap v464: 129 pgs: 129 active+clean; 1019 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-13T16:34:48.947 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:48 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/2566074376' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T16:34:50.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:50 smithi107 ceph-mon[34139]: pgmap v465: 129 pgs: 129 active+clean; 1020 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-13T16:34:50.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:50 smithi016 ceph-mon[31187]: pgmap v465: 129 pgs: 129 active+clean; 1020 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-13T16:34:50.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:50 smithi111 ceph-mon[34600]: pgmap v465: 129 pgs: 129 active+clean; 1020 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-13T16:34:52.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:52 smithi107 ceph-mon[34139]: pgmap v466: 129 pgs: 129 active+clean; 1020 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-13T16:34:52.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:52 smithi016 ceph-mon[31187]: pgmap v466: 129 pgs: 129 active+clean; 1020 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-13T16:34:52.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:52 smithi111 ceph-mon[34600]: pgmap v466: 129 pgs: 129 active+clean; 1020 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-13T16:34:53.385 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-13T16:34:53.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:34:53.386 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-13T16:34:53.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:34:53.407 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-13T16:34:53.466 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T16:34:53.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:34:53.466 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T16:34:53.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:34:53.539 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-09-13T16:34:53.653 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-13T16:34:53.674 INFO:tasks.cephadm:Teardown begin 2024-09-13T16:34:53.674 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T16:34:53.674 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:34:53.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.783+0000 7fe49302b640 1 -- 172.21.15.16:0/3260609432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe48c0fecd0 msgr2=0x7fe48c0ff0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:53.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.783+0000 7fe49302b640 1 --2- 172.21.15.16:0/3260609432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe48c0fecd0 0x7fe48c0ff0d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe480009900 tx=0x7fe48002a410 comp rx=0 tx=0).stop 2024-09-13T16:34:53.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.783+0000 7fe49302b640 1 -- 172.21.15.16:0/3260609432 shutdown_connections 2024-09-13T16:34:53.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.783+0000 7fe49302b640 1 --2- 172.21.15.16:0/3260609432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe48c100880 0x7fe48c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:53.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.783+0000 7fe49302b640 1 --2- 172.21.15.16:0/3260609432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe48c0ffec0 0x7fe48c100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:53.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.783+0000 7fe49302b640 1 --2- 172.21.15.16:0/3260609432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe48c0fecd0 0x7fe48c0ff0d0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.783+0000 7fe49302b640 1 -- 172.21.15.16:0/3260609432 >> 172.21.15.16:0/3260609432 conn(0x7fe48c0fa430 msgr2=0x7fe48c0fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.783+0000 7fe49302b640 1 -- 172.21.15.16:0/3260609432 shutdown_connections 2024-09-13T16:34:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.783+0000 7fe49302b640 1 -- 172.21.15.16:0/3260609432 wait complete. 2024-09-13T16:34:53.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.783+0000 7fe49302b640 1 Processor -- start 2024-09-13T16:34:53.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.783+0000 7fe49302b640 1 -- start start 2024-09-13T16:34:53.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.784+0000 7fe49302b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe48c0fecd0 0x7fe48c069160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:53.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.784+0000 7fe49302b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe48c0ffec0 0x7fe48c19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:53.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.784+0000 7fe49302b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe48c100880 0x7fe48c19cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:53.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.784+0000 7fe49302b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48c19d7d0 con 0x7fe48c100880 2024-09-13T16:34:53.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.784+0000 7fe49302b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48c19d940 con 0x7fe48c0fecd0 2024-09-13T16:34:53.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.784+0000 7fe49302b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48c19db00 con 0x7fe48c0ffec0 2024-09-13T16:34:53.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.784+0000 7fe4915c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe48c100880 0x7fe48c19cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:53.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.784+0000 7fe4915c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe48c100880 0x7fe48c19cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52576/0 (socket says 172.21.15.16:52576) 2024-09-13T16:34:53.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.784+0000 7fe4915c8640 1 -- 172.21.15.16:0/229492435 learned_addr learned my addr 172.21.15.16:0/229492435 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:34:53.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.784+0000 7fe490dc7640 1 --2- 172.21.15.16:0/229492435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe48c0fecd0 0x7fe48c069160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:53.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.784+0000 7fe48bfff640 1 --2- 172.21.15.16:0/229492435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe48c0ffec0 0x7fe48c19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:53.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.784+0000 7fe4915c8640 1 -- 172.21.15.16:0/229492435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe48c0ffec0 msgr2=0x7fe48c19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:53.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.784+0000 7fe4915c8640 1 --2- 172.21.15.16:0/229492435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe48c0ffec0 0x7fe48c19a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:53.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.784+0000 7fe4915c8640 1 -- 172.21.15.16:0/229492435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe48c0fecd0 msgr2=0x7fe48c069160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:53.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.784+0000 7fe4915c8640 1 --2- 172.21.15.16:0/229492435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe48c0fecd0 0x7fe48c069160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:53.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.784+0000 7fe4915c8640 1 -- 172.21.15.16:0/229492435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe480009560 con 0x7fe48c100880 2024-09-13T16:34:53.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.784+0000 7fe4915c8640 1 --2- 172.21.15.16:0/229492435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe48c100880 0x7fe48c19cff0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe48400eaa0 tx=0x7fe48400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:53.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.784+0000 7fe489ffb640 1 -- 172.21.15.16:0/229492435 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe484004d20 con 0x7fe48c100880 2024-09-13T16:34:53.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.784+0000 7fe489ffb640 1 -- 172.21.15.16:0/229492435 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe484004e80 con 0x7fe48c100880 2024-09-13T16:34:53.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.784+0000 7fe49302b640 1 -- 172.21.15.16:0/229492435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe48c19dd80 con 0x7fe48c100880 2024-09-13T16:34:53.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.784+0000 7fe489ffb640 1 -- 172.21.15.16:0/229492435 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe484011470 con 0x7fe48c100880 2024-09-13T16:34:53.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.785+0000 7fe49302b640 1 -- 172.21.15.16:0/229492435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe48c19e230 con 0x7fe48c100880 2024-09-13T16:34:53.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.786+0000 7fe49302b640 1 -- 172.21.15.16:0/229492435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe48c106180 con 0x7fe48c100880 2024-09-13T16:34:53.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.787+0000 7fe489ffb640 1 -- 172.21.15.16:0/229492435 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe48400caf0 con 0x7fe48c100880 2024-09-13T16:34:53.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.787+0000 7fe489ffb640 1 --2- 172.21.15.16:0/229492435 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe47c0a1110 0x7fe47c0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:53.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.787+0000 7fe490dc7640 1 --2- 172.21.15.16:0/229492435 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe47c0a1110 0x7fe47c0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:53.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.787+0000 7fe489ffb640 1 -- 172.21.15.16:0/229492435 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fe4840cce50 con 0x7fe48c100880 2024-09-13T16:34:53.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.788+0000 7fe490dc7640 1 --2- 172.21.15.16:0/229492435 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe47c0a1110 0x7fe47c0a35d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe4800057c0 tx=0x7fe480005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:53.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:53.791+0000 7fe489ffb640 1 -- 172.21.15.16:0/229492435 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fe484096290 con 0x7fe48c100880 2024-09-13T16:34:54.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.001+0000 7fe49302b640 1 -- 172.21.15.16:0/229492435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe48c105bf0 con 0x7fe48c100880 2024-09-13T16:34:54.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.002+0000 7fe489ffb640 1 -- 172.21.15.16:0/229492435 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10924 (secure 0 0 0) 0x7fe48401f020 con 0x7fe48c100880 2024-09-13T16:34:54.003 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:34:54.003 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3292781664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3292781664},{"type":"v1","addr":"172.21.15.16:6835","nonce":3292781664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4109039027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4109039027},{"type":"v1","addr":"172.21.15.16:6837","nonce":4109039027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T16:28:39.184734+0000","modified":"2024-09-13T16:31:06.506763+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24518,"mds_1":24490,"mds_2":24485,"mds_3":24422,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/2391618693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2391618693},{"type":"v1","addr":"172.21.15.16:6839","nonce":2391618693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/521042681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":521042681},{"type":"v1","addr":"172.21.15.16:6841","nonce":521042681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24418":{"gid":24418,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.111:6833/1984275936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1984275936},{"type":"v1","addr":"172.21.15.111:6833","nonce":1984275936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.107:6833/3797712690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3797712690},{"type":"v1","addr":"172.21.15.107:6833","nonce":3797712690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/338548468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":338548468},{"type":"v1","addr":"172.21.15.107:6835","nonce":338548468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/662156111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":662156111},{"type":"v1","addr":"172.21.15.111:6835","nonce":662156111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/1877128453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1877128453},{"type":"v1","addr":"172.21.15.111:6837","nonce":1877128453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.107:6837/2838457611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2838457611},{"type":"v1","addr":"172.21.15.107:6837","nonce":2838457611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.111:6839/1423842285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":1423842285},{"type":"v1","addr":"172.21.15.111:6839","nonce":1423842285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6839/1914689806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1914689806},{"type":"v1","addr":"172.21.15.107:6839","nonce":1914689806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T16:34:54.004 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 29 2024-09-13T16:34:54.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.004+0000 7fe49302b640 1 -- 172.21.15.16:0/229492435 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe47c0a1110 msgr2=0x7fe47c0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:54.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.004+0000 7fe49302b640 1 --2- 172.21.15.16:0/229492435 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe47c0a1110 0x7fe47c0a35d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe4800057c0 tx=0x7fe480005a20 comp rx=0 tx=0).stop 2024-09-13T16:34:54.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.004+0000 7fe49302b640 1 -- 172.21.15.16:0/229492435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe48c100880 msgr2=0x7fe48c19cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:54.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.004+0000 7fe49302b640 1 --2- 172.21.15.16:0/229492435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe48c100880 0x7fe48c19cff0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe48400eaa0 tx=0x7fe48400eea0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.004+0000 7fe49302b640 1 -- 172.21.15.16:0/229492435 shutdown_connections 2024-09-13T16:34:54.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.004+0000 7fe49302b640 1 --2- 172.21.15.16:0/229492435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe48c100880 0x7fe48c19cff0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.004+0000 7fe49302b640 1 --2- 172.21.15.16:0/229492435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe48c0ffec0 0x7fe48c19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.004+0000 7fe49302b640 1 --2- 172.21.15.16:0/229492435 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fe47c0a1110 0x7fe47c0a35d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.004+0000 7fe49302b640 1 --2- 172.21.15.16:0/229492435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe48c0fecd0 0x7fe48c069160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.004+0000 7fe49302b640 1 -- 172.21.15.16:0/229492435 >> 172.21.15.16:0/229492435 conn(0x7fe48c0fa430 msgr2=0x7fe48c103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:54.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.004+0000 7fe49302b640 1 -- 172.21.15.16:0/229492435 shutdown_connections 2024-09-13T16:34:54.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.004+0000 7fe49302b640 1 -- 172.21.15.16:0/229492435 wait complete. 2024-09-13T16:34:54.017 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:34:54.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.119+0000 7f961a29c640 1 -- 172.21.15.16:0/2684005866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96140fde10 msgr2=0x7f96140fe290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:54.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.119+0000 7f961a29c640 1 --2- 172.21.15.16:0/2684005866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96140fde10 0x7f96140fe290 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f95fc009900 tx=0x7f95fc02a410 comp rx=0 tx=0).stop 2024-09-13T16:34:54.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.119+0000 7f961a29c640 1 -- 172.21.15.16:0/2684005866 shutdown_connections 2024-09-13T16:34:54.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.119+0000 7f961a29c640 1 --2- 172.21.15.16:0/2684005866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96140fe7d0 0x7f9614108380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.119+0000 7f961a29c640 1 --2- 172.21.15.16:0/2684005866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96140fde10 0x7f96140fe290 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.119+0000 7f961a29c640 1 --2- 172.21.15.16:0/2684005866 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96141013a0 0x7f96141017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.119+0000 7f961a29c640 1 -- 172.21.15.16:0/2684005866 >> 172.21.15.16:0/2684005866 conn(0x7f96140f9b80 msgr2=0x7f96140fbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:54.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.119+0000 7f961a29c640 1 -- 172.21.15.16:0/2684005866 shutdown_connections 2024-09-13T16:34:54.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.119+0000 7f961a29c640 1 -- 172.21.15.16:0/2684005866 wait complete. 2024-09-13T16:34:54.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.120+0000 7f961a29c640 1 Processor -- start 2024-09-13T16:34:54.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.120+0000 7f961a29c640 1 -- start start 2024-09-13T16:34:54.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.120+0000 7f961a29c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96140fde10 0x7f9614195c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:54.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.120+0000 7f961a29c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96140fe7d0 0x7f96141961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:54.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.120+0000 7f961a29c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96141013a0 0x7f961419d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:54.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.120+0000 7f961a29c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961419d780 con 0x7f96140fe7d0 2024-09-13T16:34:54.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.120+0000 7f961a29c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961419d8f0 con 0x7f96140fde10 2024-09-13T16:34:54.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.120+0000 7f961a29c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961419dab0 con 0x7f96141013a0 2024-09-13T16:34:54.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.120+0000 7f960bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96140fe7d0 0x7f96141961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:54.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.120+0000 7f960bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96140fe7d0 0x7f96141961b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52588/0 (socket says 172.21.15.16:52588) 2024-09-13T16:34:54.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.120+0000 7f960bfff640 1 -- 172.21.15.16:0/1901539251 learned_addr learned my addr 172.21.15.16:0/1901539251 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:34:54.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.120+0000 7f9613fff640 1 --2- 172.21.15.16:0/1901539251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96140fde10 0x7f9614195c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:54.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.120+0000 7f9618839640 1 --2- 172.21.15.16:0/1901539251 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96141013a0 0x7f961419d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:54.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.120+0000 7f960bfff640 1 -- 172.21.15.16:0/1901539251 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96141013a0 msgr2=0x7f961419d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:54.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.120+0000 7f960bfff640 1 --2- 172.21.15.16:0/1901539251 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96141013a0 0x7f961419d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.120+0000 7f960bfff640 1 -- 172.21.15.16:0/1901539251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96140fde10 msgr2=0x7f9614195c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:54.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.120+0000 7f960bfff640 1 --2- 172.21.15.16:0/1901539251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96140fde10 0x7f9614195c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.120+0000 7f960bfff640 1 -- 172.21.15.16:0/1901539251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95fc009560 con 0x7f96140fe7d0 2024-09-13T16:34:54.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.121+0000 7f960bfff640 1 --2- 172.21.15.16:0/1901539251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96140fe7d0 0x7f96141961b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0057c0 tx=0x7f95fc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:54.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.121+0000 7f9611ffb640 1 -- 172.21.15.16:0/1901539251 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc031d70 con 0x7f96140fe7d0 2024-09-13T16:34:54.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.121+0000 7f9611ffb640 1 -- 172.21.15.16:0/1901539251 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f95fc0044e0 con 0x7f96140fe7d0 2024-09-13T16:34:54.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.121+0000 7f961a29c640 1 -- 172.21.15.16:0/1901539251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f961419dd30 con 0x7f96140fe7d0 2024-09-13T16:34:54.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.121+0000 7f9611ffb640 1 -- 172.21.15.16:0/1901539251 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc003b80 con 0x7f96140fe7d0 2024-09-13T16:34:54.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.121+0000 7f961a29c640 1 -- 172.21.15.16:0/1901539251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f961419e1d0 con 0x7f96140fe7d0 2024-09-13T16:34:54.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.122+0000 7f961a29c640 1 -- 172.21.15.16:0/1901539251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9614107470 con 0x7f96140fe7d0 2024-09-13T16:34:54.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.123+0000 7f9611ffb640 1 -- 172.21.15.16:0/1901539251 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f95fc003ce0 con 0x7f96140fe7d0 2024-09-13T16:34:54.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.124+0000 7f9611ffb640 1 --2- 172.21.15.16:0/1901539251 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f96000a1110 0x7f96000a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:54.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.124+0000 7f9611ffb640 1 -- 172.21.15.16:0/1901539251 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f95fc0ee2a0 con 0x7f96140fe7d0 2024-09-13T16:34:54.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.124+0000 7f9613fff640 1 --2- 172.21.15.16:0/1901539251 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f96000a1110 0x7f96000a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:54.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.124+0000 7f9613fff640 1 --2- 172.21.15.16:0/1901539251 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f96000a1110 0x7f96000a35d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9604005d90 tx=0x7f9604005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:54.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.127+0000 7f9611ffb640 1 -- 172.21.15.16:0/1901539251 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f95fc0b76e0 con 0x7f96140fe7d0 2024-09-13T16:34:54.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.338+0000 7f961a29c640 1 -- 172.21.15.16:0/1901539251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f961410b530 con 0x7f96140fe7d0 2024-09-13T16:34:54.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.338+0000 7f9611ffb640 1 -- 172.21.15.16:0/1901539251 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10924 (secure 0 0 0) 0x7f95fc0bad00 con 0x7f96140fe7d0 2024-09-13T16:34:54.340 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:34:54.340 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3292781664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3292781664},{"type":"v1","addr":"172.21.15.16:6835","nonce":3292781664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4109039027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4109039027},{"type":"v1","addr":"172.21.15.16:6837","nonce":4109039027}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T16:28:39.184734+0000","modified":"2024-09-13T16:31:06.506763+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24518,"mds_1":24490,"mds_2":24485,"mds_3":24422,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14586":{"gid":14586,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/2391618693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2391618693},{"type":"v1","addr":"172.21.15.16:6839","nonce":2391618693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14598":{"gid":14598,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/521042681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":521042681},{"type":"v1","addr":"172.21.15.16:6841","nonce":521042681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24418":{"gid":24418,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.111:6833/1984275936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1984275936},{"type":"v1","addr":"172.21.15.111:6833","nonce":1984275936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.107:6833/3797712690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3797712690},{"type":"v1","addr":"172.21.15.107:6833","nonce":3797712690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/338548468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":338548468},{"type":"v1","addr":"172.21.15.107:6835","nonce":338548468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/662156111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":662156111},{"type":"v1","addr":"172.21.15.111:6835","nonce":662156111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/1877128453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":1877128453},{"type":"v1","addr":"172.21.15.111:6837","nonce":1877128453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.107:6837/2838457611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2838457611},{"type":"v1","addr":"172.21.15.107:6837","nonce":2838457611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.111:6839/1423842285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":1423842285},{"type":"v1","addr":"172.21.15.111:6839","nonce":1423842285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6839/1914689806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1914689806},{"type":"v1","addr":"172.21.15.107:6839","nonce":1914689806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T16:34:54.340 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 29 2024-09-13T16:34:54.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.340+0000 7f961a29c640 1 -- 172.21.15.16:0/1901539251 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f96000a1110 msgr2=0x7f96000a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.340+0000 7f961a29c640 1 --2- 172.21.15.16:0/1901539251 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f96000a1110 0x7f96000a35d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9604005d90 tx=0x7f9604005d20 comp rx=0 tx=0).stop 2024-09-13T16:34:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.340+0000 7f961a29c640 1 -- 172.21.15.16:0/1901539251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96140fe7d0 msgr2=0x7f96141961b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.340+0000 7f961a29c640 1 --2- 172.21.15.16:0/1901539251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96140fe7d0 0x7f96141961b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0057c0 tx=0x7f95fc0042f0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.340+0000 7f961a29c640 1 -- 172.21.15.16:0/1901539251 shutdown_connections 2024-09-13T16:34:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.340+0000 7f961a29c640 1 --2- 172.21.15.16:0/1901539251 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96141013a0 0x7f961419d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.340+0000 7f961a29c640 1 --2- 172.21.15.16:0/1901539251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96140fe7d0 0x7f96141961b0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.340+0000 7f961a29c640 1 --2- 172.21.15.16:0/1901539251 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f96000a1110 0x7f96000a35d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.340+0000 7f961a29c640 1 --2- 172.21.15.16:0/1901539251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96140fde10 0x7f9614195c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.340+0000 7f961a29c640 1 -- 172.21.15.16:0/1901539251 >> 172.21.15.16:0/1901539251 conn(0x7f96140f9b80 msgr2=0x7f9614104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.341+0000 7f961a29c640 1 -- 172.21.15.16:0/1901539251 shutdown_connections 2024-09-13T16:34:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.341+0000 7f961a29c640 1 -- 172.21.15.16:0/1901539251 wait complete. 2024-09-13T16:34:54.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:54 smithi107 ceph-mon[34139]: pgmap v467: 129 pgs: 129 active+clean; 1020 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-13T16:34:54.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:54 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/229492435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:34:54.351 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:54 smithi016 ceph-mon[31187]: pgmap v467: 129 pgs: 129 active+clean; 1020 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-13T16:34:54.351 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:54 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/229492435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:34:54.353 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:34:54.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:54 smithi111 ceph-mon[34600]: pgmap v467: 129 pgs: 129 active+clean; 1020 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-13T16:34:54.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:54 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/229492435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:34:54.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.453+0000 7fb90911f640 1 -- 172.21.15.16:0/2942028214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb904105ab0 msgr2=0x7fb904107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:54.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.453+0000 7fb90911f640 1 --2- 172.21.15.16:0/2942028214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb904105ab0 0x7fb904107f20 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc009ca0 tx=0x7fb8fc02a8c0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.454+0000 7fb90911f640 1 -- 172.21.15.16:0/2942028214 shutdown_connections 2024-09-13T16:34:54.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.454+0000 7fb90911f640 1 --2- 172.21.15.16:0/2942028214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb904105ab0 0x7fb904107f20 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.454+0000 7fb90911f640 1 --2- 172.21.15.16:0/2942028214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb904103180 0x7fb904105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.454+0000 7fb90911f640 1 --2- 172.21.15.16:0/2942028214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb904100850 0x7fb904102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.454+0000 7fb90911f640 1 -- 172.21.15.16:0/2942028214 >> 172.21.15.16:0/2942028214 conn(0x7fb9040fa430 msgr2=0x7fb9040fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:54.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.454+0000 7fb90911f640 1 -- 172.21.15.16:0/2942028214 shutdown_connections 2024-09-13T16:34:54.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.454+0000 7fb90911f640 1 -- 172.21.15.16:0/2942028214 wait complete. 2024-09-13T16:34:54.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.454+0000 7fb90911f640 1 Processor -- start 2024-09-13T16:34:54.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.455+0000 7fb90911f640 1 -- start start 2024-09-13T16:34:54.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.455+0000 7fb90911f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb904100850 0x7fb904195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:54.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.455+0000 7fb90911f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb904103180 0x7fb904196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:54.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.455+0000 7fb90911f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb904105ab0 0x7fb90419d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:54.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.455+0000 7fb90911f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90419d830 con 0x7fb904103180 2024-09-13T16:34:54.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.455+0000 7fb90911f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90419d9a0 con 0x7fb904105ab0 2024-09-13T16:34:54.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.455+0000 7fb90911f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90419db60 con 0x7fb904100850 2024-09-13T16:34:54.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.455+0000 7fb90259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb904103180 0x7fb904196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:54.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.455+0000 7fb90259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb904103180 0x7fb904196290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52604/0 (socket says 172.21.15.16:52604) 2024-09-13T16:34:54.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.455+0000 7fb90259c640 1 -- 172.21.15.16:0/3789953819 learned_addr learned my addr 172.21.15.16:0/3789953819 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:34:54.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.455+0000 7fb902d9d640 1 --2- 172.21.15.16:0/3789953819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb904100850 0x7fb904195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:54.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.455+0000 7fb90359e640 1 --2- 172.21.15.16:0/3789953819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb904105ab0 0x7fb90419d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:54.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.455+0000 7fb90259c640 1 -- 172.21.15.16:0/3789953819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb904100850 msgr2=0x7fb904195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:54.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.455+0000 7fb90259c640 1 --2- 172.21.15.16:0/3789953819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb904100850 0x7fb904195d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.455+0000 7fb90259c640 1 -- 172.21.15.16:0/3789953819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb904105ab0 msgr2=0x7fb90419d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:54.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.455+0000 7fb90259c640 1 --2- 172.21.15.16:0/3789953819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb904105ab0 0x7fb90419d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.455+0000 7fb90259c640 1 -- 172.21.15.16:0/3789953819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8fc009900 con 0x7fb904103180 2024-09-13T16:34:54.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.455+0000 7fb90259c640 1 --2- 172.21.15.16:0/3789953819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb904103180 0x7fb904196290 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800c970 tx=0x7fb8f800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:54.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.456+0000 7fb8ebfff640 1 -- 172.21.15.16:0/3789953819 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8007bb0 con 0x7fb904103180 2024-09-13T16:34:54.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.456+0000 7fb8ebfff640 1 -- 172.21.15.16:0/3789953819 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb8f8007d10 con 0x7fb904103180 2024-09-13T16:34:54.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.456+0000 7fb90911f640 1 -- 172.21.15.16:0/3789953819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb90419de40 con 0x7fb904103180 2024-09-13T16:34:54.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.456+0000 7fb8ebfff640 1 -- 172.21.15.16:0/3789953819 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8018660 con 0x7fb904103180 2024-09-13T16:34:54.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.456+0000 7fb90911f640 1 -- 172.21.15.16:0/3789953819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb90419e210 con 0x7fb904103180 2024-09-13T16:34:54.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.457+0000 7fb90911f640 1 -- 172.21.15.16:0/3789953819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb904103560 con 0x7fb904103180 2024-09-13T16:34:54.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.458+0000 7fb8ebfff640 1 -- 172.21.15.16:0/3789953819 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb8f800f450 con 0x7fb904103180 2024-09-13T16:34:54.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.459+0000 7fb8ebfff640 1 --2- 172.21.15.16:0/3789953819 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb8ec0a1090 0x7fb8ec0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:54.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.459+0000 7fb902d9d640 1 --2- 172.21.15.16:0/3789953819 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb8ec0a1090 0x7fb8ec0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:54.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.459+0000 7fb8ebfff640 1 -- 172.21.15.16:0/3789953819 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb8f80ccd60 con 0x7fb904103180 2024-09-13T16:34:54.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.459+0000 7fb902d9d640 1 --2- 172.21.15.16:0/3789953819 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb8ec0a1090 0x7fb8ec0a3550 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40058c0 tx=0x7fb8f400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:54.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.463+0000 7fb8ebfff640 1 -- 172.21.15.16:0/3789953819 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb8f80960b0 con 0x7fb904103180 2024-09-13T16:34:54.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.636+0000 7fb90911f640 1 -- 172.21.15.16:0/3789953819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9040fbf20 con 0x7fb904103180 2024-09-13T16:34:54.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.638+0000 7fb8ebfff640 1 -- 172.21.15.16:0/3789953819 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24593 (secure 0 0 0) 0x7fb8f8097fe0 con 0x7fb904103180 2024-09-13T16:34:54.639 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-13T16:34:54.639 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":93,"fsid":"0d241f6a-71ec-11ef-bceb-c7b262605968","created":"2024-09-13T16:21:17.971738+0000","modified":"2024-09-13T16:30:05.681521+0000","last_up_change":"2024-09-13T16:27:54.673413+0000","last_in_change":"2024-09-13T16:27:35.718390+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:24:22.981019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T16:28:37.746027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T16:28:38.602731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T16:28:41.633322+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0e11ef8f-08d6-4e74-afd6-f49f4bd40032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6803","nonce":2079394186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6805","nonce":2079394186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6809","nonce":2079394186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":2079394186},{"type":"v1","addr":"172.21.15.16:6807","nonce":2079394186}]},"public_addr":"172.21.15.16:6803/2079394186","cluster_addr":"172.21.15.16:6805/2079394186","heartbeat_back_addr":"172.21.15.16:6809/2079394186","heartbeat_front_addr":"172.21.15.16:6807/2079394186","state":["exists","up"]},{"osd":1,"uuid":"f2cd3694-9c9e-4da7-976e-f80a8df120f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6801","nonce":4021360958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6803","nonce":4021360958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6807","nonce":4021360958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4021360958},{"type":"v1","addr":"172.21.15.107:6805","nonce":4021360958}]},"public_addr":"172.21.15.107:6801/4021360958","cluster_addr":"172.21.15.107:6803/4021360958","heartbeat_back_addr":"172.21.15.107:6807/4021360958","heartbeat_front_addr":"172.21.15.107:6805/4021360958","state":["exists","up"]},{"osd":2,"uuid":"a5bbf2eb-6599-44d5-ad96-68bb89ee19f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6801","nonce":3227179825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6803","nonce":3227179825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6807","nonce":3227179825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3227179825},{"type":"v1","addr":"172.21.15.111:6805","nonce":3227179825}]},"public_addr":"172.21.15.111:6801/3227179825","cluster_addr":"172.21.15.111:6803/3227179825","heartbeat_back_addr":"172.21.15.111:6807/3227179825","heartbeat_front_addr":"172.21.15.111:6805/3227179825","state":["exists","up"]},{"osd":3,"uuid":"d6295ffe-11bc-4964-a250-9fecca870b97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6811","nonce":2390375937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6813","nonce":2390375937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6817","nonce":2390375937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2390375937},{"type":"v1","addr":"172.21.15.16:6815","nonce":2390375937}]},"public_addr":"172.21.15.16:6811/2390375937","cluster_addr":"172.21.15.16:6813/2390375937","heartbeat_back_addr":"172.21.15.16:6817/2390375937","heartbeat_front_addr":"172.21.15.16:6815/2390375937","state":["exists","up"]},{"osd":4,"uuid":"0a8446e0-fc86-4646-beb6-a7b9d433093d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6809","nonce":1936582048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6811","nonce":1936582048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6815","nonce":1936582048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1936582048},{"type":"v1","addr":"172.21.15.107:6813","nonce":1936582048}]},"public_addr":"172.21.15.107:6809/1936582048","cluster_addr":"172.21.15.107:6811/1936582048","heartbeat_back_addr":"172.21.15.107:6815/1936582048","heartbeat_front_addr":"172.21.15.107:6813/1936582048","state":["exists","up"]},{"osd":5,"uuid":"2c97a446-357e-4975-9e51-9c95fe84a663","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6809","nonce":483858448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6811","nonce":483858448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6815","nonce":483858448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":483858448},{"type":"v1","addr":"172.21.15.111:6813","nonce":483858448}]},"public_addr":"172.21.15.111:6809/483858448","cluster_addr":"172.21.15.111:6811/483858448","heartbeat_back_addr":"172.21.15.111:6815/483858448","heartbeat_front_addr":"172.21.15.111:6813/483858448","state":["exists","up"]},{"osd":6,"uuid":"890239e6-e6fb-4f9c-b973-05cc66d199bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6819","nonce":1330214141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6821","nonce":1330214141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6825","nonce":1330214141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1330214141},{"type":"v1","addr":"172.21.15.16:6823","nonce":1330214141}]},"public_addr":"172.21.15.16:6819/1330214141","cluster_addr":"172.21.15.16:6821/1330214141","heartbeat_back_addr":"172.21.15.16:6825/1330214141","heartbeat_front_addr":"172.21.15.16:6823/1330214141","state":["exists","up"]},{"osd":7,"uuid":"ee826a9c-f441-4903-9b1b-66b627f7b712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6817","nonce":1133876754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6819","nonce":1133876754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6823","nonce":1133876754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1133876754},{"type":"v1","addr":"172.21.15.107:6821","nonce":1133876754}]},"public_addr":"172.21.15.107:6817/1133876754","cluster_addr":"172.21.15.107:6819/1133876754","heartbeat_back_addr":"172.21.15.107:6823/1133876754","heartbeat_front_addr":"172.21.15.107:6821/1133876754","state":["exists","up"]},{"osd":8,"uuid":"e19773f2-dec8-44e0-9d33-0252e07d6415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6817","nonce":837548190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6819","nonce":837548190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6823","nonce":837548190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":837548190},{"type":"v1","addr":"172.21.15.111:6821","nonce":837548190}]},"public_addr":"172.21.15.111:6817/837548190","cluster_addr":"172.21.15.111:6819/837548190","heartbeat_back_addr":"172.21.15.111:6823/837548190","heartbeat_front_addr":"172.21.15.111:6821/837548190","state":["exists","up"]},{"osd":9,"uuid":"bae4d6db-8b90-4d02-8e83-7d8f2a6f8fdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6827","nonce":4175383686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6829","nonce":4175383686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6833","nonce":4175383686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":4175383686},{"type":"v1","addr":"172.21.15.16:6831","nonce":4175383686}]},"public_addr":"172.21.15.16:6827/4175383686","cluster_addr":"172.21.15.16:6829/4175383686","heartbeat_back_addr":"172.21.15.16:6833/4175383686","heartbeat_front_addr":"172.21.15.16:6831/4175383686","state":["exists","up"]},{"osd":10,"uuid":"fd0e7740-76c8-4c99-a349-7077b4905143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6825","nonce":2659550426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6827","nonce":2659550426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6831","nonce":2659550426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":2659550426},{"type":"v1","addr":"172.21.15.107:6829","nonce":2659550426}]},"public_addr":"172.21.15.107:6825/2659550426","cluster_addr":"172.21.15.107:6827/2659550426","heartbeat_back_addr":"172.21.15.107:6831/2659550426","heartbeat_front_addr":"172.21.15.107:6829/2659550426","state":["exists","up"]},{"osd":11,"uuid":"94a1751d-b00c-4e23-815d-3a8bc3dea146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6825","nonce":1633548683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6827","nonce":1633548683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6831","nonce":1633548683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":1633548683},{"type":"v1","addr":"172.21.15.111:6829","nonce":1633548683}]},"public_addr":"172.21.15.111:6825/1633548683","cluster_addr":"172.21.15.111:6827/1633548683","heartbeat_back_addr":"172.21.15.111:6831/1633548683","heartbeat_front_addr":"172.21.15.111:6829/1633548683","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:36.359852+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:23:58.014202+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:19.753672+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:24:43.404817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:06.001677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:29.361587+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:25:52.554847+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:15.957553+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:26:39.581093+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:04.911389+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:28.971280+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:27:53.473561+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/2098378667":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/3536229016":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6801/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/2684687547":"2024-09-14T16:22:04.485277+0000","172.21.15.16:6800/1756940569":"2024-09-14T16:22:04.485277+0000","172.21.15.16:0/368410612":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/12158160":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6800/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:6801/1147765163":"2024-09-14T16:21:40.126459+0000","172.21.15.16:0/3754423379":"2024-09-14T16:21:40.126459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:34:54.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.640+0000 7fb90911f640 1 -- 172.21.15.16:0/3789953819 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb8ec0a1090 msgr2=0x7fb8ec0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:54.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.640+0000 7fb90911f640 1 --2- 172.21.15.16:0/3789953819 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb8ec0a1090 0x7fb8ec0a3550 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40058c0 tx=0x7fb8f400a2d0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.640+0000 7fb90911f640 1 -- 172.21.15.16:0/3789953819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb904103180 msgr2=0x7fb904196290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:54.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.640+0000 7fb90911f640 1 --2- 172.21.15.16:0/3789953819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb904103180 0x7fb904196290 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800c970 tx=0x7fb8f800ce30 comp rx=0 tx=0).stop 2024-09-13T16:34:54.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.640+0000 7fb90911f640 1 -- 172.21.15.16:0/3789953819 shutdown_connections 2024-09-13T16:34:54.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.640+0000 7fb90911f640 1 --2- 172.21.15.16:0/3789953819 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7fb8ec0a1090 0x7fb8ec0a3550 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.640+0000 7fb90911f640 1 --2- 172.21.15.16:0/3789953819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb904105ab0 0x7fb90419d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.640+0000 7fb90911f640 1 --2- 172.21.15.16:0/3789953819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb904103180 0x7fb904196290 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.640+0000 7fb90911f640 1 --2- 172.21.15.16:0/3789953819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb904100850 0x7fb904195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.640+0000 7fb90911f640 1 -- 172.21.15.16:0/3789953819 >> 172.21.15.16:0/3789953819 conn(0x7fb9040fa430 msgr2=0x7fb9041012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:54.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.640+0000 7fb90911f640 1 -- 172.21.15.16:0/3789953819 shutdown_connections 2024-09-13T16:34:54.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.640+0000 7fb90911f640 1 -- 172.21.15.16:0/3789953819 wait complete. 2024-09-13T16:34:54.654 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-13T16:34:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.756+0000 7f99cbfff640 1 -- 172.21.15.16:0/3301438280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99cc101260 msgr2=0x7f99cc101660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.756+0000 7f99cbfff640 1 --2- 172.21.15.16:0/3301438280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99cc101260 0x7f99cc101660 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009900 tx=0x7f99bc02a3f0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.756+0000 7f99cbfff640 1 -- 172.21.15.16:0/3301438280 shutdown_connections 2024-09-13T16:34:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.756+0000 7f99cbfff640 1 --2- 172.21.15.16:0/3301438280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99cc100150 0x7f99cc107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.756+0000 7f99cbfff640 1 --2- 172.21.15.16:0/3301438280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99cc0ff790 0x7f99cc0ffc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.756+0000 7f99cbfff640 1 --2- 172.21.15.16:0/3301438280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99cc101260 0x7f99cc101660 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.756+0000 7f99cbfff640 1 -- 172.21.15.16:0/3301438280 >> 172.21.15.16:0/3301438280 conn(0x7f99cc0fb2e0 msgr2=0x7f99cc0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.756+0000 7f99cbfff640 1 -- 172.21.15.16:0/3301438280 shutdown_connections 2024-09-13T16:34:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.756+0000 7f99cbfff640 1 -- 172.21.15.16:0/3301438280 wait complete. 2024-09-13T16:34:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.757+0000 7f99cbfff640 1 Processor -- start 2024-09-13T16:34:54.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.757+0000 7f99cbfff640 1 -- start start 2024-09-13T16:34:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.757+0000 7f99cbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99cc0ff790 0x7f99cc074260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.757+0000 7f99cbfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99cc100150 0x7f99cc0728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.757+0000 7f99cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99cc101260 0x7f99cc072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.757+0000 7f99cbfff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc073540 con 0x7f99cc101260 2024-09-13T16:34:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.757+0000 7f99cbfff640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc0736b0 con 0x7f99cc0ff790 2024-09-13T16:34:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.757+0000 7f99cbfff640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc19faf0 con 0x7f99cc100150 2024-09-13T16:34:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.758+0000 7f99cb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99cc101260 0x7f99cc072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.758+0000 7f99cb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99cc101260 0x7f99cc072de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52618/0 (socket says 172.21.15.16:52618) 2024-09-13T16:34:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.758+0000 7f99cb7fe640 1 -- 172.21.15.16:0/1417866865 learned_addr learned my addr 172.21.15.16:0/1417866865 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:34:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.758+0000 7f99caffd640 1 --2- 172.21.15.16:0/1417866865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99cc0ff790 0x7f99cc074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.758+0000 7f99ca7fc640 1 --2- 172.21.15.16:0/1417866865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99cc100150 0x7f99cc0728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.758+0000 7f99cb7fe640 1 -- 172.21.15.16:0/1417866865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99cc100150 msgr2=0x7f99cc0728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.758+0000 7f99cb7fe640 1 --2- 172.21.15.16:0/1417866865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99cc100150 0x7f99cc0728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.758+0000 7f99cb7fe640 1 -- 172.21.15.16:0/1417866865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99cc0ff790 msgr2=0x7f99cc074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.758+0000 7f99cb7fe640 1 --2- 172.21.15.16:0/1417866865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99cc0ff790 0x7f99cc074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.758+0000 7f99cb7fe640 1 -- 172.21.15.16:0/1417866865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99bc009560 con 0x7f99cc101260 2024-09-13T16:34:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.758+0000 7f99caffd640 1 --2- 172.21.15.16:0/1417866865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99cc0ff790 0x7f99cc074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:34:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.758+0000 7f99cb7fe640 1 --2- 172.21.15.16:0/1417866865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99cc101260 0x7f99cc072de0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f99c400cd10 tx=0x7f99c400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.758+0000 7f99b3fff640 1 -- 172.21.15.16:0/1417866865 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c400f450 con 0x7f99cc101260 2024-09-13T16:34:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.758+0000 7f99b3fff640 1 -- 172.21.15.16:0/1417866865 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f99c400fa40 con 0x7f99cc101260 2024-09-13T16:34:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.758+0000 7f99cbfff640 1 -- 172.21.15.16:0/1417866865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99cc19fda0 con 0x7f99cc101260 2024-09-13T16:34:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.758+0000 7f99b3fff640 1 -- 172.21.15.16:0/1417866865 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c4018a00 con 0x7f99cc101260 2024-09-13T16:34:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.758+0000 7f99cbfff640 1 -- 172.21.15.16:0/1417866865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99cc1a0200 con 0x7f99cc101260 2024-09-13T16:34:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.759+0000 7f99cbfff640 1 -- 172.21.15.16:0/1417866865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99cc0ffc10 con 0x7f99cc101260 2024-09-13T16:34:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.760+0000 7f99b3fff640 1 -- 172.21.15.16:0/1417866865 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f99c400fbb0 con 0x7f99cc101260 2024-09-13T16:34:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.761+0000 7f99b3fff640 1 --2- 172.21.15.16:0/1417866865 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f99b40a1090 0x7f99b40a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.762+0000 7f99caffd640 1 --2- 172.21.15.16:0/1417866865 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f99b40a1090 0x7f99b40a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.762+0000 7f99b3fff640 1 -- 172.21.15.16:0/1417866865 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f99c40cd210 con 0x7f99cc101260 2024-09-13T16:34:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.762+0000 7f99caffd640 1 --2- 172.21.15.16:0/1417866865 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f99b40a1090 0x7f99b40a3550 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f99bc005ab0 tx=0x7f99bc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.765+0000 7f99b3fff640 1 -- 172.21.15.16:0/1417866865 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f99c4096650 con 0x7f99cc101260 2024-09-13T16:34:54.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.979+0000 7f99cbfff640 1 -- 172.21.15.16:0/1417866865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f99cc107060 con 0x7f99cc101260 2024-09-13T16:34:54.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.993+0000 7f99b3fff640 1 -- 172.21.15.16:0/1417866865 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) v1 ==== 161+0+0 (secure 0 0 0) 0x7f99c4099c70 con 0x7f99cc101260 2024-09-13T16:34:54.994 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T16:34:54.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.995+0000 7f99cbfff640 1 -- 172.21.15.16:0/1417866865 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f99b40a1090 msgr2=0x7f99b40a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:54.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.995+0000 7f99cbfff640 1 --2- 172.21.15.16:0/1417866865 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f99b40a1090 0x7f99b40a3550 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f99bc005ab0 tx=0x7f99bc005a20 comp rx=0 tx=0).stop 2024-09-13T16:34:54.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.995+0000 7f99cbfff640 1 -- 172.21.15.16:0/1417866865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99cc101260 msgr2=0x7f99cc072de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:54.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.995+0000 7f99cbfff640 1 --2- 172.21.15.16:0/1417866865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99cc101260 0x7f99cc072de0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f99c400cd10 tx=0x7f99c400a620 comp rx=0 tx=0).stop 2024-09-13T16:34:54.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.995+0000 7f99cbfff640 1 -- 172.21.15.16:0/1417866865 shutdown_connections 2024-09-13T16:34:54.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.995+0000 7f99cbfff640 1 --2- 172.21.15.16:0/1417866865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99cc101260 0x7f99cc072de0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.995+0000 7f99cbfff640 1 --2- 172.21.15.16:0/1417866865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99cc100150 0x7f99cc0728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.995+0000 7f99cbfff640 1 --2- 172.21.15.16:0/1417866865 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f99b40a1090 0x7f99b40a3550 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.995+0000 7f99cbfff640 1 --2- 172.21.15.16:0/1417866865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99cc0ff790 0x7f99cc074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:54.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.995+0000 7f99cbfff640 1 -- 172.21.15.16:0/1417866865 >> 172.21.15.16:0/1417866865 conn(0x7f99cc0fb2e0 msgr2=0x7f99cc104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:54.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.995+0000 7f99cbfff640 1 -- 172.21.15.16:0/1417866865 shutdown_connections 2024-09-13T16:34:54.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:54.995+0000 7f99cbfff640 1 -- 172.21.15.16:0/1417866865 wait complete. 2024-09-13T16:34:55.009 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-13T16:34:55.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.123+0000 7f09c6c32640 1 -- 172.21.15.16:0/2718567239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09c0100810 msgr2=0x7f09c0102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:55.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.123+0000 7f09c6c32640 1 --2- 172.21.15.16:0/2718567239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09c0100810 0x7f09c0102c00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009ca0 tx=0x7f09b002a8c0 comp rx=0 tx=0).stop 2024-09-13T16:34:55.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.123+0000 7f09c6c32640 1 -- 172.21.15.16:0/2718567239 shutdown_connections 2024-09-13T16:34:55.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.123+0000 7f09c6c32640 1 --2- 172.21.15.16:0/2718567239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09c0105a70 0x7f09c0107ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:55.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.123+0000 7f09c6c32640 1 --2- 172.21.15.16:0/2718567239 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09c0103140 0x7f09c0105530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:55.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.123+0000 7f09c6c32640 1 --2- 172.21.15.16:0/2718567239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09c0100810 0x7f09c0102c00 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:55.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.123+0000 7f09c6c32640 1 -- 172.21.15.16:0/2718567239 >> 172.21.15.16:0/2718567239 conn(0x7f09c00fa3f0 msgr2=0x7f09c00fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:55.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.123+0000 7f09c6c32640 1 -- 172.21.15.16:0/2718567239 shutdown_connections 2024-09-13T16:34:55.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.123+0000 7f09c6c32640 1 -- 172.21.15.16:0/2718567239 wait complete. 2024-09-13T16:34:55.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.123+0000 7f09c6c32640 1 Processor -- start 2024-09-13T16:34:55.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.123+0000 7f09c6c32640 1 -- start start 2024-09-13T16:34:55.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.123+0000 7f09c6c32640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09c0100810 0x7f09c0195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:55.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.123+0000 7f09c6c32640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09c0103140 0x7f09c01961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:55.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.123+0000 7f09c6c32640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09c0105a70 0x7f09c019d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:55.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.123+0000 7f09c6c32640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c019d7c0 con 0x7f09c0100810 2024-09-13T16:34:55.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.123+0000 7f09c6c32640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c019d930 con 0x7f09c0105a70 2024-09-13T16:34:55.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.123+0000 7f09c6c32640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c019daf0 con 0x7f09c0103140 2024-09-13T16:34:55.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.124+0000 7f09c49ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09c0100810 0x7f09c0195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:55.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.124+0000 7f09c49ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09c0100810 0x7f09c0195cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52658/0 (socket says 172.21.15.16:52658) 2024-09-13T16:34:55.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.124+0000 7f09bffff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09c0103140 0x7f09c01961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:55.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.124+0000 7f09c49ce640 1 -- 172.21.15.16:0/2147107772 learned_addr learned my addr 172.21.15.16:0/2147107772 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:34:55.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.124+0000 7f09c51cf640 1 --2- 172.21.15.16:0/2147107772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09c0105a70 0x7f09c019d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:55.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.124+0000 7f09c49ce640 1 -- 172.21.15.16:0/2147107772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09c0103140 msgr2=0x7f09c01961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:55.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.124+0000 7f09c49ce640 1 --2- 172.21.15.16:0/2147107772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09c0103140 0x7f09c01961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:55.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.124+0000 7f09c49ce640 1 -- 172.21.15.16:0/2147107772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09c0105a70 msgr2=0x7f09c019d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:55.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.124+0000 7f09c49ce640 1 --2- 172.21.15.16:0/2147107772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09c0105a70 0x7f09c019d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:55.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.124+0000 7f09c49ce640 1 -- 172.21.15.16:0/2147107772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b0009900 con 0x7f09c0100810 2024-09-13T16:34:55.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.124+0000 7f09c49ce640 1 --2- 172.21.15.16:0/2147107772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09c0100810 0x7f09c0195cb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009dc0 tx=0x7f09b0030f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:55.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.124+0000 7f09bdffb640 1 -- 172.21.15.16:0/2147107772 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b0003b70 con 0x7f09c0100810 2024-09-13T16:34:55.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.124+0000 7f09c6c32640 1 -- 172.21.15.16:0/2147107772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c019dd70 con 0x7f09c0100810 2024-09-13T16:34:55.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.124+0000 7f09bdffb640 1 -- 172.21.15.16:0/2147107772 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f09b0040070 con 0x7f09c0100810 2024-09-13T16:34:55.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.124+0000 7f09bdffb640 1 -- 172.21.15.16:0/2147107772 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b0004400 con 0x7f09c0100810 2024-09-13T16:34:55.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.125+0000 7f09c6c32640 1 -- 172.21.15.16:0/2147107772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c019e1d0 con 0x7f09c0100810 2024-09-13T16:34:55.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.125+0000 7f09c6c32640 1 -- 172.21.15.16:0/2147107772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c00fbee0 con 0x7f09c0100810 2024-09-13T16:34:55.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.127+0000 7f09bdffb640 1 -- 172.21.15.16:0/2147107772 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f09b0042030 con 0x7f09c0100810 2024-09-13T16:34:55.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.127+0000 7f09bdffb640 1 --2- 172.21.15.16:0/2147107772 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f09ac0a1090 0x7f09ac0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:55.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.128+0000 7f09bdffb640 1 -- 172.21.15.16:0/2147107772 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9508+0+0 (secure 0 0 0) 0x7f09b00ee3a0 con 0x7f09c0100810 2024-09-13T16:34:55.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.128+0000 7f09bffff640 1 --2- 172.21.15.16:0/2147107772 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f09ac0a1090 0x7f09ac0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:55.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.128+0000 7f09bffff640 1 --2- 172.21.15.16:0/2147107772 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f09ac0a1090 0x7f09ac0a3550 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f09c01971b0 tx=0x7f09a8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:55.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.130+0000 7f09bdffb640 1 -- 172.21.15.16:0/2147107772 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f09b00b7630 con 0x7f09c0100810 2024-09-13T16:34:55.294 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 13 16:34:55 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-g[72459]: ignoring --setuser ceph since I am not root 2024-09-13T16:34:55.295 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 13 16:34:55 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-g[72459]: ignoring --setgroup ceph since I am not root 2024-09-13T16:34:55.295 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 13 16:34:55 smithi016 ceph-mds[72471]: -- 172.21.15.16:0/3026493439 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fdd042cd00 con 0x55fdd0401000 2024-09-13T16:34:55.295 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 13 16:34:55 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-g[72459]: starting mds.g at 2024-09-13T16:34:55.295 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 13 16:34:55 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-a[72815]: ignoring --setuser ceph since I am not root 2024-09-13T16:34:55.295 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 13 16:34:55 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-a[72815]: ignoring --setgroup ceph since I am not root 2024-09-13T16:34:55.295 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 13 16:34:55 smithi016 ceph-mds[72819]: -- 172.21.15.16:0/3798991118 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f8dd1f8d00 con 0x55f8dd1cd000 2024-09-13T16:34:55.295 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 13 16:34:55 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-a[72815]: starting mds.a at 2024-09-13T16:34:55.296 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:54 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mon-a[31183]: 2024-09-13T16:34:54.980+0000 7f761dbbd640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T16:34:55.296 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:55 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1901539251' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:34:55.296 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:55 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3789953819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:34:55.296 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:55 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1417866865' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-13T16:34:55.296 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:55 smithi016 ceph-mon[31187]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-13T16:34:55.296 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:55 smithi016 ceph-mon[31187]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T16:34:55.296 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:55 smithi016 ceph-mon[31187]: osdmap e94: 12 total, 12 up, 12 in 2024-09-13T16:34:55.296 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:55 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1417866865' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-13T16:34:55.296 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:55 smithi016 ceph-mon[31187]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-13T16:34:55.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:55.343+0000 7f09c6c32640 1 -- 172.21.15.16:0/2147107772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f09c002c410 con 0x7f09c0100810 2024-09-13T16:34:55.349 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 13 16:34:55 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-h[63177]: ignoring --setuser ceph since I am not root 2024-09-13T16:34:55.349 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 13 16:34:55 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-h[63177]: ignoring --setgroup ceph since I am not root 2024-09-13T16:34:55.349 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 13 16:34:55 smithi107 ceph-mds[63181]: -- 172.21.15.107:0/2889446365 <== mon.1 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56264b980d00 con 0x56264b954800 2024-09-13T16:34:55.349 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 13 16:34:55 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-h[63177]: starting mds.h at 2024-09-13T16:34:55.350 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 13 16:34:55 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-e[62873]: ignoring --setuser ceph since I am not root 2024-09-13T16:34:55.350 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 13 16:34:55 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-e[62873]: ignoring --setgroup ceph since I am not root 2024-09-13T16:34:55.350 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 13 16:34:55 smithi107 ceph-mds[62877]: -- 172.21.15.107:0/1629195239 <== mon.2 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562cdf254d00 con 0x562cdf229000 2024-09-13T16:34:55.350 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 13 16:34:55 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-e[62873]: starting mds.e at 2024-09-13T16:34:55.350 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 13 16:34:55 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-k[62559]: ignoring --setuser ceph since I am not root 2024-09-13T16:34:55.351 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 13 16:34:55 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-k[62559]: ignoring --setgroup ceph since I am not root 2024-09-13T16:34:55.352 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 13 16:34:55 smithi107 ceph-mds[62563]: -- 172.21.15.107:0/3012098918 <== mon.2 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556d9d31ad00 con 0x556d9d2ef400 2024-09-13T16:34:55.352 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 13 16:34:55 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-k[62559]: starting mds.k at 2024-09-13T16:34:55.353 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:55 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1901539251' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:34:55.353 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:55 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3789953819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:34:55.353 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:55 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1417866865' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-13T16:34:55.353 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:55 smithi107 ceph-mon[34139]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-13T16:34:55.353 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:55 smithi107 ceph-mon[34139]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T16:34:55.353 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:55 smithi107 ceph-mon[34139]: osdmap e94: 12 total, 12 up, 12 in 2024-09-13T16:34:55.353 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:55 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1417866865' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-13T16:34:55.353 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:55 smithi107 ceph-mon[34139]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-13T16:34:55.353 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 13 16:34:55 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-b[63478]: ignoring --setuser ceph since I am not root 2024-09-13T16:34:55.353 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 13 16:34:55 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-b[63478]: ignoring --setgroup ceph since I am not root 2024-09-13T16:34:55.354 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 13 16:34:55 smithi107 ceph-mds[63482]: -- 172.21.15.107:0/2216587574 <== mon.1 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55990cb54d00 con 0x55990cb29000 2024-09-13T16:34:55.354 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 13 16:34:55 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-b[63478]: starting mds.b at 2024-09-13T16:34:55.437 INFO:journalctl@ceph.mds.c.smithi111.stdout:Sep 13 16:34:55 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-c[65481]: ignoring --setuser ceph since I am not root 2024-09-13T16:34:55.437 INFO:journalctl@ceph.mds.c.smithi111.stdout:Sep 13 16:34:55 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-c[65481]: ignoring --setgroup ceph since I am not root 2024-09-13T16:34:55.437 INFO:journalctl@ceph.mds.c.smithi111.stdout:Sep 13 16:34:55 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-c[65481]: starting mds.c at 2024-09-13T16:34:55.438 INFO:journalctl@ceph.mds.f.smithi111.stdout:Sep 13 16:34:55 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-f[65175]: ignoring --setuser ceph since I am not root 2024-09-13T16:34:55.438 INFO:journalctl@ceph.mds.f.smithi111.stdout:Sep 13 16:34:55 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-f[65175]: ignoring --setgroup ceph since I am not root 2024-09-13T16:34:55.438 INFO:journalctl@ceph.mds.f.smithi111.stdout:Sep 13 16:34:55 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-f[65175]: starting mds.f at 2024-09-13T16:34:55.438 INFO:journalctl@ceph.mds.l.smithi111.stdout:Sep 13 16:34:55 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-l[64860]: ignoring --setuser ceph since I am not root 2024-09-13T16:34:55.438 INFO:journalctl@ceph.mds.l.smithi111.stdout:Sep 13 16:34:55 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-l[64860]: ignoring --setgroup ceph since I am not root 2024-09-13T16:34:55.438 INFO:journalctl@ceph.mds.l.smithi111.stdout:Sep 13 16:34:55 smithi111 ceph-mds[64864]: -- 172.21.15.111:0/2401979473 <== mon.1 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558e07edcd00 con 0x558e07eb1000 2024-09-13T16:34:55.438 INFO:journalctl@ceph.mds.l.smithi111.stdout:Sep 13 16:34:55 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-l[64860]: starting mds.l at 2024-09-13T16:34:55.439 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:55 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1901539251' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T16:34:55.439 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:55 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3789953819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T16:34:55.439 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:55 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1417866865' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-13T16:34:55.439 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:55 smithi111 ceph-mon[34600]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-13T16:34:55.439 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:55 smithi111 ceph-mon[34600]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T16:34:55.439 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:55 smithi111 ceph-mon[34600]: osdmap e94: 12 total, 12 up, 12 in 2024-09-13T16:34:55.439 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:55 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1417866865' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-13T16:34:55.439 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:55 smithi111 ceph-mon[34600]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-13T16:34:55.440 INFO:journalctl@ceph.mds.i.smithi111.stdout:Sep 13 16:34:55 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-i[65785]: ignoring --setuser ceph since I am not root 2024-09-13T16:34:55.440 INFO:journalctl@ceph.mds.i.smithi111.stdout:Sep 13 16:34:55 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-i[65785]: ignoring --setgroup ceph since I am not root 2024-09-13T16:34:55.440 INFO:journalctl@ceph.mds.i.smithi111.stdout:Sep 13 16:34:55 smithi111 ceph-mds[65789]: -- 172.21.15.111:0/686546686 <== mon.1 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563f2dbced00 con 0x563f2dba2800 2024-09-13T16:34:55.440 INFO:journalctl@ceph.mds.i.smithi111.stdout:Sep 13 16:34:55 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-i[65785]: starting mds.i at 2024-09-13T16:34:56.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.042+0000 7f09bdffb640 1 -- 172.21.15.16:0/2147107772 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7f09b0003e60 con 0x7f09c0100810 2024-09-13T16:34:56.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.045+0000 7f09c6c32640 1 -- 172.21.15.16:0/2147107772 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f09ac0a1090 msgr2=0x7f09ac0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:56.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.045+0000 7f09c6c32640 1 --2- 172.21.15.16:0/2147107772 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f09ac0a1090 0x7f09ac0a3550 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f09c01971b0 tx=0x7f09a8005f90 comp rx=0 tx=0).stop 2024-09-13T16:34:56.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.045+0000 7f09c6c32640 1 -- 172.21.15.16:0/2147107772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09c0100810 msgr2=0x7f09c0195cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:56.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.045+0000 7f09c6c32640 1 --2- 172.21.15.16:0/2147107772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09c0100810 0x7f09c0195cb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009dc0 tx=0x7f09b0030f20 comp rx=0 tx=0).stop 2024-09-13T16:34:56.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.045+0000 7f09c6c32640 1 -- 172.21.15.16:0/2147107772 shutdown_connections 2024-09-13T16:34:56.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.045+0000 7f09c6c32640 1 --2- 172.21.15.16:0/2147107772 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f09ac0a1090 0x7f09ac0a3550 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:56.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.045+0000 7f09c6c32640 1 --2- 172.21.15.16:0/2147107772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09c0105a70 0x7f09c019d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:56.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.045+0000 7f09c6c32640 1 --2- 172.21.15.16:0/2147107772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09c0103140 0x7f09c01961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:56.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.045+0000 7f09c6c32640 1 --2- 172.21.15.16:0/2147107772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09c0100810 0x7f09c0195cb0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:56.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.045+0000 7f09c6c32640 1 -- 172.21.15.16:0/2147107772 >> 172.21.15.16:0/2147107772 conn(0x7f09c00fa3f0 msgr2=0x7f09c01010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:56.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.046+0000 7f09c6c32640 1 -- 172.21.15.16:0/2147107772 shutdown_connections 2024-09-13T16:34:56.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.046+0000 7f09c6c32640 1 -- 172.21.15.16:0/2147107772 wait complete. 2024-09-13T16:34:56.060 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-13T16:34:56.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.168+0000 7f602b15e640 1 -- 172.21.15.16:0/3845316455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60240ffe50 msgr2=0x7f60241002d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:56.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.168+0000 7f602b15e640 1 --2- 172.21.15.16:0/3845316455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60240ffe50 0x7f60241002d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6010009900 tx=0x7f601002a3f0 comp rx=0 tx=0).stop 2024-09-13T16:34:56.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.168+0000 7f602b15e640 1 -- 172.21.15.16:0/3845316455 shutdown_connections 2024-09-13T16:34:56.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.168+0000 7f602b15e640 1 --2- 172.21.15.16:0/3845316455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6024100810 0x7f60241070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:56.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.168+0000 7f602b15e640 1 --2- 172.21.15.16:0/3845316455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60240ffe50 0x7f60241002d0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:56.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.168+0000 7f602b15e640 1 --2- 172.21.15.16:0/3845316455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60240fec60 0x7f60240ff060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:56.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.168+0000 7f602b15e640 1 -- 172.21.15.16:0/3845316455 >> 172.21.15.16:0/3845316455 conn(0x7f60240fa3e0 msgr2=0x7f60240fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:56.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.168+0000 7f602b15e640 1 -- 172.21.15.16:0/3845316455 shutdown_connections 2024-09-13T16:34:56.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.168+0000 7f602b15e640 1 -- 172.21.15.16:0/3845316455 wait complete. 2024-09-13T16:34:56.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.169+0000 7f602b15e640 1 Processor -- start 2024-09-13T16:34:56.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.169+0000 7f602b15e640 1 -- start start 2024-09-13T16:34:56.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.169+0000 7f602b15e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60240fec60 0x7f6024068110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:56.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.169+0000 7f602b15e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60240ffe50 0x7f602419a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:56.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.169+0000 7f602b15e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6024100810 0x7f602419cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:56.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.169+0000 7f602b15e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602419d6e0 con 0x7f6024100810 2024-09-13T16:34:56.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.169+0000 7f602b15e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602419d850 con 0x7f60240fec60 2024-09-13T16:34:56.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.169+0000 7f602b15e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602419da10 con 0x7f60240ffe50 2024-09-13T16:34:56.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.169+0000 7f60296fb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6024100810 0x7f602419cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:56.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.169+0000 7f60296fb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6024100810 0x7f602419cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52672/0 (socket says 172.21.15.16:52672) 2024-09-13T16:34:56.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.169+0000 7f60296fb640 1 -- 172.21.15.16:0/1815625089 learned_addr learned my addr 172.21.15.16:0/1815625089 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:34:56.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.169+0000 7f6023fff640 1 --2- 172.21.15.16:0/1815625089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60240ffe50 0x7f602419a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:56.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.169+0000 7f6028efa640 1 --2- 172.21.15.16:0/1815625089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60240fec60 0x7f6024068110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:56.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.169+0000 7f60296fb640 1 -- 172.21.15.16:0/1815625089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60240ffe50 msgr2=0x7f602419a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:56.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.169+0000 7f60296fb640 1 --2- 172.21.15.16:0/1815625089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60240ffe50 0x7f602419a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:56.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.169+0000 7f60296fb640 1 -- 172.21.15.16:0/1815625089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60240fec60 msgr2=0x7f6024068110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:56.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.169+0000 7f60296fb640 1 --2- 172.21.15.16:0/1815625089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60240fec60 0x7f6024068110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:56.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.169+0000 7f60296fb640 1 -- 172.21.15.16:0/1815625089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6010009560 con 0x7f6024100810 2024-09-13T16:34:56.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.169+0000 7f6023fff640 1 --2- 172.21.15.16:0/1815625089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60240ffe50 0x7f602419a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:34:56.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.170+0000 7f60296fb640 1 --2- 172.21.15.16:0/1815625089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6024100810 0x7f602419cf00 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f601c00cce0 tx=0x7f601c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:56.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.170+0000 7f6021ffb640 1 -- 172.21.15.16:0/1815625089 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c00f450 con 0x7f6024100810 2024-09-13T16:34:56.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.170+0000 7f602b15e640 1 -- 172.21.15.16:0/1815625089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f602419dc90 con 0x7f6024100810 2024-09-13T16:34:56.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.170+0000 7f6021ffb640 1 -- 172.21.15.16:0/1815625089 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f601c00fa40 con 0x7f6024100810 2024-09-13T16:34:56.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.170+0000 7f6021ffb640 1 -- 172.21.15.16:0/1815625089 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c0189a0 con 0x7f6024100810 2024-09-13T16:34:56.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.170+0000 7f602b15e640 1 -- 172.21.15.16:0/1815625089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f602419e120 con 0x7f6024100810 2024-09-13T16:34:56.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.171+0000 7f602b15e640 1 -- 172.21.15.16:0/1815625089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60241060a0 con 0x7f6024100810 2024-09-13T16:34:56.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.172+0000 7f6021ffb640 1 -- 172.21.15.16:0/1815625089 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f601c00fbb0 con 0x7f6024100810 2024-09-13T16:34:56.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.173+0000 7f6021ffb640 1 --2- 172.21.15.16:0/1815625089 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f60140a1090 0x7f60140a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:56.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.173+0000 7f6021ffb640 1 -- 172.21.15.16:0/1815625089 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9508+0+0 (secure 0 0 0) 0x7f601c0cd360 con 0x7f6024100810 2024-09-13T16:34:56.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.173+0000 7f6028efa640 1 --2- 172.21.15.16:0/1815625089 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f60140a1090 0x7f60140a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:56.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.173+0000 7f6028efa640 1 --2- 172.21.15.16:0/1815625089 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f60140a1090 0x7f60140a3550 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f60180058c0 tx=0x7f6018005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:56.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.177+0000 7f6021ffb640 1 -- 172.21.15.16:0/1815625089 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f601c096590 con 0x7f6024100810 2024-09-13T16:34:56.310 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:56 smithi016 ceph-mon[31187]: pgmap v468: 129 pgs: 129 active+clean; 1020 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-13T16:34:56.310 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:56 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2147107772' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-13T16:34:56.310 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:56 smithi016 ceph-mon[31187]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-13T16:34:56.310 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:56 smithi016 ceph-mon[31187]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-13T16:34:56.310 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:56 smithi016 ceph-mon[31187]: Cluster is now healthy 2024-09-13T16:34:56.310 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:56 smithi016 ceph-mon[31187]: mds.? [v2:172.21.15.16:6838/3777765682,v1:172.21.15.16:6840/3777765682] up:boot 2024-09-13T16:34:56.311 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:56 smithi016 ceph-mon[31187]: mds.? [v2:172.21.15.111:6832/50565955,v1:172.21.15.111:6834/50565955] up:boot 2024-09-13T16:34:56.311 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:56 smithi016 ceph-mon[31187]: mds.? [v2:172.21.15.16:6839/1685969224,v1:172.21.15.16:6841/1685969224] up:boot 2024-09-13T16:34:56.311 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:56 smithi016 ceph-mon[31187]: mds.? [v2:172.21.15.111:6833/2788481324,v1:172.21.15.111:6835/2788481324] up:boot 2024-09-13T16:34:56.311 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:56 smithi016 ceph-mon[31187]: mds.? [v2:172.21.15.107:6832/1722955563,v1:172.21.15.107:6834/1722955563] up:boot 2024-09-13T16:34:56.311 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:56 smithi016 ceph-mon[31187]: mds.? [v2:172.21.15.107:6833/3777340178,v1:172.21.15.107:6835/3777340178] up:boot 2024-09-13T16:34:56.311 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:56 smithi016 ceph-mon[31187]: mds.? [v2:172.21.15.111:6836/235772938,v1:172.21.15.111:6837/235772938] up:boot 2024-09-13T16:34:56.311 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:56 smithi016 ceph-mon[31187]: mds.? [v2:172.21.15.107:6837/4000005587,v1:172.21.15.107:6839/4000005587] up:boot 2024-09-13T16:34:56.311 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:56 smithi016 ceph-mon[31187]: mds.? [v2:172.21.15.111:6838/1820015513,v1:172.21.15.111:6839/1820015513] up:boot 2024-09-13T16:34:56.311 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:56 smithi016 ceph-mon[31187]: mds.? [v2:172.21.15.107:6836/1942902878,v1:172.21.15.107:6838/1942902878] up:boot 2024-09-13T16:34:56.311 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:56 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/2147107772' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-13T16:34:56.311 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:56 smithi016 ceph-mon[31187]: fsmap 12 up:standby 2024-09-13T16:34:56.311 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:56 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-13T16:34:56.311 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:56 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-13T16:34:56.312 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:56 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-13T16:34:56.312 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:56 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-13T16:34:56.312 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:56 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-13T16:34:56.312 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:56 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-13T16:34:56.312 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:56 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-13T16:34:56.312 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:56 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-13T16:34:56.312 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:56 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-13T16:34:56.312 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:56 smithi016 ceph-mon[31187]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-13T16:34:56.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:56 smithi107 ceph-mon[34139]: pgmap v468: 129 pgs: 129 active+clean; 1020 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-13T16:34:56.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:56 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/2147107772' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-13T16:34:56.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:56 smithi107 ceph-mon[34139]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-13T16:34:56.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:56 smithi107 ceph-mon[34139]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-13T16:34:56.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:56 smithi107 ceph-mon[34139]: Cluster is now healthy 2024-09-13T16:34:56.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:56 smithi107 ceph-mon[34139]: mds.? [v2:172.21.15.16:6838/3777765682,v1:172.21.15.16:6840/3777765682] up:boot 2024-09-13T16:34:56.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:56 smithi107 ceph-mon[34139]: mds.? [v2:172.21.15.111:6832/50565955,v1:172.21.15.111:6834/50565955] up:boot 2024-09-13T16:34:56.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:56 smithi107 ceph-mon[34139]: mds.? [v2:172.21.15.16:6839/1685969224,v1:172.21.15.16:6841/1685969224] up:boot 2024-09-13T16:34:56.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:56 smithi107 ceph-mon[34139]: mds.? [v2:172.21.15.111:6833/2788481324,v1:172.21.15.111:6835/2788481324] up:boot 2024-09-13T16:34:56.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:56 smithi107 ceph-mon[34139]: mds.? [v2:172.21.15.107:6832/1722955563,v1:172.21.15.107:6834/1722955563] up:boot 2024-09-13T16:34:56.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:56 smithi107 ceph-mon[34139]: mds.? [v2:172.21.15.107:6833/3777340178,v1:172.21.15.107:6835/3777340178] up:boot 2024-09-13T16:34:56.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:56 smithi107 ceph-mon[34139]: mds.? [v2:172.21.15.111:6836/235772938,v1:172.21.15.111:6837/235772938] up:boot 2024-09-13T16:34:56.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:56 smithi107 ceph-mon[34139]: mds.? [v2:172.21.15.107:6837/4000005587,v1:172.21.15.107:6839/4000005587] up:boot 2024-09-13T16:34:56.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:56 smithi107 ceph-mon[34139]: mds.? [v2:172.21.15.111:6838/1820015513,v1:172.21.15.111:6839/1820015513] up:boot 2024-09-13T16:34:56.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:56 smithi107 ceph-mon[34139]: mds.? [v2:172.21.15.107:6836/1942902878,v1:172.21.15.107:6838/1942902878] up:boot 2024-09-13T16:34:56.351 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:56 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/2147107772' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-13T16:34:56.351 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:56 smithi107 ceph-mon[34139]: fsmap 12 up:standby 2024-09-13T16:34:56.351 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:56 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-13T16:34:56.351 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:56 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-13T16:34:56.351 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:56 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-13T16:34:56.351 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:56 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-13T16:34:56.351 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:56 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-13T16:34:56.351 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:56 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-13T16:34:56.351 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:56 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-13T16:34:56.351 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:56 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-13T16:34:56.351 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:56 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-13T16:34:56.351 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:56 smithi107 ceph-mon[34139]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-13T16:34:56.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:56.351+0000 7f602b15e640 1 -- 172.21.15.16:0/1815625089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f602410a3b0 con 0x7f6024100810 2024-09-13T16:34:56.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:56 smithi111 ceph-mon[34600]: pgmap v468: 129 pgs: 129 active+clean; 1020 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-13T16:34:56.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:56 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/2147107772' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-13T16:34:56.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:56 smithi111 ceph-mon[34600]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-13T16:34:56.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:56 smithi111 ceph-mon[34600]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-13T16:34:56.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:56 smithi111 ceph-mon[34600]: Cluster is now healthy 2024-09-13T16:34:56.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:56 smithi111 ceph-mon[34600]: mds.? [v2:172.21.15.16:6838/3777765682,v1:172.21.15.16:6840/3777765682] up:boot 2024-09-13T16:34:56.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:56 smithi111 ceph-mon[34600]: mds.? [v2:172.21.15.111:6832/50565955,v1:172.21.15.111:6834/50565955] up:boot 2024-09-13T16:34:56.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:56 smithi111 ceph-mon[34600]: mds.? [v2:172.21.15.16:6839/1685969224,v1:172.21.15.16:6841/1685969224] up:boot 2024-09-13T16:34:56.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:56 smithi111 ceph-mon[34600]: mds.? [v2:172.21.15.111:6833/2788481324,v1:172.21.15.111:6835/2788481324] up:boot 2024-09-13T16:34:56.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:56 smithi111 ceph-mon[34600]: mds.? [v2:172.21.15.107:6832/1722955563,v1:172.21.15.107:6834/1722955563] up:boot 2024-09-13T16:34:56.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:56 smithi111 ceph-mon[34600]: mds.? [v2:172.21.15.107:6833/3777340178,v1:172.21.15.107:6835/3777340178] up:boot 2024-09-13T16:34:56.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:56 smithi111 ceph-mon[34600]: mds.? [v2:172.21.15.111:6836/235772938,v1:172.21.15.111:6837/235772938] up:boot 2024-09-13T16:34:56.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:56 smithi111 ceph-mon[34600]: mds.? [v2:172.21.15.107:6837/4000005587,v1:172.21.15.107:6839/4000005587] up:boot 2024-09-13T16:34:56.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:56 smithi111 ceph-mon[34600]: mds.? [v2:172.21.15.111:6838/1820015513,v1:172.21.15.111:6839/1820015513] up:boot 2024-09-13T16:34:56.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:56 smithi111 ceph-mon[34600]: mds.? [v2:172.21.15.107:6836/1942902878,v1:172.21.15.107:6838/1942902878] up:boot 2024-09-13T16:34:56.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:56 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/2147107772' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-13T16:34:56.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:56 smithi111 ceph-mon[34600]: fsmap 12 up:standby 2024-09-13T16:34:56.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:56 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-13T16:34:56.438 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:56 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-13T16:34:56.439 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:56 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-13T16:34:56.439 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:56 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-13T16:34:56.439 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:56 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-13T16:34:56.439 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:56 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-13T16:34:56.439 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:56 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-13T16:34:56.439 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:56 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-13T16:34:56.439 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:56 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-13T16:34:56.439 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:56 smithi111 ceph-mon[34600]: from='mgr.14150 172.21.15.16:0/4196472327' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-13T16:34:57.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.080+0000 7f6021ffb640 1 -- 172.21.15.16:0/1815625089 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v95) v1 ==== 181+0+0 (secure 0 0 0) 0x7f601c094e60 con 0x7f6024100810 2024-09-13T16:34:57.082 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-13T16:34:57.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.084+0000 7f602b15e640 1 -- 172.21.15.16:0/1815625089 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f60140a1090 msgr2=0x7f60140a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:57.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.084+0000 7f602b15e640 1 --2- 172.21.15.16:0/1815625089 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f60140a1090 0x7f60140a3550 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f60180058c0 tx=0x7f6018005850 comp rx=0 tx=0).stop 2024-09-13T16:34:57.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.084+0000 7f602b15e640 1 -- 172.21.15.16:0/1815625089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6024100810 msgr2=0x7f602419cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:57.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.084+0000 7f602b15e640 1 --2- 172.21.15.16:0/1815625089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6024100810 0x7f602419cf00 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f601c00cce0 tx=0x7f601c00a620 comp rx=0 tx=0).stop 2024-09-13T16:34:57.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.084+0000 7f602b15e640 1 -- 172.21.15.16:0/1815625089 shutdown_connections 2024-09-13T16:34:57.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.084+0000 7f602b15e640 1 --2- 172.21.15.16:0/1815625089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6024100810 0x7f602419cf00 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:57.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.084+0000 7f602b15e640 1 --2- 172.21.15.16:0/1815625089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60240ffe50 0x7f602419a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:57.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.084+0000 7f602b15e640 1 --2- 172.21.15.16:0/1815625089 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f60140a1090 0x7f60140a3550 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:57.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.084+0000 7f602b15e640 1 --2- 172.21.15.16:0/1815625089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60240fec60 0x7f6024068110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:57.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.084+0000 7f602b15e640 1 -- 172.21.15.16:0/1815625089 >> 172.21.15.16:0/1815625089 conn(0x7f60240fa3e0 msgr2=0x7f6024103270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:57.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.084+0000 7f602b15e640 1 -- 172.21.15.16:0/1815625089 shutdown_connections 2024-09-13T16:34:57.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.084+0000 7f602b15e640 1 -- 172.21.15.16:0/1815625089 wait complete. 2024-09-13T16:34:57.099 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-13T16:34:57.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.211+0000 7f81f0472640 1 -- 172.21.15.16:0/2527505667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81e81013b0 msgr2=0x7f81e81017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:57.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.211+0000 7f81f0472640 1 --2- 172.21.15.16:0/2527505667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81e81013b0 0x7f81e81017b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009930 tx=0x7f81e402a420 comp rx=0 tx=0).stop 2024-09-13T16:34:57.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.211+0000 7f81f0472640 1 -- 172.21.15.16:0/2527505667 shutdown_connections 2024-09-13T16:34:57.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.211+0000 7f81f0472640 1 --2- 172.21.15.16:0/2527505667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81e80fe7e0 0x7f81e8108390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:57.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.211+0000 7f81f0472640 1 --2- 172.21.15.16:0/2527505667 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81e80fde20 0x7f81e80fe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:57.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.211+0000 7f81f0472640 1 --2- 172.21.15.16:0/2527505667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81e81013b0 0x7f81e81017b0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:57.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.211+0000 7f81f0472640 1 -- 172.21.15.16:0/2527505667 >> 172.21.15.16:0/2527505667 conn(0x7f81e80f9b70 msgr2=0x7f81e80fbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:57.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.211+0000 7f81f0472640 1 -- 172.21.15.16:0/2527505667 shutdown_connections 2024-09-13T16:34:57.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.211+0000 7f81f0472640 1 -- 172.21.15.16:0/2527505667 wait complete. 2024-09-13T16:34:57.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.212+0000 7f81f0472640 1 Processor -- start 2024-09-13T16:34:57.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.212+0000 7f81f0472640 1 -- start start 2024-09-13T16:34:57.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.212+0000 7f81f0472640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81e80fde20 0x7f81e8100ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:57.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.212+0000 7f81f0472640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81e80fe7e0 0x7f81e80ff0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:57.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.212+0000 7f81f0472640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81e81013b0 0x7f81e80ff630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:57.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.212+0000 7f81f0472640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e80ffd90 con 0x7f81e80fe7e0 2024-09-13T16:34:57.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.212+0000 7f81f0472640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e80fff00 con 0x7f81e81013b0 2024-09-13T16:34:57.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.212+0000 7f81f0472640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e819fd90 con 0x7f81e80fde20 2024-09-13T16:34:57.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.212+0000 7f81eda0d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81e80fe7e0 0x7f81e80ff0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:57.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.212+0000 7f81eda0d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81e80fe7e0 0x7f81e80ff0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52688/0 (socket says 172.21.15.16:52688) 2024-09-13T16:34:57.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.212+0000 7f81eda0d640 1 -- 172.21.15.16:0/3808824695 learned_addr learned my addr 172.21.15.16:0/3808824695 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:34:57.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.212+0000 7f81eea0f640 1 --2- 172.21.15.16:0/3808824695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81e81013b0 0x7f81e80ff630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:57.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.212+0000 7f81ee20e640 1 --2- 172.21.15.16:0/3808824695 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81e80fde20 0x7f81e8100ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:57.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.212+0000 7f81eda0d640 1 -- 172.21.15.16:0/3808824695 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81e80fde20 msgr2=0x7f81e8100ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:57.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.212+0000 7f81eda0d640 1 --2- 172.21.15.16:0/3808824695 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81e80fde20 0x7f81e8100ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:57.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.212+0000 7f81eda0d640 1 -- 172.21.15.16:0/3808824695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81e81013b0 msgr2=0x7f81e80ff630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:57.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.212+0000 7f81eda0d640 1 --2- 172.21.15.16:0/3808824695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81e81013b0 0x7f81e80ff630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:57.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.212+0000 7f81eda0d640 1 -- 172.21.15.16:0/3808824695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e4009560 con 0x7f81e80fe7e0 2024-09-13T16:34:57.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.212+0000 7f81eea0f640 1 --2- 172.21.15.16:0/3808824695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81e81013b0 0x7f81e80ff630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:34:57.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.213+0000 7f81eda0d640 1 --2- 172.21.15.16:0/3808824695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81e80fe7e0 0x7f81e80ff0f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f81d800ea20 tx=0x7f81d800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:57.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.213+0000 7f81df7fe640 1 -- 172.21.15.16:0/3808824695 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d8009720 con 0x7f81e80fe7e0 2024-09-13T16:34:57.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.213+0000 7f81df7fe640 1 -- 172.21.15.16:0/3808824695 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f81d800cd80 con 0x7f81e80fe7e0 2024-09-13T16:34:57.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.213+0000 7f81f0472640 1 -- 172.21.15.16:0/3808824695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e819ffe0 con 0x7f81e80fe7e0 2024-09-13T16:34:57.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.213+0000 7f81df7fe640 1 -- 172.21.15.16:0/3808824695 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d8010430 con 0x7f81e80fe7e0 2024-09-13T16:34:57.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.213+0000 7f81f0472640 1 -- 172.21.15.16:0/3808824695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e81a04c0 con 0x7f81e80fe7e0 2024-09-13T16:34:57.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.214+0000 7f81f0472640 1 -- 172.21.15.16:0/3808824695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e81074d0 con 0x7f81e80fe7e0 2024-09-13T16:34:57.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.215+0000 7f81df7fe640 1 -- 172.21.15.16:0/3808824695 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f81d8004f10 con 0x7f81e80fe7e0 2024-09-13T16:34:57.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.216+0000 7f81df7fe640 1 --2- 172.21.15.16:0/3808824695 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f81d40a1110 0x7f81d40a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:57.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.216+0000 7f81ee20e640 1 --2- 172.21.15.16:0/3808824695 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f81d40a1110 0x7f81d40a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:57.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.216+0000 7f81df7fe640 1 -- 172.21.15.16:0/3808824695 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8954+0+0 (secure 0 0 0) 0x7f81d8014070 con 0x7f81e80fe7e0 2024-09-13T16:34:57.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.216+0000 7f81ee20e640 1 --2- 172.21.15.16:0/3808824695 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f81d40a1110 0x7f81d40a35d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009900 tx=0x7f81e4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:57.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.219+0000 7f81df7fe640 1 -- 172.21.15.16:0/3808824695 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f81d8096660 con 0x7f81e80fe7e0 2024-09-13T16:34:57.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:57 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1815625089' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T16:34:57.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:57 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1815625089' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T16:34:57.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:57.389+0000 7f81f0472640 1 -- 172.21.15.16:0/3808824695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f81e810b510 con 0x7f81e80fe7e0 2024-09-13T16:34:57.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:57 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1815625089' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T16:34:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.082+0000 7f81df7fe640 1 -- 172.21.15.16:0/3808824695 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v96) v1 ==== 169+0+0 (secure 0 0 0) 0x7f81d8099c80 con 0x7f81e80fe7e0 2024-09-13T16:34:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-13T16:34:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.086+0000 7f81f0472640 1 -- 172.21.15.16:0/3808824695 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f81d40a1110 msgr2=0x7f81d40a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.086+0000 7f81f0472640 1 --2- 172.21.15.16:0/3808824695 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f81d40a1110 0x7f81d40a35d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009900 tx=0x7f81e4005a20 comp rx=0 tx=0).stop 2024-09-13T16:34:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.086+0000 7f81f0472640 1 -- 172.21.15.16:0/3808824695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81e80fe7e0 msgr2=0x7f81e80ff0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.086+0000 7f81f0472640 1 --2- 172.21.15.16:0/3808824695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81e80fe7e0 0x7f81e80ff0f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f81d800ea20 tx=0x7f81d800ee20 comp rx=0 tx=0).stop 2024-09-13T16:34:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.086+0000 7f81f0472640 1 -- 172.21.15.16:0/3808824695 shutdown_connections 2024-09-13T16:34:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.086+0000 7f81f0472640 1 --2- 172.21.15.16:0/3808824695 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f81d40a1110 0x7f81d40a35d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.086+0000 7f81f0472640 1 --2- 172.21.15.16:0/3808824695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81e81013b0 0x7f81e80ff630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.086+0000 7f81f0472640 1 --2- 172.21.15.16:0/3808824695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81e80fe7e0 0x7f81e80ff0f0 secure :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f81d800ea20 tx=0x7f81d800ee20 comp rx=0 tx=0).stop 2024-09-13T16:34:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.086+0000 7f81f0472640 1 --2- 172.21.15.16:0/3808824695 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81e80fde20 0x7f81e8100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.086+0000 7f81f0472640 1 -- 172.21.15.16:0/3808824695 >> 172.21.15.16:0/3808824695 conn(0x7f81e80f9b70 msgr2=0x7f81e81046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.086+0000 7f81f0472640 1 -- 172.21.15.16:0/3808824695 shutdown_connections 2024-09-13T16:34:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.086+0000 7f81f0472640 1 -- 172.21.15.16:0/3808824695 wait complete. 2024-09-13T16:34:58.106 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-13T16:34:58.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.220+0000 7f7f871d2640 1 -- 172.21.15.16:0/1746486045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f800fec90 msgr2=0x7f7f800ff090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:58.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.220+0000 7f7f871d2640 1 --2- 172.21.15.16:0/1746486045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f800fec90 0x7f7f800ff090 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f7f74009900 tx=0x7f7f7402a410 comp rx=0 tx=0).stop 2024-09-13T16:34:58.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.220+0000 7f7f871d2640 1 -- 172.21.15.16:0/1746486045 shutdown_connections 2024-09-13T16:34:58.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.220+0000 7f7f871d2640 1 --2- 172.21.15.16:0/1746486045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f80100840 0x7f7f801070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:58.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.220+0000 7f7f871d2640 1 --2- 172.21.15.16:0/1746486045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f800ffe80 0x7f7f80100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:58.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.220+0000 7f7f871d2640 1 --2- 172.21.15.16:0/1746486045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f800fec90 0x7f7f800ff090 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:58.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.220+0000 7f7f871d2640 1 -- 172.21.15.16:0/1746486045 >> 172.21.15.16:0/1746486045 conn(0x7f7f800fa3f0 msgr2=0x7f7f800fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:58.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.220+0000 7f7f871d2640 1 -- 172.21.15.16:0/1746486045 shutdown_connections 2024-09-13T16:34:58.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.220+0000 7f7f871d2640 1 -- 172.21.15.16:0/1746486045 wait complete. 2024-09-13T16:34:58.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.221+0000 7f7f871d2640 1 Processor -- start 2024-09-13T16:34:58.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.221+0000 7f7f871d2640 1 -- start start 2024-09-13T16:34:58.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.221+0000 7f7f871d2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f800fec90 0x7f7f80195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:58.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.221+0000 7f7f871d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f800ffe80 0x7f7f80196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:58.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.221+0000 7f7f871d2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f80100840 0x7f7f8019d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:58.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.221+0000 7f7f871d2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8019d830 con 0x7f7f800ffe80 2024-09-13T16:34:58.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.221+0000 7f7f871d2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8019d9a0 con 0x7f7f800fec90 2024-09-13T16:34:58.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.221+0000 7f7f871d2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8019db60 con 0x7f7f80100840 2024-09-13T16:34:58.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.221+0000 7f7f7ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f800ffe80 0x7f7f80196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:58.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.221+0000 7f7f7ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f800ffe80 0x7f7f80196290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52720/0 (socket says 172.21.15.16:52720) 2024-09-13T16:34:58.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.221+0000 7f7f7ffff640 1 -- 172.21.15.16:0/494229745 learned_addr learned my addr 172.21.15.16:0/494229745 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-13T16:34:58.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.221+0000 7f7f84f6e640 1 --2- 172.21.15.16:0/494229745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f800fec90 0x7f7f80195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:58.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.221+0000 7f7f8576f640 1 --2- 172.21.15.16:0/494229745 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f80100840 0x7f7f8019d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:58.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.222+0000 7f7f7ffff640 1 -- 172.21.15.16:0/494229745 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f80100840 msgr2=0x7f7f8019d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:58.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.222+0000 7f7f7ffff640 1 --2- 172.21.15.16:0/494229745 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f80100840 0x7f7f8019d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:58.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.222+0000 7f7f7ffff640 1 -- 172.21.15.16:0/494229745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f800fec90 msgr2=0x7f7f80195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:58.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.222+0000 7f7f7ffff640 1 --2- 172.21.15.16:0/494229745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f800fec90 0x7f7f80195d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:58.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.222+0000 7f7f7ffff640 1 -- 172.21.15.16:0/494229745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f74009560 con 0x7f7f800ffe80 2024-09-13T16:34:58.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.222+0000 7f7f84f6e640 1 --2- 172.21.15.16:0/494229745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f800fec90 0x7f7f80195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:34:58.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.222+0000 7f7f7ffff640 1 --2- 172.21.15.16:0/494229745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f800ffe80 0x7f7f80196290 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00b4a0 tx=0x7f7f6c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:58.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.222+0000 7f7f7dffb640 1 -- 172.21.15.16:0/494229745 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c010610 con 0x7f7f800ffe80 2024-09-13T16:34:58.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.222+0000 7f7f7dffb640 1 -- 172.21.15.16:0/494229745 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7f6c010c00 con 0x7f7f800ffe80 2024-09-13T16:34:58.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.222+0000 7f7f871d2640 1 -- 172.21.15.16:0/494229745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f8019de40 con 0x7f7f800ffe80 2024-09-13T16:34:58.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.222+0000 7f7f7dffb640 1 -- 172.21.15.16:0/494229745 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c017ca0 con 0x7f7f800ffe80 2024-09-13T16:34:58.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.222+0000 7f7f871d2640 1 -- 172.21.15.16:0/494229745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f8019e270 con 0x7f7f800ffe80 2024-09-13T16:34:58.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.223+0000 7f7f871d2640 1 -- 172.21.15.16:0/494229745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f8010a480 con 0x7f7f800ffe80 2024-09-13T16:34:58.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.224+0000 7f7f7dffb640 1 -- 172.21.15.16:0/494229745 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7f6c010770 con 0x7f7f800ffe80 2024-09-13T16:34:58.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.225+0000 7f7f7dffb640 1 --2- 172.21.15.16:0/494229745 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7f700a1110 0x7f7f700a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:34:58.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.225+0000 7f7f84f6e640 1 --2- 172.21.15.16:0/494229745 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7f700a1110 0x7f7f700a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:34:58.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.225+0000 7f7f7dffb640 1 -- 172.21.15.16:0/494229745 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8424+0+0 (secure 0 0 0) 0x7f7f6c0cc3a0 con 0x7f7f800ffe80 2024-09-13T16:34:58.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.226+0000 7f7f84f6e640 1 --2- 172.21.15.16:0/494229745 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7f700a1110 0x7f7f700a35d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7f7402aa00 tx=0x7f7f74005e00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:34:58.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.229+0000 7f7f7dffb640 1 -- 172.21.15.16:0/494229745 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7f6c0959e0 con 0x7f7f800ffe80 2024-09-13T16:34:58.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:58 smithi107 ceph-mon[34139]: pgmap v470: 129 pgs: 129 active+clean; 1020 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-09-13T16:34:58.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:58 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/1815625089' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-13T16:34:58.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:58 smithi107 ceph-mon[34139]: osdmap e95: 12 total, 12 up, 12 in 2024-09-13T16:34:58.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:58 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3808824695' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T16:34:58.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:58 smithi016 ceph-mon[31187]: pgmap v470: 129 pgs: 129 active+clean; 1020 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-09-13T16:34:58.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:58 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/1815625089' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-13T16:34:58.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:58 smithi016 ceph-mon[31187]: osdmap e95: 12 total, 12 up, 12 in 2024-09-13T16:34:58.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:58 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3808824695' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T16:34:58.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:58.402+0000 7f7f871d2640 1 -- 172.21.15.16:0/494229745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7f8010a690 con 0x7f7f800ffe80 2024-09-13T16:34:58.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:58 smithi111 ceph-mon[34600]: pgmap v470: 129 pgs: 129 active+clean; 1020 KiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-09-13T16:34:58.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:58 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/1815625089' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-13T16:34:58.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:58 smithi111 ceph-mon[34600]: osdmap e95: 12 total, 12 up, 12 in 2024-09-13T16:34:58.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:58 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3808824695' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T16:34:59.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:59.095+0000 7f7f7dffb640 1 -- 172.21.15.16:0/494229745 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v97) v1 ==== 178+0+0 (secure 0 0 0) 0x7f7f6c097910 con 0x7f7f800ffe80 2024-09-13T16:34:59.097 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data_ec' removed 2024-09-13T16:34:59.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:59.099+0000 7f7f871d2640 1 -- 172.21.15.16:0/494229745 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7f700a1110 msgr2=0x7f7f700a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:59.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:59.099+0000 7f7f871d2640 1 --2- 172.21.15.16:0/494229745 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7f700a1110 0x7f7f700a35d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7f7402aa00 tx=0x7f7f74005e00 comp rx=0 tx=0).stop 2024-09-13T16:34:59.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:59.099+0000 7f7f871d2640 1 -- 172.21.15.16:0/494229745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f800ffe80 msgr2=0x7f7f80196290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:34:59.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:59.099+0000 7f7f871d2640 1 --2- 172.21.15.16:0/494229745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f800ffe80 0x7f7f80196290 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00b4a0 tx=0x7f7f6c00b960 comp rx=0 tx=0).stop 2024-09-13T16:34:59.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:59.099+0000 7f7f871d2640 1 -- 172.21.15.16:0/494229745 shutdown_connections 2024-09-13T16:34:59.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:59.099+0000 7f7f871d2640 1 --2- 172.21.15.16:0/494229745 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f80100840 0x7f7f8019d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:59.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:59.099+0000 7f7f871d2640 1 --2- 172.21.15.16:0/494229745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f800ffe80 0x7f7f80196290 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:59.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:59.099+0000 7f7f871d2640 1 --2- 172.21.15.16:0/494229745 >> [v2:172.21.15.16:6800/3739009683,v1:172.21.15.16:6801/3739009683] conn(0x7f7f700a1110 0x7f7f700a35d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:59.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:59.099+0000 7f7f871d2640 1 --2- 172.21.15.16:0/494229745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f800fec90 0x7f7f80195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:34:59.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:59.099+0000 7f7f871d2640 1 -- 172.21.15.16:0/494229745 >> 172.21.15.16:0/494229745 conn(0x7f7f800fa3f0 msgr2=0x7f7f800fbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:34:59.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:59.099+0000 7f7f871d2640 1 -- 172.21.15.16:0/494229745 shutdown_connections 2024-09-13T16:34:59.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-13T16:34:59.099+0000 7f7f871d2640 1 -- 172.21.15.16:0/494229745 wait complete. 2024-09-13T16:34:59.121 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-13T16:34:59.151 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-13T16:34:59.185 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-13T16:34:59.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:59 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/3808824695' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-13T16:34:59.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:59 smithi111 ceph-mon[34600]: osdmap e96: 12 total, 12 up, 12 in 2024-09-13T16:34:59.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:34:59 smithi111 ceph-mon[34600]: from='client.? 172.21.15.16:0/494229745' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T16:34:59.215 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-13T16:34:59.216 DEBUG:teuthology.orchestra.run.smithi016:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-13T16:34:59.234 DEBUG:teuthology.orchestra.run.smithi107:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-13T16:34:59.252 DEBUG:teuthology.orchestra.run.smithi111:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-13T16:34:59.278 INFO:tasks.cephadm:Stopping all daemons... 2024-09-13T16:34:59.278 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-13T16:34:59.279 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mon.a 2024-09-13T16:34:59.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:59 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/3808824695' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-13T16:34:59.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:59 smithi107 ceph-mon[34139]: osdmap e96: 12 total, 12 up, 12 in 2024-09-13T16:34:59.350 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:34:59 smithi107 ceph-mon[34139]: from='client.? 172.21.15.16:0/494229745' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T16:34:59.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:59 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/3808824695' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-13T16:34:59.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:59 smithi016 ceph-mon[31187]: osdmap e96: 12 total, 12 up, 12 in 2024-09-13T16:34:59.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:59 smithi016 ceph-mon[31187]: from='client.? 172.21.15.16:0/494229745' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T16:34:59.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:59 smithi016 systemd[1]: Stopping Ceph mon.a for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:34:59.956 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:59 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mon-a[31183]: 2024-09-13T16:34:59.604+0000 7f76233c8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T16:34:59.956 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:59 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mon-a[31183]: 2024-09-13T16:34:59.604+0000 7f76233c8640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-13T16:34:59.956 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:34:59 smithi016 podman[89161]: 2024-09-13 16:34:59.635870531 +0000 UTC m=+0.160161537 container died 0d55658237b815e9fb2a4784256f6757d6b6b8a3169ad78d6339521adbeea852 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T16:35:00.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:35:00 smithi016 podman[89161]: 2024-09-13 16:35:00.16341115 +0000 UTC m=+0.687702155 container cleanup 0d55658237b815e9fb2a4784256f6757d6b6b8a3169ad78d6339521adbeea852 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-13T16:35:00.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:35:00 smithi016 bash[89161]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mon-a 2024-09-13T16:35:00.635 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 13 16:35:00 smithi016 podman[89173]: 2024-09-13 16:35:00.415508004 +0000 UTC m=+0.777797605 container remove 0d55658237b815e9fb2a4784256f6757d6b6b8a3169ad78d6339521adbeea852 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T16:35:00.652 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mon.a.service' 2024-09-13T16:35:00.697 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T16:35:00.698 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-13T16:35:00.698 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-13T16:35:00.698 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mon.b 2024-09-13T16:35:00.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:35:00 smithi107 systemd[1]: Stopping Ceph mon.b for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:35:01.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:35:01 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mon-b[34135]: 2024-09-13T16:35:01.022+0000 7f5d36aad640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T16:35:01.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:35:01 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mon-b[34135]: 2024-09-13T16:35:01.022+0000 7f5d36aad640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-13T16:35:01.349 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:35:01 smithi107 podman[66482]: 2024-09-13 16:35:01.125362368 +0000 UTC m=+0.235735386 container died f046c66a1decc9e84f2aaeae0cd85f0f4e5b86a0e619671402fc0a3e1c5b7c37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mon-b, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-13T16:35:01.801 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:35:01 smithi107 podman[66482]: 2024-09-13 16:35:01.557074326 +0000 UTC m=+0.667447343 container cleanup f046c66a1decc9e84f2aaeae0cd85f0f4e5b86a0e619671402fc0a3e1c5b7c37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mon-b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-13T16:35:01.801 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:35:01 smithi107 bash[66482]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mon-b 2024-09-13T16:35:02.034 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mon.b.service' 2024-09-13T16:35:02.057 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:35:01 smithi107 podman[66493]: 2024-09-13 16:35:01.800497032 +0000 UTC m=+0.670634665 container remove f046c66a1decc9e84f2aaeae0cd85f0f4e5b86a0e619671402fc0a3e1c5b7c37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mon-b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, ceph=True) 2024-09-13T16:35:02.057 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:35:02 smithi107 systemd[1]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mon.b.service: Deactivated successfully. 2024-09-13T16:35:02.057 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:35:02 smithi107 systemd[1]: Stopped Ceph mon.b for 0d241f6a-71ec-11ef-bceb-c7b262605968. 2024-09-13T16:35:02.057 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 13 16:35:02 smithi107 systemd[1]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mon.b.service: Consumed 7.943s CPU time. 2024-09-13T16:35:02.101 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T16:35:02.102 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-13T16:35:02.102 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-13T16:35:02.102 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mon.c 2024-09-13T16:35:02.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:35:02 smithi111 systemd[1]: Stopping Ceph mon.c for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:35:02.929 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:35:02 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mon-c[34596]: 2024-09-13T16:35:02.477+0000 7f05457a3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T16:35:02.930 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:35:02 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mon-c[34596]: 2024-09-13T16:35:02.477+0000 7f05457a3640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-13T16:35:02.930 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:35:02 smithi111 podman[68963]: 2024-09-13 16:35:02.65211996 +0000 UTC m=+0.356094995 container died 6ae37addcae4f73bc7b02d96aae9c835597c2fc0188f2b00dee86f3d37874d47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mon-c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True) 2024-09-13T16:35:03.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:35:03 smithi111 podman[68963]: 2024-09-13 16:35:03.028031405 +0000 UTC m=+0.732006436 container cleanup 6ae37addcae4f73bc7b02d96aae9c835597c2fc0188f2b00dee86f3d37874d47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mon-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-13T16:35:03.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Sep 13 16:35:03 smithi111 bash[68963]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mon-c 2024-09-13T16:35:03.436 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mon.c.service' 2024-09-13T16:35:03.485 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T16:35:03.485 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-13T16:35:03.486 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-13T16:35:03.486 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mgr.x 2024-09-13T16:35:03.811 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:35:03 smithi016 systemd[1]: Stopping Ceph mgr.x for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:35:04.104 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:35:03 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:35:03.809+0000 7fa32c354640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T16:35:04.105 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:35:03 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x[31417]: 2024-09-13T16:35:03.809+0000 7fa32c354640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-13T16:35:04.105 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:35:03 smithi016 podman[89282]: 2024-09-13 16:35:03.840795596 +0000 UTC m=+0.156023424 container died afeb66054b3492280676fa41a48de05b46deecf1c352a19416e27a41d7a40d20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f) 2024-09-13T16:35:04.604 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:35:04 smithi016 podman[89282]: 2024-09-13 16:35:04.361535472 +0000 UTC m=+0.676763300 container cleanup afeb66054b3492280676fa41a48de05b46deecf1c352a19416e27a41d7a40d20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-13T16:35:04.604 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:35:04 smithi016 bash[89282]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x 2024-09-13T16:35:04.850 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-13T16:35:04.873 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:35:04 smithi016 podman[89295]: 2024-09-13 16:35:04.622118788 +0000 UTC m=+0.779593301 container remove afeb66054b3492280676fa41a48de05b46deecf1c352a19416e27a41d7a40d20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50bbe7f395997dcd871813949c49addbbf630361, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-x, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-13T16:35:04.874 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:35:04 smithi016 systemd[1]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mgr.x.service: Deactivated successfully. 2024-09-13T16:35:04.874 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:35:04 smithi016 systemd[1]: Stopped Ceph mgr.x for 0d241f6a-71ec-11ef-bceb-c7b262605968. 2024-09-13T16:35:04.874 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 13 16:35:04 smithi016 systemd[1]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mgr.x.service: Consumed 40.271s CPU time. 2024-09-13T16:35:04.888 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T16:35:04.888 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-13T16:35:04.889 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-13T16:35:04.889 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mgr.y 2024-09-13T16:35:05.225 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:35:04 smithi107 systemd[1]: Stopping Ceph mgr.y for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:35:05.521 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:35:05 smithi107 podman[66604]: 2024-09-13 16:35:05.224224027 +0000 UTC m=+0.136722180 container died 787cdddb9b5ceae45ab6557ef27b5c1cfca57404a4d229cd1a08afc638396fbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-13T16:35:05.849 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:35:05 smithi107 podman[66604]: 2024-09-13 16:35:05.612168719 +0000 UTC m=+0.524666872 container cleanup 787cdddb9b5ceae45ab6557ef27b5c1cfca57404a4d229cd1a08afc638396fbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-13T16:35:05.849 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 13 16:35:05 smithi107 bash[66604]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-y 2024-09-13T16:35:06.033 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-13T16:35:06.066 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T16:35:06.067 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-13T16:35:06.067 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-13T16:35:06.067 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mgr.z 2024-09-13T16:35:06.396 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:35:06 smithi111 systemd[1]: Stopping Ceph mgr.z for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:35:06.687 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:35:06 smithi111 podman[69083]: 2024-09-13 16:35:06.395650887 +0000 UTC m=+0.126747729 container died 4bfb8e2c8cc57d4b61926c77254bff27477b652038b19e847ff19fe2b0dd0547 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-z, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-13T16:35:07.060 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:35:06 smithi111 podman[69083]: 2024-09-13 16:35:06.850160212 +0000 UTC m=+0.581257049 container cleanup 4bfb8e2c8cc57d4b61926c77254bff27477b652038b19e847ff19fe2b0dd0547 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-z, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-13T16:35:07.061 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:35:06 smithi111 bash[69083]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-z 2024-09-13T16:35:07.297 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-13T16:35:07.318 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:35:07 smithi111 podman[69095]: 2024-09-13 16:35:07.060176917 +0000 UTC m=+0.660285594 container remove 4bfb8e2c8cc57d4b61926c77254bff27477b652038b19e847ff19fe2b0dd0547 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731) 2024-09-13T16:35:07.318 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:35:07 smithi111 systemd[1]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-09-13T16:35:07.318 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:35:07 smithi111 systemd[1]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-09-13T16:35:07.318 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:35:07 smithi111 systemd[1]: Stopped Ceph mgr.z for 0d241f6a-71ec-11ef-bceb-c7b262605968. 2024-09-13T16:35:07.318 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Sep 13 16:35:07 smithi111 systemd[1]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mgr.z.service: Consumed 6.242s CPU time. 2024-09-13T16:35:07.334 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T16:35:07.335 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-13T16:35:07.335 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-13T16:35:07.335 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.0 2024-09-13T16:35:07.693 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 13 16:35:07 smithi016 systemd[1]: Stopping Ceph osd.0 for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:35:08.104 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 13 16:35:07 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-0[45834]: 2024-09-13T16:35:07.692+0000 7fc701687640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T16:35:08.104 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 13 16:35:07 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-0[45834]: 2024-09-13T16:35:07.692+0000 7fc701687640 -1 osd.0 97 *** Got signal Terminated *** 2024-09-13T16:35:08.104 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 13 16:35:07 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-0[45834]: 2024-09-13T16:35:07.692+0000 7fc701687640 -1 osd.0 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T16:35:13.462 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 13 16:35:13 smithi016 podman[89405]: 2024-09-13 16:35:13.132020089 +0000 UTC m=+5.592750903 container died c049eb0fd2a25d471c1dab3b54321449350ee56ab2bf9e90377b298a26cf6615 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-13T16:35:13.854 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 13 16:35:13 smithi016 podman[89405]: 2024-09-13 16:35:13.659677391 +0000 UTC m=+6.120408194 container cleanup c049eb0fd2a25d471c1dab3b54321449350ee56ab2bf9e90377b298a26cf6615 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-13T16:35:13.854 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 13 16:35:13 smithi016 bash[89405]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-0 2024-09-13T16:35:14.341 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 13 16:35:13 smithi016 podman[89420]: 2024-09-13 16:35:13.903206248 +0000 UTC m=+0.767489498 container remove c049eb0fd2a25d471c1dab3b54321449350ee56ab2bf9e90377b298a26cf6615 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-13T16:35:14.605 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 13 16:35:14 smithi016 podman[89510]: 2024-09-13 16:35:14.242924581 +0000 UTC m=+0.016498482 image pull 152d8bc6b19e354515e8ba109d4619445de0e09b4d4a747c2f16eb5c0edb670c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a 2024-09-13T16:35:14.605 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 13 16:35:14 smithi016 podman[89510]: 2024-09-13 16:35:14.365104264 +0000 UTC m=+0.138678156 container create 499d9db13660ff6a4d3ebf8b7074bbbeaedf2afaea325b86828ceb2248ca59e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-13T16:35:15.105 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 13 16:35:14 smithi016 podman[89510]: 2024-09-13 16:35:14.623749778 +0000 UTC m=+0.397323682 container init 499d9db13660ff6a4d3ebf8b7074bbbeaedf2afaea325b86828ceb2248ca59e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-13T16:35:15.105 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 13 16:35:14 smithi016 podman[89510]: 2024-09-13 16:35:14.627463608 +0000 UTC m=+0.401037509 container start 499d9db13660ff6a4d3ebf8b7074bbbeaedf2afaea325b86828ceb2248ca59e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f) 2024-09-13T16:35:15.105 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 13 16:35:14 smithi016 podman[89510]: 2024-09-13 16:35:14.723708443 +0000 UTC m=+0.497282333 container attach 499d9db13660ff6a4d3ebf8b7074bbbeaedf2afaea325b86828ceb2248ca59e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-13T16:35:15.105 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 13 16:35:14 smithi016 podman[89510]: 2024-09-13 16:35:14.818354597 +0000 UTC m=+0.591928496 container died 499d9db13660ff6a4d3ebf8b7074bbbeaedf2afaea325b86828ceb2248ca59e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-13T16:35:15.629 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.0.service' 2024-09-13T16:35:15.675 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T16:35:15.675 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-13T16:35:15.675 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-13T16:35:15.675 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.1 2024-09-13T16:35:15.994 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 13 16:35:15 smithi107 systemd[1]: Stopping Ceph osd.1 for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:35:16.349 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 13 16:35:15 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-1[38658]: 2024-09-13T16:35:15.991+0000 7fbde701e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T16:35:16.349 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 13 16:35:15 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-1[38658]: 2024-09-13T16:35:15.991+0000 7fbde701e640 -1 osd.1 97 *** Got signal Terminated *** 2024-09-13T16:35:16.349 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 13 16:35:15 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-1[38658]: 2024-09-13T16:35:15.991+0000 7fbde701e640 -1 osd.1 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T16:35:21.599 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 13 16:35:21 smithi107 podman[66730]: 2024-09-13 16:35:21.347838002 +0000 UTC m=+5.477216895 container died f9afea0933952d96bce9e82261fa2f2003330020bcdca291db7c8691b4b13db0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-1, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T16:35:21.908 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 13 16:35:21 smithi107 podman[66730]: 2024-09-13 16:35:21.767613086 +0000 UTC m=+5.896991969 container cleanup f9afea0933952d96bce9e82261fa2f2003330020bcdca291db7c8691b4b13db0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-1, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-13T16:35:21.909 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 13 16:35:21 smithi107 bash[66730]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-1 2024-09-13T16:35:22.349 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 13 16:35:22 smithi107 podman[66747]: 2024-09-13 16:35:22.002605107 +0000 UTC m=+0.645697239 container remove f9afea0933952d96bce9e82261fa2f2003330020bcdca291db7c8691b4b13db0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-1, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-13T16:35:22.623 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 13 16:35:22 smithi107 podman[66845]: 2024-09-13 16:35:22.269742951 +0000 UTC m=+0.016230708 image pull 152d8bc6b19e354515e8ba109d4619445de0e09b4d4a747c2f16eb5c0edb670c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a 2024-09-13T16:35:22.623 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 13 16:35:22 smithi107 podman[66845]: 2024-09-13 16:35:22.395250462 +0000 UTC m=+0.141738224 container create 0a11682e324f15b8dbf7cbb3db5aad4e6c10a086ec6017369c09b4d65bd75620 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f) 2024-09-13T16:35:22.922 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 13 16:35:22 smithi107 podman[66845]: 2024-09-13 16:35:22.622836208 +0000 UTC m=+0.369323962 container init 0a11682e324f15b8dbf7cbb3db5aad4e6c10a086ec6017369c09b4d65bd75620 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True) 2024-09-13T16:35:22.923 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 13 16:35:22 smithi107 podman[66845]: 2024-09-13 16:35:22.626752314 +0000 UTC m=+0.373240065 container start 0a11682e324f15b8dbf7cbb3db5aad4e6c10a086ec6017369c09b4d65bd75620 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-13T16:35:22.923 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 13 16:35:22 smithi107 podman[66845]: 2024-09-13 16:35:22.722831984 +0000 UTC m=+0.469319736 container attach 0a11682e324f15b8dbf7cbb3db5aad4e6c10a086ec6017369c09b4d65bd75620 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-1-deactivate, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-13T16:35:22.923 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 13 16:35:22 smithi107 podman[66845]: 2024-09-13 16:35:22.833410353 +0000 UTC m=+0.579898116 container died 0a11682e324f15b8dbf7cbb3db5aad4e6c10a086ec6017369c09b4d65bd75620 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD) 2024-09-13T16:35:23.516 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.1.service' 2024-09-13T16:35:23.554 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T16:35:23.555 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-13T16:35:23.555 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-13T16:35:23.555 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.2 2024-09-13T16:35:23.937 INFO:journalctl@ceph.osd.2.smithi111.stdout:Sep 13 16:35:23 smithi111 systemd[1]: Stopping Ceph osd.2 for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:35:24.195 INFO:journalctl@ceph.osd.2.smithi111.stdout:Sep 13 16:35:23 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-2[39620]: 2024-09-13T16:35:23.952+0000 7fb4aa046640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T16:35:24.195 INFO:journalctl@ceph.osd.2.smithi111.stdout:Sep 13 16:35:23 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-2[39620]: 2024-09-13T16:35:23.952+0000 7fb4aa046640 -1 osd.2 97 *** Got signal Terminated *** 2024-09-13T16:35:24.195 INFO:journalctl@ceph.osd.2.smithi111.stdout:Sep 13 16:35:23 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-2[39620]: 2024-09-13T16:35:23.952+0000 7fb4aa046640 -1 osd.2 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T16:35:29.582 INFO:journalctl@ceph.osd.2.smithi111.stdout:Sep 13 16:35:29 smithi111 podman[69208]: 2024-09-13 16:35:29.296153489 +0000 UTC m=+5.550114724 container died 55a8f88cec6893a849ff8c7f9f3f3c999d6e63a4950f0d69eb72e6b7e5b5681a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T16:35:29.937 INFO:journalctl@ceph.osd.2.smithi111.stdout:Sep 13 16:35:29 smithi111 podman[69208]: 2024-09-13 16:35:29.679000065 +0000 UTC m=+5.932961297 container cleanup 55a8f88cec6893a849ff8c7f9f3f3c999d6e63a4950f0d69eb72e6b7e5b5681a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-13T16:35:29.937 INFO:journalctl@ceph.osd.2.smithi111.stdout:Sep 13 16:35:29 smithi111 bash[69208]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-2 2024-09-13T16:35:30.297 INFO:journalctl@ceph.osd.2.smithi111.stdout:Sep 13 16:35:29 smithi111 podman[69234]: 2024-09-13 16:35:29.955529038 +0000 UTC m=+0.659412822 container remove 55a8f88cec6893a849ff8c7f9f3f3c999d6e63a4950f0d69eb72e6b7e5b5681a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-09-13T16:35:30.577 INFO:journalctl@ceph.osd.2.smithi111.stdout:Sep 13 16:35:30 smithi111 podman[69323]: 2024-09-13 16:35:30.199260299 +0000 UTC m=+0.015722298 image pull 152d8bc6b19e354515e8ba109d4619445de0e09b4d4a747c2f16eb5c0edb670c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a 2024-09-13T16:35:30.577 INFO:journalctl@ceph.osd.2.smithi111.stdout:Sep 13 16:35:30 smithi111 podman[69323]: 2024-09-13 16:35:30.314582276 +0000 UTC m=+0.131044275 container create bf41e7c17fed9212bafaacdeab97a7240a32f461ee4dba7a59be882ada3b7141 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f) 2024-09-13T16:35:30.578 INFO:journalctl@ceph.osd.2.smithi111.stdout:Sep 13 16:35:30 smithi111 podman[69323]: 2024-09-13 16:35:30.576986192 +0000 UTC m=+0.393448192 container init bf41e7c17fed9212bafaacdeab97a7240a32f461ee4dba7a59be882ada3b7141 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-13T16:35:30.937 INFO:journalctl@ceph.osd.2.smithi111.stdout:Sep 13 16:35:30 smithi111 podman[69323]: 2024-09-13 16:35:30.580595416 +0000 UTC m=+0.397057418 container start bf41e7c17fed9212bafaacdeab97a7240a32f461ee4dba7a59be882ada3b7141 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f) 2024-09-13T16:35:30.937 INFO:journalctl@ceph.osd.2.smithi111.stdout:Sep 13 16:35:30 smithi111 podman[69323]: 2024-09-13 16:35:30.650435156 +0000 UTC m=+0.466897155 container attach bf41e7c17fed9212bafaacdeab97a7240a32f461ee4dba7a59be882ada3b7141 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-13T16:35:30.937 INFO:journalctl@ceph.osd.2.smithi111.stdout:Sep 13 16:35:30 smithi111 podman[69323]: 2024-09-13 16:35:30.767341038 +0000 UTC m=+0.583803047 container died bf41e7c17fed9212bafaacdeab97a7240a32f461ee4dba7a59be882ada3b7141 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-13T16:35:31.336 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.2.service' 2024-09-13T16:35:31.375 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T16:35:31.375 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-13T16:35:31.376 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-13T16:35:31.376 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.3 2024-09-13T16:35:31.708 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 13 16:35:31 smithi016 systemd[1]: Stopping Ceph osd.3 for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:35:32.104 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 13 16:35:31 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-3[52258]: 2024-09-13T16:35:31.705+0000 7f6691145640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T16:35:32.105 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 13 16:35:31 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-3[52258]: 2024-09-13T16:35:31.705+0000 7f6691145640 -1 osd.3 97 *** Got signal Terminated *** 2024-09-13T16:35:32.105 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 13 16:35:31 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-3[52258]: 2024-09-13T16:35:31.705+0000 7f6691145640 -1 osd.3 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T16:35:34.664 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:34 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:35:34.399+0000 7fc8886e9640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:12.023578+0000 front 2024-09-13T16:35:12.023786+0000 (oldest deadline 2024-09-13T16:35:34.323571+0000) 2024-09-13T16:35:34.665 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:34 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:34.572+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:35:35.619 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:35 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:35:35.428+0000 7fc8886e9640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:12.023578+0000 front 2024-09-13T16:35:12.023786+0000 (oldest deadline 2024-09-13T16:35:34.323571+0000) 2024-09-13T16:35:35.937 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:35 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:35.617+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:35:36.354 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:36 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:36.078+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:35:36.648 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:36 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:35:36.396+0000 7fc8886e9640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:12.023578+0000 front 2024-09-13T16:35:12.023786+0000 (oldest deadline 2024-09-13T16:35:34.323571+0000) 2024-09-13T16:35:36.937 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:36 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:36.646+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:35:37.354 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:37 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:37.029+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:35:37.355 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 13 16:35:37 smithi016 podman[89612]: 2024-09-13 16:35:37.101928878 +0000 UTC m=+5.498576291 container died d03fd39381535bb3148d3109498349b376f3c4f8b06bf695b6d1b6f62ef1b006 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-3, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-13T16:35:37.854 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 13 16:35:37 smithi016 podman[89612]: 2024-09-13 16:35:37.60708741 +0000 UTC m=+6.003734864 container cleanup d03fd39381535bb3148d3109498349b376f3c4f8b06bf695b6d1b6f62ef1b006 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-3, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-13T16:35:37.854 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 13 16:35:37 smithi016 bash[89612]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-3 2024-09-13T16:35:37.937 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:37 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:37.641+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:35:37.937 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:37 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:35:37.445+0000 7fc8886e9640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:12.023578+0000 front 2024-09-13T16:35:12.023786+0000 (oldest deadline 2024-09-13T16:35:34.323571+0000) 2024-09-13T16:35:37.937 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:37 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:37.685+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:35:38.248 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 13 16:35:37 smithi016 podman[89641]: 2024-09-13 16:35:37.91707959 +0000 UTC m=+0.806437546 container remove d03fd39381535bb3148d3109498349b376f3c4f8b06bf695b6d1b6f62ef1b006 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-13T16:35:38.248 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:37 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:37.983+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:35:38.538 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 13 16:35:38 smithi016 podman[89729]: 2024-09-13 16:35:38.150793247 +0000 UTC m=+0.016580052 image pull 152d8bc6b19e354515e8ba109d4619445de0e09b4d4a747c2f16eb5c0edb670c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a 2024-09-13T16:35:38.538 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 13 16:35:38 smithi016 podman[89729]: 2024-09-13 16:35:38.295771907 +0000 UTC m=+0.161558712 container create 3efcf2e6a441e05082266e24240ed1b1a867622f2b023b444fd8042f232ac349 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f) 2024-09-13T16:35:38.661 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:38 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:35:38.404+0000 7fc8886e9640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:12.023578+0000 front 2024-09-13T16:35:12.023786+0000 (oldest deadline 2024-09-13T16:35:34.323571+0000) 2024-09-13T16:35:38.854 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 13 16:35:38 smithi016 podman[89729]: 2024-09-13 16:35:38.537535491 +0000 UTC m=+0.403322301 container init 3efcf2e6a441e05082266e24240ed1b1a867622f2b023b444fd8042f232ac349 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-13T16:35:38.855 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 13 16:35:38 smithi016 podman[89729]: 2024-09-13 16:35:38.541491405 +0000 UTC m=+0.407278206 container start 3efcf2e6a441e05082266e24240ed1b1a867622f2b023b444fd8042f232ac349 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-3-deactivate, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-13T16:35:38.855 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 13 16:35:38 smithi016 podman[89729]: 2024-09-13 16:35:38.620866004 +0000 UTC m=+0.486652815 container attach 3efcf2e6a441e05082266e24240ed1b1a867622f2b023b444fd8042f232ac349 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-13T16:35:38.855 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 13 16:35:38 smithi016 podman[89729]: 2024-09-13 16:35:38.726938641 +0000 UTC m=+0.592725444 container died 3efcf2e6a441e05082266e24240ed1b1a867622f2b023b444fd8042f232ac349 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_CLEAN=True) 2024-09-13T16:35:38.937 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:38 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:38.677+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:35:38.937 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:38 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:38.660+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:35:39.354 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:38 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:38.983+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:35:39.589 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.3.service' 2024-09-13T16:35:39.630 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:39 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:35:39.355+0000 7fc8886e9640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:12.023578+0000 front 2024-09-13T16:35:12.023786+0000 (oldest deadline 2024-09-13T16:35:34.323571+0000) 2024-09-13T16:35:39.638 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T16:35:39.638 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-13T16:35:39.638 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-13T16:35:39.638 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.4 2024-09-13T16:35:39.920 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 13 16:35:39 smithi107 systemd[1]: Stopping Ceph osd.4 for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:35:39.937 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:39 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:39.628+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:35:39.937 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:39 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:39.676+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:35:40.349 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 13 16:35:39 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-4[45062]: 2024-09-13T16:35:39.918+0000 7f4074fc5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T16:35:40.350 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 13 16:35:39 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-4[45062]: 2024-09-13T16:35:39.918+0000 7f4074fc5640 -1 osd.4 97 *** Got signal Terminated *** 2024-09-13T16:35:40.350 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 13 16:35:39 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-4[45062]: 2024-09-13T16:35:39.918+0000 7f4074fc5640 -1 osd.4 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T16:35:40.354 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:40 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:40.012+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:35:40.677 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:40 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:35:40.355+0000 7fc8886e9640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:12.023578+0000 front 2024-09-13T16:35:12.023786+0000 (oldest deadline 2024-09-13T16:35:34.323571+0000) 2024-09-13T16:35:40.937 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:40 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:40.675+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:35:40.938 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:40 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:40.695+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:35:41.354 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:41 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:41.062+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:35:41.650 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:41 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:35:41.352+0000 7fc8886e9640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:12.023578+0000 front 2024-09-13T16:35:12.023786+0000 (oldest deadline 2024-09-13T16:35:34.323571+0000) 2024-09-13T16:35:41.937 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:41 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:41.649+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:35:41.937 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:41 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:41.668+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:35:42.354 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:42 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:42.090+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:35:42.687 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:42 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:35:42.329+0000 7fc8886e9640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:12.023578+0000 front 2024-09-13T16:35:12.023786+0000 (oldest deadline 2024-09-13T16:35:34.323571+0000) 2024-09-13T16:35:43.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:42 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:42.691+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:35:43.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:42 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:42.690+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:35:43.354 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:43 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:43.096+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:35:43.354 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:43 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:43.096+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:35:43.685 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:43 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:35:43.344+0000 7fc8886e9640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:12.023578+0000 front 2024-09-13T16:35:12.023786+0000 (oldest deadline 2024-09-13T16:35:34.323571+0000) 2024-09-13T16:35:43.854 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:35:43 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:35:43.445+0000 7f8b1145b640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:16.696549+0000 front 2024-09-13T16:35:16.696670+0000 (oldest deadline 2024-09-13T16:35:42.596506+0000) 2024-09-13T16:35:43.937 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:43 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:43.712+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:35:43.937 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:43 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:43.684+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:35:44.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:35:43 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:35:43.923+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.787129+0000 front 2024-09-13T16:35:17.787242+0000 (oldest deadline 2024-09-13T16:35:43.687327+0000) 2024-09-13T16:35:44.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:35:43 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:35:43.977+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:35:44.428 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:44 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:44.141+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:35:44.429 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:44 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:44.141+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:35:44.687 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:44 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:35:44.330+0000 7fc8886e9640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:12.023578+0000 front 2024-09-13T16:35:12.023786+0000 (oldest deadline 2024-09-13T16:35:34.323571+0000) 2024-09-13T16:35:44.687 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:44 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:35:44.330+0000 7fc8886e9640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.025828+0000 front 2024-09-13T16:35:20.025910+0000 (oldest deadline 2024-09-13T16:35:43.525951+0000) 2024-09-13T16:35:44.854 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:35:44 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:35:44.427+0000 7f8b1145b640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:16.696549+0000 front 2024-09-13T16:35:16.696670+0000 (oldest deadline 2024-09-13T16:35:42.596506+0000) 2024-09-13T16:35:44.962 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 13 16:35:44 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-4[45062]: 2024-09-13T16:35:44.691+0000 7f40715d9640 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:18.590090+0000 front 2024-09-13T16:35:18.590138+0000 (oldest deadline 2024-09-13T16:35:43.890203+0000) 2024-09-13T16:35:44.962 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:35:44 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:35:44.925+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.787129+0000 front 2024-09-13T16:35:17.787242+0000 (oldest deadline 2024-09-13T16:35:43.687327+0000) 2024-09-13T16:35:45.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:44 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:44.689+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:35:45.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:44 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:44.725+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:35:45.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:35:44 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:35:44.961+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:35:45.467 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:45 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:45.138+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:35:45.467 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:45 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:45.138+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:35:45.687 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:45 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:35:45.345+0000 7fc8886e9640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:12.023578+0000 front 2024-09-13T16:35:12.023786+0000 (oldest deadline 2024-09-13T16:35:34.323571+0000) 2024-09-13T16:35:45.687 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:45 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:35:45.345+0000 7fc8886e9640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.025828+0000 front 2024-09-13T16:35:20.025910+0000 (oldest deadline 2024-09-13T16:35:43.525951+0000) 2024-09-13T16:35:45.695 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 13 16:35:45 smithi107 podman[66951]: 2024-09-13 16:35:45.413339575 +0000 UTC m=+5.572468094 container died 3249a8b86719309bcc66044e3fbaa4edbfec6acb8ed149568c4f0b9e3ebfb615 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-4, ceph=True, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T16:35:45.854 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:35:45 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:35:45.466+0000 7f8b1145b640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:16.696549+0000 front 2024-09-13T16:35:16.696670+0000 (oldest deadline 2024-09-13T16:35:42.596506+0000) 2024-09-13T16:35:46.054 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:35:45 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:35:45.943+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.787129+0000 front 2024-09-13T16:35:17.787242+0000 (oldest deadline 2024-09-13T16:35:43.687327+0000) 2024-09-13T16:35:46.054 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:35:45 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:35:45.941+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:35:46.054 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 13 16:35:45 smithi107 podman[66951]: 2024-09-13 16:35:45.793671828 +0000 UTC m=+5.952800337 container cleanup 3249a8b86719309bcc66044e3fbaa4edbfec6acb8ed149568c4f0b9e3ebfb615 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-13T16:35:46.054 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 13 16:35:45 smithi107 bash[66951]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-4 2024-09-13T16:35:46.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:45 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:45.691+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:35:46.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:45 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:45.769+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:35:46.349 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 13 16:35:46 smithi107 podman[66968]: 2024-09-13 16:35:46.053335453 +0000 UTC m=+0.634438067 container remove 3249a8b86719309bcc66044e3fbaa4edbfec6acb8ed149568c4f0b9e3ebfb615 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-4, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-13T16:35:46.473 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:46 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:46.170+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:35:46.473 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:46 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:46.170+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:35:46.660 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 13 16:35:46 smithi107 podman[67058]: 2024-09-13 16:35:46.311603981 +0000 UTC m=+0.016132322 image pull 152d8bc6b19e354515e8ba109d4619445de0e09b4d4a747c2f16eb5c0edb670c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a 2024-09-13T16:35:46.660 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 13 16:35:46 smithi107 podman[67058]: 2024-09-13 16:35:46.412340545 +0000 UTC m=+0.116868887 container create 5856233c9c3cc53447a2f066aeb9d543e931685fb7c7fd8d5fb05fe93f2dcb0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T16:35:46.660 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 13 16:35:46 smithi107 podman[67058]: 2024-09-13 16:35:46.656269838 +0000 UTC m=+0.360798174 container init 5856233c9c3cc53447a2f066aeb9d543e931685fb7c7fd8d5fb05fe93f2dcb0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-13T16:35:46.660 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 13 16:35:46 smithi107 podman[67058]: 2024-09-13 16:35:46.659599167 +0000 UTC m=+0.364127511 container start 5856233c9c3cc53447a2f066aeb9d543e931685fb7c7fd8d5fb05fe93f2dcb0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True) 2024-09-13T16:35:46.687 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:46 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:35:46.352+0000 7fc8886e9640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:12.023578+0000 front 2024-09-13T16:35:12.023786+0000 (oldest deadline 2024-09-13T16:35:34.323571+0000) 2024-09-13T16:35:46.687 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:46 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:35:46.352+0000 7fc8886e9640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.025828+0000 front 2024-09-13T16:35:20.025910+0000 (oldest deadline 2024-09-13T16:35:43.525951+0000) 2024-09-13T16:35:46.854 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:35:46 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:35:46.471+0000 7f8b1145b640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:16.696549+0000 front 2024-09-13T16:35:16.696670+0000 (oldest deadline 2024-09-13T16:35:42.596506+0000) 2024-09-13T16:35:46.964 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 13 16:35:46 smithi107 podman[67058]: 2024-09-13 16:35:46.756355778 +0000 UTC m=+0.460884122 container attach 5856233c9c3cc53447a2f066aeb9d543e931685fb7c7fd8d5fb05fe93f2dcb0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T16:35:46.964 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 13 16:35:46 smithi107 podman[67058]: 2024-09-13 16:35:46.848591628 +0000 UTC m=+0.553119966 container died 5856233c9c3cc53447a2f066aeb9d543e931685fb7c7fd8d5fb05fe93f2dcb0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T16:35:47.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:46 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:46.707+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:35:47.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:46 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:46.720+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:35:47.227 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:35:46 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:35:46.962+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.787129+0000 front 2024-09-13T16:35:17.787242+0000 (oldest deadline 2024-09-13T16:35:43.687327+0000) 2024-09-13T16:35:47.227 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:35:46 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:35:46.982+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:35:47.494 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:47 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:47.177+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:35:47.494 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:47 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:47.177+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:35:47.554 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.4.service' 2024-09-13T16:35:47.595 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T16:35:47.595 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-13T16:35:47.595 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-13T16:35:47.595 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.5 2024-09-13T16:35:47.622 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:47 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:35:47.369+0000 7fc8886e9640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:12.023578+0000 front 2024-09-13T16:35:12.023786+0000 (oldest deadline 2024-09-13T16:35:34.323571+0000) 2024-09-13T16:35:47.623 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:47 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:35:47.369+0000 7fc8886e9640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.025828+0000 front 2024-09-13T16:35:20.025910+0000 (oldest deadline 2024-09-13T16:35:43.525951+0000) 2024-09-13T16:35:47.854 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:35:47 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:35:47.493+0000 7f8b1145b640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:16.696549+0000 front 2024-09-13T16:35:16.696670+0000 (oldest deadline 2024-09-13T16:35:42.596506+0000) 2024-09-13T16:35:47.915 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:47 smithi111 systemd[1]: Stopping Ceph osd.5 for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:35:47.916 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:47 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:47.750+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:35:47.916 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:47 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:47.675+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:35:48.187 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:47 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:35:47.914+0000 7fc88c0d5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T16:35:48.187 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:47 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:35:47.914+0000 7fc88c0d5640 -1 osd.5 97 *** Got signal Terminated *** 2024-09-13T16:35:48.187 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:47 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:35:47.914+0000 7fc88c0d5640 -1 osd.5 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T16:35:48.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:35:47 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:35:47.922+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.787129+0000 front 2024-09-13T16:35:17.787242+0000 (oldest deadline 2024-09-13T16:35:43.687327+0000) 2024-09-13T16:35:48.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:35:47 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:35:47.952+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:35:48.463 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:48 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:48.191+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:35:48.463 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:48 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:48.191+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:35:48.627 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:48 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:35:48.327+0000 7fc8886e9640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:12.023578+0000 front 2024-09-13T16:35:12.023786+0000 (oldest deadline 2024-09-13T16:35:34.323571+0000) 2024-09-13T16:35:48.627 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:48 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:35:48.327+0000 7fc8886e9640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.025828+0000 front 2024-09-13T16:35:20.025910+0000 (oldest deadline 2024-09-13T16:35:43.525951+0000) 2024-09-13T16:35:48.854 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:35:48 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:35:48.462+0000 7f8b1145b640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:16.696549+0000 front 2024-09-13T16:35:16.696670+0000 (oldest deadline 2024-09-13T16:35:42.596506+0000) 2024-09-13T16:35:48.937 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:48 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:48.729+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:35:48.937 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:48 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:48.626+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:35:49.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:35:48 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:35:48.973+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:35:49.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:35:48 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:35:48.965+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.787129+0000 front 2024-09-13T16:35:17.787242+0000 (oldest deadline 2024-09-13T16:35:43.687327+0000) 2024-09-13T16:35:49.481 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:49 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:49.215+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:35:49.481 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:49 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:49.215+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:35:49.673 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:49 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:35:49.296+0000 7fc8886e9640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:12.023578+0000 front 2024-09-13T16:35:12.023786+0000 (oldest deadline 2024-09-13T16:35:34.323571+0000) 2024-09-13T16:35:49.673 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:49 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:35:49.296+0000 7fc8886e9640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.025828+0000 front 2024-09-13T16:35:20.025910+0000 (oldest deadline 2024-09-13T16:35:43.525951+0000) 2024-09-13T16:35:49.854 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:35:49 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:35:49.480+0000 7f8b1145b640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:16.696549+0000 front 2024-09-13T16:35:16.696670+0000 (oldest deadline 2024-09-13T16:35:42.596506+0000) 2024-09-13T16:35:49.937 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:49 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:49.691+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:35:49.937 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:49 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:49.691+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:25.620883+0000 front 2024-09-13T16:35:25.620914+0000 (oldest deadline 2024-09-13T16:35:49.621782+0000) 2024-09-13T16:35:49.937 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:49 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:49.672+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:35:50.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:35:49 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:35:49.960+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.787129+0000 front 2024-09-13T16:35:17.787242+0000 (oldest deadline 2024-09-13T16:35:43.687327+0000) 2024-09-13T16:35:50.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:35:49 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:35:49.995+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:35:50.486 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:50 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:50.171+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:35:50.486 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:50 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:50.171+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:35:50.486 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:50 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:50.171+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:35:50.644 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:50 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:35:50.340+0000 7fc8886e9640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:12.023578+0000 front 2024-09-13T16:35:12.023786+0000 (oldest deadline 2024-09-13T16:35:34.323571+0000) 2024-09-13T16:35:50.645 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:50 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:35:50.340+0000 7fc8886e9640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.025828+0000 front 2024-09-13T16:35:20.025910+0000 (oldest deadline 2024-09-13T16:35:43.525951+0000) 2024-09-13T16:35:50.854 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:35:50 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:35:50.484+0000 7f8b1145b640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:16.696549+0000 front 2024-09-13T16:35:16.696670+0000 (oldest deadline 2024-09-13T16:35:42.596506+0000) 2024-09-13T16:35:50.937 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:50 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:50.693+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:35:50.937 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:50 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:50.693+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:25.620883+0000 front 2024-09-13T16:35:25.620914+0000 (oldest deadline 2024-09-13T16:35:49.621782+0000) 2024-09-13T16:35:50.937 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:50 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:50.643+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:35:51.250 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:35:50 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:35:50.924+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.787129+0000 front 2024-09-13T16:35:17.787242+0000 (oldest deadline 2024-09-13T16:35:43.687327+0000) 2024-09-13T16:35:51.251 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:35:50 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:35:50.991+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:35:51.251 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:35:50 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:35:50.991+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:35:51.442 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:51 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:51.190+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:35:51.442 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:51 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:51.190+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:35:51.443 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:51 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:51.190+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:35:51.687 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:51 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:51.616+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:35:51.687 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:51 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:35:51.387+0000 7fc8886e9640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:12.023578+0000 front 2024-09-13T16:35:12.023786+0000 (oldest deadline 2024-09-13T16:35:34.323571+0000) 2024-09-13T16:35:51.687 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:51 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:35:51.387+0000 7fc8886e9640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.025828+0000 front 2024-09-13T16:35:20.025910+0000 (oldest deadline 2024-09-13T16:35:43.525951+0000) 2024-09-13T16:35:51.854 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:35:51 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:35:51.441+0000 7f8b1145b640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:16.696549+0000 front 2024-09-13T16:35:16.696670+0000 (oldest deadline 2024-09-13T16:35:42.596506+0000) 2024-09-13T16:35:51.854 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:35:51 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:35:51.441+0000 7f8b1145b640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.697456+0000 front 2024-09-13T16:35:26.697698+0000 (oldest deadline 2024-09-13T16:35:51.397340+0000) 2024-09-13T16:35:52.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:51 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:51.731+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:35:52.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:51 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:51.731+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:25.620883+0000 front 2024-09-13T16:35:25.620914+0000 (oldest deadline 2024-09-13T16:35:49.621782+0000) 2024-09-13T16:35:52.266 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:35:51 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:35:51.968+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.787129+0000 front 2024-09-13T16:35:17.787242+0000 (oldest deadline 2024-09-13T16:35:43.687327+0000) 2024-09-13T16:35:52.267 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:35:51 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:35:51.970+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:35:52.267 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:35:51 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:35:51.970+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:35:52.420 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:52 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:52.144+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:35:52.420 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:52 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:52.144+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:35:52.421 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:52 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:52.144+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:35:52.687 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:52 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:52.575+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:35:52.687 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:52 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:35:52.418+0000 7fc8886e9640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:12.023578+0000 front 2024-09-13T16:35:12.023786+0000 (oldest deadline 2024-09-13T16:35:34.323571+0000) 2024-09-13T16:35:52.687 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:52 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5[46490]: 2024-09-13T16:35:52.418+0000 7fc8886e9640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.025828+0000 front 2024-09-13T16:35:20.025910+0000 (oldest deadline 2024-09-13T16:35:43.525951+0000) 2024-09-13T16:35:52.854 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:35:52 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:35:52.419+0000 7f8b1145b640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:16.696549+0000 front 2024-09-13T16:35:16.696670+0000 (oldest deadline 2024-09-13T16:35:42.596506+0000) 2024-09-13T16:35:52.854 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:35:52 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:35:52.419+0000 7f8b1145b640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.697456+0000 front 2024-09-13T16:35:26.697698+0000 (oldest deadline 2024-09-13T16:35:51.397340+0000) 2024-09-13T16:35:53.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:52 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:52.765+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:35:53.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:52 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:52.765+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:25.620883+0000 front 2024-09-13T16:35:25.620914+0000 (oldest deadline 2024-09-13T16:35:49.621782+0000) 2024-09-13T16:35:53.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:35:52 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:35:52.997+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.787129+0000 front 2024-09-13T16:35:17.787242+0000 (oldest deadline 2024-09-13T16:35:43.687327+0000) 2024-09-13T16:35:53.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:35:52 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:35:52.991+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:35:53.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:35:52 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:35:52.991+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:35:53.465 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:53 smithi111 podman[69426]: 2024-09-13 16:35:53.272249308 +0000 UTC m=+5.476650778 container died c56eba6e0e1f99552f4392f57de0716b44d18f99dcc7cb0fbd4f4649293c08ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, maintainer=Guillaume Abrioux ) 2024-09-13T16:35:53.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:53 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:53.164+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:35:53.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:53 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:53.164+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:35:53.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:53 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:53.164+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:35:53.605 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:35:53 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:35:53.392+0000 7f8b1145b640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:16.696549+0000 front 2024-09-13T16:35:16.696670+0000 (oldest deadline 2024-09-13T16:35:42.596506+0000) 2024-09-13T16:35:53.605 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:35:53 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:35:53.392+0000 7f8b1145b640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.697456+0000 front 2024-09-13T16:35:26.697698+0000 (oldest deadline 2024-09-13T16:35:51.397340+0000) 2024-09-13T16:35:53.770 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:53 smithi111 podman[69426]: 2024-09-13 16:35:53.664690406 +0000 UTC m=+5.869091877 container cleanup c56eba6e0e1f99552f4392f57de0716b44d18f99dcc7cb0fbd4f4649293c08ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-13T16:35:53.770 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:53 smithi111 bash[69426]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5 2024-09-13T16:35:53.770 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:53 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:53.576+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:35:54.187 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:53 smithi111 podman[69441]: 2024-09-13 16:35:53.908159335 +0000 UTC m=+0.631922449 container remove c56eba6e0e1f99552f4392f57de0716b44d18f99dcc7cb0fbd4f4649293c08ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-13T16:35:54.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:53 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:53.769+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:35:54.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:53 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:53.769+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:25.620883+0000 front 2024-09-13T16:35:25.620914+0000 (oldest deadline 2024-09-13T16:35:49.621782+0000) 2024-09-13T16:35:54.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:35:54 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:35:54.031+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.787129+0000 front 2024-09-13T16:35:17.787242+0000 (oldest deadline 2024-09-13T16:35:43.687327+0000) 2024-09-13T16:35:54.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:35:53 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:35:53.967+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:35:54.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:35:53 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:35:53.967+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:35:54.510 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:54 smithi111 podman[69540]: 2024-09-13 16:35:54.158893903 +0000 UTC m=+0.021725323 image pull 152d8bc6b19e354515e8ba109d4619445de0e09b4d4a747c2f16eb5c0edb670c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a 2024-09-13T16:35:54.511 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:54 smithi111 podman[69540]: 2024-09-13 16:35:54.300709345 +0000 UTC m=+0.163540750 container create 144cd0632d8209c4292272bfc24fb2f160071de84b53fc8fd66ea3bd71524e15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T16:35:54.604 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:35:54 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:35:54.379+0000 7f8b1145b640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:16.696549+0000 front 2024-09-13T16:35:16.696670+0000 (oldest deadline 2024-09-13T16:35:42.596506+0000) 2024-09-13T16:35:54.604 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:35:54 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:35:54.379+0000 7f8b1145b640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.697456+0000 front 2024-09-13T16:35:26.697698+0000 (oldest deadline 2024-09-13T16:35:51.397340+0000) 2024-09-13T16:35:54.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:54 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:54.169+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:35:54.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:54 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:54.169+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:35:54.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:54 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:54.169+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:35:54.777 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:54 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:54.589+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:35:54.777 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:54 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:54.589+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:35:54.777 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:54 smithi111 podman[69540]: 2024-09-13 16:35:54.545681443 +0000 UTC m=+0.408512849 container init 144cd0632d8209c4292272bfc24fb2f160071de84b53fc8fd66ea3bd71524e15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-13T16:35:54.777 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:54 smithi111 podman[69540]: 2024-09-13 16:35:54.549263542 +0000 UTC m=+0.412094948 container start 144cd0632d8209c4292272bfc24fb2f160071de84b53fc8fd66ea3bd71524e15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-09-13T16:35:54.778 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:54 smithi111 podman[69540]: 2024-09-13 16:35:54.653298233 +0000 UTC m=+0.516129654 container attach 144cd0632d8209c4292272bfc24fb2f160071de84b53fc8fd66ea3bd71524e15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-13T16:35:54.778 INFO:journalctl@ceph.osd.5.smithi111.stdout:Sep 13 16:35:54 smithi111 podman[69540]: 2024-09-13 16:35:54.74061806 +0000 UTC m=+0.603449466 container died 144cd0632d8209c4292272bfc24fb2f160071de84b53fc8fd66ea3bd71524e15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T16:35:55.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:54 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:54.775+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:35:55.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:54 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:54.775+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:25.620883+0000 front 2024-09-13T16:35:25.620914+0000 (oldest deadline 2024-09-13T16:35:49.621782+0000) 2024-09-13T16:35:55.303 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.5.service' 2024-09-13T16:35:55.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:35:55 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:35:55.017+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.787129+0000 front 2024-09-13T16:35:17.787242+0000 (oldest deadline 2024-09-13T16:35:43.687327+0000) 2024-09-13T16:35:55.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:35:55 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:35:55.017+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:23.687843+0000 front 2024-09-13T16:35:23.687932+0000 (oldest deadline 2024-09-13T16:35:54.888392+0000) 2024-09-13T16:35:55.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:35:54 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:35:54.985+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:35:55.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:35:54 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:35:54.985+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:35:55.384 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T16:35:55.385 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-13T16:35:55.385 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-13T16:35:55.385 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.6 2024-09-13T16:35:55.436 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:35:55 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:35:55.332+0000 7f8b1145b640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:16.696549+0000 front 2024-09-13T16:35:16.696670+0000 (oldest deadline 2024-09-13T16:35:42.596506+0000) 2024-09-13T16:35:55.436 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:35:55 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:35:55.332+0000 7f8b1145b640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.697456+0000 front 2024-09-13T16:35:26.697698+0000 (oldest deadline 2024-09-13T16:35:51.397340+0000) 2024-09-13T16:35:55.437 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:55 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:55.181+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:35:55.437 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:55 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:55.181+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:35:55.437 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:55 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:55.181+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:35:55.707 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:35:55 smithi016 systemd[1]: Stopping Ceph osd.6 for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:35:55.937 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:55 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:55.608+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:35:55.937 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:55 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:55.608+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:35:55.937 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:55 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:55.770+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:35:55.938 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:55 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:55.770+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:25.620883+0000 front 2024-09-13T16:35:25.620914+0000 (oldest deadline 2024-09-13T16:35:49.621782+0000) 2024-09-13T16:35:56.104 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:35:55 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:35:55.706+0000 7f8b18e4f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T16:35:56.113 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:35:55 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:35:55.706+0000 7f8b18e4f640 -1 osd.6 97 *** Got signal Terminated *** 2024-09-13T16:35:56.114 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:35:55 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:35:55.706+0000 7f8b18e4f640 -1 osd.6 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T16:35:56.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:35:56 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:35:56.062+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.787129+0000 front 2024-09-13T16:35:17.787242+0000 (oldest deadline 2024-09-13T16:35:43.687327+0000) 2024-09-13T16:35:56.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:35:56 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:35:56.062+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:23.687843+0000 front 2024-09-13T16:35:23.687932+0000 (oldest deadline 2024-09-13T16:35:54.888392+0000) 2024-09-13T16:35:56.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:35:55 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:35:55.958+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:35:56.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:35:55 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:35:55.958+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:35:56.604 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:35:56 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:35:56.367+0000 7f8b1145b640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:16.696549+0000 front 2024-09-13T16:35:16.696670+0000 (oldest deadline 2024-09-13T16:35:42.596506+0000) 2024-09-13T16:35:56.605 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:35:56 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:35:56.367+0000 7f8b1145b640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.697456+0000 front 2024-09-13T16:35:26.697698+0000 (oldest deadline 2024-09-13T16:35:51.397340+0000) 2024-09-13T16:35:56.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:56 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:56.159+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:35:56.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:56 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:56.159+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:35:56.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:56 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:56.159+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:35:56.937 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:56 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:56.774+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:35:56.937 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:56 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:56.774+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:25.620883+0000 front 2024-09-13T16:35:25.620914+0000 (oldest deadline 2024-09-13T16:35:49.621782+0000) 2024-09-13T16:35:56.937 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:56 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:56.634+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:35:56.937 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:56 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:56.634+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:35:57.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:35:57 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:35:57.049+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.787129+0000 front 2024-09-13T16:35:17.787242+0000 (oldest deadline 2024-09-13T16:35:43.687327+0000) 2024-09-13T16:35:57.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:35:57 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:35:57.049+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:23.687843+0000 front 2024-09-13T16:35:23.687932+0000 (oldest deadline 2024-09-13T16:35:54.888392+0000) 2024-09-13T16:35:57.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:35:56 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:35:56.943+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:35:57.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:35:56 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:35:56.943+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:35:57.604 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:35:57 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:35:57.364+0000 7f8b1145b640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:16.696549+0000 front 2024-09-13T16:35:16.696670+0000 (oldest deadline 2024-09-13T16:35:42.596506+0000) 2024-09-13T16:35:57.605 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:35:57 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:35:57.364+0000 7f8b1145b640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.697456+0000 front 2024-09-13T16:35:26.697698+0000 (oldest deadline 2024-09-13T16:35:51.397340+0000) 2024-09-13T16:35:57.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:57 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:57.166+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:35:57.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:57 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:57.166+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:35:57.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:57 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:57.166+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:35:57.937 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:57 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:57.659+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:35:57.937 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:57 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:57.659+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:35:57.937 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:57 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:57.772+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:35:57.937 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:57 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:57.772+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:25.620883+0000 front 2024-09-13T16:35:25.620914+0000 (oldest deadline 2024-09-13T16:35:49.621782+0000) 2024-09-13T16:35:57.938 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:57 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:57.772+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:32.423464+0000 front 2024-09-13T16:35:32.423193+0000 (oldest deadline 2024-09-13T16:35:57.723270+0000) 2024-09-13T16:35:58.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:35:58 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:35:58.001+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.787129+0000 front 2024-09-13T16:35:17.787242+0000 (oldest deadline 2024-09-13T16:35:43.687327+0000) 2024-09-13T16:35:58.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:35:58 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:35:58.001+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:23.687843+0000 front 2024-09-13T16:35:23.687932+0000 (oldest deadline 2024-09-13T16:35:54.888392+0000) 2024-09-13T16:35:58.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:35:57 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:35:57.940+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:35:58.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:35:57 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:35:57.940+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:35:58.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:35:57 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:35:57.940+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:35:58.604 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:35:58 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:35:58.389+0000 7f8b1145b640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:16.696549+0000 front 2024-09-13T16:35:16.696670+0000 (oldest deadline 2024-09-13T16:35:42.596506+0000) 2024-09-13T16:35:58.605 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:35:58 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:35:58.389+0000 7f8b1145b640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.697456+0000 front 2024-09-13T16:35:26.697698+0000 (oldest deadline 2024-09-13T16:35:51.397340+0000) 2024-09-13T16:35:58.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:58 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:58.193+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:35:58.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:58 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:58.193+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:35:58.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:58 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:58.193+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:35:58.937 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:58 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:58.822+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:35:58.937 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:58 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:58.822+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:25.620883+0000 front 2024-09-13T16:35:25.620914+0000 (oldest deadline 2024-09-13T16:35:49.621782+0000) 2024-09-13T16:35:58.937 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:58 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:58.822+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:32.423464+0000 front 2024-09-13T16:35:32.423193+0000 (oldest deadline 2024-09-13T16:35:57.723270+0000) 2024-09-13T16:35:58.937 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:58 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:58.659+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:35:58.938 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:58 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:58.659+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:35:58.938 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:58 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:58.659+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:35:59.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:35:58 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:35:58.987+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.787129+0000 front 2024-09-13T16:35:17.787242+0000 (oldest deadline 2024-09-13T16:35:43.687327+0000) 2024-09-13T16:35:59.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:35:58 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:35:58.987+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:23.687843+0000 front 2024-09-13T16:35:23.687932+0000 (oldest deadline 2024-09-13T16:35:54.888392+0000) 2024-09-13T16:35:59.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:35:58 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:35:58.960+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:35:59.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:35:58 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:35:58.960+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:35:59.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:35:58 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:35:58.960+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:35:59.604 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:35:59 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:35:59.391+0000 7f8b1145b640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:16.696549+0000 front 2024-09-13T16:35:16.696670+0000 (oldest deadline 2024-09-13T16:35:42.596506+0000) 2024-09-13T16:35:59.604 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:35:59 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:35:59.391+0000 7f8b1145b640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.697456+0000 front 2024-09-13T16:35:26.697698+0000 (oldest deadline 2024-09-13T16:35:51.397340+0000) 2024-09-13T16:35:59.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:59 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:59.187+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:35:59.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:59 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:59.187+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:35:59.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:35:59 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:35:59.187+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:35:59.937 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:59 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:59.800+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:35:59.937 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:59 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:59.800+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:25.620883+0000 front 2024-09-13T16:35:25.620914+0000 (oldest deadline 2024-09-13T16:35:49.621782+0000) 2024-09-13T16:35:59.937 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:35:59 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:35:59.800+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:32.423464+0000 front 2024-09-13T16:35:32.423193+0000 (oldest deadline 2024-09-13T16:35:57.723270+0000) 2024-09-13T16:35:59.937 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:59 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:59.674+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:35:59.938 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:59 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:59.674+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:35:59.938 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:35:59 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:35:59.674+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:00.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:35:59 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:35:59.974+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.787129+0000 front 2024-09-13T16:35:17.787242+0000 (oldest deadline 2024-09-13T16:35:43.687327+0000) 2024-09-13T16:36:00.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:35:59 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:35:59.974+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:23.687843+0000 front 2024-09-13T16:35:23.687932+0000 (oldest deadline 2024-09-13T16:35:54.888392+0000) 2024-09-13T16:36:00.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:35:59 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:35:59.974+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.888970+0000 front 2024-09-13T16:35:34.889163+0000 (oldest deadline 2024-09-13T16:35:59.588937+0000) 2024-09-13T16:36:00.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:35:59 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:35:59.927+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:00.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:35:59 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:35:59.927+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:00.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:35:59 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:35:59.927+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:00.604 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:36:00 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:36:00.374+0000 7f8b1145b640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:16.696549+0000 front 2024-09-13T16:35:16.696670+0000 (oldest deadline 2024-09-13T16:35:42.596506+0000) 2024-09-13T16:36:00.604 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:36:00 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6[59175]: 2024-09-13T16:36:00.374+0000 7f8b1145b640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.697456+0000 front 2024-09-13T16:35:26.697698+0000 (oldest deadline 2024-09-13T16:35:51.397340+0000) 2024-09-13T16:36:00.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:00 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:00.213+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:36:00.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:00 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:00.213+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:36:00.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:00 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:00.213+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:36:00.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:00 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:00.213+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.563842+0000 front 2024-09-13T16:35:33.563887+0000 (oldest deadline 2024-09-13T16:35:59.464030+0000) 2024-09-13T16:36:00.937 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:00 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:00.817+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:36:00.937 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:00 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:00.817+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:25.620883+0000 front 2024-09-13T16:35:25.620914+0000 (oldest deadline 2024-09-13T16:35:49.621782+0000) 2024-09-13T16:36:00.937 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:00 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:00.817+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:32.423464+0000 front 2024-09-13T16:35:32.423193+0000 (oldest deadline 2024-09-13T16:35:57.723270+0000) 2024-09-13T16:36:00.937 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:00 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:00.674+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:00.937 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:00 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:00.674+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:00.938 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:00 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:00.674+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:01.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:00 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:00.973+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.787129+0000 front 2024-09-13T16:35:17.787242+0000 (oldest deadline 2024-09-13T16:35:43.687327+0000) 2024-09-13T16:36:01.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:00 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:00.973+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:23.687843+0000 front 2024-09-13T16:35:23.687932+0000 (oldest deadline 2024-09-13T16:35:54.888392+0000) 2024-09-13T16:36:01.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:00 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:00.973+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.888970+0000 front 2024-09-13T16:35:34.889163+0000 (oldest deadline 2024-09-13T16:35:59.588937+0000) 2024-09-13T16:36:01.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:00 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:00.935+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:01.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:00 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:00.935+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:01.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:00 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:00.935+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:01.506 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:01 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:01.163+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:36:01.506 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:01 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:01.163+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:36:01.506 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:01 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:01.163+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:36:01.506 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:01 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:01.163+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.563842+0000 front 2024-09-13T16:35:33.563887+0000 (oldest deadline 2024-09-13T16:35:59.464030+0000) 2024-09-13T16:36:01.507 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:36:01 smithi016 podman[89832]: 2024-09-13 16:36:01.157192238 +0000 UTC m=+5.570694686 container died 2ce0d96ea6e4ad0ed40fc75660d469a49c40c37260341319c847d6db744965c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T16:36:02.104 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:36:01 smithi016 podman[89832]: 2024-09-13 16:36:01.789557026 +0000 UTC m=+6.203059474 container cleanup 2ce0d96ea6e4ad0ed40fc75660d469a49c40c37260341319c847d6db744965c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-13T16:36:02.104 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:36:01 smithi016 bash[89832]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6 2024-09-13T16:36:02.104 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:36:02 smithi016 podman[89848]: 2024-09-13 16:36:02.024686797 +0000 UTC m=+0.856725023 container remove 2ce0d96ea6e4ad0ed40fc75660d469a49c40c37260341319c847d6db744965c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-13T16:36:02.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:01 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:01.715+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:02.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:01 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:01.715+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:02.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:01 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:01.715+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:02.188 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:01 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:01.839+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:36:02.188 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:01 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:01.839+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:25.620883+0000 front 2024-09-13T16:35:25.620914+0000 (oldest deadline 2024-09-13T16:35:49.621782+0000) 2024-09-13T16:36:02.188 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:01 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:01.839+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:32.423464+0000 front 2024-09-13T16:35:32.423193+0000 (oldest deadline 2024-09-13T16:35:57.723270+0000) 2024-09-13T16:36:02.353 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:01 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:01.944+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.787129+0000 front 2024-09-13T16:35:17.787242+0000 (oldest deadline 2024-09-13T16:35:43.687327+0000) 2024-09-13T16:36:02.353 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:01 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:01.944+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:23.687843+0000 front 2024-09-13T16:35:23.687932+0000 (oldest deadline 2024-09-13T16:35:54.888392+0000) 2024-09-13T16:36:02.353 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:01 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:01.944+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.888970+0000 front 2024-09-13T16:35:34.889163+0000 (oldest deadline 2024-09-13T16:35:59.588937+0000) 2024-09-13T16:36:02.354 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:01 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:01.897+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:02.354 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:01 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:01.897+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:02.354 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:01 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:01.897+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:02.498 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:02 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:02.181+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:36:02.498 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:02 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:02.181+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:36:02.498 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:02 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:02.181+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:36:02.498 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:02 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:02.181+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.563842+0000 front 2024-09-13T16:35:33.563887+0000 (oldest deadline 2024-09-13T16:35:59.464030+0000) 2024-09-13T16:36:02.770 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:36:02 smithi016 podman[89947]: 2024-09-13 16:36:02.400672518 +0000 UTC m=+0.016449693 image pull 152d8bc6b19e354515e8ba109d4619445de0e09b4d4a747c2f16eb5c0edb670c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a 2024-09-13T16:36:02.770 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:36:02 smithi016 podman[89947]: 2024-09-13 16:36:02.533860498 +0000 UTC m=+0.149637658 container create df589c15f5f72e859b5a173f2cefb849b41c6b59417b99860273cf0d99db00d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_BRANCH=HEAD) 2024-09-13T16:36:02.770 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:36:02 smithi016 podman[89947]: 2024-09-13 16:36:02.769563052 +0000 UTC m=+0.385340213 container init df589c15f5f72e859b5a173f2cefb849b41c6b59417b99860273cf0d99db00d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-13T16:36:03.105 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:36:02 smithi016 podman[89947]: 2024-09-13 16:36:02.773211751 +0000 UTC m=+0.388988920 container start df589c15f5f72e859b5a173f2cefb849b41c6b59417b99860273cf0d99db00d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-13T16:36:03.105 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:36:02 smithi016 podman[89947]: 2024-09-13 16:36:02.852941225 +0000 UTC m=+0.468718387 container attach df589c15f5f72e859b5a173f2cefb849b41c6b59417b99860273cf0d99db00d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-13T16:36:03.105 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 13 16:36:02 smithi016 podman[89947]: 2024-09-13 16:36:02.962196255 +0000 UTC m=+0.577973426 container died df589c15f5f72e859b5a173f2cefb849b41c6b59417b99860273cf0d99db00d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f) 2024-09-13T16:36:03.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:02 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:02.853+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:36:03.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:02 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:02.853+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:25.620883+0000 front 2024-09-13T16:35:25.620914+0000 (oldest deadline 2024-09-13T16:35:49.621782+0000) 2024-09-13T16:36:03.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:02 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:02.853+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:32.423464+0000 front 2024-09-13T16:35:32.423193+0000 (oldest deadline 2024-09-13T16:35:57.723270+0000) 2024-09-13T16:36:03.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:02 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:02.689+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:03.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:02 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:02.689+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:03.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:02 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:02.689+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:03.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:02 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:02.954+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.787129+0000 front 2024-09-13T16:35:17.787242+0000 (oldest deadline 2024-09-13T16:35:43.687327+0000) 2024-09-13T16:36:03.350 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:02 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:02.954+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:23.687843+0000 front 2024-09-13T16:35:23.687932+0000 (oldest deadline 2024-09-13T16:35:54.888392+0000) 2024-09-13T16:36:03.350 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:02 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:02.954+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.888970+0000 front 2024-09-13T16:35:34.889163+0000 (oldest deadline 2024-09-13T16:35:59.588937+0000) 2024-09-13T16:36:03.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:02 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:02.850+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:03.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:02 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:02.850+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:03.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:02 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:02.850+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:03.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:03 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:03.189+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:36:03.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:03 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:03.189+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:36:03.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:03 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:03.189+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:36:03.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:03 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:03.189+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.563842+0000 front 2024-09-13T16:35:33.563887+0000 (oldest deadline 2024-09-13T16:35:59.464030+0000) 2024-09-13T16:36:03.835 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.6.service' 2024-09-13T16:36:03.882 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T16:36:03.882 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-13T16:36:03.883 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-13T16:36:03.883 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.7 2024-09-13T16:36:03.937 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:03 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:03.684+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:03.937 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:03 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:03.684+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:03.937 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:03 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:03.684+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:03.938 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:03 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:03.842+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:36:03.938 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:03 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:03.842+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:25.620883+0000 front 2024-09-13T16:35:25.620914+0000 (oldest deadline 2024-09-13T16:35:49.621782+0000) 2024-09-13T16:36:03.938 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:03 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:03.842+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:32.423464+0000 front 2024-09-13T16:35:32.423193+0000 (oldest deadline 2024-09-13T16:35:57.723270+0000) 2024-09-13T16:36:04.196 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:03 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:03.869+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:04.196 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:03 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:03.869+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:04.196 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:03 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:03.869+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:04.197 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:03 smithi107 systemd[1]: Stopping Ceph osd.7 for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:36:04.197 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:03 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:03.969+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.787129+0000 front 2024-09-13T16:35:17.787242+0000 (oldest deadline 2024-09-13T16:35:43.687327+0000) 2024-09-13T16:36:04.197 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:03 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:03.969+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:23.687843+0000 front 2024-09-13T16:35:23.687932+0000 (oldest deadline 2024-09-13T16:35:54.888392+0000) 2024-09-13T16:36:04.197 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:03 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:03.969+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.888970+0000 front 2024-09-13T16:35:34.889163+0000 (oldest deadline 2024-09-13T16:35:59.588937+0000) 2024-09-13T16:36:04.197 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:04 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:04.194+0000 7fda11908640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T16:36:04.197 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:04 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:04.194+0000 7fda11908640 -1 osd.7 97 *** Got signal Terminated *** 2024-09-13T16:36:04.197 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:04 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:04.194+0000 7fda11908640 -1 osd.7 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T16:36:04.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:04 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:04.223+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:36:04.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:04 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:04.223+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:36:04.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:04 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:04.223+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:36:04.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:04 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:04.223+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.563842+0000 front 2024-09-13T16:35:33.563887+0000 (oldest deadline 2024-09-13T16:35:59.464030+0000) 2024-09-13T16:36:04.937 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:04 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:04.664+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:04.937 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:04 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:04.664+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:04.937 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:04 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:04.664+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:04.938 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:04 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:04.816+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:36:04.938 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:04 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:04.816+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:25.620883+0000 front 2024-09-13T16:35:25.620914+0000 (oldest deadline 2024-09-13T16:35:49.621782+0000) 2024-09-13T16:36:04.938 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:04 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:04.816+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:32.423464+0000 front 2024-09-13T16:35:32.423193+0000 (oldest deadline 2024-09-13T16:35:57.723270+0000) 2024-09-13T16:36:05.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:04 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:04.958+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.787129+0000 front 2024-09-13T16:35:17.787242+0000 (oldest deadline 2024-09-13T16:35:43.687327+0000) 2024-09-13T16:36:05.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:04 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:04.958+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:23.687843+0000 front 2024-09-13T16:35:23.687932+0000 (oldest deadline 2024-09-13T16:35:54.888392+0000) 2024-09-13T16:36:05.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:04 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:04.958+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.888970+0000 front 2024-09-13T16:35:34.889163+0000 (oldest deadline 2024-09-13T16:35:59.588937+0000) 2024-09-13T16:36:05.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:04 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:04.875+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:05.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:04 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:04.875+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:05.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:04 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:04.875+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:05.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:05 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:05.179+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:36:05.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:05 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:05.179+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:36:05.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:05 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:05.179+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:36:05.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:05 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:05.179+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.563842+0000 front 2024-09-13T16:35:33.563887+0000 (oldest deadline 2024-09-13T16:35:59.464030+0000) 2024-09-13T16:36:06.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:05 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:05.833+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:36:06.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:05 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:05.833+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:25.620883+0000 front 2024-09-13T16:35:25.620914+0000 (oldest deadline 2024-09-13T16:35:49.621782+0000) 2024-09-13T16:36:06.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:05 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:05.833+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:32.423464+0000 front 2024-09-13T16:35:32.423193+0000 (oldest deadline 2024-09-13T16:35:57.723270+0000) 2024-09-13T16:36:06.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:05 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:05.695+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:06.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:05 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:05.695+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:06.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:05 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:05.695+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:06.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:05 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:05.955+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.787129+0000 front 2024-09-13T16:35:17.787242+0000 (oldest deadline 2024-09-13T16:35:43.687327+0000) 2024-09-13T16:36:06.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:05 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:05.955+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:23.687843+0000 front 2024-09-13T16:35:23.687932+0000 (oldest deadline 2024-09-13T16:35:54.888392+0000) 2024-09-13T16:36:06.350 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:05 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:05.955+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.888970+0000 front 2024-09-13T16:35:34.889163+0000 (oldest deadline 2024-09-13T16:35:59.588937+0000) 2024-09-13T16:36:06.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:05 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:05.889+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:06.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:05 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:05.889+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:06.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:05 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:05.889+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:06.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:06 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:06.217+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:36:06.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:06 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:06.217+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:36:06.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:06 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:06.217+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:36:06.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:06 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:06.217+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.563842+0000 front 2024-09-13T16:35:33.563887+0000 (oldest deadline 2024-09-13T16:35:59.464030+0000) 2024-09-13T16:36:07.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:06 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:06.835+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:36:07.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:06 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:06.835+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:25.620883+0000 front 2024-09-13T16:35:25.620914+0000 (oldest deadline 2024-09-13T16:35:49.621782+0000) 2024-09-13T16:36:07.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:06 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:06.835+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:32.423464+0000 front 2024-09-13T16:35:32.423193+0000 (oldest deadline 2024-09-13T16:35:57.723270+0000) 2024-09-13T16:36:07.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:06 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:06.704+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:07.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:06 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:06.704+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:07.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:06 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:06.704+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:07.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:06 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:06.704+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:07.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:06 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:06.950+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.787129+0000 front 2024-09-13T16:35:17.787242+0000 (oldest deadline 2024-09-13T16:35:43.687327+0000) 2024-09-13T16:36:07.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:06 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:06.950+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:23.687843+0000 front 2024-09-13T16:35:23.687932+0000 (oldest deadline 2024-09-13T16:35:54.888392+0000) 2024-09-13T16:36:07.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:06 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:06.950+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.888970+0000 front 2024-09-13T16:35:34.889163+0000 (oldest deadline 2024-09-13T16:35:59.588937+0000) 2024-09-13T16:36:07.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:06 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:06.950+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:43.089996+0000 front 2024-09-13T16:35:43.089964+0000 (oldest deadline 2024-09-13T16:36:05.990180+0000) 2024-09-13T16:36:07.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:06 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:06.863+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:07.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:06 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:06.863+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:07.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:06 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:06.863+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:07.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:06 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:06.863+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:43.915178+0000 front 2024-09-13T16:35:43.915157+0000 (oldest deadline 2024-09-13T16:36:06.815354+0000) 2024-09-13T16:36:07.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:07 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:07.196+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:36:07.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:07 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:07.196+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:36:07.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:07 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:07.196+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:36:07.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:07 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:07.196+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.563842+0000 front 2024-09-13T16:35:33.563887+0000 (oldest deadline 2024-09-13T16:35:59.464030+0000) 2024-09-13T16:36:07.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:07 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:07.196+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:42.965239+0000 front 2024-09-13T16:35:42.965185+0000 (oldest deadline 2024-09-13T16:36:07.065197+0000) 2024-09-13T16:36:08.099 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:07 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:07.957+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.787129+0000 front 2024-09-13T16:35:17.787242+0000 (oldest deadline 2024-09-13T16:35:43.687327+0000) 2024-09-13T16:36:08.099 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:07 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:07.957+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:23.687843+0000 front 2024-09-13T16:35:23.687932+0000 (oldest deadline 2024-09-13T16:35:54.888392+0000) 2024-09-13T16:36:08.099 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:07 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:07.957+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.888970+0000 front 2024-09-13T16:35:34.889163+0000 (oldest deadline 2024-09-13T16:35:59.588937+0000) 2024-09-13T16:36:08.099 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:07 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:07.957+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:43.089996+0000 front 2024-09-13T16:35:43.089964+0000 (oldest deadline 2024-09-13T16:36:05.990180+0000) 2024-09-13T16:36:08.099 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:07 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:07.820+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:08.100 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:07 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:07.820+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:08.100 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:07 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:07.820+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:08.100 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:07 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:07.820+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:43.915178+0000 front 2024-09-13T16:35:43.915157+0000 (oldest deadline 2024-09-13T16:36:06.815354+0000) 2024-09-13T16:36:08.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:07 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:07.727+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:08.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:07 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:07.727+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:08.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:07 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:07.727+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:08.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:07 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:07.727+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:08.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:07 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:07.792+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:36:08.188 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:07 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:07.792+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:25.620883+0000 front 2024-09-13T16:35:25.620914+0000 (oldest deadline 2024-09-13T16:35:49.621782+0000) 2024-09-13T16:36:08.188 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:07 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:07.792+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:32.423464+0000 front 2024-09-13T16:35:32.423193+0000 (oldest deadline 2024-09-13T16:35:57.723270+0000) 2024-09-13T16:36:08.188 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:07 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:07.792+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:44.525858+0000 front 2024-09-13T16:35:44.525805+0000 (oldest deadline 2024-09-13T16:36:07.326288+0000) 2024-09-13T16:36:08.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:08 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:08.168+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:36:08.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:08 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:08.168+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:36:08.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:08 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:08.168+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:36:08.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:08 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:08.168+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.563842+0000 front 2024-09-13T16:35:33.563887+0000 (oldest deadline 2024-09-13T16:35:59.464030+0000) 2024-09-13T16:36:08.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:08 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:08.168+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:42.965239+0000 front 2024-09-13T16:35:42.965185+0000 (oldest deadline 2024-09-13T16:36:07.065197+0000) 2024-09-13T16:36:09.099 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:08 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:08.834+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:09.099 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:08 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:08.834+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:09.099 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:08 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:08.834+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:09.099 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:08 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:08.834+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:43.915178+0000 front 2024-09-13T16:35:43.915157+0000 (oldest deadline 2024-09-13T16:36:06.815354+0000) 2024-09-13T16:36:09.100 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:08 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:08.918+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.787129+0000 front 2024-09-13T16:35:17.787242+0000 (oldest deadline 2024-09-13T16:35:43.687327+0000) 2024-09-13T16:36:09.100 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:08 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:08.918+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:23.687843+0000 front 2024-09-13T16:35:23.687932+0000 (oldest deadline 2024-09-13T16:35:54.888392+0000) 2024-09-13T16:36:09.100 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:08 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:08.918+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.888970+0000 front 2024-09-13T16:35:34.889163+0000 (oldest deadline 2024-09-13T16:35:59.588937+0000) 2024-09-13T16:36:09.100 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:08 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7[51508]: 2024-09-13T16:36:08.918+0000 7fda0df1c640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:43.089996+0000 front 2024-09-13T16:35:43.089964+0000 (oldest deadline 2024-09-13T16:36:05.990180+0000) 2024-09-13T16:36:09.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:08 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:08.776+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:09.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:08 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:08.776+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:09.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:08 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:08.776+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:09.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:08 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:08.776+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:09.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:08 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:08.814+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:36:09.188 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:08 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:08.814+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:25.620883+0000 front 2024-09-13T16:35:25.620914+0000 (oldest deadline 2024-09-13T16:35:49.621782+0000) 2024-09-13T16:36:09.188 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:08 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:08.814+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:32.423464+0000 front 2024-09-13T16:35:32.423193+0000 (oldest deadline 2024-09-13T16:35:57.723270+0000) 2024-09-13T16:36:09.188 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:08 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:08.814+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:44.525858+0000 front 2024-09-13T16:35:44.525805+0000 (oldest deadline 2024-09-13T16:36:07.326288+0000) 2024-09-13T16:36:09.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:09 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:09.158+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:36:09.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:09 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:09.158+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:36:09.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:09 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:09.158+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:36:09.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:09 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:09.158+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.563842+0000 front 2024-09-13T16:35:33.563887+0000 (oldest deadline 2024-09-13T16:35:59.464030+0000) 2024-09-13T16:36:09.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:09 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:09.158+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:42.965239+0000 front 2024-09-13T16:35:42.965185+0000 (oldest deadline 2024-09-13T16:36:07.065197+0000) 2024-09-13T16:36:10.037 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:09 smithi107 podman[67172]: 2024-09-13 16:36:09.737387403 +0000 UTC m=+5.653319684 container died a4c4d1143f31ddb58fff4e290ba0ef40ba59f585ba587761d9b7fe4449d542a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f) 2024-09-13T16:36:10.037 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:09 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:09.856+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:10.038 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:09 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:09.856+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:10.038 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:09 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:09.856+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:10.038 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:09 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:09.856+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:43.915178+0000 front 2024-09-13T16:35:43.915157+0000 (oldest deadline 2024-09-13T16:36:06.815354+0000) 2024-09-13T16:36:10.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:09 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:09.783+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:36:10.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:09 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:09.783+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:25.620883+0000 front 2024-09-13T16:35:25.620914+0000 (oldest deadline 2024-09-13T16:35:49.621782+0000) 2024-09-13T16:36:10.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:09 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:09.783+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:32.423464+0000 front 2024-09-13T16:35:32.423193+0000 (oldest deadline 2024-09-13T16:35:57.723270+0000) 2024-09-13T16:36:10.188 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:09 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:09.783+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:44.525858+0000 front 2024-09-13T16:35:44.525805+0000 (oldest deadline 2024-09-13T16:36:07.326288+0000) 2024-09-13T16:36:10.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:09 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:09.795+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:10.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:09 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:09.795+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:10.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:09 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:09.795+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:10.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:09 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:09.795+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:10.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:10 smithi107 podman[67172]: 2024-09-13 16:36:10.110517263 +0000 UTC m=+6.026449533 container cleanup a4c4d1143f31ddb58fff4e290ba0ef40ba59f585ba587761d9b7fe4449d542a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T16:36:10.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:10 smithi107 bash[67172]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7 2024-09-13T16:36:10.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:10 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:10.174+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:36:10.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:10 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:10.174+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:36:10.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:10 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:10.174+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:36:10.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:10 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:10.174+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.563842+0000 front 2024-09-13T16:35:33.563887+0000 (oldest deadline 2024-09-13T16:35:59.464030+0000) 2024-09-13T16:36:10.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:10 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:10.174+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:42.965239+0000 front 2024-09-13T16:35:42.965185+0000 (oldest deadline 2024-09-13T16:36:07.065197+0000) 2024-09-13T16:36:10.740 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:10 smithi107 podman[67186]: 2024-09-13 16:36:10.403611138 +0000 UTC m=+0.665621992 container remove a4c4d1143f31ddb58fff4e290ba0ef40ba59f585ba587761d9b7fe4449d542a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T16:36:11.041 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:10 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:10.877+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:11.041 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:10 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:10.877+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:11.041 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:10 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:10.877+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:11.041 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:10 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:10.877+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:43.915178+0000 front 2024-09-13T16:35:43.915157+0000 (oldest deadline 2024-09-13T16:36:06.815354+0000) 2024-09-13T16:36:11.041 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:10 smithi107 podman[67279]: 2024-09-13 16:36:10.737767891 +0000 UTC m=+0.109968186 container create d006b23fa0d051548e739f26a5ddd47c74d2dcf67f84432eb2bc7797747a0f8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f) 2024-09-13T16:36:11.041 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:10 smithi107 podman[67279]: 2024-09-13 16:36:10.643640346 +0000 UTC m=+0.015840604 image pull 152d8bc6b19e354515e8ba109d4619445de0e09b4d4a747c2f16eb5c0edb670c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a 2024-09-13T16:36:11.041 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:10 smithi107 podman[67279]: 2024-09-13 16:36:10.964937764 +0000 UTC m=+0.337138015 container init d006b23fa0d051548e739f26a5ddd47c74d2dcf67f84432eb2bc7797747a0f8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f) 2024-09-13T16:36:11.041 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:10 smithi107 podman[67279]: 2024-09-13 16:36:10.968093897 +0000 UTC m=+0.340294148 container start d006b23fa0d051548e739f26a5ddd47c74d2dcf67f84432eb2bc7797747a0f8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-13T16:36:11.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:10 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:10.785+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:36:11.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:10 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:10.785+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:25.620883+0000 front 2024-09-13T16:35:25.620914+0000 (oldest deadline 2024-09-13T16:35:49.621782+0000) 2024-09-13T16:36:11.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:10 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:10.785+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:32.423464+0000 front 2024-09-13T16:35:32.423193+0000 (oldest deadline 2024-09-13T16:35:57.723270+0000) 2024-09-13T16:36:11.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:10 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:10.785+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:44.525858+0000 front 2024-09-13T16:35:44.525805+0000 (oldest deadline 2024-09-13T16:36:07.326288+0000) 2024-09-13T16:36:11.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:10 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:10.764+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:11.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:10 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:10.764+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:11.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:10 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:10.764+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:11.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:10 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:10.764+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:11.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:11 smithi107 podman[67279]: 2024-09-13 16:36:11.039782984 +0000 UTC m=+0.411983234 container attach d006b23fa0d051548e739f26a5ddd47c74d2dcf67f84432eb2bc7797747a0f8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.vendor=CentOS) 2024-09-13T16:36:11.349 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 13 16:36:11 smithi107 podman[67279]: 2024-09-13 16:36:11.156102066 +0000 UTC m=+0.528302317 container died d006b23fa0d051548e739f26a5ddd47c74d2dcf67f84432eb2bc7797747a0f8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-13T16:36:11.606 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:11 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:11.155+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:36:11.606 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:11 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:11.155+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:36:11.606 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:11 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:11.155+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:36:11.606 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:11 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:11.155+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.563842+0000 front 2024-09-13T16:35:33.563887+0000 (oldest deadline 2024-09-13T16:35:59.464030+0000) 2024-09-13T16:36:11.606 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:11 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:11.155+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:42.965239+0000 front 2024-09-13T16:35:42.965185+0000 (oldest deadline 2024-09-13T16:36:07.065197+0000) 2024-09-13T16:36:11.743 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.7.service' 2024-09-13T16:36:11.780 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T16:36:11.781 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-13T16:36:11.781 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-13T16:36:11.781 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.8 2024-09-13T16:36:12.099 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:11 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:11.897+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:12.100 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:11 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:11.897+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:12.100 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:11 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:11.897+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:12.100 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:11 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:11.897+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:43.915178+0000 front 2024-09-13T16:35:43.915157+0000 (oldest deadline 2024-09-13T16:36:06.815354+0000) 2024-09-13T16:36:12.121 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:11 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:11.815+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:36:12.121 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:11 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:11.815+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:25.620883+0000 front 2024-09-13T16:35:25.620914+0000 (oldest deadline 2024-09-13T16:35:49.621782+0000) 2024-09-13T16:36:12.121 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:11 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:11.815+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:32.423464+0000 front 2024-09-13T16:35:32.423193+0000 (oldest deadline 2024-09-13T16:35:57.723270+0000) 2024-09-13T16:36:12.121 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:11 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:11.815+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:44.525858+0000 front 2024-09-13T16:35:44.525805+0000 (oldest deadline 2024-09-13T16:36:07.326288+0000) 2024-09-13T16:36:12.121 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:11 smithi111 systemd[1]: Stopping Ceph osd.8 for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:36:12.121 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:12 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:12.119+0000 7ff3c4397640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T16:36:12.121 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:12 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:12.119+0000 7ff3c4397640 -1 osd.8 97 *** Got signal Terminated *** 2024-09-13T16:36:12.122 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:11 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:11.810+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:12.122 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:11 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:11.810+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:12.122 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:11 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:11.810+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:12.122 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:11 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:11.810+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:12.437 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:12 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:12.119+0000 7ff3c4397640 -1 osd.8 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T16:36:12.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:12 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:12.195+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:36:12.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:12 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:12.195+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:36:12.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:12 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:12.195+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:36:12.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:12 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:12.195+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.563842+0000 front 2024-09-13T16:35:33.563887+0000 (oldest deadline 2024-09-13T16:35:59.464030+0000) 2024-09-13T16:36:12.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:12 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:12.195+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:42.965239+0000 front 2024-09-13T16:35:42.965185+0000 (oldest deadline 2024-09-13T16:36:07.065197+0000) 2024-09-13T16:36:13.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:12 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:12.816+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:13.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:12 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:12.816+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:13.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:12 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:12.816+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:13.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:12 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:12.816+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:13.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:12 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:12.775+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:36:13.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:12 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:12.775+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:25.620883+0000 front 2024-09-13T16:35:25.620914+0000 (oldest deadline 2024-09-13T16:35:49.621782+0000) 2024-09-13T16:36:13.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:12 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:12.775+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:32.423464+0000 front 2024-09-13T16:35:32.423193+0000 (oldest deadline 2024-09-13T16:35:57.723270+0000) 2024-09-13T16:36:13.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:12 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:12.775+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:44.525858+0000 front 2024-09-13T16:35:44.525805+0000 (oldest deadline 2024-09-13T16:36:07.326288+0000) 2024-09-13T16:36:13.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:12 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:12.892+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:13.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:12 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:12.892+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:13.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:12 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:12.892+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:13.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:12 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:12.892+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:43.915178+0000 front 2024-09-13T16:35:43.915157+0000 (oldest deadline 2024-09-13T16:36:06.815354+0000) 2024-09-13T16:36:13.354 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:13 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:13.168+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:36:13.354 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:13 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:13.168+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:36:13.354 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:13 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:13.168+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:36:13.354 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:13 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:13.168+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.563842+0000 front 2024-09-13T16:35:33.563887+0000 (oldest deadline 2024-09-13T16:35:59.464030+0000) 2024-09-13T16:36:13.354 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:13 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:13.168+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:42.965239+0000 front 2024-09-13T16:35:42.965185+0000 (oldest deadline 2024-09-13T16:36:07.065197+0000) 2024-09-13T16:36:14.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:13 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:13.800+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:36:14.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:13 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:13.800+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:25.620883+0000 front 2024-09-13T16:35:25.620914+0000 (oldest deadline 2024-09-13T16:35:49.621782+0000) 2024-09-13T16:36:14.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:13 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:13.800+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:32.423464+0000 front 2024-09-13T16:35:32.423193+0000 (oldest deadline 2024-09-13T16:35:57.723270+0000) 2024-09-13T16:36:14.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:13 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:13.800+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:44.525858+0000 front 2024-09-13T16:35:44.525805+0000 (oldest deadline 2024-09-13T16:36:07.326288+0000) 2024-09-13T16:36:14.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:13 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:13.858+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:14.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:13 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:13.858+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:14.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:13 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:13.858+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:14.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:13 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:13.858+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:14.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:13 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:13.927+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:14.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:13 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:13.927+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:14.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:13 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:13.927+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:14.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:13 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:13.927+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:43.915178+0000 front 2024-09-13T16:35:43.915157+0000 (oldest deadline 2024-09-13T16:36:06.815354+0000) 2024-09-13T16:36:14.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:14 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:14.125+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:36:14.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:14 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:14.125+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:36:14.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:14 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:14.125+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:36:14.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:14 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:14.125+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.563842+0000 front 2024-09-13T16:35:33.563887+0000 (oldest deadline 2024-09-13T16:35:59.464030+0000) 2024-09-13T16:36:14.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:14 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:14.125+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:42.965239+0000 front 2024-09-13T16:35:42.965185+0000 (oldest deadline 2024-09-13T16:36:07.065197+0000) 2024-09-13T16:36:15.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:14 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:14.778+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:36:15.188 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:14 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:14.778+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:25.620883+0000 front 2024-09-13T16:35:25.620914+0000 (oldest deadline 2024-09-13T16:35:49.621782+0000) 2024-09-13T16:36:15.188 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:14 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:14.778+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:32.423464+0000 front 2024-09-13T16:35:32.423193+0000 (oldest deadline 2024-09-13T16:35:57.723270+0000) 2024-09-13T16:36:15.188 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:14 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:14.778+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:44.525858+0000 front 2024-09-13T16:35:44.525805+0000 (oldest deadline 2024-09-13T16:36:07.326288+0000) 2024-09-13T16:36:15.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:14 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:14.892+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:15.189 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:14 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:14.892+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:15.189 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:14 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:14.892+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:15.189 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:14 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:14.892+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:15.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:14 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:14.957+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:15.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:14 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:14.957+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:15.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:14 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:14.957+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:15.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:14 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:14.957+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:43.915178+0000 front 2024-09-13T16:35:43.915157+0000 (oldest deadline 2024-09-13T16:36:06.815354+0000) 2024-09-13T16:36:15.354 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:15 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:15.082+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:36:15.354 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:15 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:15.082+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:36:15.355 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:15 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:15.082+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:36:15.355 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:15 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:15.082+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.563842+0000 front 2024-09-13T16:35:33.563887+0000 (oldest deadline 2024-09-13T16:35:59.464030+0000) 2024-09-13T16:36:15.355 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:15 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:15.082+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:42.965239+0000 front 2024-09-13T16:35:42.965185+0000 (oldest deadline 2024-09-13T16:36:07.065197+0000) 2024-09-13T16:36:16.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:15 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:15.845+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:16.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:15 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:15.845+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:16.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:15 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:15.845+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:16.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:15 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:15.845+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:16.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:15 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:15.766+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:36:16.188 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:15 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:15.766+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:25.620883+0000 front 2024-09-13T16:35:25.620914+0000 (oldest deadline 2024-09-13T16:35:49.621782+0000) 2024-09-13T16:36:16.188 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:15 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:15.766+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:32.423464+0000 front 2024-09-13T16:35:32.423193+0000 (oldest deadline 2024-09-13T16:35:57.723270+0000) 2024-09-13T16:36:16.188 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:15 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:15.766+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:44.525858+0000 front 2024-09-13T16:35:44.525805+0000 (oldest deadline 2024-09-13T16:36:07.326288+0000) 2024-09-13T16:36:16.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:15 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:15.993+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:16.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:15 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:15.993+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:16.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:15 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:15.993+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:16.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:15 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:15.993+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:43.915178+0000 front 2024-09-13T16:35:43.915157+0000 (oldest deadline 2024-09-13T16:36:06.815354+0000) 2024-09-13T16:36:16.354 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:16 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:16.092+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:36:16.354 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:16 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:16.092+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:36:16.354 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:16 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:16.092+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:36:16.355 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:16 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:16.092+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.563842+0000 front 2024-09-13T16:35:33.563887+0000 (oldest deadline 2024-09-13T16:35:59.464030+0000) 2024-09-13T16:36:16.355 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:16 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:16.092+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:42.965239+0000 front 2024-09-13T16:35:42.965185+0000 (oldest deadline 2024-09-13T16:36:07.065197+0000) 2024-09-13T16:36:17.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:16 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:16.769+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.618692+0000 front 2024-09-13T16:35:11.618752+0000 (oldest deadline 2024-09-13T16:35:37.518697+0000) 2024-09-13T16:36:17.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:16 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:16.769+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:25.620883+0000 front 2024-09-13T16:35:25.620914+0000 (oldest deadline 2024-09-13T16:35:49.621782+0000) 2024-09-13T16:36:17.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:16 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:16.769+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:32.423464+0000 front 2024-09-13T16:35:32.423193+0000 (oldest deadline 2024-09-13T16:35:57.723270+0000) 2024-09-13T16:36:17.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:16 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8[53375]: 2024-09-13T16:36:16.769+0000 7ff3bc9a3640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:44.525858+0000 front 2024-09-13T16:35:44.525805+0000 (oldest deadline 2024-09-13T16:36:07.326288+0000) 2024-09-13T16:36:17.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:16 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:16.834+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:17.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:16 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:16.834+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:17.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:16 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:16.834+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:17.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:16 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:16.834+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:17.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:16 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:16.968+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:17.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:16 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:16.968+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:17.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:16 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:16.968+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:17.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:16 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:16.968+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:43.915178+0000 front 2024-09-13T16:35:43.915157+0000 (oldest deadline 2024-09-13T16:36:06.815354+0000) 2024-09-13T16:36:17.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:16 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:16.968+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:51.516251+0000 front 2024-09-13T16:35:51.516333+0000 (oldest deadline 2024-09-13T16:36:16.216193+0000) 2024-09-13T16:36:17.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:17 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:17.136+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:36:17.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:17 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:17.136+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:36:17.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:17 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:17.136+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:36:17.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:17 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:17.136+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.563842+0000 front 2024-09-13T16:35:33.563887+0000 (oldest deadline 2024-09-13T16:35:59.464030+0000) 2024-09-13T16:36:17.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:17 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:17.136+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:42.965239+0000 front 2024-09-13T16:35:42.965185+0000 (oldest deadline 2024-09-13T16:36:07.065197+0000) 2024-09-13T16:36:17.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:17 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:17.136+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:47.065752+0000 front 2024-09-13T16:35:47.065867+0000 (oldest deadline 2024-09-13T16:36:16.466356+0000) 2024-09-13T16:36:17.869 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:17 smithi111 podman[69644]: 2024-09-13 16:36:17.60527335 +0000 UTC m=+5.633078256 container died e38d24c92ddfdf82b53d790cf520323f15379797373be11a0d8b0982c948f575 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-13T16:36:17.869 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:17 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:17.834+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:17.869 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:17 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:17.834+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:17.869 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:17 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:17.834+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:17.869 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:17 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:17.834+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:18.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:17 smithi111 podman[69644]: 2024-09-13 16:36:17.975802801 +0000 UTC m=+6.003607719 container cleanup e38d24c92ddfdf82b53d790cf520323f15379797373be11a0d8b0982c948f575 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T16:36:18.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:17 smithi111 bash[69644]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8 2024-09-13T16:36:18.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:17 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:17.919+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:18.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:17 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:17.919+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:18.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:17 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:17.919+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:18.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:17 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:17.919+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:43.915178+0000 front 2024-09-13T16:35:43.915157+0000 (oldest deadline 2024-09-13T16:36:06.815354+0000) 2024-09-13T16:36:18.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:17 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:17.919+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:51.516251+0000 front 2024-09-13T16:35:51.516333+0000 (oldest deadline 2024-09-13T16:36:16.216193+0000) 2024-09-13T16:36:18.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:18 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:18.164+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:36:18.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:18 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:18.164+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:36:18.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:18 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:18.164+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:36:18.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:18 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:18.164+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.563842+0000 front 2024-09-13T16:35:33.563887+0000 (oldest deadline 2024-09-13T16:35:59.464030+0000) 2024-09-13T16:36:18.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:18 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:18.164+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:42.965239+0000 front 2024-09-13T16:35:42.965185+0000 (oldest deadline 2024-09-13T16:36:07.065197+0000) 2024-09-13T16:36:18.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:18 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:18.164+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:47.065752+0000 front 2024-09-13T16:35:47.065867+0000 (oldest deadline 2024-09-13T16:36:16.466356+0000) 2024-09-13T16:36:18.628 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:18 smithi111 podman[69660]: 2024-09-13 16:36:18.318987968 +0000 UTC m=+0.706260409 container remove e38d24c92ddfdf82b53d790cf520323f15379797373be11a0d8b0982c948f575 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f) 2024-09-13T16:36:18.931 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:18 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:18.806+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:18.931 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:18 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:18.806+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:18.931 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:18 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:18.806+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:18.932 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:18 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:18.806+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:18.932 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:18 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:18.806+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:52.840617+0000 front 2024-09-13T16:35:52.840629+0000 (oldest deadline 2024-09-13T16:36:18.740698+0000) 2024-09-13T16:36:18.932 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:18 smithi111 podman[69749]: 2024-09-13 16:36:18.628017893 +0000 UTC m=+0.098185341 container create e43622649eee04391d733635398c99ae19569655ebcbd0b4082cadc54d72cc5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-13T16:36:18.932 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:18 smithi111 podman[69749]: 2024-09-13 16:36:18.545752043 +0000 UTC m=+0.015919501 image pull 152d8bc6b19e354515e8ba109d4619445de0e09b4d4a747c2f16eb5c0edb670c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a 2024-09-13T16:36:18.932 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:18 smithi111 podman[69749]: 2024-09-13 16:36:18.830896131 +0000 UTC m=+0.301063580 container init e43622649eee04391d733635398c99ae19569655ebcbd0b4082cadc54d72cc5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-13T16:36:18.932 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:18 smithi111 podman[69749]: 2024-09-13 16:36:18.834240547 +0000 UTC m=+0.304408005 container start e43622649eee04391d733635398c99ae19569655ebcbd0b4082cadc54d72cc5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-13T16:36:18.932 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:18 smithi111 podman[69749]: 2024-09-13 16:36:18.930767064 +0000 UTC m=+0.400934523 container attach e43622649eee04391d733635398c99ae19569655ebcbd0b4082cadc54d72cc5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-13T16:36:19.187 INFO:journalctl@ceph.osd.8.smithi111.stdout:Sep 13 16:36:19 smithi111 podman[69749]: 2024-09-13 16:36:19.022265407 +0000 UTC m=+0.492432855 container died e43622649eee04391d733635398c99ae19569655ebcbd0b4082cadc54d72cc5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T16:36:19.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:18 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:18.914+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:19.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:18 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:18.914+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:19.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:18 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:18.914+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:19.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:18 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:18.914+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:43.915178+0000 front 2024-09-13T16:35:43.915157+0000 (oldest deadline 2024-09-13T16:36:06.815354+0000) 2024-09-13T16:36:19.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:18 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:18.914+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:51.516251+0000 front 2024-09-13T16:35:51.516333+0000 (oldest deadline 2024-09-13T16:36:16.216193+0000) 2024-09-13T16:36:19.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:19 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:19.203+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:36:19.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:19 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:19.203+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:36:19.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:19 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:19.203+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:36:19.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:19 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:19.203+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.563842+0000 front 2024-09-13T16:35:33.563887+0000 (oldest deadline 2024-09-13T16:35:59.464030+0000) 2024-09-13T16:36:19.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:19 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:19.203+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:42.965239+0000 front 2024-09-13T16:35:42.965185+0000 (oldest deadline 2024-09-13T16:36:07.065197+0000) 2024-09-13T16:36:19.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:19 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:19.203+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:47.065752+0000 front 2024-09-13T16:35:47.065867+0000 (oldest deadline 2024-09-13T16:36:16.466356+0000) 2024-09-13T16:36:19.654 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.8.service' 2024-09-13T16:36:19.693 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T16:36:19.693 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-13T16:36:19.693 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-13T16:36:19.693 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.9 2024-09-13T16:36:19.937 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:19 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:19.838+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:19.937 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:19 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:19.838+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:19.938 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:19 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:19.838+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:19.938 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:19 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:19.838+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:19.938 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:19 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:19.838+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:52.840617+0000 front 2024-09-13T16:35:52.840629+0000 (oldest deadline 2024-09-13T16:36:18.740698+0000) 2024-09-13T16:36:20.011 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:19 smithi016 systemd[1]: Stopping Ceph osd.9 for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:36:20.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:19 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:19.932+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:20.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:19 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:19.932+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:20.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:19 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:19.932+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:20.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:19 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:19.933+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:43.915178+0000 front 2024-09-13T16:35:43.915157+0000 (oldest deadline 2024-09-13T16:36:06.815354+0000) 2024-09-13T16:36:20.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:19 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:19.933+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:51.516251+0000 front 2024-09-13T16:35:51.516333+0000 (oldest deadline 2024-09-13T16:36:16.216193+0000) 2024-09-13T16:36:20.354 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:20 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:20.009+0000 7f4ab44bd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T16:36:20.354 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:20 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:20.009+0000 7f4ab44bd640 -1 osd.9 97 *** Got signal Terminated *** 2024-09-13T16:36:20.355 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:20 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:20.009+0000 7f4ab44bd640 -1 osd.9 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T16:36:20.355 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:20 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:20.162+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:36:20.355 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:20 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:20.162+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:36:20.355 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:20 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:20.162+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:36:20.355 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:20 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:20.162+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.563842+0000 front 2024-09-13T16:35:33.563887+0000 (oldest deadline 2024-09-13T16:35:59.464030+0000) 2024-09-13T16:36:20.355 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:20 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:20.162+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:42.965239+0000 front 2024-09-13T16:35:42.965185+0000 (oldest deadline 2024-09-13T16:36:07.065197+0000) 2024-09-13T16:36:20.355 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:20 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:20.162+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:47.065752+0000 front 2024-09-13T16:35:47.065867+0000 (oldest deadline 2024-09-13T16:36:16.466356+0000) 2024-09-13T16:36:21.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:20 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:20.815+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:21.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:20 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:20.815+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:21.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:20 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:20.815+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:21.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:20 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:20.815+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:21.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:20 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:20.815+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:52.840617+0000 front 2024-09-13T16:35:52.840629+0000 (oldest deadline 2024-09-13T16:36:18.740698+0000) 2024-09-13T16:36:21.251 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:20 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:20.948+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:21.251 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:20 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:20.948+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:21.251 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:20 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:20.948+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:21.251 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:20 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:20.948+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:43.915178+0000 front 2024-09-13T16:35:43.915157+0000 (oldest deadline 2024-09-13T16:36:06.815354+0000) 2024-09-13T16:36:21.251 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:20 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:20.948+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:51.516251+0000 front 2024-09-13T16:35:51.516333+0000 (oldest deadline 2024-09-13T16:36:16.216193+0000) 2024-09-13T16:36:21.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:21 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:21.131+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:36:21.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:21 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:21.131+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:36:21.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:21 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:21.131+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:36:21.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:21 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:21.131+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.563842+0000 front 2024-09-13T16:35:33.563887+0000 (oldest deadline 2024-09-13T16:35:59.464030+0000) 2024-09-13T16:36:21.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:21 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:21.131+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:42.965239+0000 front 2024-09-13T16:35:42.965185+0000 (oldest deadline 2024-09-13T16:36:07.065197+0000) 2024-09-13T16:36:21.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:21 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:21.131+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:47.065752+0000 front 2024-09-13T16:35:47.065867+0000 (oldest deadline 2024-09-13T16:36:16.466356+0000) 2024-09-13T16:36:22.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:21 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:21.802+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:22.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:21 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:21.802+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:22.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:21 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:21.802+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:22.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:21 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:21.802+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:22.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:21 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:21.802+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:52.840617+0000 front 2024-09-13T16:35:52.840629+0000 (oldest deadline 2024-09-13T16:36:18.740698+0000) 2024-09-13T16:36:22.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:21 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:21.984+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:22.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:21 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:21.984+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:22.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:21 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:21.984+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:22.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:21 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:21.984+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:43.915178+0000 front 2024-09-13T16:35:43.915157+0000 (oldest deadline 2024-09-13T16:36:06.815354+0000) 2024-09-13T16:36:22.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:21 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:21.984+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:51.516251+0000 front 2024-09-13T16:35:51.516333+0000 (oldest deadline 2024-09-13T16:36:16.216193+0000) 2024-09-13T16:36:22.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:22 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:22.137+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:36:22.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:22 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:22.137+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:36:22.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:22 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:22.137+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:36:22.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:22 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:22.137+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.563842+0000 front 2024-09-13T16:35:33.563887+0000 (oldest deadline 2024-09-13T16:35:59.464030+0000) 2024-09-13T16:36:22.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:22 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:22.137+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:42.965239+0000 front 2024-09-13T16:35:42.965185+0000 (oldest deadline 2024-09-13T16:36:07.065197+0000) 2024-09-13T16:36:22.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:22 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:22.137+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:47.065752+0000 front 2024-09-13T16:35:47.065867+0000 (oldest deadline 2024-09-13T16:36:16.466356+0000) 2024-09-13T16:36:23.088 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:22 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:22.836+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:23.089 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:22 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:22.836+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:23.089 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:22 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:22.836+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:23.089 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:22 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:22.836+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:23.089 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:22 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:22.836+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:52.840617+0000 front 2024-09-13T16:35:52.840629+0000 (oldest deadline 2024-09-13T16:36:18.740698+0000) 2024-09-13T16:36:23.089 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:22 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:22.836+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-13T16:35:59.241670+0000 front 2024-09-13T16:35:59.241644+0000 (oldest deadline 2024-09-13T16:36:22.741612+0000) 2024-09-13T16:36:23.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:22 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:22.958+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:23.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:22 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:22.958+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:23.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:22 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:22.958+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:23.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:22 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:22.958+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:43.915178+0000 front 2024-09-13T16:35:43.915157+0000 (oldest deadline 2024-09-13T16:36:06.815354+0000) 2024-09-13T16:36:23.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:22 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:22.958+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:51.516251+0000 front 2024-09-13T16:35:51.516333+0000 (oldest deadline 2024-09-13T16:36:16.216193+0000) 2024-09-13T16:36:23.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:23 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:23.156+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:36:23.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:23 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:23.156+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:36:23.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:23 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:23.156+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:36:23.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:23 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:23.156+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.563842+0000 front 2024-09-13T16:35:33.563887+0000 (oldest deadline 2024-09-13T16:35:59.464030+0000) 2024-09-13T16:36:23.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:23 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:23.157+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:42.965239+0000 front 2024-09-13T16:35:42.965185+0000 (oldest deadline 2024-09-13T16:36:07.065197+0000) 2024-09-13T16:36:23.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:23 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:23.157+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:47.065752+0000 front 2024-09-13T16:35:47.065867+0000 (oldest deadline 2024-09-13T16:36:16.466356+0000) 2024-09-13T16:36:24.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:23 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:23.885+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:24.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:23 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:23.885+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:24.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:23 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:23.885+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:24.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:23 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:23.885+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:24.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:23 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:23.885+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:52.840617+0000 front 2024-09-13T16:35:52.840629+0000 (oldest deadline 2024-09-13T16:36:18.740698+0000) 2024-09-13T16:36:24.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:23 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:23.885+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-13T16:35:59.241670+0000 front 2024-09-13T16:35:59.241644+0000 (oldest deadline 2024-09-13T16:36:22.741612+0000) 2024-09-13T16:36:24.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:23 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:23.960+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:24.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:23 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:23.960+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:24.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:23 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:23.960+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:24.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:23 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:23.960+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:43.915178+0000 front 2024-09-13T16:35:43.915157+0000 (oldest deadline 2024-09-13T16:36:06.815354+0000) 2024-09-13T16:36:24.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:23 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:23.960+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:51.516251+0000 front 2024-09-13T16:35:51.516333+0000 (oldest deadline 2024-09-13T16:36:16.216193+0000) 2024-09-13T16:36:24.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:24 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:24.198+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:11.360789+0000 front 2024-09-13T16:35:11.360654+0000 (oldest deadline 2024-09-13T16:35:35.460784+0000) 2024-09-13T16:36:24.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:24 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:24.198+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:17.761914+0000 front 2024-09-13T16:35:17.761863+0000 (oldest deadline 2024-09-13T16:35:43.061783+0000) 2024-09-13T16:36:24.604 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:24 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:24.198+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:26.563148+0000 front 2024-09-13T16:35:26.563018+0000 (oldest deadline 2024-09-13T16:35:49.463057+0000) 2024-09-13T16:36:24.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:24 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:24.198+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.563842+0000 front 2024-09-13T16:35:33.563887+0000 (oldest deadline 2024-09-13T16:35:59.464030+0000) 2024-09-13T16:36:24.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:24 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:24.198+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:42.965239+0000 front 2024-09-13T16:35:42.965185+0000 (oldest deadline 2024-09-13T16:36:07.065197+0000) 2024-09-13T16:36:24.605 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:24 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9[66134]: 2024-09-13T16:36:24.198+0000 7f4ab0ad1640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:47.065752+0000 front 2024-09-13T16:35:47.065867+0000 (oldest deadline 2024-09-13T16:36:16.466356+0000) 2024-09-13T16:36:25.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:24 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:24.853+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:25.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:24 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:24.853+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:25.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:24 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:24.853+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:25.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:24 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:24.853+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:25.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:24 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:24.853+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:52.840617+0000 front 2024-09-13T16:35:52.840629+0000 (oldest deadline 2024-09-13T16:36:18.740698+0000) 2024-09-13T16:36:25.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:24 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:24.853+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-13T16:35:59.241670+0000 front 2024-09-13T16:35:59.241644+0000 (oldest deadline 2024-09-13T16:36:22.741612+0000) 2024-09-13T16:36:25.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:24 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:24.971+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:25.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:24 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:24.971+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:25.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:24 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:24.971+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:25.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:24 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:24.971+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:43.915178+0000 front 2024-09-13T16:35:43.915157+0000 (oldest deadline 2024-09-13T16:36:06.815354+0000) 2024-09-13T16:36:25.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:24 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:24.971+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:51.516251+0000 front 2024-09-13T16:35:51.516333+0000 (oldest deadline 2024-09-13T16:36:16.216193+0000) 2024-09-13T16:36:25.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:24 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:24.971+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-13T16:35:58.517179+0000 front 2024-09-13T16:35:58.517150+0000 (oldest deadline 2024-09-13T16:36:24.417044+0000) 2024-09-13T16:36:25.778 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:25 smithi016 podman[90052]: 2024-09-13 16:36:25.489436521 +0000 UTC m=+5.590773106 container died 70cae2916efe86b87d0ce53e434de277511d946c8ebfb0e71d8427c8a4aa7686 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-13T16:36:26.104 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:25 smithi016 podman[90052]: 2024-09-13 16:36:25.859861481 +0000 UTC m=+5.961198064 container cleanup 70cae2916efe86b87d0ce53e434de277511d946c8ebfb0e71d8427c8a4aa7686 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-09-13T16:36:26.105 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:25 smithi016 bash[90052]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9 2024-09-13T16:36:26.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:25 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:25.882+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:26.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:25 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:25.882+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:26.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:25 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:25.882+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:26.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:25 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:25.882+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:26.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:25 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:25.882+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:52.840617+0000 front 2024-09-13T16:35:52.840629+0000 (oldest deadline 2024-09-13T16:36:18.740698+0000) 2024-09-13T16:36:26.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:25 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:25.882+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-13T16:35:59.241670+0000 front 2024-09-13T16:35:59.241644+0000 (oldest deadline 2024-09-13T16:36:22.741612+0000) 2024-09-13T16:36:26.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:26 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:26.014+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:26.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:26 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:26.014+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:26.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:26 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:26.014+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:26.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:26 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:26.014+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:43.915178+0000 front 2024-09-13T16:35:43.915157+0000 (oldest deadline 2024-09-13T16:36:06.815354+0000) 2024-09-13T16:36:26.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:26 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:26.014+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:51.516251+0000 front 2024-09-13T16:35:51.516333+0000 (oldest deadline 2024-09-13T16:36:16.216193+0000) 2024-09-13T16:36:26.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:26 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:26.014+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-13T16:35:58.517179+0000 front 2024-09-13T16:35:58.517150+0000 (oldest deadline 2024-09-13T16:36:24.417044+0000) 2024-09-13T16:36:26.486 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:26 smithi016 podman[90069]: 2024-09-13 16:36:26.136329753 +0000 UTC m=+0.639839047 container remove 70cae2916efe86b87d0ce53e434de277511d946c8ebfb0e71d8427c8a4aa7686 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-13T16:36:26.798 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:26 smithi016 podman[90153]: 2024-09-13 16:36:26.388110224 +0000 UTC m=+0.016690011 image pull 152d8bc6b19e354515e8ba109d4619445de0e09b4d4a747c2f16eb5c0edb670c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a 2024-09-13T16:36:26.798 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:26 smithi016 podman[90153]: 2024-09-13 16:36:26.495425167 +0000 UTC m=+0.124004983 container create eda13b01de9776f0808aaafec82489239a41118fe8056430e82ea40cb41a2f1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-13T16:36:26.798 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:26 smithi016 podman[90153]: 2024-09-13 16:36:26.722717107 +0000 UTC m=+0.351296887 container init eda13b01de9776f0808aaafec82489239a41118fe8056430e82ea40cb41a2f1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9-deactivate, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-13T16:36:26.798 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:26 smithi016 podman[90153]: 2024-09-13 16:36:26.726020984 +0000 UTC m=+0.354600763 container start eda13b01de9776f0808aaafec82489239a41118fe8056430e82ea40cb41a2f1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f) 2024-09-13T16:36:26.798 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:26 smithi016 podman[90153]: 2024-09-13 16:36:26.79756136 +0000 UTC m=+0.426141131 container attach eda13b01de9776f0808aaafec82489239a41118fe8056430e82ea40cb41a2f1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-13T16:36:27.105 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:26 smithi016 podman[90153]: 2024-09-13 16:36:26.91292644 +0000 UTC m=+0.541506211 container died eda13b01de9776f0808aaafec82489239a41118fe8056430e82ea40cb41a2f1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-13T16:36:27.459 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:26 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:26.979+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:27.459 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:26 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:26.979+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:27.459 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:26 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:26.979+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:27.459 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:26 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:26.979+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:43.915178+0000 front 2024-09-13T16:35:43.915157+0000 (oldest deadline 2024-09-13T16:36:06.815354+0000) 2024-09-13T16:36:27.459 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:26 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:26.979+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:51.516251+0000 front 2024-09-13T16:35:51.516333+0000 (oldest deadline 2024-09-13T16:36:16.216193+0000) 2024-09-13T16:36:27.459 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:26 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:26.979+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-13T16:35:58.517179+0000 front 2024-09-13T16:35:58.517150+0000 (oldest deadline 2024-09-13T16:36:24.417044+0000) 2024-09-13T16:36:27.459 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:26 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:26.879+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:27.459 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:26 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:26.879+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:27.459 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:26 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:26.879+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:27.459 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:26 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:26.879+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:27.460 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:26 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:26.879+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:52.840617+0000 front 2024-09-13T16:35:52.840629+0000 (oldest deadline 2024-09-13T16:36:18.740698+0000) 2024-09-13T16:36:27.460 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:26 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:26.879+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-13T16:35:59.241670+0000 front 2024-09-13T16:35:59.241644+0000 (oldest deadline 2024-09-13T16:36:22.741612+0000) 2024-09-13T16:36:27.460 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 13 16:36:27 smithi016 podman[90176]: 2024-09-13 16:36:27.425925481 +0000 UTC m=+0.506592728 container remove eda13b01de9776f0808aaafec82489239a41118fe8056430e82ea40cb41a2f1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-9-deactivate, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-13T16:36:27.461 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.9.service' 2024-09-13T16:36:27.506 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T16:36:27.506 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-13T16:36:27.507 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-13T16:36:27.507 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.10 2024-09-13T16:36:27.837 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:27 smithi107 systemd[1]: Stopping Ceph osd.10 for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:36:28.099 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:27 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:27.836+0000 7ff82bb08640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T16:36:28.099 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:27 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:27.836+0000 7ff82bb08640 -1 osd.10 97 *** Got signal Terminated *** 2024-09-13T16:36:28.099 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:27 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:27.836+0000 7ff82bb08640 -1 osd.10 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T16:36:28.099 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:27 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:27.936+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:28.100 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:27 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:27.936+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:28.100 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:27 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:27.936+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:28.100 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:27 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:27.936+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:43.915178+0000 front 2024-09-13T16:35:43.915157+0000 (oldest deadline 2024-09-13T16:36:06.815354+0000) 2024-09-13T16:36:28.100 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:27 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:27.936+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:51.516251+0000 front 2024-09-13T16:35:51.516333+0000 (oldest deadline 2024-09-13T16:36:16.216193+0000) 2024-09-13T16:36:28.100 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:27 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:27.936+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-13T16:35:58.517179+0000 front 2024-09-13T16:35:58.517150+0000 (oldest deadline 2024-09-13T16:36:24.417044+0000) 2024-09-13T16:36:28.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:27 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:27.882+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:28.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:27 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:27.882+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:28.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:27 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:27.882+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:28.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:27 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:27.882+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:28.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:27 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:27.882+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:52.840617+0000 front 2024-09-13T16:35:52.840629+0000 (oldest deadline 2024-09-13T16:36:18.740698+0000) 2024-09-13T16:36:28.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:27 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:27.882+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-13T16:35:59.241670+0000 front 2024-09-13T16:35:59.241644+0000 (oldest deadline 2024-09-13T16:36:22.741612+0000) 2024-09-13T16:36:29.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:28 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:28.904+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:29.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:28 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:28.904+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:29.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:28 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:28.904+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:29.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:28 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:28.904+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:29.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:28 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:28.904+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:52.840617+0000 front 2024-09-13T16:35:52.840629+0000 (oldest deadline 2024-09-13T16:36:18.740698+0000) 2024-09-13T16:36:29.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:28 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:28.904+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-13T16:35:59.241670+0000 front 2024-09-13T16:35:59.241644+0000 (oldest deadline 2024-09-13T16:36:22.741612+0000) 2024-09-13T16:36:29.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:28 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:28.929+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:29.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:28 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:28.929+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:29.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:28 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:28.929+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:29.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:28 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:28.929+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:43.915178+0000 front 2024-09-13T16:35:43.915157+0000 (oldest deadline 2024-09-13T16:36:06.815354+0000) 2024-09-13T16:36:29.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:28 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:28.929+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:51.516251+0000 front 2024-09-13T16:35:51.516333+0000 (oldest deadline 2024-09-13T16:36:16.216193+0000) 2024-09-13T16:36:29.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:28 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:28.929+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-13T16:35:58.517179+0000 front 2024-09-13T16:35:58.517150+0000 (oldest deadline 2024-09-13T16:36:24.417044+0000) 2024-09-13T16:36:30.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:29 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:29.872+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:30.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:29 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:29.872+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:30.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:29 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:29.872+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:30.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:29 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:29.872+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:30.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:29 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:29.872+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:52.840617+0000 front 2024-09-13T16:35:52.840629+0000 (oldest deadline 2024-09-13T16:36:18.740698+0000) 2024-09-13T16:36:30.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:29 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:29.872+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-13T16:35:59.241670+0000 front 2024-09-13T16:35:59.241644+0000 (oldest deadline 2024-09-13T16:36:22.741612+0000) 2024-09-13T16:36:30.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:29 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:29.960+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:30.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:29 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:29.960+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:30.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:29 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:29.960+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:30.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:29 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:29.960+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:43.915178+0000 front 2024-09-13T16:35:43.915157+0000 (oldest deadline 2024-09-13T16:36:06.815354+0000) 2024-09-13T16:36:30.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:29 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:29.960+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:51.516251+0000 front 2024-09-13T16:35:51.516333+0000 (oldest deadline 2024-09-13T16:36:16.216193+0000) 2024-09-13T16:36:30.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:29 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:29.960+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-13T16:35:58.517179+0000 front 2024-09-13T16:35:58.517150+0000 (oldest deadline 2024-09-13T16:36:24.417044+0000) 2024-09-13T16:36:31.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:30 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:30.831+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:31.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:30 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:30.831+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:31.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:30 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:30.831+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:31.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:30 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:30.831+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:31.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:30 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:30.831+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:52.840617+0000 front 2024-09-13T16:35:52.840629+0000 (oldest deadline 2024-09-13T16:36:18.740698+0000) 2024-09-13T16:36:31.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:30 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:30.831+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-13T16:35:59.241670+0000 front 2024-09-13T16:35:59.241644+0000 (oldest deadline 2024-09-13T16:36:22.741612+0000) 2024-09-13T16:36:31.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:30 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:30.831+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-09-13T16:36:08.542889+0000 front 2024-09-13T16:36:08.542872+0000 (oldest deadline 2024-09-13T16:36:30.242787+0000) 2024-09-13T16:36:31.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:30 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:30.946+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:31.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:30 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:30.946+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:31.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:30 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:30.946+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:31.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:30 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:30.946+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:43.915178+0000 front 2024-09-13T16:35:43.915157+0000 (oldest deadline 2024-09-13T16:36:06.815354+0000) 2024-09-13T16:36:31.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:30 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:30.946+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:51.516251+0000 front 2024-09-13T16:35:51.516333+0000 (oldest deadline 2024-09-13T16:36:16.216193+0000) 2024-09-13T16:36:31.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:30 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:30.946+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-13T16:35:58.517179+0000 front 2024-09-13T16:35:58.517150+0000 (oldest deadline 2024-09-13T16:36:24.417044+0000) 2024-09-13T16:36:31.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:30 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:30.946+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-09-13T16:36:05.517997+0000 front 2024-09-13T16:36:05.518016+0000 (oldest deadline 2024-09-13T16:36:30.818022+0000) 2024-09-13T16:36:32.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:31 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:31.798+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:32.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:31 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:31.798+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:32.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:31 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:31.798+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:32.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:31 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:31.798+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:32.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:31 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:31.798+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:52.840617+0000 front 2024-09-13T16:35:52.840629+0000 (oldest deadline 2024-09-13T16:36:18.740698+0000) 2024-09-13T16:36:32.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:31 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:31.798+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-13T16:35:59.241670+0000 front 2024-09-13T16:35:59.241644+0000 (oldest deadline 2024-09-13T16:36:22.741612+0000) 2024-09-13T16:36:32.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:31 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:31.798+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-09-13T16:36:08.542889+0000 front 2024-09-13T16:36:08.542872+0000 (oldest deadline 2024-09-13T16:36:30.242787+0000) 2024-09-13T16:36:32.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:31 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:31.983+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-13T16:35:20.011636+0000 front 2024-09-13T16:35:20.011615+0000 (oldest deadline 2024-09-13T16:35:43.511780+0000) 2024-09-13T16:36:32.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:31 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:31.983+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.812771+0000 front 2024-09-13T16:35:28.812716+0000 (oldest deadline 2024-09-13T16:35:50.512765+0000) 2024-09-13T16:36:32.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:31 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:31.983+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:33.413937+0000 front 2024-09-13T16:35:33.413970+0000 (oldest deadline 2024-09-13T16:35:57.513800+0000) 2024-09-13T16:36:32.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:31 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:31.983+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:43.915178+0000 front 2024-09-13T16:35:43.915157+0000 (oldest deadline 2024-09-13T16:36:06.815354+0000) 2024-09-13T16:36:32.349 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:31 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:31.983+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:51.516251+0000 front 2024-09-13T16:35:51.516333+0000 (oldest deadline 2024-09-13T16:36:16.216193+0000) 2024-09-13T16:36:32.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:31 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:31.983+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-13T16:35:58.517179+0000 front 2024-09-13T16:35:58.517150+0000 (oldest deadline 2024-09-13T16:36:24.417044+0000) 2024-09-13T16:36:32.350 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:31 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10[58326]: 2024-09-13T16:36:31.983+0000 7ff824114640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-09-13T16:36:05.517997+0000 front 2024-09-13T16:36:05.518016+0000 (oldest deadline 2024-09-13T16:36:30.818022+0000) 2024-09-13T16:36:33.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:32 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:32.825+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:33.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:32 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:32.825+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:33.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:32 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:32.825+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:33.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:32 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:32.825+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:33.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:32 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:32.825+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:52.840617+0000 front 2024-09-13T16:35:52.840629+0000 (oldest deadline 2024-09-13T16:36:18.740698+0000) 2024-09-13T16:36:33.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:32 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:32.825+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-13T16:35:59.241670+0000 front 2024-09-13T16:35:59.241644+0000 (oldest deadline 2024-09-13T16:36:22.741612+0000) 2024-09-13T16:36:33.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:32 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:32.825+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-09-13T16:36:08.542889+0000 front 2024-09-13T16:36:08.542872+0000 (oldest deadline 2024-09-13T16:36:30.242787+0000) 2024-09-13T16:36:33.512 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:33 smithi107 podman[67393]: 2024-09-13 16:36:33.256746026 +0000 UTC m=+5.548288335 container died 396277c04f0f5436f386cab129cb36ea415cfbb39aedbbdcf68bb8cd2a01db91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, io.buildah.version=1.37.2) 2024-09-13T16:36:33.849 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:33 smithi107 podman[67393]: 2024-09-13 16:36:33.610659881 +0000 UTC m=+5.902202196 container cleanup 396277c04f0f5436f386cab129cb36ea415cfbb39aedbbdcf68bb8cd2a01db91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-13T16:36:33.849 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:33 smithi107 bash[67393]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10 2024-09-13T16:36:34.180 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:33 smithi107 podman[67408]: 2024-09-13 16:36:33.86166261 +0000 UTC m=+0.607073924 container remove 396277c04f0f5436f386cab129cb36ea415cfbb39aedbbdcf68bb8cd2a01db91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-13T16:36:34.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:33 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:33.779+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:34.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:33 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:33.779+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:34.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:33 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:33.779+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:34.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:33 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:33.779+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:34.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:33 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:33.779+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:52.840617+0000 front 2024-09-13T16:35:52.840629+0000 (oldest deadline 2024-09-13T16:36:18.740698+0000) 2024-09-13T16:36:34.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:33 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:33.779+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-13T16:35:59.241670+0000 front 2024-09-13T16:35:59.241644+0000 (oldest deadline 2024-09-13T16:36:22.741612+0000) 2024-09-13T16:36:34.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:33 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:33.779+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-09-13T16:36:08.542889+0000 front 2024-09-13T16:36:08.542872+0000 (oldest deadline 2024-09-13T16:36:30.242787+0000) 2024-09-13T16:36:34.499 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:34 smithi107 podman[67493]: 2024-09-13 16:36:34.082336767 +0000 UTC m=+0.015517652 image pull 152d8bc6b19e354515e8ba109d4619445de0e09b4d4a747c2f16eb5c0edb670c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a 2024-09-13T16:36:34.499 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:34 smithi107 podman[67493]: 2024-09-13 16:36:34.204246843 +0000 UTC m=+0.137427717 container create 5c3e73b9bc116068c3d855d16a761e71a2cc6b49c2e12f8b9c61a238835ca77e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T16:36:34.499 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:34 smithi107 podman[67493]: 2024-09-13 16:36:34.414676492 +0000 UTC m=+0.347857379 container init 5c3e73b9bc116068c3d855d16a761e71a2cc6b49c2e12f8b9c61a238835ca77e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T16:36:34.499 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:34 smithi107 podman[67493]: 2024-09-13 16:36:34.418018001 +0000 UTC m=+0.351198878 container start 5c3e73b9bc116068c3d855d16a761e71a2cc6b49c2e12f8b9c61a238835ca77e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, maintainer=Guillaume Abrioux ) 2024-09-13T16:36:34.499 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:34 smithi107 podman[67493]: 2024-09-13 16:36:34.498109204 +0000 UTC m=+0.431290087 container attach 5c3e73b9bc116068c3d855d16a761e71a2cc6b49c2e12f8b9c61a238835ca77e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10-deactivate, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T16:36:34.849 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 13 16:36:34 smithi107 podman[67493]: 2024-09-13 16:36:34.604415642 +0000 UTC m=+0.537596517 container died 5c3e73b9bc116068c3d855d16a761e71a2cc6b49c2e12f8b9c61a238835ca77e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-13T16:36:35.178 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.10.service' 2024-09-13T16:36:35.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:34 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:34.822+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:35.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:34 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:34.822+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:35.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:34 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:34.822+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:35.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:34 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:34.822+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:35.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:34 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:34.822+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:52.840617+0000 front 2024-09-13T16:35:52.840629+0000 (oldest deadline 2024-09-13T16:36:18.740698+0000) 2024-09-13T16:36:35.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:34 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:34.822+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-13T16:35:59.241670+0000 front 2024-09-13T16:35:59.241644+0000 (oldest deadline 2024-09-13T16:36:22.741612+0000) 2024-09-13T16:36:35.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:34 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:34.822+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-09-13T16:36:08.542889+0000 front 2024-09-13T16:36:08.542872+0000 (oldest deadline 2024-09-13T16:36:30.242787+0000) 2024-09-13T16:36:35.216 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T16:36:35.216 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-13T16:36:35.216 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-13T16:36:35.216 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.11 2024-09-13T16:36:35.546 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:35 smithi111 systemd[1]: Stopping Ceph osd.11 for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:36:35.798 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:35 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:35.544+0000 7fa9cda39640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T16:36:35.798 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:35 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:35.544+0000 7fa9cda39640 -1 osd.11 97 *** Got signal Terminated *** 2024-09-13T16:36:35.798 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:35 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:35.544+0000 7fa9cda39640 -1 osd.11 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T16:36:35.798 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:35 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:35.796+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:35.798 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:35 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:35.796+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:35.798 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:35 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:35.796+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:35.798 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:35 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:35.796+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:35.798 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:35 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:35.796+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:52.840617+0000 front 2024-09-13T16:35:52.840629+0000 (oldest deadline 2024-09-13T16:36:18.740698+0000) 2024-09-13T16:36:35.798 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:35 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:35.796+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-13T16:35:59.241670+0000 front 2024-09-13T16:35:59.241644+0000 (oldest deadline 2024-09-13T16:36:22.741612+0000) 2024-09-13T16:36:35.798 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:35 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:35.796+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-09-13T16:36:08.542889+0000 front 2024-09-13T16:36:08.542872+0000 (oldest deadline 2024-09-13T16:36:30.242787+0000) 2024-09-13T16:36:37.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:36 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:36.762+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:37.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:36 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:36.762+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:37.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:36 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:36.762+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:37.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:36 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:36.762+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:37.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:36 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:36.762+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:52.840617+0000 front 2024-09-13T16:35:52.840629+0000 (oldest deadline 2024-09-13T16:36:18.740698+0000) 2024-09-13T16:36:37.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:36 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:36.762+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-13T16:35:59.241670+0000 front 2024-09-13T16:35:59.241644+0000 (oldest deadline 2024-09-13T16:36:22.741612+0000) 2024-09-13T16:36:37.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:36 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:36.762+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-09-13T16:36:08.542889+0000 front 2024-09-13T16:36:08.542872+0000 (oldest deadline 2024-09-13T16:36:30.242787+0000) 2024-09-13T16:36:38.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:37 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:37.783+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:38.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:37 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:37.783+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:38.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:37 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:37.783+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:38.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:37 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:37.783+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:38.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:37 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:37.783+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:52.840617+0000 front 2024-09-13T16:35:52.840629+0000 (oldest deadline 2024-09-13T16:36:18.740698+0000) 2024-09-13T16:36:38.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:37 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:37.783+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-13T16:35:59.241670+0000 front 2024-09-13T16:35:59.241644+0000 (oldest deadline 2024-09-13T16:36:22.741612+0000) 2024-09-13T16:36:38.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:37 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:37.783+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-09-13T16:36:08.542889+0000 front 2024-09-13T16:36:08.542872+0000 (oldest deadline 2024-09-13T16:36:30.242787+0000) 2024-09-13T16:36:39.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:38 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:38.793+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:39.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:38 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:38.793+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:39.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:38 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:38.793+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:39.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:38 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:38.793+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:39.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:38 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:38.793+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:52.840617+0000 front 2024-09-13T16:35:52.840629+0000 (oldest deadline 2024-09-13T16:36:18.740698+0000) 2024-09-13T16:36:39.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:38 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:38.793+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-13T16:35:59.241670+0000 front 2024-09-13T16:35:59.241644+0000 (oldest deadline 2024-09-13T16:36:22.741612+0000) 2024-09-13T16:36:39.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:38 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:38.793+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-09-13T16:36:08.542889+0000 front 2024-09-13T16:36:08.542872+0000 (oldest deadline 2024-09-13T16:36:30.242787+0000) 2024-09-13T16:36:40.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:39 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:39.809+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-13T16:35:09.332696+0000 front 2024-09-13T16:35:09.332623+0000 (oldest deadline 2024-09-13T16:35:34.032751+0000) 2024-09-13T16:36:40.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:39 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:39.809+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-09-13T16:35:28.535688+0000 front 2024-09-13T16:35:28.535826+0000 (oldest deadline 2024-09-13T16:35:53.736400+0000) 2024-09-13T16:36:40.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:39 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:39.809+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-13T16:35:34.837391+0000 front 2024-09-13T16:35:34.837371+0000 (oldest deadline 2024-09-13T16:35:57.737319+0000) 2024-09-13T16:36:40.187 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:39 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:39.809+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-13T16:35:41.139311+0000 front 2024-09-13T16:35:41.139217+0000 (oldest deadline 2024-09-13T16:36:05.839255+0000) 2024-09-13T16:36:40.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:39 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:39.809+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-09-13T16:35:52.840617+0000 front 2024-09-13T16:35:52.840629+0000 (oldest deadline 2024-09-13T16:36:18.740698+0000) 2024-09-13T16:36:40.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:39 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:39.809+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-13T16:35:59.241670+0000 front 2024-09-13T16:35:59.241644+0000 (oldest deadline 2024-09-13T16:36:22.741612+0000) 2024-09-13T16:36:40.188 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:39 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11[60359]: 2024-09-13T16:36:39.809+0000 7fa9ca04d640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-09-13T16:36:08.542889+0000 front 2024-09-13T16:36:08.542872+0000 (oldest deadline 2024-09-13T16:36:30.242787+0000) 2024-09-13T16:36:41.288 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:40 smithi111 podman[69865]: 2024-09-13 16:36:40.9991435 +0000 UTC m=+5.584911774 container died e5bf2a1c5f2794f217a26937eee628da9b96626b5adc31c86d80e4b9419b8e48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-13T16:36:41.629 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:41 smithi111 podman[69865]: 2024-09-13 16:36:41.369219222 +0000 UTC m=+5.954987488 container cleanup e5bf2a1c5f2794f217a26937eee628da9b96626b5adc31c86d80e4b9419b8e48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-13T16:36:41.629 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:41 smithi111 bash[69865]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11 2024-09-13T16:36:41.937 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:41 smithi111 podman[69881]: 2024-09-13 16:36:41.628725865 +0000 UTC m=+0.628365612 container remove e5bf2a1c5f2794f217a26937eee628da9b96626b5adc31c86d80e4b9419b8e48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T16:36:42.265 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:41 smithi111 podman[69973]: 2024-09-13 16:36:41.850084645 +0000 UTC m=+0.016144486 image pull 152d8bc6b19e354515e8ba109d4619445de0e09b4d4a747c2f16eb5c0edb670c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a 2024-09-13T16:36:42.266 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:41 smithi111 podman[69973]: 2024-09-13 16:36:41.954510856 +0000 UTC m=+0.120570709 container create 6bc4ded5861d814f1ec48deb61f7493d10fc8ef991b9012231ce2fcb83cc5cbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T16:36:42.266 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:42 smithi111 podman[69973]: 2024-09-13 16:36:42.181688119 +0000 UTC m=+0.347747961 container init 6bc4ded5861d814f1ec48deb61f7493d10fc8ef991b9012231ce2fcb83cc5cbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11-deactivate, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-13T16:36:42.266 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:42 smithi111 podman[69973]: 2024-09-13 16:36:42.185097289 +0000 UTC m=+0.351157134 container start 6bc4ded5861d814f1ec48deb61f7493d10fc8ef991b9012231ce2fcb83cc5cbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-13T16:36:42.266 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:42 smithi111 podman[69973]: 2024-09-13 16:36:42.265062134 +0000 UTC m=+0.431121980 container attach 6bc4ded5861d814f1ec48deb61f7493d10fc8ef991b9012231ce2fcb83cc5cbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-13T16:36:42.668 INFO:journalctl@ceph.osd.11.smithi111.stdout:Sep 13 16:36:42 smithi111 podman[69973]: 2024-09-13 16:36:42.371824671 +0000 UTC m=+0.537884522 container died 6bc4ded5861d814f1ec48deb61f7493d10fc8ef991b9012231ce2fcb83cc5cbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-osd-11-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T16:36:43.003 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@osd.11.service' 2024-09-13T16:36:43.041 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T16:36:43.042 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-13T16:36:43.042 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-13T16:36:43.042 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.a 2024-09-13T16:36:43.354 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 13 16:36:43 smithi016 systemd[1]: Stopping Ceph mds.a for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:36:43.854 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 13 16:36:43 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-a[72815]: 2024-09-13T16:36:43.353+0000 7fa722fbb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T16:36:43.854 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 13 16:36:43 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-a[72815]: 2024-09-13T16:36:43.353+0000 7fa722fbb640 -1 mds.a *** got signal Terminated *** 2024-09-13T16:36:47.354 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 13 16:36:47 smithi016 podman[90268]: 2024-09-13 16:36:47.05098295 +0000 UTC m=+3.804358726 container died 25504be1b45da5315437bd4db20ffc5a815f5b7f04165e5851232f373162b4af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-a, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-13T16:36:47.705 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 13 16:36:47 smithi016 podman[90268]: 2024-09-13 16:36:47.444741373 +0000 UTC m=+4.198117147 container cleanup 25504be1b45da5315437bd4db20ffc5a815f5b7f04165e5851232f373162b4af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-a, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-13T16:36:47.705 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 13 16:36:47 smithi016 bash[90268]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-a 2024-09-13T16:36:47.882 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.a.service' 2024-09-13T16:36:47.957 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T16:36:47.958 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-13T16:36:47.958 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-13T16:36:47.958 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.d 2024-09-13T16:36:48.283 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 13 16:36:48 smithi016 systemd[1]: Stopping Ceph mds.d for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:36:48.604 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 13 16:36:48 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-d[71868]: 2024-09-13T16:36:48.282+0000 7f5abcdeb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T16:36:48.604 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 13 16:36:48 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-d[71868]: 2024-09-13T16:36:48.282+0000 7f5abcdeb640 -1 mds.d *** got signal Terminated *** 2024-09-13T16:36:50.492 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 13 16:36:50 smithi016 podman[90392]: 2024-09-13 16:36:50.222055893 +0000 UTC m=+2.042820497 container died 47749514ffd28e2056c7e484e67039385062e5b015fe07ee20a26486f4cfafca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-d, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-13T16:36:50.854 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 13 16:36:50 smithi016 podman[90392]: 2024-09-13 16:36:50.573645482 +0000 UTC m=+2.394410085 container cleanup 47749514ffd28e2056c7e484e67039385062e5b015fe07ee20a26486f4cfafca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-d, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-13T16:36:50.854 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 13 16:36:50 smithi016 bash[90392]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-d 2024-09-13T16:36:51.066 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.d.service' 2024-09-13T16:36:51.099 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T16:36:51.100 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-13T16:36:51.100 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-13T16:36:51.100 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.g 2024-09-13T16:36:51.420 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 13 16:36:51 smithi016 systemd[1]: Stopping Ceph mds.g for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:36:51.854 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 13 16:36:51 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-g[72459]: 2024-09-13T16:36:51.419+0000 7ff2e6e15640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T16:36:51.854 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 13 16:36:51 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-g[72459]: 2024-09-13T16:36:51.419+0000 7ff2e6e15640 -1 mds.g *** got signal Terminated *** 2024-09-13T16:36:55.337 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 13 16:36:55 smithi016 podman[90515]: 2024-09-13 16:36:55.056540676 +0000 UTC m=+3.738195957 container died 8d4e0e08d9fab43c017a9e36c63899dbd7aef9c5bbf1fee8e442896c261d927d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-g, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-13T16:36:55.604 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 13 16:36:55 smithi016 podman[90515]: 2024-09-13 16:36:55.427636679 +0000 UTC m=+4.109291964 container cleanup 8d4e0e08d9fab43c017a9e36c63899dbd7aef9c5bbf1fee8e442896c261d927d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-13T16:36:55.604 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 13 16:36:55 smithi016 bash[90515]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-g 2024-09-13T16:36:55.875 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.g.service' 2024-09-13T16:36:55.908 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T16:36:55.908 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-13T16:36:55.908 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-13T16:36:55.909 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.j 2024-09-13T16:36:56.241 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 13 16:36:55 smithi016 systemd[1]: Stopping Ceph mds.j for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:36:56.604 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 13 16:36:56 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-j[71524]: 2024-09-13T16:36:56.239+0000 7fcab4537640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T16:36:56.605 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 13 16:36:56 smithi016 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-j[71524]: 2024-09-13T16:36:56.239+0000 7fcab4537640 -1 mds.j *** got signal Terminated *** 2024-09-13T16:37:00.041 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 13 16:36:59 smithi016 podman[90635]: 2024-09-13 16:36:59.682289852 +0000 UTC m=+3.549090633 container died 216caf59eb38152ceaf1f71f153c5268526fcd3a91e1c6a95ade5cb10d2d0588 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-j, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, ceph=True) 2024-09-13T16:37:00.354 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 13 16:37:00 smithi016 podman[90635]: 2024-09-13 16:37:00.139597372 +0000 UTC m=+4.006398154 container cleanup 216caf59eb38152ceaf1f71f153c5268526fcd3a91e1c6a95ade5cb10d2d0588 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f) 2024-09-13T16:37:00.354 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 13 16:37:00 smithi016 bash[90635]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-j 2024-09-13T16:37:00.629 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.j.service' 2024-09-13T16:37:00.662 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T16:37:00.662 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-13T16:37:00.663 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-13T16:37:00.663 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.b 2024-09-13T16:37:00.975 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 13 16:37:00 smithi107 systemd[1]: Stopping Ceph mds.b for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:37:01.289 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 13 16:37:00 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-b[63478]: 2024-09-13T16:37:00.974+0000 7fed8d3a9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T16:37:01.290 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 13 16:37:00 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-b[63478]: 2024-09-13T16:37:00.974+0000 7fed8d3a9640 -1 mds.b *** got signal Terminated *** 2024-09-13T16:37:03.349 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 13 16:37:03 smithi107 podman[67597]: 2024-09-13 16:37:03.065786289 +0000 UTC m=+2.209364114 container died 5a0a934d5a9ee524a772f55cf809b59b520dcd87217ed4405ea7f95dd32ea159 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-09-13T16:37:03.683 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 13 16:37:03 smithi107 podman[67597]: 2024-09-13 16:37:03.440119561 +0000 UTC m=+2.583697387 container cleanup 5a0a934d5a9ee524a772f55cf809b59b520dcd87217ed4405ea7f95dd32ea159 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T16:37:03.684 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 13 16:37:03 smithi107 bash[67597]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-b 2024-09-13T16:37:03.837 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.b.service' 2024-09-13T16:37:03.874 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T16:37:03.874 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-13T16:37:03.874 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-13T16:37:03.875 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.e 2024-09-13T16:37:04.203 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 13 16:37:03 smithi107 systemd[1]: Stopping Ceph mds.e for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:37:04.599 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 13 16:37:04 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-e[62873]: 2024-09-13T16:37:04.201+0000 7f5588011640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T16:37:04.599 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 13 16:37:04 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-e[62873]: 2024-09-13T16:37:04.201+0000 7f5588011640 -1 mds.e *** got signal Terminated *** 2024-09-13T16:37:07.328 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 13 16:37:07 smithi107 podman[67729]: 2024-09-13 16:37:07.070685609 +0000 UTC m=+2.973574352 container died b50793f01f1926f2c81823a85d43b59f612f1e7749a8d4d5875584ebea278c11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-e, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-09-13T16:37:07.600 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 13 16:37:07 smithi107 podman[67729]: 2024-09-13 16:37:07.435293018 +0000 UTC m=+3.338181763 container cleanup b50793f01f1926f2c81823a85d43b59f612f1e7749a8d4d5875584ebea278c11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T16:37:07.600 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 13 16:37:07 smithi107 bash[67729]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-e 2024-09-13T16:37:07.870 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.e.service' 2024-09-13T16:37:07.903 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T16:37:07.903 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-13T16:37:07.903 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-13T16:37:07.903 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.h 2024-09-13T16:37:08.216 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 13 16:37:07 smithi107 systemd[1]: Stopping Ceph mds.h for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:37:08.599 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 13 16:37:08 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-h[63177]: 2024-09-13T16:37:08.212+0000 7f18d233b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T16:37:08.599 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 13 16:37:08 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-h[63177]: 2024-09-13T16:37:08.212+0000 7f18d233b640 -1 mds.h *** got signal Terminated *** 2024-09-13T16:37:11.315 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 13 16:37:11 smithi107 podman[67850]: 2024-09-13 16:37:11.059031529 +0000 UTC m=+2.934365513 container died db4523075397bd49374053864aa87e9b11d837f77c95ec8207d82f9fc537d013 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T16:37:11.590 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 13 16:37:11 smithi107 podman[67850]: 2024-09-13 16:37:11.396724829 +0000 UTC m=+3.272058855 container cleanup db4523075397bd49374053864aa87e9b11d837f77c95ec8207d82f9fc537d013 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-h, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T16:37:11.590 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 13 16:37:11 smithi107 bash[67850]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-h 2024-09-13T16:37:11.821 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.h.service' 2024-09-13T16:37:11.842 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 13 16:37:11 smithi107 podman[67864]: 2024-09-13 16:37:11.589402812 +0000 UTC m=+0.519821111 container remove db4523075397bd49374053864aa87e9b11d837f77c95ec8207d82f9fc537d013 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-13T16:37:11.843 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 13 16:37:11 smithi107 systemd[1]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.h.service: Deactivated successfully. 2024-09-13T16:37:11.843 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 13 16:37:11 smithi107 systemd[1]: Stopped Ceph mds.h for 0d241f6a-71ec-11ef-bceb-c7b262605968. 2024-09-13T16:37:11.843 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 13 16:37:11 smithi107 systemd[1]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.h.service: Consumed 1.283s CPU time. 2024-09-13T16:37:11.854 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T16:37:11.854 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-13T16:37:11.854 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-13T16:37:11.854 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.k 2024-09-13T16:37:12.099 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 13 16:37:11 smithi107 systemd[1]: Stopping Ceph mds.k for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:37:12.599 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 13 16:37:12 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-k[62559]: 2024-09-13T16:37:12.190+0000 7f4b477eb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T16:37:12.599 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 13 16:37:12 smithi107 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-k[62559]: 2024-09-13T16:37:12.190+0000 7f4b477eb640 -1 mds.k *** got signal Terminated *** 2024-09-13T16:37:15.334 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 13 16:37:15 smithi107 podman[67971]: 2024-09-13 16:37:15.067099719 +0000 UTC m=+2.995284583 container died 9fb9b76f26adca92252d49b8495bc90340a302ae885df9c66001958c1afb0a07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-k, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=) 2024-09-13T16:37:15.599 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 13 16:37:15 smithi107 podman[67971]: 2024-09-13 16:37:15.433082327 +0000 UTC m=+3.361267176 container cleanup 9fb9b76f26adca92252d49b8495bc90340a302ae885df9c66001958c1afb0a07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-k, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True) 2024-09-13T16:37:15.599 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 13 16:37:15 smithi107 bash[67971]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-k 2024-09-13T16:37:15.813 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.k.service' 2024-09-13T16:37:15.845 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T16:37:15.846 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-13T16:37:15.846 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-13T16:37:15.846 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.c 2024-09-13T16:37:16.187 INFO:journalctl@ceph.mds.c.smithi111.stdout:Sep 13 16:37:15 smithi111 systemd[1]: Stopping Ceph mds.c for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:37:16.438 INFO:journalctl@ceph.mds.c.smithi111.stdout:Sep 13 16:37:16 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-c[65481]: 2024-09-13T16:37:16.225+0000 7f9dd6243640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T16:37:16.439 INFO:journalctl@ceph.mds.c.smithi111.stdout:Sep 13 16:37:16 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-c[65481]: 2024-09-13T16:37:16.225+0000 7f9dd6243640 -1 mds.c *** got signal Terminated *** 2024-09-13T16:37:19.344 INFO:journalctl@ceph.mds.c.smithi111.stdout:Sep 13 16:37:19 smithi111 podman[70075]: 2024-09-13 16:37:19.064301542 +0000 UTC m=+3.024115347 container died 1ae73ad6ef9689571ff8cce6fcf122db89ad545fc1007c14138642e70b9b8a93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-c, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-13T16:37:19.687 INFO:journalctl@ceph.mds.c.smithi111.stdout:Sep 13 16:37:19 smithi111 podman[70075]: 2024-09-13 16:37:19.450252062 +0000 UTC m=+3.410065867 container cleanup 1ae73ad6ef9689571ff8cce6fcf122db89ad545fc1007c14138642e70b9b8a93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-c, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.build-date=20240731) 2024-09-13T16:37:19.687 INFO:journalctl@ceph.mds.c.smithi111.stdout:Sep 13 16:37:19 smithi111 bash[70075]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-c 2024-09-13T16:37:19.839 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.c.service' 2024-09-13T16:37:19.877 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T16:37:19.877 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-13T16:37:19.877 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-13T16:37:19.877 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.f 2024-09-13T16:37:20.437 INFO:journalctl@ceph.mds.f.smithi111.stdout:Sep 13 16:37:19 smithi111 systemd[1]: Stopping Ceph mds.f for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:37:20.437 INFO:journalctl@ceph.mds.f.smithi111.stdout:Sep 13 16:37:20 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-f[65175]: 2024-09-13T16:37:20.178+0000 7f50bc7c7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T16:37:20.437 INFO:journalctl@ceph.mds.f.smithi111.stdout:Sep 13 16:37:20 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-f[65175]: 2024-09-13T16:37:20.178+0000 7f50bc7c7640 -1 mds.f *** got signal Terminated *** 2024-09-13T16:37:23.356 INFO:journalctl@ceph.mds.f.smithi111.stdout:Sep 13 16:37:23 smithi111 podman[70207]: 2024-09-13 16:37:23.068239988 +0000 UTC m=+2.969327255 container died 19749af66c83e3b0f774b86a4cb51b62d989980924771edd504b2d72f739225a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, ceph=True) 2024-09-13T16:37:23.687 INFO:journalctl@ceph.mds.f.smithi111.stdout:Sep 13 16:37:23 smithi111 podman[70207]: 2024-09-13 16:37:23.437501886 +0000 UTC m=+3.338589139 container cleanup 19749af66c83e3b0f774b86a4cb51b62d989980924771edd504b2d72f739225a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-13T16:37:23.687 INFO:journalctl@ceph.mds.f.smithi111.stdout:Sep 13 16:37:23 smithi111 bash[70207]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-f 2024-09-13T16:37:23.866 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.f.service' 2024-09-13T16:37:23.899 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T16:37:23.899 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-13T16:37:23.899 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-13T16:37:23.900 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.i 2024-09-13T16:37:24.217 INFO:journalctl@ceph.mds.i.smithi111.stdout:Sep 13 16:37:23 smithi111 systemd[1]: Stopping Ceph mds.i for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:37:24.687 INFO:journalctl@ceph.mds.i.smithi111.stdout:Sep 13 16:37:24 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-i[65785]: 2024-09-13T16:37:24.216+0000 7f0e8cb61640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T16:37:24.687 INFO:journalctl@ceph.mds.i.smithi111.stdout:Sep 13 16:37:24 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-i[65785]: 2024-09-13T16:37:24.216+0000 7f0e8cb61640 -1 mds.i *** got signal Terminated *** 2024-09-13T16:37:27.187 INFO:journalctl@ceph.mds.i.smithi111.stdout:Sep 13 16:37:27 smithi111 podman[70331]: 2024-09-13 16:37:27.052550628 +0000 UTC m=+2.940271591 container died d958eadab8860f3ca941aed13e916a331b502ce8e63041a17d80f3566d19d196 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-i, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-13T16:37:27.667 INFO:journalctl@ceph.mds.i.smithi111.stdout:Sep 13 16:37:27 smithi111 podman[70331]: 2024-09-13 16:37:27.449411934 +0000 UTC m=+3.337132897 container cleanup d958eadab8860f3ca941aed13e916a331b502ce8e63041a17d80f3566d19d196 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-i, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f) 2024-09-13T16:37:27.668 INFO:journalctl@ceph.mds.i.smithi111.stdout:Sep 13 16:37:27 smithi111 bash[70331]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-i 2024-09-13T16:37:27.806 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.i.service' 2024-09-13T16:37:27.839 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T16:37:27.840 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-13T16:37:27.840 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-13T16:37:27.840 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.l 2024-09-13T16:37:28.179 INFO:journalctl@ceph.mds.l.smithi111.stdout:Sep 13 16:37:27 smithi111 systemd[1]: Stopping Ceph mds.l for 0d241f6a-71ec-11ef-bceb-c7b262605968... 2024-09-13T16:37:28.437 INFO:journalctl@ceph.mds.l.smithi111.stdout:Sep 13 16:37:28 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-l[64860]: 2024-09-13T16:37:28.177+0000 7f3a6230b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T16:37:28.437 INFO:journalctl@ceph.mds.l.smithi111.stdout:Sep 13 16:37:28 smithi111 ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-l[64860]: 2024-09-13T16:37:28.177+0000 7f3a6230b640 -1 mds.l *** got signal Terminated *** 2024-09-13T16:37:31.337 INFO:journalctl@ceph.mds.l.smithi111.stdout:Sep 13 16:37:31 smithi111 podman[70450]: 2024-09-13 16:37:31.051935286 +0000 UTC m=+2.998972265 container died aad66e58c1dccfadeecef3774715ac4de96bcaf827395fb6edb6e9618971a840 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-l, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-13T16:37:31.687 INFO:journalctl@ceph.mds.l.smithi111.stdout:Sep 13 16:37:31 smithi111 podman[70450]: 2024-09-13 16:37:31.403188852 +0000 UTC m=+3.350225830 container cleanup aad66e58c1dccfadeecef3774715ac4de96bcaf827395fb6edb6e9618971a840 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:195bcc5557a21b1fc5da9f42b2147dd7b8d9c3a0edf068f394db39472062247a, name=ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-l, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.095512-reef-50bbe7f, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-13T16:37:31.687 INFO:journalctl@ceph.mds.l.smithi111.stdout:Sep 13 16:37:31 smithi111 bash[70450]: ceph-0d241f6a-71ec-11ef-bceb-c7b262605968-mds-l 2024-09-13T16:37:31.822 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0d241f6a-71ec-11ef-bceb-c7b262605968@mds.l.service' 2024-09-13T16:37:31.853 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T16:37:31.854 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-13T16:37:31.854 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-13T16:37:32.013 INFO:teuthology.orchestra.run.smithi016.stdout:Deleting cluster with fsid: 0d241f6a-71ec-11ef-bceb-c7b262605968 2024-09-13T16:37:34.290 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-13T16:37:34.442 INFO:teuthology.orchestra.run.smithi107.stdout:Deleting cluster with fsid: 0d241f6a-71ec-11ef-bceb-c7b262605968 2024-09-13T16:37:36.746 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-13T16:37:36.900 INFO:teuthology.orchestra.run.smithi111.stdout:Deleting cluster with fsid: 0d241f6a-71ec-11ef-bceb-c7b262605968 2024-09-13T16:37:39.049 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-13T16:37:39.083 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-13T16:37:39.116 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-13T16:37:39.142 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-13T16:37:39.145 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903803/remote/smithi016/crash 2024-09-13T16:37:39.145 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/crash -- . 2024-09-13T16:37:39.191 INFO:teuthology.orchestra.run.smithi016.stderr:tar: /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-13T16:37:39.191 INFO:teuthology.orchestra.run.smithi016.stderr:tar: Error is not recoverable: exiting now 2024-09-13T16:37:39.193 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903803/remote/smithi107/crash 2024-09-13T16:37:39.194 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/crash -- . 2024-09-13T16:37:39.237 INFO:teuthology.orchestra.run.smithi107.stderr:tar: /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-13T16:37:39.237 INFO:teuthology.orchestra.run.smithi107.stderr:tar: Error is not recoverable: exiting now 2024-09-13T16:37:39.239 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903803/remote/smithi111/crash 2024-09-13T16:37:39.240 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/crash -- . 2024-09-13T16:37:39.276 INFO:teuthology.orchestra.run.smithi111.stderr:tar: /var/lib/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-13T16:37:39.277 INFO:teuthology.orchestra.run.smithi111.stderr:tar: Error is not recoverable: exiting now 2024-09-13T16:37:39.278 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-13T16:37:39.278 DEBUG:teuthology.orchestra.run.smithi016:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-13T16:37:39.313 INFO:teuthology.orchestra.run.smithi016.stderr:grep: /var/log/ceph/0d241f6a-71ec-11ef-bceb-c7b262605968/ceph.log: No such file or directory 2024-09-13T16:37:39.316 INFO:tasks.cephadm:Compressing logs... 2024-09-13T16:37:39.316 DEBUG:teuthology.orchestra.run.smithi016:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T16:37:39.358 DEBUG:teuthology.orchestra.run.smithi107:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T16:37:39.360 DEBUG:teuthology.orchestra.run.smithi111:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T16:37:39.383 INFO:teuthology.orchestra.run.smithi016.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-13T16:37:39.384 INFO:teuthology.orchestra.run.smithi111.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-13T16:37:39.385 INFO:teuthology.orchestra.run.smithi107.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-13T16:38:16.844 INFO:tasks.cephadm:Archiving logs... 2024-09-13T16:38:16.845 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903803/remote/smithi016/log 2024-09-13T16:38:16.846 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T16:38:20.833 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903803/remote/smithi107/log 2024-09-13T16:38:20.834 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T16:38:23.691 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903803/remote/smithi111/log 2024-09-13T16:38:23.692 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T16:38:26.469 INFO:tasks.cephadm:Removing cluster... 2024-09-13T16:38:26.469 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 --force 2024-09-13T16:38:26.628 INFO:teuthology.orchestra.run.smithi016.stdout:Deleting cluster with fsid: 0d241f6a-71ec-11ef-bceb-c7b262605968 2024-09-13T16:38:26.930 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 --force 2024-09-13T16:38:27.083 INFO:teuthology.orchestra.run.smithi107.stdout:Deleting cluster with fsid: 0d241f6a-71ec-11ef-bceb-c7b262605968 2024-09-13T16:38:27.368 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0d241f6a-71ec-11ef-bceb-c7b262605968 --force 2024-09-13T16:38:27.519 INFO:teuthology.orchestra.run.smithi111.stdout:Deleting cluster with fsid: 0d241f6a-71ec-11ef-bceb-c7b262605968 2024-09-13T16:38:27.805 INFO:tasks.cephadm:Removing cephadm ... 2024-09-13T16:38:27.805 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-13T16:38:27.824 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-13T16:38:27.840 DEBUG:teuthology.orchestra.run.smithi111:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-13T16:38:27.856 INFO:tasks.cephadm:Teardown complete 2024-09-13T16:38:27.856 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-13T16:38:27.871 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-13T16:38:27.871 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T16:38:27.874 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T16:38:27.883 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T16:38:28.014 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-13T16:38:28.024 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-13T16:38:28.024 DEBUG:teuthology.orchestra.run.smithi016:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T16:38:28.027 DEBUG:teuthology.orchestra.run.smithi107:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T16:38:28.029 DEBUG:teuthology.orchestra.run.smithi111:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T16:38:28.041 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-09-13T16:38:28.042 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-09-13T16:38:28.043 INFO:teuthology.orchestra.run.smithi111.stderr:bash: line 1: ntpq: command not found 2024-09-13T16:38:28.045 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T16:38:28.046 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-09-13T16:38:28.046 INFO:teuthology.orchestra.run.smithi016.stdout:^* hv01.front.sepia.ceph.com 3 6 377 26 +213us[ +213us] +/- 58ms 2024-09-13T16:38:28.046 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 89 +138us[ +132us] +/- 58ms 2024-09-13T16:38:28.046 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 -465us[ -465us] +/- 77ms 2024-09-13T16:38:28.046 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T16:38:28.047 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T16:38:28.047 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-09-13T16:38:28.048 INFO:teuthology.orchestra.run.smithi107.stdout:^* hv01.front.sepia.ceph.com 3 6 377 26 +219us[ +215us] +/- 58ms 2024-09-13T16:38:28.048 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 25 +117us[ +117us] +/- 59ms 2024-09-13T16:38:28.048 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 -454us[ -458us] +/- 77ms 2024-09-13T16:38:28.048 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T16:38:28.049 INFO:teuthology.orchestra.run.smithi111.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T16:38:28.049 INFO:teuthology.orchestra.run.smithi111.stdout:=============================================================================== 2024-09-13T16:38:28.049 INFO:teuthology.orchestra.run.smithi111.stdout:^* hv01.front.sepia.ceph.com 3 6 377 22 +217us[ +222us] +/- 58ms 2024-09-13T16:38:28.049 INFO:teuthology.orchestra.run.smithi111.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 24 +127us[ +132us] +/- 59ms 2024-09-13T16:38:28.049 INFO:teuthology.orchestra.run.smithi111.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 -467us[ -462us] +/- 77ms 2024-09-13T16:38:28.049 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T16:38:28.049 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-13T16:38:28.059 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-13T16:38:28.060 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-13T16:38:28.070 DEBUG:teuthology.orchestra.run.smithi016:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-13T16:38:28.172 DEBUG:teuthology.orchestra.run.smithi107:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-13T16:38:28.240 DEBUG:teuthology.orchestra.run.smithi111:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-13T16:38:28.306 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-13T16:38:28.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:38:28.338 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-13T16:38:28.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:38:28.365 DEBUG:teuthology.orchestra.run.smithi111:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-13T16:38:28.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:38:28.393 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-13T16:38:28.406 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-13T16:38:28.449 INFO:teuthology.task.internal:Duration was 1425.547302 seconds 2024-09-13T16:38:28.449 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-13T16:38:28.460 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-13T16:38:28.460 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T16:38:28.463 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T16:38:28.465 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T16:38:28.501 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T16:38:28.503 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T16:38:28.504 INFO:teuthology.orchestra.run.smithi111.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T16:38:28.882 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-13T16:38:28.882 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi016.front.sepia.ceph.com 2024-09-13T16:38:28.882 DEBUG:teuthology.orchestra.run.smithi016:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T16:38:28.904 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi107.front.sepia.ceph.com 2024-09-13T16:38:28.905 DEBUG:teuthology.orchestra.run.smithi107:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T16:38:28.947 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi111.front.sepia.ceph.com 2024-09-13T16:38:28.947 DEBUG:teuthology.orchestra.run.smithi111:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T16:38:28.968 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-13T16:38:28.968 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T16:38:28.971 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T16:38:28.989 DEBUG:teuthology.orchestra.run.smithi111:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T16:38:29.034 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-13T16:38:29.034 DEBUG:teuthology.orchestra.run.smithi016:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T16:38:29.040 DEBUG:teuthology.orchestra.run.smithi107:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T16:38:29.055 DEBUG:teuthology.orchestra.run.smithi111:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T16:38:29.475 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-13T16:38:29.490 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-13T16:38:29.490 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T16:38:29.516 DEBUG:teuthology.orchestra.run.smithi107:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T16:38:29.542 DEBUG:teuthology.orchestra.run.smithi111:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T16:38:29.567 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-13T16:38:29.578 DEBUG:teuthology.orchestra.run.smithi016:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T16:38:29.581 DEBUG:teuthology.orchestra.run.smithi107:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T16:38:29.584 DEBUG:teuthology.orchestra.run.smithi111:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T16:38:29.602 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = core 2024-09-13T16:38:29.605 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = core 2024-09-13T16:38:29.630 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern = core 2024-09-13T16:38:29.656 DEBUG:teuthology.orchestra.run.smithi016:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T16:38:29.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:38:29.672 DEBUG:teuthology.orchestra.run.smithi107:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T16:38:29.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:38:29.686 DEBUG:teuthology.orchestra.run.smithi111:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T16:38:29.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:38:29.711 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-13T16:38:29.721 INFO:teuthology.task.internal:Transferring archived files... 2024-09-13T16:38:29.724 DEBUG:teuthology.misc:Transferring archived files from smithi016:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903803/remote/smithi016 2024-09-13T16:38:29.724 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T16:38:29.767 DEBUG:teuthology.misc:Transferring archived files from smithi107:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903803/remote/smithi107 2024-09-13T16:38:29.768 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T16:38:29.804 DEBUG:teuthology.misc:Transferring archived files from smithi111:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903803/remote/smithi111 2024-09-13T16:38:29.805 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T16:38:29.847 INFO:teuthology.task.internal:Removing archive directory... 2024-09-13T16:38:29.847 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T16:38:29.850 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T16:38:29.852 DEBUG:teuthology.orchestra.run.smithi111:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T16:38:29.897 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-13T16:38:29.915 INFO:teuthology.task.internal:Not uploading archives. 2024-09-13T16:38:29.915 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-13T16:38:29.926 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-13T16:38:29.926 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T16:38:29.929 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T16:38:29.934 DEBUG:teuthology.orchestra.run.smithi111:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T16:38:29.942 INFO:teuthology.orchestra.run.smithi016.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 16:38 /home/ubuntu/cephtest 2024-09-13T16:38:29.947 INFO:teuthology.orchestra.run.smithi107.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 16:38 /home/ubuntu/cephtest 2024-09-13T16:38:29.951 INFO:teuthology.orchestra.run.smithi111.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 16:38 /home/ubuntu/cephtest 2024-09-13T16:38:29.952 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-13T16:38:29.964 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-13T16:38:30.031 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/iozone}} duration: 1425.547301530838 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-09-13T16:38:30.031 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T16:38:30.133 INFO:teuthology.run:pass